Is Pulsara Down Right Now? Check if there is a current outage ongoing.

This incident has been resolved.

Pulsara's third-party video platform has restored all services, and Pulsara Calls are working. Pulsara will continue to monitor this issue.

The Pulsara Calling feature is temporarily unavailable because our third-party partner is down. Pulsara is investigating this issue and will update the Pulsara Status Page when more information is available.

The Pulsara platform was temporarily unavailable from approximately 15:44 to 15:46 US Mountain time today. A crash of a cloud vendor component resulted in errors for all application functionality. Automatic monitoring and recovery systems were initiated. All systems are back online, and normal operations have resumed.

Google FCM incident status page has announced that they have mitigated this issue and are currently monitoring it. As Pulsara opened this incident as a helpful way of providing transparency to this event, we will close out this incident now. For further information, please visit https://status.firebase.google.com/incidents/PTWWAYPjkjPapM4qkGY2. As always, Pulsara will continue monitoring all third-party services to ensure our product is running.

We are continuing to monitor for any further issues.

We have identified and are monitoring an issue with Google Firebase Cloud Messaging (FCM) affecting a small number of notifications sent to Android devices. FCM is aware of this issue and is providing status information on their Firebase Status Dashboard (https://status.firebase.google.com/incidents/PTWWAYPjkjPapM4qkGY2) We will post updates here as they become available from Firebase. Please Note: this does not affect any Apple devices.

**Incident Summary:** On 08/27/2024, our web application experienced an error that prevented some users from accessing their account information. This issue occurred due to a mismatch between the API responses from our older web application and the newer server. **Impact:** Some users could not log in to access their account information or perform actions that required user authentication. **Resolution:** The issue was resolved after completing the web upgrade, including a parsing error fix. We apologize for any inconvenience this incident may have caused. We have already taken steps and remain committed to improving our systems and processes to prevent similar issues in the future.

This incident has been resolved. Pulsara will provide RCA information on this incident when it becomes available.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating this issue.

The Pulsara platform was temporarily unavailable from approximately 18:04 to 18:07 US Mountain time today. A crash of a cloud vendor component resulted in errors for all application functionality. Automatic monitoring and recovery systems were initiated. All systems are back online and normal operations have resumed.

The Pulsara platform experienced a service outage due to a database failure from 13:41 MDT until 13:48 MDT. The Pulsara team was notified by internal monitoring systems about the issue and resolved the issue. All systems and services have recovered and are operating normally. The team continues to monitor all potentially impacted services.

This incident has been resolved.

All calendar filters have now been restored. At this time, all calendar integrations should be functioning normally. We are continuing to monitor this issue.

We are currently working on restoring the calendar filters on a customer-by-customer basis.

The root cause of this issue with the calendar integration has been identified. We are actively working on a resolution and will provide a status update within the next 1-2 hours. Please monitor the call status of your team members who are using the calendar integration during this time.

Our development team is investigating an issue where our calendar integration is not appropriately applying the calendar filters to the on-call status. This may result in team members that use the calendar integration to be placed on call when they should not be.

An issue with the On Call Callendar integration has been identified and is being investigated.

Microsoft has resolved its DNS issue.

Beginning 07:30 US Mountain time on 10/25/2023, we began seeing intermittent failures with Edge push notifications. There appears to be a global issue with Microsoft Windows Notification Service (WNS) related to DNS entries for a portion of their infrastructure. This is not a problem specific to Pulsara or a problem that is in Pulsara's control to fix. We recommend using Google Chrome if possible during this degraded performance of Microsoft Edge.

According to our monitoring, Microsoft resolved an issue on Friday, August 11th at approximately 0500 MST. We will continue to monitor the situation. At this time, all Microsoft Edge browser push notification functionality appears to be functioning normally.

We are monitoring this issue to ensure that the issue is resolved.

Microsoft recently released an update that disabled some previous settings for notification services in the Edge Browser. This issue has been identified and tied to this recent update from Microsoft Edge. To resolve this issue, please adjust your Edge browser settings to “enable” the Enable system notification setting. You can find this setting by navigating to edge://flags/#enable-system-notifications in your Edge browser. When you navigate this page, you will see the setting “Enable system notifications.” Please make sure that this setting is enabled. After you adjust this setting, we recommend sending yourself a test alert to ensure that alerts are working correctly. Please work with your IT staff if you have problems adjusting this setting.

Issue description: We have identified a Microsoft issue that started at approximately 1100 hrs CDT on 08/09/23. Browser Push notifications sent to customers using Microsoft Edge are not delivered approximately 50% of the time. Impact: Customers using Microsoft Edge EDs and other teams may not receive notifications when logged into Pulsara HQ using the Microsoft Edge browser for inbound patients, consult and transfer requests, etc. Note: New patients in Pulsara HQ will still flash red to notify you of new patients. What do I need to do about it? We strongly recommend that customers relying on Pulsara browser push notifications (audible alerts) for patient awareness switch to Google Chrome until this Microsoft issue is resolved.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently experiencing a service disruption with our browser application. This only affects customers that are using our Federated Authentication integration. Our Development team is working to identify the root cause and implement a solution. If you are a customer utilizing Federated Authentication and see a black screen when logged in, the immediate workaround to get logged into Pulsara is to clear the browser data and then log back into Pulsara.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are continuing to investigate this issue. While the root cause is still being determined, the Pulsara Patient app can still be downloaded directly from the Apple App Store or the Google PlayStore.

We are investigating an issue with our third-party service that delivers SMS Pulsara Patient invites. Currently, SMS invites are being delayed. Our engineering team is working on a resolution, and we will provide another update in 2 hours or as soon as more information becomes available.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are continuing to investigate this issue.

We have identified an intermittent Pulsara MANAGER (Web application) issue that is impacting the ability of some Hospital Admin users to be able to manage the call status of other users. The Mobile application is not impacted and users are able to manage their own call status from within the HQ page or Android / iOS Pulsara MOBILE applications. We are currently working to resolve this issue and will update this page with additional information as it becomes available.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are continuing to work on a fix for this issue.

The root cause of the issue has been identified. A fix is in progress, will be tested, and then deployed to production as soon as testing is complete.

We have identified an issue that has been reported by a few customers. These customers have reported that they have not been able to toggle on call from Pulsara MOBILE. Pulsara Administrators, should any of your users report this issue, our recommendation at this time is that you manage the affected user’s call status via Pulsara MANAGER. Pulsara is currently working to resolve this issue and will follow up with you when the resolution is deployed.

This incident has been resolved.

We are continuing to monitor for any further issues.

As a result of issues with our cloud vendor, we're seeing intermittent issues with requests to our API endpoints.

We are currently experiencing a service disruption. Our Development team is working to identify the root cause and implement a solution.

This incident has been resolved. We will update this incident with the full root cause along with resolution shortly.

A fix has been implemented and we are monitoring the results.

Pulsara customers are having intermittent issues connecting to the Pulsara platform due to a partial outage of our upstream DNS provider. We are actively investigating and have started the process to switch to our backup DNS provider.

Delayed alerting for US customers from 12:27 MT to 14:12 MT

Today, at roughly 14:35 MST our internal monitoring systems alerted us of push notification failures. At 14:38 MST our engineering team discovered that the root cause of this incident was a failure with one of our third party vendors. During this time, no push notifications on either Android or iOS were sent. Pulsara video calls and all other data components continued to work during this time. Our development team continued to monitor this event and prepared backup alternatives to deploy to our production environment if needed. At 14:55 MST our third party vendor resumed working status and push notifications began to work again. At this time, all new push notifications were sent as expected. As of 14:55 MST, Pulsara was completely back up and running at 100%.

This incident has been resolved.

Push notifications have now been restored. We are currently monitoring this event and will have more details to follow.

We have identified an issue with one of our third party vendors. We are actively working on a fix for this. Currently, no push notification are being sent.

We are actively investigating delays in push notification delivery.

Pulsara experienced a brief service outage this morning between 8:45 AM - 9:05 AM Mountain Time. This outage resulted in sporadic errors and timeouts on certain pages/screens during this timeframe, but Pulsara was not completely offline. The issue has now been resolved and all production systems are currently online without errors. A primary root cause has been identified and additional troubleshooting and research is ongoing to determine additional, potential safeguards and remediations that we can put in place.

On Friday February 22nd, we had an unexpected database failure that caused the app to be unresponsive for 2.5 minutes. As we run a full database cluster, no data was lost and a full root cause analysis is being performed to prevent the same failures from occurring again.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We have identified an issue within the system and are currently deploying a fix.

We are continuing to investigate this issue.

We are continuing to monitor this event and will update with more information when available.

We are continuing to investigate this issue.

We have identified delays in message delivery, and are actively investigating. Messages are currently delayed and will update once we resolve the issue. We will send an additional update in 10 minutes.

Pulsara experienced sporadic network outages in the Amazon Web Services US West Oregon region. Here are the full details in chronological order. All times are US Mountain. --- Updated 4:00 PM 10/18/17 We've confirmed that the issue has been resolved and the service is operating normally. --- Updated 3:33 PM 10/18/17 AWS says the issue is resolved. We continue to monitor the situation. --- Updated 3:28 PM 10/18/17 Pulsara services are operating sporadically for our production customers. Sometimes requests succeed, and sometimes they fail. This affects all requests to the Pulsara platform in production. AWS Status page acknowledged errors with AWS connections at 2:59 US Mountain time. See https://status.aws.amazon.com/ Manual testing of the Pulsara app between 3:00 - 3:20 PM US Mountain showed sporadic errors. Some requests from the mobile app resulted in "Previous operation failed" messages.

Pulsara experienced push notification issues for some Android users due to an error with Google's Firebase Messaging System. Here's the full details in chronological order. All times are US Mountain. --- Updated 4:50 PM 5/16/17 Pulsara engineers have updated the Google messaging key from the legacy server key, to a the FCM key per Google’s recommendation, verified the change in test environments, and finally rolled this change out to Production. This issue is fully resolved to the best of our knowledge. With the additional changes to our update our messaging token, we expect that operations are fully restored, and push notifications to all Android users will be sent and delivered as normal going forward. We will keep monitoring server logs this evening to keep an eye on things. --- Updated 3:22 PM 5/16/17 Google has posted an update that the outage is affecting customers using legacy GCM keys. Pulsara Engineering is initiating steps to upgrade our GCM keys to FCM keys, per Google’s recommendation. --- Updated 3:02 PM 5/16/17 Google has officially acknowledged the service disruption, and has posted updated (and backdated) status at the Firebase status console at https://status.firebase.google.com Apologies for the service disruption and for not updating the Firebase status earlier. The rate of errors has been small enough that we didn't believe was having an impact on FCM users. Clearly some developers are being affected disproportionately. We updated the Firebase status at this page: https://status.firebase.google.com/incident/Cloud%20Messaging/17005 --- Updated 2:51 PM 5/16/17 Pulsara Engineering is continuing to monitor the issue. --- Updated 1:49 PM 5/16/17 Pulsara has released hotfix Server 44.2 to production, which will resolve the errors being seen in our mobile applications, by continuing past any sporadic errors from Google FCM. NOTE: some Android users will continue to have intermittent push notification delivery until Google resolves their issues. --- Updated 12:36 AM 5/16/17 ETA for build complete is 1:20 PM. ETA for hotfix deployed to production is about 2 PM. --- Updated 12:35 AM 5/16/17 Google has a Google FCM service outage on public message boards. However, they have not yet updated the Firebase status console at https://status.firebase.google.com --- Updated 12:10 AM 5/16/17 We have confirmed on developer message boards that we are not the only company to be experiencing this issue with Google messaging services. --- Updated 11:43 AM 5/16/17 Next Steps: Pulsara Engineering is working on a code change that will stop fatal errors from happening when our connection to Google FCM fails. After confirming the fix, we will roll out the changes to test stacks (environments), and then to Production. --- Updated 10:54 AM 5/16/17 Errors from Google FCM have started occurring again. Our engineering team is continuing to monitor the issue. --- Updated 10:27 AM 5/16/17 Issues at Google FCM appear to be resolved. We have not observed an error in 20+ minutes. --- Updated 9:51 AM 5/16/17 We are currently experiencing intermittent errors connecting to Google Firebase Cloud Messaging (FCM). Pulsara Engineering is investigating the root cause. On an intermittent basis, affected users will not receive push notifications and other mobile application users will see errors when performing operations that send push notifications as a side effect. More info will be posted here when available.