Is Uberflip Down Right Now? Check if there is a current outage ongoing.

Our team has identified the cause of this issue and successfully implemented a solution to fully resolve it. All Uberflip systems are back online and fully operational. We thank you for your patience while we worked towards a resolution.

We are currently experiencing an issue which is impacting availability of client facing hub URLs for some of our customers. Our team is investigating the root cause and working to resolve as quickly as possible. We will continue to provide updates via our System Status page as this issue progresses.

Our team has identified the cause of this issue and successfully implemented a solution to fully resolve it. All Uberflip systems are back online and fully operational. We thank you for your patience while we worked towards a resolution.

We are currently experiencing an issue which is impacting availability of client facing hub URLs for some of our customers. Our team is investigating the root cause and working to resolve as quickly as possible. We will continue to provide updates via our System Status page as this issue progresses.

The incident has been resolved.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved.

This issue has been resolved, and flipbook processing is now operational.

We are currently investigating this issue. Only Flipbook processing is affected, existing Flipbooks continue to function normally.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are currently experiencing an issue which is impacting the availability of our Marketplace. The team is investigating and working to resolve as quickly as possible. This issue does not appear to impact any installed apps within Uberflip customers' accounts and they should continue to function as expected.

This incident has been resolved.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This issue has now been resolved and our Marketplace is now functioning as expected.

We are currently experiencing an issue which is impacting the availability of our Marketplace. The team is investigating and has identified the cause. We are working to resolve as quickly as possible. This issue does not impact any installed apps within Uberflip customers' accounts and they will continue to function as expected.

This incident has been resolved and all systems are operating as expected.

We are continuing to work on a fix for this issue.

We are currently experiencing an issue which is impacting the availability of our Marketplace. The team is investigating and has identified the cause. We are working to resolve as quickly as possible.

A fix has been implemented for Flipbook processing on new uploads, this incident is now resolved. The team is still investigating instances of Flipbooks failing that were uploaded during the hours in which the processing issue surfaced. If you require the asset immediately, re-upload the Flipbook and it will process successfully. If you need any assistance, please reach out to support@uberflip.com.

A fix has been implemented for Flipbook processing on new uploads, and we are currently monitoring it closely. The team is still investigating instances of Flipbooks failing that were uploaded during the hours in which the processing issue surfaced.

We have become aware of an issue affecting Flipbook processing. Existing Flipbooks will continue to display without issue. Any attempt to upload a new Flipbook or make changes to an existing Flipbook will fail. The team is currently working on a solution.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We are investigating an issue that is causing reduced performance of our Hubs platform.

This incident has been resolved.

We are investigating an issue that is causing reduced performance of our Hubs platform.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We have identified the cause of the issue and a fix is currently being developed/implemented.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently experiencing an issue where portions of the back end (IE: Content section, CTA's, Custom Code Section, User Section, etc) of the Uberflip application are loading intermittently. Refreshing these appears to resolve the issue as a workaround. Front end hubs continue to function as expected. We are investigating the cause of this issue and working to resolve as quickly as possible.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

Uberflip Analytics is temporarily unavailable. We have identified the root cause and are working towards a full resolution.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We have identified the cause of the issue and a fix is currently being developed/implemented.

# **Technical Summary** At 14:14 ET on July 27, 2021 our systems reported an increase in latency to our primary database cluster due to an abnormal and significant increase in traffic. This increase outpaced our scaling capabilities, and resulted in a cascading failure that brought down the database cluster responsible for our Hub Platform. Recovery efforts were initiated quickly once the cause was identified, however a complication during recovery extended the outage. By 16:19 ET the Hub Platform was back online, and all systems were brought back to normal operation shortly thereafter. Remediation efforts on our infrastructure and product performance are underway to prevent a recurrence of this issue.

This incident has been resolved.

A fix has now been implemented and operations are returning to normal. The team will continue to monitor and further updates will be posted here.

Our Engineering team has identified the issue and are currently preparing a fix. We will provide additional updates as soon as the fix has been deployed and validated.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

A fix was deployed late in the afternoon yesterday, and the backlog was processed overnight. All Marketo CTA data should now be updated.

Our Engineering team has identified the issue and are currently preparing a fix. We will provide additional updates as soon as the fix has been deployed and validated.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We are continuing to investigate this issue.

We have become aware of an issue affecting the submission of CTA Data for customers leveraging our Marketo integration. Some customers may notice that only the email address is being submitted to Marketo (leaving out the name and other fields). Our Engineering teams are working on this issue as a top priority, and we will post updates here as we learn more. If you are encountering issues, please submit a ticket to our support team at https://help.uberflip.com as they will be able to provide a temporary workaround.

This incident has been resolved.

A fix has been implemented for Flipbook processing, and we are currently monitoring it closely.

We have become aware of an issue affecting Flipbook processing. Existing Flipbooks will continue to display without issue. Any attempt to upload a new Flipbook or make changes to an existing Flipbook will fail. The team is currently working on a solution.

Uberflip Analytics is now fully functional and we do not expect any further issues at this time.

A fix for Uberflip Analytics has been applied and we are now monitoring the results.

Uberflip Analytics is temporarily unavailable. We are performing an investigation to identify and resolve the root cause.

This incident has been resolved and all systems are operating as expected.

We experienced a brief interruption of service this afternoon which affected the availability of the platform between 12:47pm-12:50pm ET. The cause has been identified and a fix has been put into place. All services are back to normal and we are monitoring the results.

We have deployed a fix for this issue and can confirm that all Standard Reports are now appearing as expected. The workaround is no longer required.

As we continue to investigate a workaround has been identified: The Shared Reports are available from within the UI (in addition to the left hand navigation panel which is at the core of this issue). In the top right menu, you can go to the "Shared" section, and there will be links to the reports. This will provide access access the data in the interim.

We are currently investigating an issue where Uberflip Analytics Standard Reports appear to be missing for some cusotmers. Not all customers are impacted. Uberflip Analytics Dashboards remain operational.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We have identified the cause of the issue and a fix is currently being developed/implemented.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved.

We have identified the issue, implemented a fix and are now monitoring. At present, all systems are performing as expected.

We have identified an issue where Uberflip Analytics data is not properly being displayed for any date after September 8, 2020. Data is being collected in the backend, however not being displayed. The team is investigating and will post updates here as information becomes known.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We are currently experiencing an issue which is impacting availability of the backend/admin portion of the platform. Front end Hubs remain operational. The team is investigating.

This incident has been resolved.

We are continuing to monitor for any further issues.

A fix has been implemented and we are monitoring the results.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

We identified some issues with the TLS/SSL cert renewal process, and implemented a fix.

The issue has been identified and a fix is being implemented.

A number of domains are no longer loading securely. We are currently investigating the issue.

This incident has been resolved and all systems are operating as expected.

A fix has been implemented and we are monitoring the results.

We have identified the cause of the issue and a fix is currently being developed/implemented.

We are currently experiencing an issue which is impacting performance of the platform. The team is investigating.

Earlier this morning, around 11:20am ET, we experienced a temporary interruption to service. This incident has been resolved and all systems are operating as expected.

All systems have returned to normal operational levels.

Legacy Hub Metrics are back online. There appears to be no metric data loss, however customers will have experienced delays in receiving their Hub Metrics on our legacy metric dashboards. Legacy Flipbook Metrics are being brought back online still, with an estimation of approximately 3 hours (5pm ET) for full restoration.

We have identified the cause of the issue with our legacy metrics processing pipeline. We are working on restoring the pipeline functionality, targeting EOD (June 30, 2020) for a fix. We will also continue to work on verifying if there was any metric data loss.

We are observing issues with our Legacy Metrics (Flipbook and Hub Metrics). Starting on June 26th, the collection and displaying of legacy metrics encountered an issue which has resulted in no data being displayed in the hub. The team is currently investigating and will post updates here.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are seeing a major degradation in service resulting in most pages to not load. We are investigating, and will update as we know more.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are again seeing degraded performance and continue to investigate.

A fix has been implemented and we are monitoring the results.

We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We are continuing to investigate this issue.

Our infrastructure teams are currently investigating the issue and working to restore services as quickly as possible.

This incident has been resolved.

A fix has been implemented for Flipbook processing, and we are currently monitoring it closely.

We have become aware of an issue affecting Flipbook processing. Existing Flipbooks will continue to display without issue. Any attempt to upload a new Flipbook or make changes to an existing Flipbook will fail. The team is currently working on a solution.

The issue is resolved, but we continue to communicate with our vendor to identify the root cause of the earlier outage.

A fix has been implemented and we are monitoring the results.

We are currently investigating this issue.

This incident has now been resolved and the platform is functioning as expected.

A fix has now been implemented and the team is monitoring the platform.

We have observed an issue impacting hubs and the team is actively investigating the cause.

A fix was implemented earlier this evening and all systems have returned to normal operational status.

We are again seeing slower than normal processing times on Flipbook uploads. The team continues to investigate.

The team has implemented a fix and processing has resumed. We will continue to monitor for any reoccurrence.

We are currently experiencing an issue which is impacting the ability to upload Flipbooks. The team is reviewing and will provide updates as soon as available.

This incident has been resolved. All services have returned to normal operational status.

Response times are returning to expected levels. The team continues to monitor closely.

Our OPS team has detected an increase in response time for some actions and is actively investigating.

This incident has now been resolved.

The team has corrected the issue and is monitoring performance.

Our infrastructure teams are currently investigating the issue and working to restore services as quickly as possible.

This incident has been resolved. All services are functioning normally.

A fix has been implemented and we are monitoring the results.

We are observing an issue that is causing intermittent request failures with the platform. The team is actively investigating and indications are that things are now stabilizing.

We observed an issue for a brief period of time this afternoon causing intermittent service issues on our platform. The platform has now been stabilized and the team is monitoring for re-occurrence.

The issues have been confirmed to have been fixed. The outage was intermittent from 5:30 to 6:30.

We have identified the root cause for an intermittent outage that started at 5:30PM Eastern. We believe we have a fix in place, and are monitoring the situation.

Between 6:54am ET and 7:54am ET, there were intermittent issues loading custom CSS and JS assets. While Hubs were accessible, any custom look/feel/functional components would not have loaded resulting in presentation/appearance issues.

## What Happened On September 23rd, beginning around 9:30 AM EDT, we began to experience database performance problems which affected all parts of the Uberflip platform. As a result of these issues, customer Hubs and the Uberflip application backend were intermittently inaccessible for a period of approximately 14 minutes. We immediately took action to identify the cause of these issues. By 9:40 AM our systems began to recover, and by 9:44 AM the outage was fully resolved, with all parts of the Uberflip platform functioning normally once again. We sincerely apologize for the impact that this interruption in our service may have had on your business. Following a thorough examination of the causes of this issue, we would like to share with you what we have learned, and what steps we are taking to prevent it from happening again. ## What Caused the Issue Starting at around 9:25 AM, our servers suddenly began to receive an extremely high volume of traffic over a short amount of time. This unusual volume of requests caused the load on our primary database to exceed its capacity, at which point it failed over to our backup database. The backup database also rapidly reached its capacity limit, causing it to revert back to the primary database \(which had recovered in the interim\). As the abnormally high volume continued, this cycle repeated several times, with each database becoming overloaded, failing over, recovering, then becoming overloaded again and so forth. Each time the platform switched between the two databases, it took a few moments for all parts of the application to reconnect. At the same time, any requests in progress at the time of the switch were lost. Combined, these factors are what caused the intermittent inaccessibility and lack of responsiveness in the Uberflip application and customer Hubs. ## What We’ve Learned, and How We’re Responding As with all incidents that affect our platform and its reliability, we have performed a thorough investigation. Here’s what we’ve learned, and what we're doing to make sure it can't happen again. The root cause of this incident was a sudden, sustained increase in the amount of traffic to our platform. This traffic was highly unusual: it was significantly higher than the amount we would typically expect, ramped up almost instantly, and stopped just as suddenly. We've investigated the source of this traffic, and have determined that there was probably no malicious intent behind it. Given that this kind of anomalous traffic would not occur organically, we believe this should be a very rare occurrence. Nonetheless, we are of course taking measures to prevent our platform from being affected should something similar happen in the future. Primarily, we will be implementing rate limiting. Rate limiting will allow us to automatically control the volume of traffic to our servers, and prevent unusual bursts of traffic like this one from impacting the reliability of our platform. While we believe rate limiting will be effective in this regard, we are also aware that it could potentially affect legitimate traffic. As a result, we're taking a considered approach, and are currently investigating what the most appropriate limits are before setting them. At the same time, we are also looking at whether there are any ways for us to improve our existing warning systems so that we can detect and respond to anomalies faster. In this instance our technical teams very quickly pinpointed the cause of the issue and were ready to respond, but the anomalous traffic actually ceased by itself before they had a chance to do so. That said, while we had the capability to resolve this incident rapidly, our focus is of course on preventing it from happening in the first place. To that end, we are currently examining how we can identify and react to similar situations before our customers are affected. Finally, we are also performing a thorough review of our application, database, and network architecture, as we do following all significant incidents. In this particular review, we will be placing a specific focus on how we can make the entire platform more resilient to issues such as this one. Once again, we are deeply sorry that this issue occurred. We are committed to making sure it can’t happen again, and we appreciate your patience and trust.

This incident has now been resolved. All systems are operating as expected.

The platform has now stabilized and the team continues to monitor for any re-occurence.

We are observing an issue that is causing intermittent connectivity issues with the platform. The team is actively investigating and indications are that things are now stabilizing.