Historical record of incidents for Front
Report: "[us-west-2] [by_company_00007] Customers may experience slowness in app loading"
Last updateWe are currently investigating reports of increased loading times affecting a subset of our customers. Some users may experience slower than normal response times when accessing the app or navigating between pages.
Report: "Web app not loading"
Last updateA configuration change was made to the Content Distribution Network that caused the web clients to lose access to files that were needed for startup. The web app couldn't be loaded between 1:48 PM and 2:17 PM PDT. Impact was limited to users who tried to start the app between those times.
Report: "[us-west-2] [by_company_00008] Delays in analytics data ingestion"
Last updateWe are currently processing a backlog of analytics data. This may cause delays in data availability and temporarily skewed metrics. Our team is actively working to resolve the issue.
Report: "App not loading [us-west-2][by_company_00008]"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue where some users may be experiencing difficulties with the app not loading and some API calls timing out or returning errors. We are working to identify the cause and will provide an update shortly.
Report: "App not loading [us-west-2][by_company_00008]"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue where some users may be experiencing difficulties with the app not loading and some API calls timing out or returning errors. We are working to identify the cause and will provide an update shortly.
Report: "2025-04-29 Application unavailable for some customers"
Last updateOn Tuesday, April 29, Front experienced an application outage lasting roughly 4 hours for a subset of US-based customers—roughly 3% of users. This was the result of a human error during a planned database maintenance operation that resulted in one database shard becoming unrecoverable. Front was able to restore this database shard from an up-to-date backup and then catch up any missed messages during that period, which resulted in no data lost as a result of this issue. Front customers are divided into database shards so that an issue with one shard does not affect all others. Earlier this week Front performed an upgrade of all database shards as a routine procedure, and while the upgrade was successful with no customer impact, there was an error in the cleanup step that caused one shard to be irrevocably invalidated. This occurred at 4:16 pm PDT \(23:16 UTC\). Front maintains up-to-date database replicas for this purpose, from which we were able to initiate a restoration. This involves copying data, and therefore took nearly four hours to complete. Once the database was restored at 8:07 pm PDT \(03:07 UTC Wednesday\), the application became available again, and Front was able to initiate the next task of applying all changes since the incident began, like syncing in emails from Gmail and O365. Front apologizes to the affected customers for the inconvenience caused by this extended outage. We strive to be always available to customers and have made great investments over the last several years to minimize the possibility and the impact of this kind of error. We can always do more, and are taking the following steps based on lessons from this incident. We have modified our database upgrade procedures and the permissions granted to the operators to significantly reduce the likelihood of this same error. At the same time, we are accelerating our upgrade frequency, which means we’ll have more practice and more automation with this procedure, further reducing risk. More upgrades also means faster adoption of performance and security improvements. Front is also investigating opportunities to improve the recovery time from the backup. Most of the time during this outage was waiting for data to be replicated from one storage region to another, and there may be ways to cut that down. Once again, Front would like to apologize for any disruption caused by this incident and emphasize our commitment to high availability and transparency.
This incident has been resolved.
As of 8:11 pm PDT (03:11 Wednesday UTC), the application is functioning for all customers. Front began applying all changes since the incident started, like syncing messages from email channels, which are now caught up. Customers may experience delays for a few more minutes as caches clear. Front is now monitoring the application for further issues. All functionality should be restored; please contact support if you continue to see application issues. [us-west-1] [by_company_00005]
The database recovery has completed and application functionality is now restored. Messages from the last few hours since the incident started are now being re-synced. This will take some time to catch up, but we anticipate no data loss as a result of this issue. [us-west-1] [by_company_00005]
The restoration process is still underway, but nearing completion. We are targeting less than 60 minutes to restore limited application functionality. [us-west-1] [by_company_00005]
At 4:16 pm PDT (23:16 UTC), during a routine maintenance operation on one of Front’s customer databases, the database was inadvertently disabled and made unrecoverable. Front immediately began the restoration procedure from our up-to-date backup, but as it has to copy the entire database this procedure takes time to complete. We expect this to take roughly two hours, during which time the application will be unavailable for customers on this database. This issue affects a fraction of Front's customers, but for those customers the application will be unavailable until the restoration is complete. [us-west-1] [by_company_00005]
Front is investigating a major database outage affecting one shard of customers. [us-west-1] [by_company_00005]
Report: "[office365] Some customers experiencing delays receiving messages on Office365 channels"
Last updateIssue is not resolved, opening a partner incident instead: https://front-partners.statuspage.io/incidents/sdln9wvbqdb7 Please contact Microsoft for assistance with an affected mailbox.
We are continuing to investigate this issue.
We are currently investigating delays for inbound messages on some Office365 channels.
Report: "[office365] Message Attachment Size Limit Error"
Last updateThere has been a significant reduction in the number of failures for Office 365 requests with attachments larger than 3 MB. We will continue to monitor.
Office 365 servers may sometimes fail to accept attachments larger than 3 MB. As a temporary alternative, we suggest using another method of sharing files with recipients, such as OneDrive. We have communicated the issue to Microsoft and are investigating additional mitigations.
We received reports from a small number of customers that some drafts error when sending, with the message: "Unexpected problem sending attachment larger than 3MB to Office 365."
Report: "[us-west-2] [by_company_00008] Customers may experience slowness in app loading"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring performance for customers.
We are currently investigating the issue
Report: "Errors sending and receiving mail for [Office365] channels"
Last updateThis incident has been resolved.
We have deployed a fix and are monitoring the situation.
We are currently deploying a fix for this issue and expect to restore service within 30 minutes
We have identified the issue causing errors when sending and receiving mail for Office365 channels and are working towards a resolution.
We are currently investigating errors sending and receiving email for Office365 channels
Report: "Elevated error rates for [office365] channels"
Last updateThis incident has been resolved.
Office seem to have resolved the issue. We're working through a backlog of imports.
Office365 is currently experiencing an outage preventing users to login. This is also preventing us from sending and importing messages.
Our monitoring is suggesting there are elevated error rates for Office 365 channels. We are currently investigating.
Report: "Delays ingesting emails [gmail]"
Last updateThis incident has been resolved.
We're currently processing a backlog of Gmail messages. Messages received after 8:25pm UTC are not impacted.
Report: "AI answers for Chat errors"
Last updateThis incident has been resolved.
We are seeing AI Answers in Chat back to being fully operational. We will continue monitoring.
We are seeing AI Answers in Chat back to being fully operational. We will continue monitoring.
AI Answers functionality in Chat is degraded due to an outage with an upstream provider. We are working on a mitigation. In the meantime, AI Answers chatbot nodes will follow the "AI doesn't resolve the inquiry" branch.
We're investigating some errors only affecting our AI answers functionality with Chat
Report: "Automated emails from Front are delayed - notifications, invites to your teammates, calendar events, etc."
Last updateThis incident has been relsolved.
We've worked with our provider and remediated the issue. We'll continue to monitor.
We have identified the issue and are working with our provider to remediate.
We are currently investigating this issue.
Report: "Front AI features outage"
Last updateThis caused numerous Front AI features to be unavailable, including AI Answers, AI Tagging (beta), Suggested Replies(beta), AI Compose, and AI Translate (beta). Additionally, the AI Tagging failures blocked subsequent rules from being run.
Report: "Exchange Online mailboxes sync delay [office365]"
Last updateThis incident has been resolved.
Our systems have fully recovered. We are now retrying failed tasks.
Microsoft deployed a fix. Our systems are recovering. Latest update from Microsoft: > At 20:52 CET > We're continuing to redirect traffic and apply targeted remediation to impacted servers. > We've observed initially positive results in some areas, and we're monitoring service health closely as our mitigation workstreams continue in parallel.
Microsoft is deploying a fix > At 15:27 CET > We’re monitoring the progress of the fix, which has been deployed to approximately 60% of the affected environments. We’re continuing our manual restarts on the remaining impacted machines. We are closely monitoring our systems for signs of recovery; however, we have not observed any significant progress at this time.
Microsoft is still investigating the issue; here is their latest update. > We’re continuing to investigate recent changes, service telemetry and the failure paths to determine the root cause of impact. Source: https://portal.office.com/Adminportal/Home#/servicehealth/:/alerts/MO941162 (You need to be an O365 admin to be able to access it)
Our partner Microsoft is experiencing a service degradation affecting Exchange Online mailboxes. You can follow Microsoft Issue ID MO941162 for more details. This may cause outbound/inbound delays for customers using Office 365, as Front cannot connect to Microsoft servers.
Report: "[us-west-2] Increased failure rate on public API"
Last updateThis incident has been resolved.
The increased rate of errors has subsided as of approximately 10 minutes ago, but we are still determining the root cause.
We are seeing an increase in 5xx errors on our public API in us-west-2 region, which may also affect the ability to send/receive messages on some channels.
Report: "[by_company_00017] Elevated loading times for some customers in us-west-1"
Last updateThis incident has been resolved.
We have put in place mitigations and are seeing a return to normal performance. We're continuing to monitor.
We are receiving reports of elevated loading times for some customers in the us-west-1 region. We're currently investigating the root cause.
Report: "Increased error rate sending and receiving messages on email-based channels"
Last updateThis incident has been resolved.
Customers in us-west-2 and eu-west-1 regions have fully recovered. We are still working through the incoming message backlog on us-west-1.
The fix has been deployed and we are seeing email delivery services have recovered. We're monitoring and catching up on backlog so there might be residual delays.
We have validated the fix and are in the process of doing a manual deploy to circumvent the current Docker outage.
We are validating the fix in our staging environment.
We have identified the cause of the issue and are working on a fix. The issue affects some but not all channels.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Difficulties accessing Front on desktop and mobile in eu-west-1 region [by_company_00001] [by_company_00002] [by_company_00003] [by_company_00004]"
Last updateApplication performance has appeared to return to normal for eu-west-1 users.
We believe we've identified the problem which caused a temporary spike in API errors. We're seeing a return to normal levels, and are working on a root-cause fix.
We're receiving some reports of customers having difficulties accessing Front, or delays in sending and receiving messages, in our eu-west-1 region. We're currently investigating.
Report: "Delays in sending and receiving Gmail messages [gmail]"
Last updateAll channels should be back to normal now.
Some channels are still experiencing delays as we import the backlog of messages received while Gmail was experiencing an outage
Continuing to drain the backlog
We are continuing to drain the backlog and will be doing so for the next couple hours.
We are on the recovery path now and are focused on draining the backlog of delayed messages.
We are beginning to see a gradual increase in successful deliveries. We are continuing to monitor the situation, as well as replaying deliveries that have been delayed by the original incident.
We are continuing to see message delays, and are rescaling our internal queues globally to help mitigate the issue.
We are continuing to investigate this issue, which seems to be related to Gmail's rate limiting mechanisms. We're seeing some impact across multiple regions.
We're observing problems sending and receiving some messages on Gmail channels in the us-west-2 region. We're working on identifying the root cause.
Report: "Degraded performance in us-west-2 [us-west-2]"
Last updateThis incident has been resolved.
We're beginning to see new deployments coming online. We're continuing to monitor the health of the cluster.
We are experiencing issues launching new deployments in our us-west-2 region, which may degrade the app performance and lead to delays in receiving messages for some customers.
Report: "Unable to sign in using Google"
Last updateBetween our workaround, and potentially changes by Google, this issue has been resolved.
A workaround has been implemented and we are monitoring the results.
We are deploying a potential workaround for the issue.
We are continuing to investigate the issue, which seems to affect only a small number of users.
We are getting reports of customers being unable to sign in using Google with a "500. That's an error" page and are investigating.
Report: "Failures to send emails on Gmail channels [gmail]"
Last updateA workaround has been implemented and we've observed a full recovery.
We have identified an issue and filed a bug report with Google. In the meantime, we are deploying a potential mitigation to alleviate the problem.
We are continuing to investigate this issue.
We are continuing to see failures sending emails on Gmail channels for a small number of customers following Gmail's incident from yesterday and are investigating.
Report: "Failures to send emails on Gmail channels [gmail]"
Last updateAlmost all Gmail traffic has now returned to a healthy state, with our retry mechanisms expected to restore full service to the small number of channels that are experiencing delays, in the coming hours.
The vast majority of Gmail traffic has returned to a healthy state; a very small number of Gmail channels appear to still be in the process of self repairing. We're continuing to monitor the situation.
We are continuing to observe errors for a small percentage of Gmail traffic, and are continuing to investigate.
We believe the incident is related to an ongoing global incident disrupting aspects of Gmail's service and platform (https://www.google.com/appsstatus/dashboard/incidents/G4vfJGa9LyWQjK4hiiHA). We're continuing to investigate possible mitigations to restore full service to Front's customers.
Our engineering team is continuing to investigate this issue.
We are seeing failures to send emails on Gmail channels due to an authentication issue and we are investigating.
We are seeing failures to send emails on Gmail channels due to an authentication issue and we are investigating.
Report: "App is intermittently slow [us-west-1][by_company_00009]"
Last updateThis incident is currently resolved. We've scheduled a database maintenance for later today to prevent the issue from reoccurring: https://www.frontstatus.com/incidents/w4v2qjtb2dqy
The issue has subsided due to a natural reduction in database load. We are continuing to work on a database upgrade to prevent this issue from reoccurring tomorrow.
We identified a low-level bug in a database service that appears to be causing this issue, and are working on a short-term mitigation.
We are continuing to investigate a database performance issue that is causing intermittent user-visible slowness and errors.
We are currently investigating this issue.
Report: "App is intermittently slow [us-west-1][by_company_00009]"
Last updateThis incident has been resolved.
We have found something that appears to mitigate this issue. We are continuing to investigate.
We are continuing to investigate this issue.
We are continuing to investigate the issue, which affects some users for up to 2 minutes every 15 minutes.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Some customers are experiencing delays in sending emails [sendgrid]"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Automated emails from Front are delayed - notifications, invites to your teammates, calendar event etc."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Critical transactional emails have been restored. We're working on the rest of our transactional emails, like notifications.
We have identified the source of the issue and are working on fixing it. We're working with our provider to resolve this issue.
Report: "[us-west-1][us-west-2][eu-west-1] Chat Inaccesible"
Last updateIssue resolved, chat functional for all customers.
Identified, mitigated, and continuing to monitor chat for customers.
Issue identified, currently mitigating issue
Report: "[eu-west-1][by_company_00004] Front inaccessible"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified the issue and are working on a fix.
We are aware that some users cannot connect to Front. We are investigating.
Report: "[us-west-2][by_company_00004] Front is not responsive"
Last updateWe are aware of issues preventing some customers from using Front. We have identified the issue and are working on a fix.
This incident has been resolved.
A fix has been implemented and we are seeing less failures. We are monitoring.
Report: "Intermittent API errors"
Last updateFront has resolved this incident. It was caused by a large unexpected increase in web traffic. We have put additional safeguards in place and will continue to monitor the application for any errors.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
Since 17:40 UTC (9:40am PST) some users have experience problems loading the application. Front is investigating. [us-west-1|eu-west-1] [by_company_00001|
Report: "Partner Incident Impacting New Facebook and Instagram Channels"
Last updateThis incident has been resolved.
Monitoring the issue based on partner updates. Impacting only new channel creations for Facebook and Instagram channels in Front. Existing channels should be functional but may see errors during reauthorization.
Report: "Unstable infrastructure is causing Degraded performances across Front for some users [us-west-1][by_company_00009]"
Last updateThis incident has been resolved.
The performance of the app should now be back to normal. We are still monitoring the situation.
We are deploying a fix and are monitoring the results.
We are continuing to investigate the issue
We are currently investigating this issue.
Report: "Front Chat Inaccessible"
Last updateFront Chat was unavailable for approximately 20 minutes. The issue was quickly identified and service has been fully restored.
Report: "[office365] Channel Reauthentication Errors"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
Front has run a command to restore the Office365 channels that were disconnected, and is now monitoring for any additional errors. Any channels that were successfully manually reconnected during the last few hours were untouched, and should still be operational. If and channels were manually reconnected but received an error, Front is unable to automatically reconnect them. This is rare. Please follow the standard instructions for reconnecting these channels.
We are continuing to work on automatically reconnecting all the Office265 channels that were disconnected. Note that if you attempted to reconnect a channel and were unable to, Front will no longer be able to restore the authenticate (as any attempt to manually re-authenticate will overwrite the stored tokens). Customers can still manually reconnect any channels at any time. Please make sure to follow the guidance in https://help.front.com/en/articles/2043
We're seeing improvements for manual reauthentication attempts. When reauthenticating, please follow the steps outlined on https://help.front.com/en/articles/2043 and ensure that you are reauthenticating with the correct credentials for the channel.
We are getting reports that some customers are still seeing unexpected permissions errors when reauthenticating O365 channels manually. We're investigating why these reauthentication attempts are failing.
We are working on automatically reauthenticating any channels that were affected. In the meantime, customers should be able to manually reauthenticate their O365 channels.
We identified an issue where Office365 channels that needed authentication were not able to do so and put the channel into a blocked state. We found the root cause and have deployed a fix.
Report: "[eu-west-1] Failures in dropdown search suggestions"
Last updateThis incident has been resolved.
A fix has been deployed and appears to have resolved issues with search suggestions and contact autocompletion. We are monitoring to ensure the fix is stable.
We have identified the issue and are actively working on mitigations
We are investigating an issue with search suggestions and contact autocompletions in EU.
Report: "Errors sending Intercom replies [intercom]"
Last updateThis incident has been resolved.
A fix has been deployed for this issue and we are monitoring sending and receiving for Intercom channels.
We have identified an issue with sending Intercom replies to conversations created before 04:00 UTC. Our engineers are working on a solution and estimate that the issue will be resolved in the next hour.
Report: "Permission errors on some O365 channels [office365]"
Last updateAll Office365 channels that still need to be re-authorized are now correctly identified by in-app warnings. Re-authorizing those channels will resolve send/receive issues. Note: re-authorization typically needs to be done by an administrator and/or someone with an Office365 login that has the appropriate permissions.
All Office365 channels that still need to be re-authorized are now correctly identified by in-app warnings. Re-authorizing those channels will resolve send/receive issues. Note: re-authorization typically needs to be done by an administrator and/or someone with an Office365 login that has the appropriate permissions.
As a side effect of https://www.frontstatus.com/incidents/1q7f4p99943g , some Office365 channels are failing to send with a "The Office365 account does not have permission to access this mailbox" error, but in actuality these channels need to be re-authorized. Re-authorizing affected Office365 channels using an Office365 account that has permissions for that inbox should resolve the issue. We are working to update the UI to correctly prompt for these channels to be re-authorized.
As a side effect of https://www.frontstatus.com/incidents/1q7f4p99943g some customers reported that their Office365 channels are now showing "The Office365 Account does not have permission to access this mailbox" Our team is working to find the cause of this error. There is currently no estimated time for a resolution.
Report: "Office365 channels incorrectly marked offline [office365]"
Last updateThis incident has been resolved.
We backfilled all the concerned channels back online.
We've received multiple reports of customers having Office365 channels in an offline status. To put the channel back online you can manually re-auth. In the meantime, we are working on a script to return all Office365 channels online.
Report: "Front is less responsive for some customers in us-west-1 [us-west-1][by-company-00017]"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We're seeing increased API latency and delays in receiving new messages for a subset of our US customers, leading to a slower experience when using Front's app.
Report: "Delays ingesting Gmail messages [gmail]"
Last updateAll services have been fully restored and are stable.
Sending and receiving emails through Gmail has been restored and the system is fully recovered. We will continue to monitor the situation for any changes.
Sending and receiving emails through Gmail has been restored, but the system is still recovering. Customers should expect to see delays drop significantly. We will continue to monitor the situation for any changes.
We are observing errors and rate limiting when ingesting Gmail messages. Some service has been restored but we are still working to resolve rate limiting issues with Gmail's API. Our engineering team is continuing to work around these issues with the Gmail API, but there is currently no timeline to restore service.
We are observing errors and rate limiting when ingesting incoming Gmail messages. Our engineering team attempting to work around these issues with the Gmail API, but there is currently no timeline to restore service.
Report: "Front less responsive or inaccessible for some customers [us-west-2][by_company_00004]"
Last updateThis incident has been resolved. We will continue to monitor for issues.
Error rates have subsided, but Front will continue to monitor for any issues.
Front application less responsive or inaccessible for some customers starting at 18:00 UTC. Front team is investigating.
Report: "Delay in rules and deliverability of emails for customers [us-west-1]"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and the delay is going down. We are continuing to monitor the results.
We are experiencing a 30 minute delay. We are currently investigating this issue.
A fix has been implemented and we are monitoring the results.
Customers should expect some delay in delivery of new inbounds.
Report: "Front less responsive or inaccessible for some customers [us-west-2][by_company_00002]"
Last updateThis incident has been resolved.
Front application less responsive or inaccessible for some customers due to high latency / timeouts from Front's backend. A fix has been implementing and we are monitoring.
Front application should be operational - messages are continuing to experience degraded performance
We are continuing to monitor for any further issues.
Front application less responsive or inaccessible for some customers due to high latency / timeouts from Front's backend. A fix has been implementing and we are monitoring.
Report: "Front less responsive or inaccessible for some customers [us-west-2][by_company_00004]"
Last updateFront application less responsive or inaccessible for some customers between 18:00 and 18:30 UTC due to high latency / timeouts from Front's backend.
Report: "Rules settings page unavailable"
Last updateFrom 09:09 to 10:15 UTC on Wednesday, Nov. 8, the Rules settings page was not accessible. Users are unable to view, create, or update rules. Evaluation of existing rules was unaffected. This issue was caused by a code change that introduced a UI bug. Front identified the issue within minutes, but an error in our revert procedure caused a delay in deploying the fix. Along with additional testing for the original issue, Front has also addressed this problem during revert.
Report: "Delays in rules and deliverability of emails for customers"
Last updateAll impacted regions updated, customer impact has been resolved.
All US West 1 customer events are resolved.
All EU West 1 customer events are resolved.
Replayed failed events are resolving. Continuing to replay all events in backlog for resolution
Replaying failed events for impacted customers. Continuing to monitor impact
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "App is intermittently slow"
Last updateThis incident has been resolved.
App performance has returned to normal. We are continuing to investigate and monitor app performance.
We are investigating degraded app performance
Report: "Front is less responsive for some customers in us-west-1 [us-west-1][by-company-00017]"
Last updateThis incident has been resolved.
The team has implemented measures, and we are beginning to see improvements.
We've identified the issue, and our team is actively working on measures to mitigate it.
We're seeing increased API latency and delays in receiving new messages for a subset of our US customers, leading to a slower experience when using Front's app. (Actions in Front are taking 10-20 seconds to complete in some cases).
Report: "Front is less responsive for some customers in us-west-1 [us-west-1][by-company-00017]"
Last updateThis incident has been resolved.
The team has implemented measures, and we are beginning to see improvements.
We've identified the issue, and our team is actively working on measures to mitigate it.
We're seeing increased API latency and delays in receiving new messages for a subset of our US customers, leading to a slower experience when using Front's app. (Actions in Front are taking 10-20 seconds to complete in some cases).
Report: "Front is less responsive for some customers in us-west-1 [us-west-1][by-company-00017]"
Last updateThis incident has been resolved.
We have identified a potential cause, and are beginning to see the app's responsiveness improving.
We're seeing increased API latency for a subset of our US customers, leading to a slower experience when using Front's app. (Actions in Front are taking 10-20 seconds to complete in some cases). We're working on identify the root cause and returning the app's responsiveness to normal levels.
Report: "Partner incident impacting Front Channel Creation and Onboarding"
Last updatePartner APIs are now operational and will continue to be monitored. We will provide further updates if further impact occurs.
We are continuing to monitor partner incident status updates. No timeline currently provided. Specific to SMTP channel creation
Monitoring our partner incident page for further updates