Phrase Incident History

1. resolvedOct 15 · 10:24 UTC

   This incident has been resolved.

Reported by Phrase on their status page.

1. postmortemSep 30 · 12:12 UTC

   Introduction

   We would like to share details about an incident that affected access to Phrase TMS on September 23, 2026. Between 9:02 AM and 11:27 AM CEST, users opening Phrase TMS from Phrase Platform could not complete the sign-in. They were either caught in a redirect loop or returned to the Phrase TMS login page with a sign-in error.

   Users who already had an active Phrase TMS session were not affected, and no customer data was lost or modified. Signing in to Phrase Platform itself kept working throughout the incident, both with a password and with single sign-on. So did signing in with a password directly on the Phrase TMS login page.

   This post-mortem explains what happened, how it was resolved, and what we are doing to prevent it from happening again.

   Timeline

   • Sep 23, 2026 at 9:02 AM CEST – An update to the Phrase TMS web application went live in production. It included a security improvement to how the sign-in flow handles redirects after login. From this point, users opening Phrase TMS from Phrase Platform could not complete sign-in. Shorty after, the issues was investigated internally.
   • Sep 23, 2026 at 10:59 AM CEST – The update from 9:02 AM CEST was identified as the cause, and we began preparing a rollback.
   • Sep 23, 2026 at 11:13 AM CEST – The rollback was verified in our test environment, and deployment to production started.
   • Sep 23, 2026 at 11:27 AM CEST – The rollback was live in all production regions. Customer-facing impact ended.
   • Sep 23, 2026 at 11:28 AM CEST – We confirmed that signing in to Phrase TMS from Phrase Platform worked again.
   • Sep 23, 2026 at 12:04 PM CEST – The incident was marked as resolved on our status page.

   Root Cause

   The incident was caused by a security improvement to the Phrase TMS web application. Its goal was to make sure users are only sent to trusted destinations after they sign in. To do this, the web application only accepted redirects to the Phrase TMS web address itself. However, when a user opens Phrase TMS from Phrase Platform, the sign-in process legitimately moves the user between different Phrase web addresses. Our servers already check each of these redirects against a list of trusted Phrase domains and allow them. The new check in the TMS core application was stricter than this server-side check and overrode it. As a result, legitimate redirects were replaced with the Phrase TMS start page, which started the sign-in process again. Users either looped through sign-in repeatedly or landed back on the Phrase TMS login page with an error.

   Signing in to Phrase Platform itself kept working throughout the incident, both with a password and with single sign-on. So did signing in with a password directly on the Phrase TMS login page. The failure happened at the step where a signed-in user moves from Phrase Platform into Phrase TMS. Because of this, some affected users reported the problem as a password issue: they entered their credentials successfully but were then sent back to a login screen.

   Actions to Prevent Recurrence

   1. Change rolled back – The update that caused the issue was rolled back in all production regions at 11:27 AM CEST, which restored normal sign-in.
   2. Continuous sign-in monitoring – We are adding automated checks that run regularly in all production regions and in our test environment. Each check performs the full sign-in from Phrase Platform into Phrase TMS and immediately alerts our on-call engineers if it fails. Before we rely on them, we will confirm that these checks detect the exact failure that occurred in this incident.

Reported by Phrase on their status page.

1. ResolvedSep 23 · 10:03 UTC

   This incident has been resolved.

Reported by Phrase on their status page.

1. postmortemOct 6 · 08:14 UTC

   Introduction

   We would like to share more details about the events that occurred with Phrase between 2:11 PM CEST and 4:10 PM CEST on August 19, 2026, which led to degraded Translation Memory functionality in Phrase TMS (EU) and what Phrase engineers are doing to prevent these issues from recurring.

   Timeline

   • August 19, 2026 at 2:11 PM CEST: Phrase identified an issue affecting Translation Memory functionality in Phrase TMS (EU). Users experienced errors when creating translation memories and performing related operations, including confirming segments, elevating error rates across dependant services and use-cases.
   • August 19, 2026 at 2:32 PM CEST: Root cause was identified and engineers began applying a fix to the affected environment by rebalancing the cluster and restarting affected nodes to speed up recovery.
   • August 19, 2026 at 3:10 PM CEST: Translation memory creation and related operations were confirmed to be functioning normally again.
   • August 19, 2026 at 4:10 PM CEST: After a sustained monitoring period showed no recurrence, the incident was marked resolved.

   Root Cause

   Investigation revealed that the incident originated in the database cluster that stores Translation Memory data. A single Translation Memory had, over time, grown very large and required a disproportionate share of read activity to be served by the node that also coordinated the health and metadata management of the wider cluster.

   Because the cluster was undergoing zero-downtime maintenance spanning days, there was a higher amount of coordination actions and an elevated load on coordination nodes in the cluster.

   Those two combined with peak load caused multiple coordination nodes performance to slowly deteriorate, leading to elevated error rate and degraded performance.

   Actions to Prevent Recurrence

   To prevent similar incidents in the future and mitigate any negative effects, we:

   • Rebalanced the cluster and  its backing infrastructure to create more headroom for Translation Memory searches and updates.
   • Paused the ongoing maintenance on impacted Translation Memory cluster.
   • Improved capacity of coordination node and will be improving cluster’s topology to prevent cluster impact during sudden requests peaks.

Reported by Phrase on their status page.