Historical record of incidents for Temporal
Report: "External metrics outage"
Last updateUsers ingesting metrics via Temporal Cloud's metrics endpoint will see no data
Report: "Investigating issue for namespaces in GCP regions"
Last updateWe are currently investigating this issue.
Report: "Some customers in AWS regions may experience delays in visibility task processing"
Last updateWe are currently investigating this issue. Workflow execution is unaffected.
Report: "Audit log delivery delayed"
Last updateThis incident has been resolved.
The delivery of audit logs for cloud events stopped publishing at 2025-05-30T17:54:00Z. The problem has been identified and a fix is actively being deployed. Once deployed, all events will be backfilled.
Report: "Audit log delivery delayed"
Last updateThis incident has been resolved.
The delivery of audit logs for cloud events stopped publishing at 2025-05-30T17:54:00Z. The problem has been identified and a fix is actively being deployed. Once deployed, all events will be backfilled.
Report: "Some customers in ap-south-2 (AWS) may experience elevated API latencies"
Last updateThis incident has been resolved.
A network issue has been identified by the cloud provider that is actively being investigated.
Report: "Some customers in ap-south-2 (AWS) may experience elevated API latencies"
Last updateThis incident has been resolved.
A network issue has been identified by the cloud provider that is actively being investigated.
Report: "New Customer Onboarding Issue"
Last updateThe issue is fixed. New customers should be able to onboard without a problem.
New customers may experience temporary delays signing up for Temporal Cloud.
Report: "Services running hot in US-E-1"
Last updateNote that the system had no errors; this was due to pure demand spike. Performance has returned to nominal levels, we will watch carefully as always.
We have added capacity, performance has improved, and we are continuing to monitor.
Some customers may experience higher-than-usual latencies
Report: "New customer onboarding issue"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
The new customer onboarding has some issue.
Report: "Some customers in us-east-1 (AWS) may experience elevated API latencies"
Last updateThis incident has been resolved.
We have resolved the original source of this problem, and are working to isolate the workflows involved, and will be contacting any affected customers with more specific information.
In addition, we currently believe a small number of workflows may be stuck. Research is ongoing.
We are currently investigating this issue.
Report: "Customers are experiencing issues logging into Temporal Cloud UI"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
This is due to our downstream provider Auth0: https://status.auth0.com/incidents We are working closely with them to get updates and mitigate this issue
We are currently investigating this issue.
Report: "Some customers in us-east-1 (AWS) may experience elevated API latencies"
Last updateThis incident has been resolved.
Some customers in us-east-1 (AWS) may experience elevated API latencies.
Report: "Customers in AWS eu-west-1 may experience elevated latencies and error rates"
Last updateOn April 4th, 2025, Temporal Cloud in AWS eu-west-1 experienced a service outage from 1647 UTC to 1717 UTC. During this 30 minute window all customers of this region saw high API latencies and a peak gRPC error rate of 87%. The root cause was a latent internal TLS configuration change that had previous rolled out to the cell, with the trigger being an unrelated History service restart during a WAL rollout. As each History service restarted more and more of the overall service pool was unable to connect to the WAL and cell performance and availability deteriorated. The issue was mitigated by reverting the latent configuration change. After which we identified the sequence of events which lead to the configuration change laying dormant instead of being immediately applied and have instituted process improvements to prevent it from happening in the future.
This incident has been resolved.
Some customers in AWS eu-west-1 may experience elevated API latencies
Report: "Some customers in AWS ap-south-1 may experience elevated latencies and error rates"
Last updateThis incident has been resolved.
The fix has been rolled out and we are seeing recoveries to latencies/errors
The hotfix is currently being rolled out to the impacted cluster
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue
Report: "Some customers may experience errors when attempting to log in to cloud.temporal.io"
Last updateThis incident has been resolved.
Some users may experience errors when attempting to log in to cloud.temporal.io
Report: "Some customers in AWS us-west-2 may experience elevated visibility persistence latencies"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
Some customers in aws us-west-2 may experience elevated visibility persistence latencies.
Report: "Some customers in AWS us-west-2 may experience elevated visibility persistence latencies"
Last updateThis incident has been resolved.
Latencies have returned to normal and we are monitoring the system.
Some customers in aws us-west-2 may experience elevated visibility persistence latencies.
Report: "Audit Log Delivery Delay"
Last updateThe delivery of audit logs for any Cloud events that occurred between 3/26/2025 15:30 UTC and 3/27/2025 17:30 UTC was delayed. These audit logs have since been delivered to their destinations as of 3/28/2025 21:00 UTC
Report: "User Permission Updates Delay"
Last updateAny permission changs related to user creation/deletion/update were accepted by the system, but were not processed until 19:15 UTC
Report: "Some customers in AWS us-west-2 may experience elevated latencies"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Some customers in the AWS us-west-2 region might notice a delay in the visibility API"
Last updateThis incident has been resolved.
We are seeing Visibility API latency improvements and anticipate an accelerating full recovery.
The issue has been identified and we are working to scale the cluster further.
We are currently investigating this issue.
Report: "Some customers in the AWS us-west-2 region might notice a delay in the visibility API"
Last updateVisibility API latencies are fully recovered and the visibility task backlog has been drained.
After a slight wobble Visibility API latencies are mostly recovered and we are working to process the backlog of visibility tasks ASAP. We will continue to monitor this until the backlog is fully drained.
We are currently investigating a regression back towards degraded visibility API performance. We will update again when we know more.
Visibility API latency has improved and we are now working through the backlog of visibility tasks.
We are waiting on the fix to fully rollout and anticipate that we will begin to see recovery soon.
The issue has been identified and a fix is being implemented.
The issue is only with the visibility API and is not affecting workflows in anyway. We are investigating the issue.
Report: "Some customer of us-east-2 may see degraded Create/Delete/Update Namespace performance"
Last updateThis incident has been resolved.
The original incident was incorrectly reported as affecting us-east-1 when in fact it was only affecting us-east-2. We apologize for misreporting the incident, but it has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Higher latencies and error rate observed for some customers in us-east-1"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the result.
We are continuing to investigate this issue.
We are currently investigating the issue.
Report: "Some customers in AWS us-west-2 may experience elevated latencies"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the result.
Some customers in AWS us-west-2 may experience elevated latencies related to visibility task processing. The core system is not impacted.
Report: "Customers in GCP us-east4 experience unavailability of service"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Customers in GCP us-east4 experience unavailability of service
Report: "Customers in GCP us-west1 experience unavailability of service"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Customers in GCP us-west1 experience unavailability of service
Report: "Some customers in AWS us-east-1 may experience elevated API latencies"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Some customers in AWS us-east-1 may experience elevated API latencies.
Report: "temporal.io website temporarily unavailable"
Last updateThis is resolved.
Issue is resolved, will monitor a bit.
http://temporal.io website is temporarily unavailable. Note: https://cloud.temporal.io/ is NOT impacted. Temporal Cloud service is NOT impacted.
Report: "Temporal Workflow Web UI not accessible for a subset of namespaces in AWS us-east-1"
Last updateWeb UI functionality has been fully restored.
We have remedied the issue and are monitoring to make sure web traffic remains healthy
We are continuing to investigate this issue.
We are currently investigating the issue
Report: "Some customers in AWS us-east-1 may experience elevated API latencies"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Namespace creation failed for some customers"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the system.
We are continuing to work on a fix for this issue.
A recent release of our control plane led to namespace creation failure for some of our customers. We have identified the issue and are rolling out a fix.
Report: "Increased latency and reduced availability for some customers in the AWS ap-south-1 (Asia Pacific (Mumbai))"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Degraded response for some namespaces in ap-south-1 (Asia Pacific (Mumbai))"
Last updateThis incident has been resolved.
We believe that the service has fully recovered and are closing this incident
Customers should be seeing significant recovery (not yet complete)
We are continuing to work on a fix for this issue.
We believe we have identified the cause of this incident and are taking remedial steps
Report: "Updates of namespace configuration via web UI are failing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Updates via tcld or API are working with no issue. Only web UI is affected.
We are currently investigating this issue.
Report: "Temporal service degradation in aws us-east-1 region for some customers"
Last updateSome Temporal cloud customers in the AWS us-east-1 region experienced elevated latency and reduced availability
Report: "Some customers in AWS us-east-2 may experience elevated API latencies"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the issue.
Some customers in AWS us-east-2 may experience elevated API latencies
Report: "Small number of customers in AWS ap-southeast-1 may experience an elevated rate of API errors"
Last updateThis incident has been resolved.
Please note that this is exclusively impacting requests from the Temporal Cloud UI.
Small number of customers in AWS ap-southeast-1 may experience an elevated rate of API errors
Report: "Namespace creation failed for some customers"
Last updateA recent release of our control plane led to namespace creation failure for some of our customers. We have since rolled back the release which mitigated the issue.
Report: "Storage billing for namespaces on GCP has been temporarily disabled."
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Storage billing for namespaces on GCP has been temporarily disabled. We are working on a fix. In the meantime, storage costs might be inaccurate. Please know that there will be no charge for GCP namespace storage until this is resolved.
Report: "Some customers in AWS ap-south-1 may experience elevated API latencies"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Some customers in AWS ap-south-1 may experience elevated API latencies.
Report: "Some customers might experience an inability to configure some aspects of their namespace."
Last updateThis incident has been resolved.
We are in the process of deploying a fix for this issue.
We are continuing to work on a fix for this issue.
Some customers might experience an inability to configure some aspects of their namespace. The issue has been identified and a fix is being implemented.
Report: "Permission updates to newly created namespaces are delayed."
Last updateThis incident has been resolved.
The issue has been identified and a fix is being deployed.
Namespace permission updates are delayed.
Report: "Customers in us-east-1 may experience elevated rate of API errors"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
The issue has been identified and we are working on the fix
Customers in us-east-1 may experience elevated rate of API errors since 17:30 UTC
Report: "Customers in us-east-1 may experience elevated API latencies"
Last updateThis incident has been resolved.
A fix has been implemented and we see recovery
We have identified the issue and applying the fix to mitigate it
Customers in us-east-1 may experience elevated API latencies.
Report: "Customers in us-east-1 may experience elevated API latencies"
Last updateThis incident has been resolved.
The fix has been implemented, and we are seeing signs of recovery.
Customers in us-east-1 may experience elevated API latencies starting 18:30 UTC
Report: "High error rate and latency observed in us-east-1"
Last updateThis incident has been resolved.
Report: "Namespace creation, updates and deletes are currently broken"
Last updateThis incident has been resolved.
We are starting to see successful namespace operations going through
We are continuing to investigate this issue. This appears to be related to an ongoing AWS STS outage
We are continuing to investigate this issue. This appears to be related to an ongoing AWS STS outage
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "High latency and timeouts in us-east-1."
Last updateThis incident has been resolved.
Our vendor has identified and resolved the issue in us-east-1.
Some customers may be experiencing high latency and occasional timeouts in us-east-1. We are investigating with our vendor.
Report: "High latency and timeouts in us-east-1"
Last updateThe issue has been fully resolved and all customer should be experiencing normal latency in us-east-1.
Our vendor has identified and resolved the issue and we are seeing full recovery across us-east-1. We will continue to monitor the regions health for a while longer.
Some customers may be experiencing high latency and occasional timeouts in us-east-1. We are investigating with our vendor.
Report: "Customers in us-east-1 may experience elevated API latencies and error rates."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate the issue.
We are continuing to investigate the issue.
Customers in us-east-1 may experience elevated API latencies and error rates.
Report: "Web UI shows 500 errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Users experience 500 errors when trying to access web UI.
Report: "Elevated latencies observed in us-east-1"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Some customers in us-east-1 may experience elevated API latencies"
Last updateSome customers in us-east-1 may experience elevated API latencies. Impacted time window: 13:09 ~ 13:17 UTC