Historical record of incidents for Retool
Report: "Intermittent Certificate Errors for Retool Requests"
Last updateWe've rolled out the fix and verified that customers should no longer be seeing self-signed TLS certificate errors and that custom-domain provisioning capability has been restored.
We've confirmed that the issue has stabilized. We're monitoring for unexpected spikes in errors, but please anticipate a very low rate of errors until a permanent fix has been executed.
We've implemented a stopgap to prevent this issue from reoccurring while we wait to execute a permanent resolution. You may continue to very rarely see this self-signed cert issue.
Retool occasionally incorrectly returns a self-signed TLS certificate on a small subset of HTTP requests. This can manifest on any request, affecting normal usage as well as Workflow runs that run resource queries. Our team has identified the issue and is working on a resolution. To address this, the Retool team has intentionally degraded the functionality to create a new custom domain. When creating a custom domain, certificate creation will take longer than usual while this incident is active. Your custom domain creation will progress automatically once this issue is resolved.
Report: "Planned Maintenance for Retool Networking"
Last updateWe are performing maintenance to resolve a networking issue that is currently causing self-signed SSL certificates to be returned intermittently. During this time, you may experience self-signed certificate errors and there may be downtime.
We are performing maintenance to resolve a networking issue that is currently causing self-signed SSL certificates to be returned intermittently. During this time, you may experience self-signed certificate errors and there may be downtime.
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Report: "Downstream Email Service Provider Outage"
Last updateProvider has fully recovered.
Platform features relying on email sending (invites, password reset etc) will be degraded until the outage is resolved Data connectors using the Retool Email resource type are NOT affected at this time. They should continue to be fully functional.
Report: "Downstream Email Service Provider Outage"
Last updateFeatures relying on email sending will be degraded until the outage is resolved
Report: "Intermittent Certificate Errors for Retool Requests"
Last updateRetool occasionally incorrectly returns a self-signed TLS certificate on a small subset of HTTP requests. This can manifest on any request, affecting normal usage as well as Workflow runs that run resource queries. Our team has identified the issue and is working on a resolution.To address this, the Retool team has intentionally degraded the functionality to create a new custom domain. When creating a custom domain, certificate creation will take longer than usual while this incident is active. Your custom domain creation will progress automatically once this issue is resolved.
Report: "Investigating slowness and crashing when editing apps"
Last updateThis incident has been resolved.
We are looking into reports of users experiencing increased slowness and eventual crashes when editing apps
Report: "Investigating slowness and crashing when editing apps"
Last updateWe are looking into reports of users experiencing increased slowness and eventual crashes when editing apps
Report: "Database Maintenance"
Last updateWe expect to experience 5-10 minutes of downtime as we conduct database maintenance. Workflows scheduled during this period may fail to run and will be picked up after maintenance has concluded.
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Report: "Retool Mobile App Editor experiencing frontend errors"
Last updateThis incident has been resolved.
We've identified an issue affecting the Retool Mobile App editor and have deployed a fix. We're currently monitoring to ensure everything is working as expected. We'll provide another update once we've confirmed the issue is fully resolved.
Report: "Retool Mobile App Editor experiencing frontend errors"
Last updateWe've identified an issue affecting the Retool Mobile App editor and have deployed a fix. We're currently monitoring to ensure everything is working as expected. We'll provide another update once we've confirmed the issue is fully resolved.
Report: "Retool Mobile on Android v3.191.0 facing crashes on load"
Last updateWe have resolved the issue that was causing Retool Mobile v3.191.0 to crash on launch for Android users. The patched version (v3.191.1) has been reviewed and is now available in the Google Play Store for users who had updated to the 3.191.0 version. If you were impacted by this issue please update to the latest version of Retool Mobile via the Play Store to restore full functionality. If problems persist after updating, please contact our support team for assistance. Users who never updated to the 3.191.0 version should see the 3.191.1 within the next hour or so, and for the time being may see 1.83.0 on their device still, this is the same build and should not be an issue.
We’ve moved this incident to Monitoring as no new users should be impacted and the issue has been resolved in the patched build (v3.191.1). However, the update is still pending approval in the Google Play Store. Until the Play Store version becomes available, we recommend impacted users continue using the temporary workaround of directly downloading the APK provided in our previous update: 👉 https://retool-mobile-releases-public.s3.us-west-2.amazonaws.com/3079.apk This manual installation will allow you to resume use of Retool Mobile on affected Android devices
We’ve identified the root cause of the crash affecting Retool Mobile v3.191.0 on Android and have submitted a patched version (v3.191.1) to the Google Play Store for review that will auto release when approved. Based on typical review timelines, we expect the update to be approved and available for download within the next few hours. In the meantime, for users currently blocked by the crash, we’re providing a temporary workaround: you can manually install the patched version by downloading the APK directly from the link below. Please note that this version has not yet been verified by the Play Store, so you will see a warning during installation. Follow on-screen instructions to allow the installation if you choose to proceed, and agree to update the current app. 👉 https://retool-mobile-releases-public.s3.us-west-2.amazonaws.com/3079.apk
We’ve identified an issue in the latest version of Retool Mobile (v3.191.0) for Android that is causing the app to crash immediately upon launch. The rollout was promptly halted after being downloaded by a limited number of users and will no longer be downloaded by new users. Our engineering team is actively investigating the root cause and working on fast tracking a recovery release for devices already impacted. We’ll provide our next update by 11:30pm Pacific Time, or sooner if more information becomes available.
Report: "Retool Mobile on Android v3.191.0 facing crashes on load"
Last updateWe’ve identified an issue in the latest version of Retool Mobile (v3.191.0) for Android that is causing the app to crash immediately upon launch. The rollout was promptly halted after being downloaded by a limited number of users and will no longer be downloaded by new users.Our engineering team is actively investigating the root cause and working on fast tracking a recovery release for devices already impacted.We’ll provide our next update by 11:30pm Pacific Time, or sooner if more information becomes available.
Report: "Retool Resources: Slack files.upload API Deprecation"
Last updateThis incident has been resolved.
Slack is ending support for their files.upload API on March 11, 2025. You will now need to use the files.getUploadURLExternal and files.completeUploadExternal APIs to continue using upload functionality. Slack's changelog provides more information on this change as well as migration steps: https://api.slack-gov.com/changelog/2024-04-a-better-way-to-upload-files-is-here-to-stay Our team is aware of this change and is working to update our Slack integration as quickly as possible. In the meantime, you should be able to use a REST Query to utilize the new APIs. https://docs.retool.com/queries/guides/api/rest We apologize for any inconvenience and will keep you posted.
Report: "Intermittent failures for source control deployments on Cloud"
Last updateThis incident has been resolved.
Service has been fully restored for nearly all customers. We’re assisting the remaining affected customers to ensure everything is back to normal.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
Some customers are experiencing an inability to deploy from source control on cloud (retool.com) starting around 12am PST. We're investigating the root cause at the moment.
Report: "Intermittent failures for source control deployments on Cloud"
Last updateThis incident has been resolved.
Service has been fully restored for nearly all customers. We’re assisting the remaining affected customers to ensure everything is back to normal.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
Some customers are experiencing an inability to deploy from source control on cloud (retool.com) starting around 12am PST. We're investigating the root cause at the moment.
Report: "Some apps not loading"
Last updateThis incident has been resolved.
We've received some reports of apps not loading for users. We've identified the issue and are deploying a fix.
Report: "Some apps not loading"
Last updateThis incident has been resolved.
We've received some reports of apps not loading for users. We've identified the issue and are deploying a fix.
Report: "Slack incident impacting slack-based support services"
Last updateThe incident impacting the Slack Events API impacting accessibility of our Slack Support integration has been resolved from Slack's end, and full functionality has been restored.
A second slack incident may be impacting our Support offering via slack (available to Enterprise customers with Premium Support). Replies may be intermittent during the outage. Please leverage https://help.retool.com and support@retool.com as fallback contact channels in the meantime.
Report: "Slack downtime possibly impacting Retool support services"
Last updateThe slack incident has been resolved and Retool support via slack is now operational as normal.
Due to the ongoing slack api outage, our Support offering via slack (available to Enterprise customers with Premium Support) may be impacted. Replies may be intermittent during the outage. Please leverage https://help.retool.com and support@retool.com as fallback contact channels in the meantime.
We are currently investigating the impact of an on-going slack incident on our support services.
Report: "community.retool.com down"
Last updateWe're identified the root cause. The website should be back to normal now.
We are continuing to investigate this issue.
We're aware of the issue and are actively investigating.
Report: "Javascript custom libraries broken for workflows"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Adding new javascript custom libraries to workflows is currently broken. Existing workflows with custom libraries are still functional.
Report: "Investigating reports of performance degradation during peak traffic hours"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Investigation is under way for increased latencies during 6-9am PST hours for some customers. Further information will be shared once we verify a root cause. Components affected will be updated to reflect any ongoing performance degradations
Report: "Performance degradation"
Last updateThis issue has been resolved
We are currently observing performance issues around loading apps in Retool Cloud.
Report: "Errors when deploying new versions of Retool apps with source control"
Last updateThis incident has been resolved.
We are currently investigating "Unable to Deploy" errors when pushing new versions of Retool apps via source control.
Report: "Slowness for Retool backend requests"
Last updateThis incident has been resolved.
We've identified an anomalous traffic pattern causing high response times, and our latency has recovered.
We're investigating high response times for some backend requests.
Report: "Issues running queries that used SSH tunnelling"
Last update25% of resources that used SSH Tunneling were failing between 21:00 UTC and 22:45 UTC.
Report: "Networking outage"
Last updateThe configuration change has been rolled back and systems are stable. Incident is resolved at this time.
A configuration issue caused a networking outage that impacted critical retool components from 3:41 PST to 4:00 PST. Systems have recovered and we are actively monitoring.
Report: "Custom domain provisioning delayed"
Last updateThis incident has been resolved.
Requested domains will be provisioned after maintenance is complete
Report: "Retool Workflows Partial Outage"
Last updateThis incident has been resolved
A fix has been implemented and we are monitoring.
We are currently investigating the issue
Report: "Upstream email provider is down"
Last updateThis incident has been resolved.
Our upstream provider is recovering and emails are successfully sending again.
Our upstream email provider is having a major outage which impacts all outgoing Retool email (invites, password resets, etc...). We are closely monitoring progress.
Report: "Slow loading times for EU"
Last updateThis incident has been resolved.
We have received reports of Retool cloud accounts experiencing slow load times through the app from EU based users. We have confirmed all Retool infrastructure is operating normally, and have seen that using a US VPN currently seems the best way to work around such networking issues.
Report: "Apps with "expand content to fit" not loading"
Last updateBetween 3:01pm PDT and 3:50pm PDT, Retool apps with the "expand content to fit" property were unable to load.
Report: "Issues with Spaces and Custom Domains provisioning"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Some customers might experience issues with provisioning new Spaces or Custom Domains. We're investigating this issue.
Report: "Retool-managed Temporal Partial Outage"
Last updateThis incident has been resolved.
Some Retool-managed Temporal namespaces are experiencing degraded service levels due to an ongoing incident with our third-party vendor, Temporal.io. A small number of self-hosted customers will have their workflow executions stalled until the incident is resolved.
Report: "Issues with OpenAPI query performance"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "High resource query error rate"
Last updateThis incident has been resolved. We apologize for the inconvenience caused. This was due to an infrastructure code change that was not well-validated and we will be putting systems in place to ensure this classification of issue cannot reach our production environment without proper validation.
Resource queries should be fully operational as of ~17:42 UTC. We will continue monitor closely to be sure there are no further issues.
We have identified and implemented a fix. We are monitoring error rates as they return to normal.
We are investigating abnormally high error rates when running resource queries.
Report: "Elevated backend error rate"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for the root cause of the issue, in the meantime the issue has been mitigated and all services are operational.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Elevated backend error rates"
Last updateThis incident has been resolved.
We are currently not observing the previously reported issue. Our team is actively monitoring the system to ensure ongoing stability and to promptly address any reoccurrences. We appreciate your understanding and will provide further updates as necessary.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Elevated backend error rate"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Page Freezing on App Load"
Last updateWe rolled back cloud and the issue has been resolved.
We are currently investigating an issue causing freezing/unresponsiveness on app load. This seems to be affecting users on Chrome with hardware acceleration. As a workaround for the time being, try using another browser or disabling 'hardware acceleration' in Chrome.
Report: "Investigating increased latency and errors"
Last updateThis incident has been resolved
A fix has been implemented and we are monitoring performance
We are investigating increased latency and errors
Report: "RetoolDB down"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
RetoolDB experienced a temporary outage between 6:24am and 7:40am PT as the third party database provider we use was unavailable
Report: "Workflows performance degraded"
Last updateThis incident has been resolved
A fix has been implemented and we're monitoring the results.
We are currently investigating the issue.
Report: "retool.com is inaccessible"
Last updateThis incident has been resolved.
We've put out a fix and we're monitoring the issue
retool.com is currently down, we're investigating the issue
Report: "RetoolDB down"
Last updateRetoolDB experienced a temporary outage between 10:04am and 10:18am as the third party database provider we use was unavailable.
Report: "Slow response time in tryretool.com"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Users with tryretool.com subdomains are experiencing slower than usual response times. We're currently investigating the issue.
Report: "Degraded Docker Hub performance"
Last updateThis incident has been resolved.
Accessibility of Docker images for self-hosted Retool may be affected by an ongoing Docker Hub incident: https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/6515dbc89fabff05350ae18d
Report: "Single managed_db instance shard unavailable"
Last updateThis incident has been resolved.
The new database instance is up and we are monitoring. Thank you for bearing with us through this incident, please contact us through support if continue to see any issues.
We are still monitoring the database recreation.
We are still monitoring the database recreation.
We are still monitoring the database recreation.
We are in the process of restoring a database backup from the point at which this particular instance went down. Unfortunately due to the size of the database we cannot give an ETA when it will be back up yet. We apologize for the inconvenience and will have an update in 3 hours or less.
We are continuing to investigate this issue and will provide updates as soon as we have them.
We have identified this to be an upstream issue with our database vendor (AWS) and we are working with them to resolve.
We are currently investigating reports of a subset of Cloud orgs no longer able to access a specific instance of the managed_db starter postgres resource.
Report: "Query errors for script clients and some public apps"
Last updateThe fix has been deployed and resolution confirmed.
We have identified an issue causing non-browser clients (i.e. custom scripts) to be unable to make resource queries as well as a small percentage of public apps to fail to load. We have identified the cause and will deploy a fix shortly.
Report: "Elevated query latency and error rates"
Last updateCustomers may have experienced elevated rates of 502's and significantly higher query latency on a subset of queries for a small period of time (12:50PM -> 2PM PST) We've identified the cause and are continuing to monitor while rolling out a longer-term fix.
Report: "Chunk loading errors on self-hosted Retool instances"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Self-hosted deployments that are configured to automatically pull new Docker images may experience frontend chunk loading issues when some but not all containers are started / restarted. To fix these issues, please restart the remaining containers in the deployment.
Report: "Email service interruptions"
Last updateUsers on self-hosted Retool instances might experience interruptions on sending user invites
Report: "Elevated API error rate"
Last updateThis incident has been resolved.
We have addressed an issue with our data cache and our monitors have returned to normal. Workflows runs were delayed and possibly failed if there were multiple scheduled runs of the same workflow between ~13:30 UTC and ~14:30 UTC.
We are investigating an issue with elevated error rates on our backend API, possibly data cache related.
Report: "AWS domains resolving as private IP addresses"
Last updateThis incident has been resolved.
We have applied a fix and are monitoring an issue where DNS resolution for AWS subdomains of *.execute-api.us-west-2.amazonaws.com were incorrectly resolving to private IP addresses.
We are continuing to monitor for any further issues.
We have applied a fix and are monitoring an issue where DNS resolution for some AWS (sub)domains were incorrectly resolving to private IP addresses.
Report: "Some apps unable to run queries"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Some Retool services affected by Azure outage"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
Retool services affected by the Azure networking outage appear to be recovering.
Some Retool services are being affected by an Azure networking outage. This includes our licensing server and a subset of database connector servers hosted in Azure.
Report: "Increased latency when loading app list"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "tryretool.com domains unable to load"
Last updateThe DNS change has been propagated. If you're still encountering issues try flushing your DNS cache.
We've identified the issue and are waiting for the DNS change to propagate to all customers.
The issue has been identified and a fix is being implemented.
We are currently investigating domains on tryretool.com failing to load.
Report: "analytics and licensing servers are down"
Last updateThe DNS change has been propagated. If you're still encountering issues try flushing your DNS cache.
We've identified the issue and are waiting for the DNS change to propagate to all customers.
On-prem customers are currently unable to access our analytics and licensing servers. We are currently remediating.