Historical record of incidents for Tuple
Report: "Interimittent TURN errors"
Last updateTURN connections are currently impacted by Cloudflare's outage: https://www.cloudflarestatus.com/incidents/25r9t0vz99rp We are investigating temporary mitigations and monitoring the Cloudflare incident status.
Report: "Database performance issues"
Last updateThis incident has been resolved. Please get in touch with support@tuple.app if you experience any issues.
We've implemented a fix and are monitoring the results.
We are currently investigating degraded performance across the web dashboard and native applications due to an underlying issue with our database infrastructure.
Report: "App Crash with Latest Release"
Last updateCrash reports have ceased. We're considering this incident fully resolved!
We've reverted to the previous release. If you're experiencing issues, please update your app. We'll resolve this once we're confident no further crashes are occurring.
We're seeing issues with our latest release, deployed today, that's causing a subset of users to experience app crashes while calling. We're reverting the release to get things back to normal; that should be ready and deployed in the next 15 minutes.
Report: "Web Backend Request Failures"
Last updateAll client response times are back to normal and we're considering this incident resolved. Please get in touch with support@tuple.app if you experience any issues.
A fix has been implemented for affected teams, and response times for the affected API route are back to normal.
We've identified a performance issue with our Call URLs API route that is affecting a small number of teams. We've put a temporary fix in place for those teams that will restore application access with some diminished functionality around Call URLs, and the team is actively working on a permanent fix. The majority of Tuple users remain unaffected.
We are seeing failed requests to our call URLs endpoint for specific teams. Not all users are affected, and we're currently investigating the issue to determine the root cause.
Report: "Tuple Backend Outage"
Last updateThis incident has been resolved.
Services are operational, and we're continuing to monitor our web application metrics before considering this fully resolved.
Presence service traffic was scaled to zero to allow the backend to stabilize. Things appear to be operational, but we're actively monitoring the situation to ensure the incident is fully resolved.
We're experiencing a large number of requests through our presence service that are overwhelming our web backend.
Report: "Native client connection issues"
Last updateWe're fully back online and client connections are succeeding.
System health is recovering, and some users are successfully reconnecting. We're increasing infrastructure capacity slowly to accommodate the backlog of reconnecting clients.
We are currently investigating an issue with clients being unable to connect after a planned production deployment. Ongoing calls appear to be unaffected.
Report: "Presence Signaling Service Issues"
Last updateDirectory status for all users is up-to-date, and the signaling server has continued to process requests successfully.
We've seen some instances of stale presence data persisting in the directory and are monitoring the relevant queue as it is processed.
Tuple contact directories are loading. We're actively monitoring the service while clients reconnect to ensure that the issue is resolved.
We've attempted to roll back our signaling service to a previous release, but are still experiencing issues with our load balancer correctly negotiating connections. Our team is still actively working to resolve this issue.
We're continuing to work on a fix for this issue.
We've identified an issue with our signaling server that is preventing the team directory from loading. We're working to resolve the issue now.
Report: "Email Provider Outage"
Last updateThis incident has been resolved.
Our Email provider is experiencing an outage. Application emails are delayed.
Report: "App down"
Last updateService appears to have been restored.
We are currently down due to downtime at Heroku, an underlying provider of ours. You will likely be unable to sign in or make calls at this time. Their incident response can be tracked here: https://status.heroku.com/incidents/2402 (Unfortunately, their status page appears to be loading only intermittently.)
Report: "Login loop/crash after update"
Last updateThis issue has been resolved. Sorry about that folks!
We've released a fix. Please restart Tuple to force an update to the latest version. This should resolve the issue.
We are continuing to work on a fix for this issue.
We believe we have identified the issue. We're building a new release and are deploying it imminently.
We're aware that users are getting stuck in a login loop/crash after this morning's update. We're investigating.
Report: "Intermittent authentication issues"
Last updateA weekend deploy temporarily prevented unpaid guest users \(and users on a small number of older teams\) from signing in. We investigated Monday morning after reports started to roll in and resolved the issue approximately 20 minutes later. If you are still having trouble signing in, please contact support: support@tuple.app.
This incident has been resolved.
Some users are reporting that they cannot log in to the Tuple client.