Historical record of incidents for Sunsama
Report: "Gmail & Google Calendar Outage"
Last updateThis incident has been resolved.
The issue has been fixed however the root cause is undetermined. We will continue monitoring things in case it returns.
The Gmail and Google Calendar APIs have started responding with errors. We are investigating the issue and will report back when we know more.
Report: "Todoist Integration Degraded"
Last updateThis incident has been resolved.
The Todoist integration is currently degraded. We are investigating the root cause and looking for fixes. In the meantime please use import by URL for Todoist tasks.
Report: "AI Services Outage"
Last updateThe problem occurred because of an outage in a third party service. That service has been restarted and all issues have been resolved.
We are continuing to investigate this issue.
Both the planned time prediction and channel prediction services are currently offline. We are investigating the issue.
Report: "Sunsama is down"
Last updateAll systems go! We should be totally back on line now.
Sunsama is back up for 99% of customers. There are some power users who may still experience issues with today's tasks loading (we expect that to resolve shortly).
We fixed the primary issue and expect normal performance for most users. However there are a small number of requests that are still timing out. We are sending out a possible fix for the remaining issues and we will continue to monitor the situation.
We are aware of an outage and hope to have Sunsama back online within the hour.
Sunsama is currently down, extremely slow to load, or not loading tasks. We're aware of the situation and aware of a possible fix and hope it will be back up within the hour.
Report: "Archive tasks not showing"
Last updateThis incident has been resolved.
We're testing a fix right now.
There is a fault in the archive view, which causes archived tasks not to show up. Don't worry, you're tasks are fine, they just don't show.
Report: "Calendar events not loading"
Last updateThis incident has been resolved.
This issue is now resolved. For about an hour today 4PM PST to 5PM PST, calendar events did not load in the app.
We're experiencing an outage with calendar events, we are investigating and working on a patch.
Report: "Performance degradation with collaborative editing"
Last updateThis incident has been resolved.
We are experiencing intermittent issues with our notes editor, specifically when multiple people are typing.
Report: "Calendar Sync Performance Degraded"
Last updateWe have resolved the issue that prevented calendars from syncing. The issue was caused by a change to one of our software libraries that resulted in a new warning message. The warning message was not handled gracefully and resulted in sync requests dropping details.
We're experiencing some performance degradation with calendar syncing caused by some unintended regressions. Solutions are being investigated
Report: "General app outage"
Last updateThis outage is now resolved and we've been monitoring our systems for the past hour to confirm stability. Thanks for your patience. It's been a tough day at Sunsama HQ. All of the outages we've had this year, all on the same day 🥵
We are aware of an issue that's made the app unable to load and for users where it does load, the app generally does not work (load events/tasks). We've identified the root cause and are working on a fix. We believe its a second order impact from our most recent fix to work around ClickUp's false positive ban of our IP for our ClickUp integration.
Report: "ClickUp integration cannot be added or used"
Last updateOur ClickUp integration is back up. After further investigation, we learned that ClickUp's API Infrastructure banned all IP addresses associated with the Sunsama application. We believe that ClickUp's security teams and infrastructure banned our IP in response to actions taken by a nefarious actor who uses the same cloud service provider as us. When they banned the IP of the nefarious user, they also banned all users who use the same cloud provider. In order to mitigate this, we've moved all of our outbound traffic to a static IP address so we can't share IP level bans with nefarious users of our cloud provider.
We're investigating an issue with the ClickUp integration. New integrations can't be added and ClickUp tasks can't be imported from the integration panel.
Report: "Old/stopped recurring tasks showing up"
Last updateAll issues have been identified and affected customers will be notified with their options within the next 24h.
We've been doing some database cleanup of some recurring tasks that had invalid data, and in the process have decided to mark old/stopped ones as valid tasks to repeat, when we weren't sure if they should or should not keep repeating. We're investigating how we can automatically fix this situation.
We're aware some recurring tasks have re-appeared and can be showing up as duplicates for "Today". We're working on a fix to make sure they don't show up again, and expect to have an update in the next hour.
Report: "Issue with our notes editor"
Last updateThe web and desktop editors are both stable again. We will be adding in more robust solutions to prevent this issue again in the future.
We've deployed a fix. Please refresh your browser or restart the desktop app for the fix to take affect. We'll keep monitoring the situation to make sure this resolves the issue for everyone
We've deployed a first fix that limits the impact of the issue, re-enabling the writing of daily planning and shutdowns. Some users can still problems in editing notes of tasks. We've identified a way to resolve this as well and are working on deploying a fix.
In an update to the web and desktop apps, we've noticed an issue in writing notes for tasks, and daily planning / shutdown. You might experience trouble in typing certain characters, like spaces. We're working on resolving the issue as we speak.
Report: "Cannot create events from within Sunsama's web/desktop apps"
Last updateA fix has been deployed and the fix verified.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Todoist integration issue"
Last updateThis incident has been resolved.
There's currently an error in the Todoist integration, rendering it unusable. We are investigating the issue and resolve this as quickly as possible.
Report: "Performance Issue (App freezes)"
Last updateThis performance issue was caused by a third party library we used for error reporting and diagnosis. The offending library has patched the underlying issue and rolled out a new version. We've also removed this library entirely since it's caused several performance degradations over the past few months.
We've received reports that the web/desktop app are freezing up. If this happens to you, please let us know, we're looking for more clues to the root cause right now.
Report: "ClickUp and Outlook Outage"
Last updateThe outage of our ClickUp and Outlook integrations is now resolved.
We are aware of issues with the ClickUp integration and Outlook integrations not loading. We are working on fix and hope to have this resolved ASAP.
Report: "Tasks disappearing"
Last updateA heavy operation in our production database was started an hour ago that wasn't supposed to impact task loading. Unfortunately it did, so some days might have had missing tasks for a few moments (our app's retry mechanism should eventually fetch them). If you find them missing after a few moments, please refresh the app. Everything should be stable now, and no data should've been lost.
Report: "Degraded service due to AWS outage"
Last updateAWS has let it known the situation is resolved. All back to normal again.
It looks like Sunsama is coming back to life again. We're continuing to monitor the situation so please let us know if you still have trouble working with or logging into Sunsama.
Currently one of our service providers is offline due to an AWS failure. This is out of our hands but we are currently monitoring the situation.
Report: "Service interruption"
Last updateEverything has been stable since the last update, and we simply aren't able to safely deploy new improvements and fixes for now.
It seems we're gradually recovering now. Still monitoring. Google shared a report at https://status.cloud.google.com/incidents/6PM5mNd43NbMqjCZ5REh
It seems our main systems provider (Google Cloud Platform) is having issues right now. We'll keep investigating. - https://downdetector.com/status/google-cloud/ - https://status.cloud.google.com/
Our systems appear to be overwhelmed with connections and are malfunctioning. We're investigating the issue.
Report: "Major reliability release, please clear your browser cache"
Last updateWe released a major reliability update this morning. If you're experiencing a white screen (or the blue update banner that won't go away), please try clearing your browser/app cache and reloading (CTRL + SHIFT + R or CMD + SHIFT + R), as stale cache would cause it. While there was no downtime, stale cache would cause the app to appear broken.
Report: "Outlook Email previews broken"
Last updateOutlook emails were not being displayed properly and an error screen was showing in their place. The bug that introduced this issue has been repaired and deployed
Report: "Some calendar events might not show"
Last updateWe have now deployed a fix and the issues should not persist after a refresh.
We've identified a problem with some calendar events not showing and are working on verifying and deploying a fix already.
Report: "Customize trial not working"
Last updateThis incident has been resolved.
The "Onward" button is now working, sorry for the trouble!
We are aware the "Onward" button from the "Customize trial" page isn't working right now. We're investigating.
Report: "App not loading"
Last updateThe app should be back up if you refresh now. We're sorry about the inconvenience this caused.
We are continuing to work on a fix for this issue.
We just released an update that caused many clients to not load successfully. We're already releasing a fix for it and expect the app to be back to normal in less than 15 minutes.
Report: "Slack integration"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently experiencing issues with Sunsama tasks created from within Slack.
Report: "Missing tasks"
Last updateWe've finally restored visibility for all of the 2M tasks affected, and systems are looking normal again. Thank you so much for your patience, and no data should've been lost!
We have confidently fixed the cause of problem and confirm it is fixed. Right now we're working on restoring all data visibility with minimal customer action required.
While testing we noticed the fix was incomplete and are working on restoring the data visibility, while verifying the new fix.
We've identified what seems to be the culprit, an update from Friday which would be hiding expired recurring tasks. We'll revert it for now and will investigate the problematic logic in that update. We'll also revert all tasks that were erroneously hidden, which can bring back a few deleted tasks that were intentionally deleted.
We're seeing many reports of missing tasks from customers but are unable to find a cause yet. We're currently investigating and will try to publish an update in 30 minutes or less.
Report: "App Errors"
Last updateWe've patched the issue and everything is back up!
We're experiencing some in app errors. We expect to push a fix in the next 5-10 minutes.
Report: "Performance degradation"
Last updateWe're experiencing some performance degradation due to some new changes we pushed. We're already reverting them and it should only affect some people who got this new version for the next 10 minutes maximum.
Report: "New Slack Integration being published"
Last updateMost customers should be seeing the new banner now to inform about this and experience no more hiccups.
We've just published a new Slack integration that will require everyone to re-add it. The app might have some hiccups but refreshing should work fine.
Report: "Infrastructure Upgrade"
Last update✨ Infrastructure upgraded! We saw about ~10 minutes where data didn't load intermittently for some users. We're back in action now!
We're doing a massive upgrade of our infrastructure tonight to better support everyone. We expect minimal to no downtime but you may notice a few minutes of tasks, events or integrations not loading.
Report: "Login Failing"
Last updateThis incident has been resolved.
We've had reports of users being unable to login, we are actively investigating.
Report: "Hosting Service (Vercel) is experiencing degraded availability"
Last updateWe are currently monitoring an outage with our hosting provider Vercel: https://twitter.com/vercel_status/status/1360026810365464578 They claim the issue is now resolved but it's possible our users may have seen intermittent failures or disconnections for a period of 40 minutes. In this case, this incident impacts all of their customers and is not something we have specific control over.
Report: "Performance degradation on app"
Last updateThe app loaded but it was slow and interactions would frequently fail, due to higher load in our servers. We temporarily disabled some unnecessary features and the slowness lasted for less than one hour. This affected the web app, desktop app, and mobile app.
Report: "Performance degradation on app"
Last updateThe app loaded but it was slow and interactions would frequently fail, due to higher load in our servers. We increased our infrastructure capabilities and the slowness lasted for less than one hour. This affected the web app, desktop app, and mobile app.
Report: "Blank page on web and desktop apps"
Last updateBlank page preventing the usage of the app, caused by a code cleanup. It lasted for almost 1 hour.
Report: "Performance degradation"
Last updatePerformance degradation related to higher usage and less-performant code being deployed. The app was accessible but slow for roughly 4 hours, with a few periods of normal performance in between. This affected the web app, desktop app, and mobile app.
Report: "Performance degradation with integrations"
Last updateSlowness and instability loading integration data from all integrations, due to higher usage. This lasted for roughly 15 minutes and affected the web app and desktop app.
Report: "Calendar integration unresponsive"
Last updateCalendar events and event information stopped showing because of Google blocking our API requests due to excess usage. This lasted for 2.5h until Google raised our limit. This affected the web app, desktop app, and mobile app.