Historical record of incidents for Tettra
Report: "Heroku server issues"
Last updateWe are experience server issues with our hosting service Heroku (Salesforce). We are actively investigating the issue, and will update as soon as we have identified the issue. Customers will have issues logging in and using Tettra.
Report: "Full Outage"
Last updateWe are currently investigating a major outage impacting our application. This is due to an upstream infrastructure failure with our hosting provider, Heroku. Users may be unable to access any part of the application at this time. We’ve reached out to Heroku support and are closely monitoring their incident updates.
Report: "Partial Outage - Pages Not Loading Properly [Investigating]"
Last updateWe have identified and fixed the issue. We are currently monitoring for on-going problems.
We are continuing to investigate this issue.
We are currently investigating an issue casuing a partial outage to the Tettra app. The Tettra app itself still loads, but pages are not loading properly some of the time.
Report: "Partial Outage - Some paid accounts defaulting to billing screen"
Last updateWe have restored access to all the affected accounts and are monitoring the situation.
We have identified the issue with our billing system that is blocking some paid teams from accessing their customers. We are currently in the process of fixing it.
We are investigating an issue with our billing system that is defaulting some paid teams to the billing page.
Report: "Search not working for a small % of teams"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Report: "Pages and sidebar not loading for read-only users"
Last updateThis incident has been resolved.
A fix for this issue has been deployed.
Read-only users are not able to view pages or the sidebar right now. We're working on rolling back the deploy the caused this issue.
Report: "Some users unable to create new team"
Last updateThis incident has been resolved.
Report: "Intermittent issues with SSO Login"
Last updateWe were having issues with our SSO provider integration. This has now been resolved.
Report: "Tettra web app is down"
Last updateA workaround for the DNS issue has been implemented and the app is accessible as per normal
Our hosting provider Heroku is currently experiencing DNS issues, which is affecting the Tettra app’s ability to properly load. We are currently working to identify a potential workaround. https://status.heroku.com/incidents/2453
Report: "Webapp is down"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
Connectivity to the webapp is recovering.
Our hosting provider, Heroku, is experiencing an outage.
The Tettra webapp is not responding to requests. We are investigating the issue.
Report: "Email delivery delayed due to service provider errors"
Last updateThis incident has been resolved.
Our transactional email service provider, Mailgun, is experiencing API issues causing our email delivery to fail. We are monitoring the situation. https://status.mailgun.com/incidents/0hgvxpzzhvcq
Report: "Slack Instability"
Last updateWe believe slack's outage is now contained to parts of Slack that do not impact Tettra. Please let us know if you see any further issues with the Slack integration.
Slack is starting to come back online, so we hope all issues with our integration will resolve soon.
We are continuing to monitor Slack. You can see their own status page, here: https://status.slack.com/
Slack is currently experiencing system-wide issues. Tettra users may be unable to sign in using Slack or use the Slack integration within Tettra. We are monitoring the situation in case there is anything we can do to mitigate these issues with Slack.
Report: "Email delivery disabled due to spam attack"
Last updateEmail delivery and links are operational again. The 'share' page via email functionality has been temporarily disabled while we mitigate further spam attempts.
Links in existing/delivered emails are also disabled while we continue to work on a fix.
All emails from our application have been suspended since Dec 24 approximately 22:24 UTC due to a spam attack. This includes all automated notification emails, invitation emails, and page share emails. We have identified and suspended the malicious account(s) and are in the process of re-enabling emails.
Report: "Content embeds are failing"
Last updateContent embedding functionality has been restored. Thanks for your patience.
Embedly is still working on a fix. At this time there is no ETA as to when content embedding will be available again. We will update here as we know more. Original incident: http://status.embed.ly/incidents/tdkr78g2qq5x
We've identified an issue with our content embedding provider that is causing content embedding in pages to fail. We'll update this issue once the problem is resolved.
Report: "Readonly users unable to access Tettra pages"
Last updateFrom approximately Friday 4/17 5pm EST to Saturday 11am EST, readonly users were not able to access Tettra pages on the webapp.
Report: "Webapp unavailable"
Last updateAccess to the webapp has returned to normal. We will continue to keep an eye on the underlying Heroku incident that caused this and the outage earlier today: https://status.heroku.com/incidents/1974
We are continuing to monitor the app as it recovers.
app.tettra.co is unavailable due to problems with our service provider. We are investigating the issue.
Report: "Web app unavailable"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
The app is loading normally again. We'll keep monitoring until we hear from our hosting provider.
Our service provider Heroku is experiencing widespread platform issues. We will update as we have more information.
Visitors to app.tettra.co are seeing a Cloudflare error. We are investigating the cause.
Report: "Slowness and Instability"
Last updateWe have successfully mitigated the connection issues, and will work to assess what we can do to prevent such issues in the future.
We have narrowed down the issue to problems connecting to where we store data. We have contacted our service provider's support team but are also further investigating ourselves.
We are getting alerts of degraded performance to the point where some users are unable to view their content or log in. We are investigating.
Report: "Intermittent 500 errors"
Last updateOur billing provider has recovered. Please let us know if you see any more 500 errors.
We're aware of intermittent 500 errors being caused by issues with our billing provider. We will update you again when the problems have been resolved.
Report: "Web application unavailable"
Last updateCloudflare has implemented a fix for the network issue and is currently monitoring the results. The application is available again. We are continuing to monitor the situation.
We are continuing to investigate this issue.
We are currently investigating an issue with our DNS provider Cloudflare.
Report: "Intermittent Webapp Connectivity Issue"
Last updateEarly Sunday morning 5:30AM - 8AM EST the Tettra webapp was intermittently unavailable due to network issues on one of our application servers.
Report: "Broken links in email notifications"
Last updateFrom about 2018-10-22 3am EDT to 10:30am EDT links in Tettra email notifications were broken due to a DNS error. Our DNS settings have been updated and this issue is now resolved.
Report: "Web App Down"
Last updateThis issue has been resolved. We saw a spike in Slack events due to changing our App's name in the Slack directory. We're taking steps to make sure this doesn't happen again.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Web App Down"
Last updateOur web app was down for 8 minutes between 4:07-4:25pm EST. The incident is resolved and we are investigating the cause. Sorry for the issues.
Report: "SSL errors on app.tettra.co"
Last updateFrom 10:36 - 10:42 am EDT this morning we experienced unexpected downtime due to an SSL certificate migration issue.
Report: "Email notification outage"
Last updateFrom Friday morning at 7:15 AM EDT until Monday morning, we had an outage of our email notification system. Customers did not receive notifications via email including daily digests, mentions, suggestion updates, and invitation emails. Slack notifications were not affected.
Report: "Users unable to log in with Slack"
Last updateBetween 4pm EST and 10:45am EST users were unable to log in with their Slack account. The offending code has been rolled back and all systems are working properly at this time.
Report: "Web app connectivity issues"
Last updateFrom 20:10 to 20:30 EST customers experienced intermittent timeouts while accessing Tettra.
Report: "Web app connectivity issues"
Last updateThe web app had sustained connectivity issues and degraded performance for about 2 hours Monday morning.
Report: "Web App Down"
Last updateOur web app was down for about 2 hours on Thursday morning.
Report: "Degraded Web App Performance"
Last updateThis incident has been resolved.
We've found the underlying issue and have fixed it. You should see normal performance going forward.
We're investigating degraded web app performance.