Historical record of incidents for Stacker
Report: "Stacker Classic Down"
Last updateWe're investigating an issue which is affecting all customers on Stacker Classic not using custom domains.
Report: "Unscheduled database maintenance for Astra"
Last updateThe scheduled maintenance has been completed.
Scheduled maintenance is in progress
As part of preventing an upcoming incident we're having to restart the database cluster which serves our Astra product. The whole of Astra will be unavailable fr 2-3 minutes while we make this change. I apologies for the short notice for this maintenance.Customers using the Classic product are unaffected.
Report: "Scheduled maintenance"
Last updateThe scheduled maintenance has been completed.
Scheduled maintenance on Astra is complete. Thank you.
We will be undergoing scheduled maintenance during this time.
Astra Only (Stacker Classic unaffected)We have a scheduled infrastructure update on Monday, May 26 at 9:00 AM PST. We expect only brief and partial downtime, lasting no more than a minute or two. In most cases, any disruption—such as to Stacker data loading—is barely noticeable. However, there is a chance some automation events will be missed during this short window.
Report: "Stacker Classic Outage"
Last updateThis incident has now been resolved and everything is working as expected.
We have fixed the issue and we are investigating the underlying causes. We're monitoring the system while everything returns to normal.
We are currently urgently investigating an issue impacting Stacker Classic. This is causing Stacker apps to load slowly or to show loading spinners for long periods of time. Our engineers are engaged to resolve this issue as quickly as possible. We'll update here when we have an ETA for resolution.
Report: "Stacker stuck on the loading screen"
Last updateThis has been resolved, you should now be able to access your apps. Please let us know if you have any questions or remaining issues.
This is impacting only Astra users. When opening Stacker it is stuck on the loading page, we are working on finding a fix for the issue. Thank you for your patience while we work through the issue.
Report: "Some users unable to login to Stacker (12:20UTC-13:40UTC)"
Last updateWe experienced an issue that impacted some users' ability to log into Stacker between 12:20 UTC and 13:40 UTC. The issue has since been resolved. If you're still encountering any problems, please try refreshing your page. If the issue persists, feel free to reach out to us at support@stackerhq.com, and we'll be happy to assist you. We apologize for any inconvenience this may have caused and are actively investigating the cause to prevent this from happening in the future. If you have any questions or concerns, please don’t hesitate to contact our support team. Thank you for your understanding.
Report: "Short unplanned outages due to database maintenance"
Last updateStacker V5 "Astra" had three short outage periods where all requests to the service failed on Monday between 12:30 and 14:00 UTC. Each of the three outage periods lasted 1-2 minutes. This was a result of some routine database changes causing an unforseen conflict which prevented normal queries from being processed. Because it was part of changes we were making manually we were able to identify and resolve the issue almost immediately. We apologies for the disrption this would have caused to any users online at the time. We're reviewing our processes which should have protected aginst thins kind of conflict.
Report: "File uploads are not working"
Last updateThis incident has been resolved.
The issue seems to be resolved and uploads are now working. We will continue to monitor for any further issues. If you still have issues after a page refresh please reach out to support and we'll be able to help you out.
Our file upload service is not working correctly. This prevents any files being uploaded into Stacker: in attachment fields, rich text fields, app or workspace logos, and so on. We're continuing to investigate the issue. This is affecting both our Classic (V4) and Astra (V5) versions.
Report: "Admins are unable to preview as a user"
Last updateWe have released a fix for this issue! Please contact support@stackerhq.com if you have questions or continue to experience difficulties.
We have identified the issue and are working on a fix.
We are currently investigating an issue where admins are unable to preview as a user in their apps.
Report: "File uploads not available"
Last updateAttachment fields in Stacker weren't able to add new files. This was caused by an issue with our file storage provider which we have now resolved.
Report: "Airtable tables not appearing in some apps"
Last updateWe have emailed all customers who need to take action to re-authorize their Airtable connections.
We have resolved the issues with the Stacker apps, however you may need to repair your authorization with Airtable in order to continue to sync data. Please go to the data connection settings in each application and click "Repair". During the re-authorization process, you need to authorize every Base used in Stacker, across all of your Stacker apps. The simplest way to do this is to select the option labelled "All current and future bases in all current and future workspaces”.
We've identified the issue, and have taken Stacker offline in order to safely apply the necessary fix. We're working hard to get it back online as soon as possible.
We are investigating reports that tables from Airtable are not appearing in some users' apps
Report: "Schema sync from Airtable is not syncing or is slow"
Last updateThis incident has now been resolved! Thank you.
We have identified the issue. The following features are experiencing issues: our data source schema syncs (Airtable, Gsheets, Postgres, Salesforce) and Stacker realtime data. Workaround: If you run a schema sync then close the schema sync pop up and refresh your app you should see your changes.
We are currently investigation an issue where Airtable schema sync is not working or is slow. We will keep you updated as we continue to look into this.
Report: "Stacker navigation is hidden from app view"
Last updateThe incident of the navigation being hidden has now been resolved.
A fix has been implemented and we are currently monitoring the fix.
The issue has been identified and we are implementing a fix for the navigation being hidden.
We are currently investigating an issue where the navigation and tables appear to be hidden from the app view. We will update when we have more information. Thank you!
Report: "Apps are taking longer than normal to load"
Last updateThe incident with slow loading times has now been resolved.
A fix for the long load times in Stacker apps has been deployed and we are continuing to monitor.
We are currently investigating an issue with apps taking longer than normal to load.
Report: "Schema Sync Intermittently Fails for Password-Protected Airtable Bases"
Last updateSchema sync should now work consistently for password-protected sharing links. There was an unexpected change in the schema sync responses we were getting from Airtable, and we needed some time to adapt. Thanks for your patience while we sorted this out.
We're aware of an ongoing issue affecting schema syncing for Airtable bases with passwords. We'll be looking into this matter and will provide further updates as soon as possible. We apologize for the inconvenience and thank you for your patience.
Report: "Attachments no longer working"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Errors when schema syncing an Airtable base"
Last updateThis incident has now been resolved.
The issue has been identified and we are implementing a fix.
We are currently investigating an issue with schema syncing Airtable bases that have passwords or are using old sharing link types.
Report: "New Airtable connections are not syncing"
Last updateNo other issues being seen or reported. Will continue to work with Airtable to resolve issue with password protected bases.
A fix has been implemented and we are monitoring the results. We are still seeing issues with Password-protected bases not schema syncing.
We are currently investigating an issue where adding a new Airtable connection is not syncing for some users.
Report: "Blank screen when loading the app"
Last updateIssue is resolved. We'll be implementing additional monitoring and working on making sure this particular issue doesn't happen again. Thanks for your patience.
We are looking into an issue where some customers are seeing a blank screen when loading the app.
Report: "Load balancer issues"
Last updateThis incident has now been resolved!
A fix has been implemented and we are continuing to monitor the incident.
Everything in Stacker is currently operational with slightly degraded performance. However, the root cause is currently being investigated by one of our providers as it may be stemming from them. We are working with them to find a solution. Further outages may be possible as we continue working on the issue, but we will do our best to keep them at a minimum.
We believe Stacker should now be working for most people, but we're continuing to investigate the situation.
It seems like an issue with our load balancer has brought Stacker down. You will most likely not be able to load your apps at the moment. We're looking into it with priority, and we'll let you know as soon as we have more information.
Report: "Potential sync time issue"
Last updateWe should be in the clear, thanks for your patience.
It seems like there was a change in the responses from Airtable—we have made a few steps to take these changes into account, and we should be back to normal. We'll continue monitoring the situation to make sure everything is indeed normal.
We are still investigating, and will let you know when we have more to share.
We are seeing slower response times when interacting with Airtable. We believe this could cause slower or non-working data or schema syncs for some customers. Due to the way our sync queues work, you could be impacted even if you use different data sources. We’re investigating the situation and we’ll let you know as soon as we have more to share.
Report: "Airtable sync outage"
Last updateThe issue has been resolved, and we'll continue to monitor performance.
Airtable sync is returning 'SCHEMA_IMPORT_ERROR' for some customers, and we are currently investigating this issue.
Report: "Issue with Google Sheets Schema Sync"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We have identified the problem and are working on a fix.
Report: "Issue with Google Sheets integration"
Last updateWe're in the clear. Thanks so much for your reports and patience.
We have implemented the fix and are monitoring the issue. If your layouts are still not loading, please try reloading your app.
We have identified the issue and are working on a fix.
We've received reports of Google Sheets data not loading for users and admins, and are investigating these reports now.
Report: "Users unable to access Stacker"
Last updateThis incident has been resolved.
We've finished monitoring the fix and this is now resolved.
Fix is in place and we're monitoring.
We've received reports that some users aren't able to access Stacker. We're currently investigating the issue.
Report: "Schema and user sync outage"
Last updateSchema and user syncs were experiencing degraded/slow performance at around 04:40GMT on the 22nd of December. A fix was implemented, and the incident was resolved at 07:58 GMT on the 22nd of December.
Report: "Schema and user sync outage"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are continuing to monitor the results.
Schema and user syncs are experiencing degraded/slow performance. We are currently investigating the issue and apologize for any inconvenience.
Report: "Issues with loading the app"
Last updateWe're in the clear. The issue was caused by a bug in today's update. The issue wasn't showing on our preview environments, which is why we haven't caught it before the update. Fortunately, the alarms from our monitoring tools rang instantly and we could fix the problem. Thanks for your understanding and patience.
The issue should now be fixed. We're continuing to monitor the situation until we are confident that we are in the clear.
Some users might not be able to use their apps. We have identified the issue and are working on a fix. We will let you know more details as soon as we're in the clear. Thanks for your patience.
Report: "Emergency maintenance - Sync will not work for a small portion of customers"
Last updateMaintenance is done, and all is well. Thanks for your patience.
Hey folks, We are performing an emergency maintenance on one of our servers. As a part of that maintenance, we will need to disable syncing on that server. This will affect only a small portion of Stacker Workspaces, and we'll let you know as soon as the maintenance is done.
Report: "Google Sheets popup affecting some users"
Last updateThis incident has been resolved.
Fix has been made live and we're monitoring the situation.
The issue has been identified and a fix is being implemented.
We're investigating an issue that some users have reported where a Google Sheet popup is making some apps inaccessible on mobile.
Report: "Airtable Schema Sync Outage"
Last updateThank you for your patience, the issue is now resolved.
We have deployed a fix and are now monitoring this.
We have found the issue and we are actively working on a fix.
Airtable Schema Sync is currently not working with an 'IMPORT_EXCEPTION' error. We are currently investigating this issue
Report: "Login not working for some users"
Last updateWe are in the clear—thanks for your reports and patience.
Thanks for your patience—we've released a fix, and are now monitoring the situation.
We have identified the issue and are working on a fix. We'll let you know as soon as the fix is live.
We're aware of an issue that is affecting some users ability to login. The team is currently investigating it.
Report: "Filter not working for some users"
Last updateThis issue has been resolved.
We're currently investigating an issue that is affecting the "contains current user" filter for some users.
Report: "Workspaces not loading"
Last updateBetween 15:35 and 15:41 BST we had an issue with loading your workspaces. If you tried logging in during that time, you would have seen the "No healthy upstream" error. This was a mistake on our end. It happened because our new server scheduling rules briefly excluded all workers. This meant your apps couldn't find a server to run on. Thanks for your patience while we fixed this one—we will put process changes in place to make sure this doesn’t happen again.
Report: "Sync is not working"
Last updateWe've implemented a fix, and everything seems to be working now. We'll continue keeping an eye on the issue. Thanks for your patience. 🙇♂️
We're investigating reports where data and schema sync is not working. We have identified that there is indeed an issue, and will let you know as soon as we make progress.
Report: "Some visibility filters are not working as expected"
Last updateThis incident has been resolved and all visibility filters should be working as expected. Users will need to refresh their browser in order to reflect the latest update. Once again, we apologize for this issue and thank you so much for your patience.
The team has been alerted to an issue affecting the end user visibility filters in the UI for some apps. We've identified the issue and are currently deploying a fix. Our sincere apologies for this inconvenience.
Report: "Slow actions"
Last updateThanks for your patience—we've found the root cause and fixed it. It seems like one of the data source processing scripts running on our servers was unintentionally slowing down performance.
We've detected significant slowdown of certain actions from today around 12 BST until 14:20 BST - Manually initiated schema and data syncs - Connecting new data sources - Adding new users These actions worked, but were taking way too long to finish. We've cleared the symptom so the speed should back to what it was. We're now investigating the cause of the issue to make sure it doesn't happen again. We'll let you know as soon as we have more information.
Report: "Temporary sync issue for some customers"
Last updateAll good—thanks for your patience!
We have performed the maintenance, and we are now monitoring the situation. Automatic sync should work for everyone again.
We're still working on the maintenance. We'll let you know as soon as it's done.
Thanks to alerting we implemented after the last incident, we have noticed that one of our servers is starting to have trouble performing. In order to prevent an issue similar to the incident on April 12th, we will be performing some emergency maintenance on that server. We expect this to take around an hour. During this time, automatic data syncing will not work for some customers.
Report: "Slow app loading and schema syncing issues"
Last updateThe issue is now fully resolved. It seems like Airtable slightly changed the way their schema syncs works, so our schema syncs were failing. This caused a knock-on effect where our API calls got delayed too. We've adjusted to Airtable's change and everything should be back to normal. If you're still seeing any data loading or schema sync issues, please get in touch with support—they are most likely not caused by this issue.
The issue seems to be resolved, but we'll continue monitoring the situation. We will update the status page with the cause of the issue once we're confident the issue is resolved.
We believe we have identified the issue and have implemented a fix. We're now testing the fix to make sure it actually fixes the issue.
We're investigating reports of apps loading slowly, and Airtable schema updates working intermittently. Initial investigation suggests that the issue might have started on April 11th, around 20:45 UTC. We'll update as we know more.
Report: "Service Interruption"
Last updateWe are in the clear, the incident is resolved. Now that was intense—sorry for the hassle, and thanks for your incredible patience. The issue was caused by an overzealous security update that slowly took all of the server resources for itself. We will rest up, and then take steps to avoid this type of situation in the future.
The fix is now live, and data should now sync automatically. We’ll be monitoring the situation to make sure that we are in the clear.
Thanks for your ongoing patience. We have a fix ready, and have started rolling it out as an update. The update is expected to take around 4 hours. During this time you may see intermittent issues such as: - Slower loading of the app - Record loading request timing out (the turtle icon) We will update the status page if the timeline changes, or if anything new happens.
We have identified the root cause and are working on a fix. We will update as soon as the fix is live. In the meantime, please continue using manual data and schema syncs.
We are still looking into the issue with automatic data syncing. You can still use manual data and schema syncs, but automatic syncing will not work.
Our engineering team is still actively investigating this issue to find the root cause. At the moment, apps are expected to be able to do manual schema/data syncs (including new app onboardings). No apps will have automatic syncs at the moment. All other app functionalities are not affected.
We are currently experiencing a service degradation which is causing interruptions to onboarding, data syncs, and schema syncs for some apps. Our engineering team is actively investigating the issue. We will update this incident as we learn more and identify the cause and resolution.
Report: "Pages not loading"
Last updateThe issue was fixed at 8:51 UTC when we mentioned that it was resolved, but we haven't properly updated the status page—sorry about that. 😅
The issue should now be resolved—one of our app components was taking a bit more time updating during today's release. If you're still seeing a blank screen, please reload the page and you should be good to go.
We are investigating reports of pages not loading, or showing as blank. We'll update the status as soon as we know more.
Report: "Airtable schema sync failing"
Last updateWe've issued new certificates for our Airtable schema sync servers, and schema sync should now work. If you're still seeing any schema sync issues, please get in touch with our support at support@stackerhq.com.
The issue seems to be due to a certificate issue. We're working on a fix, and we will update when resolved.
It seems like Airtable schema sync is failing on some Workspaces. We're investigating, and will update this page as soon as we have more information.
Report: "SSL certificate errors"
Last updateThis incident has been resolved. If you continue to experience issues, please refresh your browser or use a different browser.
A fix has been implemented and we are monitoring the results.
We have confirmed the issue and started fixing the issue.
A small number of customers using custom domains are affected by this issue, users will not be able to access apps using custom domains. If you think you are affected, please contact customer support. Otherwise we will keep everyone updated here.
Report: "User lists unavailable for certain apps"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified an issue with some changes shipped earlier today which is causing a subset of apps not to be able to access their user lists. This prevents app admins from being able to list their users, or preview, stops @mentions from working in comments, and affects permissions (but does not allow any non-permitted users to see data they shouldn't). We are working on a fix which we hope to be deployed soon.
Report: "Oct 11"
Last updateEarlier today, we received reports that Stacker apps were not loading. A fix has been implemented, and this incident is now resolved. Thank you very much for your patience. If you have any questions, please get in touch with support@stackerhq.com.
We have identified the root cause, and we are working on a fix. As we resolve the incident, we will be providing updates for all of our customers here. If you have any further questions, feel free to contact support@stackerhq.com. Thank you for your patience.
We are experiencing downtime. We are investigating the issue and will update you as soon as possible. We apologize for the inconvenience.
Report: "Service interruption"
Last updateEarlier today, we received reports from customers who were experiencing slow performance in their Stacker apps. A fix has been implemented, and this incident is now resolved. Thank you very much for your patience.
We are currently experiencing a service interruption causing slow performance in most apps. Our engineering team is currently investigating the issue and working to restore full service as quickly as possible. We sincerely apologize for the disruption.
Report: "Performance Degradation"
Last updateEarlier today, we received reports from customers who were experiencing slow performance in their Stacker apps. A fix has been implemented, and this incident is now resolved. Thank you very much for your patience.
Our engineering team has identified the cause of the performance slowdown. We expect normal service to be restored to apps shortly. We sincerely apologize for the disruption.
We are experiencing an issue that is causing slow performance and service interruption to some apps. Our engineering team is currently investigating the issue and working to restore full service as quickly as possible. We sincerely apologize for the disruption.
Report: "Service Interruption"
Last updateService has been restored to all apps. We are continuing to monitor performance and apologize for any inconvenience this has caused.
We are currently experiencing a service degradation which is causing interruptions to onboarding, data syncs, and schema syncs in all apps. Our engineering team is actively investigating the issue. We will update this incident as we learn more and identify the cause and resolution.
Report: "Service interruption"
Last updateWe had an unexpected service interruption for around 20 minutes due to a release. Our engineering team noticed the issue as it happened and made sure that it was back up as quickly as possible. We apologize for this disruption.
Report: "Some Stacker Apps Not Loading Data"
Last updateAll customer apps should now be back working as normal. We are continuing to investigate the issue and upgrade our infrastructure to make sure we can tackle these issues before they affect customers in future. Thanks again for the issue reports and everyone's patience while we got this under control.
We have now got all Stacker apps to a state where they are loading data as normal. We are continuing to work on resolving the underlying issues whihc caused this outage. Thanks for your patience and understanding while we get everything straightened out. Please let us know if you are still experiencing problems loading your data.
We have identified the issue with the remaining apps which are still not functional. Our engineering team is fully engaged on restoring access to all customers. In terms of a timeline for resolution, we are now hoping that all apps will be restored within a couple of hours. We apologize sincerely for the inconvenience caused to our customers.
Access has been restored for the majority of affected apps. A small number of customers are still experiencing service disruptions. We are continuing to work on restoring full service to all apps. We apologize sincerely for the inconvenience caused to our customers.
We are continuing to investigate this issue. We don't have a fix available yet. We apologize sincerely for the inconvenience caused to our customers.
Some customers will be experiencing data not loading in their apps. We are urgently investigating this and will update this incident as we learn more and identify the cause and resolution. We apologize sincerely for the inconvenience caused to our customers.
We are currently experiencing delays with some processes in the Stacker platform which may be delaying data syncs into some apps. Our engineering team is actively engaged with the issue. We will update this incident as we learn more and identify the cause and resolution.
Report: "Airtable and Google Sheets schema sync failing intermittently"
Last updateThis incident has been resolved.
We are investigating reports of an issue causing our Airtable and Google Sheets schema updates to fail intermittently.
Report: "Stacker service degradation"
Last updateThis incident has been resolved.
The service performance has recovered, we are continuing to monitor
We currently are experiencing a service degradation on some features of Stacker and our Airtable Schema update service. We are urgently investigating and will update this page with more details as we have them.
Report: "Airtable schema import not completing"
Last updateWe have resolved the issues around the schema imports.
We are investigating an issue causing schema imports and updates from Airtable not to complete