Historical record of incidents for Sigma Computing
Report: "GCP Outage affecting Sigma organizations hosted on multiple cloud providers"
Last updateGCP Outage affecting Sigma organizations hosted on multiple cloud providers We are investigating an issue with Sigma organizations hosted on GCP experiencing errors owing to GCP outage (https://status.cloud.google.com/). A number of services and providers we use are also down so there may be some impact in other cloud providers as well. We are treating this as our highest priority. Our apologies for any inconvenience caused. User Experience: Increased error rates when performing regular operations. Incident Start Time: Approximately 17:55 UTC June 12, 2025
Report: "404 Not Found error trying to log-in to Sigma in Europe/UK"
Last updateOur upstream provider has confirmed the issue has resolved as of 11:06 AM UTC on January 29, 2025. Users may have experienced "404 NoSuchBucket" errors when accessing Sigma from EU/UK regions between the hours of 08:55 AM UTC and 10:01 AM UTC. This issue is now resolved and all services are operating normally.
Incident Start Time: 08:55 AM UTC January 29, 2025 Incident End Time: 10:01 AM UTC January 29, 2025 An upstream dependency issue has been mitigated, and we are actively monitoring for any further impact. Please let us know if you encounter any lingering issues.
Users in the UK and Europe are currently experiencing an issue accessing the Sigma application or embeds. Engineering is investigating the cause of this issue and we will have an update as soon as possible.
Report: "Errored Elements in Workbooks"
Last updateOn December 17, some Sigma users encountered column errors—such as "Reference to errored column" and "Expression contains errors"—when loading visualizations and tables. This issue surfaced as a result of introducing a workbook performance enhancement to the product. The issue has been resolved by reverting the recent change, and the Sigma application is now fully operational. We will conduct a thorough root cause analysis and share a detailed postmortem in the Sigma Community in the coming days. Incident Start Time: 15:49 UTC Incident End Time: 16:55 UTC
Report: "Unable to Publish Workbooks"
Last updateAt approximately 22:38 UTC on September 10, 2024, some Sigma users were unable to publish workbooks and upload CSVs due to a spike in error rates caused by a third-party service network rule. The incident has been resolved by disabling the rule that was triggering the issue and Sigma is actively working with the service to mitigate this in the future. Users should now be able to publish workbooks and the Sigma app is fully operational. We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. Incident Start Time: Approximately 22:38 UTC September 10, 2024 Incident End Time: Approximately 23:50 UTC September 10, 2024
Report: "Snowflake tables not loading on AWS US West 2"
Last updateThe mitigation measures have proven effective, and the issue is resolved. We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. Impacted Cloud Regions & Services: Sigma users hosted on AWS US West 2 User Experience: We expect the rate of errors to be negligible for all Sigma users hosted on AWS US West 2 Incident Start Time: Approximately 14:00 UTC September 4, 2024 Incident End Time: Approximately 14:30 UTC September 4, 2024
Our engineers have implemented temporary mitigation measures to address the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: Sigma users hosted on AWS US West 2 User Experience: We expect the rate of errors for long running queries on Snowflake tables to decrease while working in Sigma Workbooks. Incident Start Time: Approximately 14:00 UTC September 4, 2024 Incident End Time: Approximately 14:30 UTC September 4, 2024
We are continuing to investigate this issue.
At approximately 7 am PST, Sigma users on AWS US West experienced long running workbook queries on Snowflake tables that may result in a timeout. This incident is now actively under investigation. We are treating this as our highest priority. Our apologies for any inconvenience caused. Impacted Cloud Regions & Services: Sigma users hosted on AWS US West Incident Start Time: Approximately 14:00 UTC September 4, 2024
Report: "Users Unable To Contact Live Chat Support"
Last updateOur third party chat service has resolved the issue that was causing users to not be able to contact our live support.
Our live chat service is currently experiencing difficulties and Sigma users are unable to contact live support through the Sigma application. We are continuing to monitor the status of the issue. In the meantime please reach out to support@sigmacomputing.com for any questions or inquiries.
Our live chat service is currently experiencing difficulties and Sigma users are unable to contact live support through the Sigma application. We are monitoring the status of the issue. In the meantime please reach out to support@sigmacomputing.com for any questions or inquiries.
Report: "Issues with production services accessing databases on Sigma AWS"
Last updateAt approximately 17:22 UTC, we had applied an erroneous network security change which prevented incoming network connections to our services. This was detected by our internal monitors and we quickly reverted it at 17:26 UTC. Customers on AWS us-west-2 may have experienced intermittent slowness or erroring queries in loading their workbooks during that time. The mitigation measures have proven effective, and the issue is resolved. Incident Start Time: Approximately 17:22 UTC Incident End Time: Approximately 18:27 UTC We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "Outage affecting Sigma organizations hosted on GCP"
Last updateThe mitigation measures have proven effective, and the issue is resolved. Impacted Cloud Regions & Services: Organizations hosted on GCP User Experience: Unable to log in or use Sigma services Incident Start Time: Approximately 18:00 UTC Incident Resolution Time: 18:35 UTC
We have received reports of errors causing users to be unable to utilize all portions of the Sigma application. The issue has been mitigated and we are currently monitoring the issue. Our preliminary investigations indicate the following: Impacted Cloud Regions & Services: Organizations hosted on GCP User Experience: Unable to log in or use Sigma services Incident Start Time: Approximately 18:00 UTC Incident Mitigation Time: 18:35 UTC We will provide updates on this page when we have pertinent status updates
Report: "Search not returning results on AWS West"
Last updateWe identified an issue in our infrastructure impacting Sigma organizations hosted on AWS US-WEST-2 resulting in Search and API endpoint v2/workbooks returning no results. We implemented mitigation measures and the issue is now resolved. Incident Start Time: Approximately 16:40 UTC January 21, 2023 Incident End Time: Approximately 17:40 UTC January 21, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "Sigma organizations hosted on AWS US are not loading"
Last updateThe mitigation measures have proven effective, and the issue is resolved. We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. Impacted Cloud Regions & Services: Sigma users hosted on AWS US User Experience: Users will be able to load the Sigma app successfully Incident Start Time: Approximately 20:39 UTC November 08, 2023 Incident End Time: Approximately 20:57 UTC November 08, 2023
Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: Sigma users hosted on AWS US User Experience: Users will be able to load the Sigma app successfully Incident Start Time: Approximately 20:39 UTC November 08, 2023 Incident End Time: Approximately 20:57 UTC November 08, 2023
We are investigating an issue in our infrastructure causing Sigma organizations hosted on AWS US to be in a loading state. We are treating this as our highest priority. Our apologies for any inconvenience caused. Impacted Cloud Regions & Services: Sigma users hosted on AWS US User Experience: Attempting to load Sigma app shows a loading screen Incident Start Time: Approximately 20:39 UTC November 08, 2023
Report: "Missing search results from last 3 months affecting all Sigma deployments except AWS Canada"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for Sept 29, 2023 Incident: Missing search results from last 3 months affecting all Sigma deployments except AWS Canada](https://community.sigmacomputing.com/t/postmortem-for-sept-29-2023-incident-missing-search-results-from-last-3-months-affecting-all-sigma-deployments-except-aws-canada/2660)
At approximately 1:44PM PDT (20:44 UTC), for Sigma users on all clouds excepting AWS Canada, search functionality within Sigma (home page search, datasource search and API search) were missing results from the last 3 months. This meant that Sigma application users may have been unable to find their documents, or saw older documents which no longer existed. Users of embedded applications which use the Sigma search API may have experienced those applications loading incorrectly or failing to load. Search results were brought back up to date as follows: 2:38 PM (21:38 UTC) - Azure eastus2 resolved 3:50 PM (22:50 UTC) - GCP us-central1 resolved 4:07 PM (23:07 UTC) - AWS us-west-2 resolved 4:17 PM (23:17 UTC) - AWS eu-central-1 resolved Incident Start Time: 2023-09-29 20:44 UTC Incident End Time: 2023-09-29 23:17 UTC We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "Query failures for select organizations connected to Snowflake"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for Oct 03, 2023 Incident: Query failures for select organizations connected to Snowflake ](https://community.sigmacomputing.com/t/postmortem-for-oct-03-2023-incident-query-failures-for-select-organizations-connected-to-snowflake/2634)
We have received confirmation from Snowflake that they've rolled back globally for all impacted users. At this time we believe full functionality has been restored and expect all Snowflake users can now access their workbooks and data warehouse connections. Impacted Cloud Regions & Services: Snowflake connections for a subset of Sigma users hosted on GCP and AWS User Experience: Affected users can now access their workbooks and data warehouse connections. Incident Start Time: Approximately 13:00 UTC October 3, 2023 Incident End Time: Approximately 18:03 UTC October 3, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
We were able to to reproduce locally and issue is fixed in our environment after Snowflake rolled back a server change on their side. We're now working with them to apply the rollback to all accounts.
We are investigating an issue with queries resulting in the error message "Bad request; operation not supported." We have engaged with Snowflake support and are treating this as our highest priority. Our apologies for any inconvenience caused.
We are investigating an issue with queries resulting in the error message "Bad request; operation not supported." We are treating this as our highest priority. Our apologies for any inconvenience caused.
Report: "Users Unable To Contact Live Chat Support"
Last updateOur third party chat service provider has resolved the issue that was causing users to not be able to contact our live support.
Our third party chat service is currently experiencing an incident. We are monitoring the status of the issue. In the meantime please reach out to support@sigmacomputing.com for any questions or inquiries.
Report: "Exports with Color Conditional Formatting of "None" Failing"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for Aug 14th: Exports with Color Conditional Formatting of None](https://community.sigmacomputing.com/t/postmortem-for-aug-14th-exports-with-color-conditional-formatting-of-none/2619)
This incident regarding exports to Excel and CSV failing on some workbooks has been resolved. Incident End Time: Approximately 16:34 UTC August 14, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Sigma Computing is currently addressing a Severity 0 incident with some exports to Excel, CSV consistently failing.
Report: "Incorrect timestamps returned from warehouse for some customers"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for Aug 10th, 2023: Incorrect results for timezone-based calculations](https://community.sigmacomputing.com/t/postmortem-for-aug-10th-2023-incorrect-results-for-timezone-based-calculations/2618)
Backdated Incident: Some customers observed that timestamps in data returned from the warehouse were incorrect. Specifically the timestamp was off by the UTC offset (hours the customer timezone is ahead or behind UTC). Customers may have also observed that date_trunc() would return incorrect values. This was due to a change made during a code refactoring; no behavioral difference was anticipated, but certain customer configurations experienced the described discrepancy. Impacted Cloud Regions & Services: Snowflake and DataBricks customers with a particular combination of timezone configuration in Sigma and their warehouse were impacted by this issue. Note the refactored code path was enabled for some customers on August 7, and then progressively enabled for more customers and warehouses over several days; notably: - August 8, 17:30 UTC, code path enabled for all Databricks customers - August 10, 15:58 UTC, code path enabled for all Snowflake customers - August 10, 18:49 UTC, code path disabled for all Snowflake customers - August 14, 15:40 UTC, code path disabled for all customers Incident Start Time: 2023-08-07 18:10 UTC Incident End Time: 2023-08-14 15:40 UTC We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "Users unable to enter edits to Workbooks"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for Aug 9th, 2023: Users unable to enter edits to workbooks](https://community.sigmacomputing.com/t/postmortem-for-aug-9th-2023-users-unable-to-enter-edits-to-workbooks/2617)
Users who had recently made edits to a workbook received an error while attempting to make further edits to that Workbook.
Report: "Workbooks not loading for Sigma users on AWS Snowflake"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for September 14th, 2023; Workbooks not loading for Sigma users on AWS Snowflake](https://community.sigmacomputing.com/t/post-mortem-for-september-14th-2023-workbooks-not-loading-for-sigma-users-on-aws-snowflake/2604)
The mitigation measures have proven effective, and the issue is resolved. We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs. Impacted Cloud Regions & Services: Sigma users hosted on AWS US & AWS EU using Snowflake CDW Incident Start Time: Approximately 12:49 UTC September 14, 2023 Incident End Time: Approximately 14:39 UTC September 14, 2023
Our engineers have implemented mitigation measures and we are monitoring the effect closely. Impacted Cloud Regions & Services: Sigma users hosted on AWS US & AWS EU using Snowflake CDW User Experience: We expected workbooks to load normally starting 14:48 UTC for AWS US users and at 15:01 UTC for AWS EU users. Incident Start Time: Approximately 14:12 UTC Sep 14, 2023 Note: A correction was made for the incident start time to be in 24-Hour time
We are continuing to investigate this issue.
We are investigating an issue with workbooks not loading and queries hanging. We are treating this as our highest priority. Our apologies for any inconvenience caused. Impacted Cloud Regions & Services: Sigma users hosted on AWS US & AWS EU using Snowflake CDW Incident Start Time: Approximately 14:12 UTC September 14, 2023
Report: "Materialization failures impacting GCP cloud-hosted orgs"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Post Mortem for Sep 6, 2023: Materialization failures impacting GCP cloud-hosted orgs](https://community.sigmacomputing.com/t/post-mortem-for-sep-6-2023-materialization-failures-impacting-gcp-cloud-hosted-orgs/2600).
Current Status: The mitigation measures have proven effective, and the issue is resolved. Impacted Cloud Regions & Services: Sigma users hosted on GCP User Experience: We expect affected materializations to be resolved on the next scheduled run or when manually retriggered for all customers leveraging materialization and hosted on GCP. Incident Start Time: Approximately 16:30 UTC Sep 6, 2023 Incident End Time: Approximately 19:40 UTC Sep 6, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: Sigma users hosted on GCP User Experience: We expect the rate of errors for affected materializations to decrease on the next scheduled run or when manually retriggered for all customers leveraging materialization and hosted on GCP. Incident Start Time: Approximately 17:00 UTC Sep 6, 2023 Incident End Time: Approximately 19:40 UTC Sep 6, 2023
Some customers hosted on Google Cloud Platform (GCP) have observed Materialization and CSV upload failures beginning at 1:00pm EST / 10:00am PST. Our Engineering team is currently conducting an investigation to identify and mitigate the root cause, and Technical Support is actively monitoring the situation. We are currently investigating an issue with materialization and CSV upload failure for customers on Google Cloud Platform (GCP). This issue is being treated with the highest priority and we will update the status accordingly. Impacted Cloud Regions & Services: Sigma organizations hosted on GCP utilizing materialization User Experience: Errors related to materialization and CSV upload failure Incident Start Time: Approximately 17:00 UTC Sep 6, 2023
Report: "Workbooks cannot be opened for all Azure-hosted orgs"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Post Mortem for Sep 6, 2023: Workbooks cannot be opened for all Azure-hosted orgs](https://community.sigmacomputing.com/t/post-mortem-for-sep-6-2023-workbooks-cannot-be-opened-for-all-azure-hosted-orgs/2587).
Current Status: Production HMS Key Vault is back into healthy state as of 5:22PM EST and we expect all Azure customers can now access their workbooks and data warehouse connections. Impacted Cloud Regions & Services: Sigma users hosted on Azure cloud User Experience: All users on Azure hosted organizations can now access their workbooks and data warehouse connections. Incident Start Time: Approximately 19:50 UTC Sep 6, 2023 Incident End Time: Approximately 21:45 UTC Sep 7, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Engineering is actively debugging our internal Production organization, data warehouse connections setup, and new HSM Key Vault to verify recovery of our test workbooks.
Our Infrastructure team is verifying and testing the internal workaround solution on internal Production testing organizations. Further investigation is underway on the scope of other Sigma assets that may require system updates in the event that we proceed with the internal workaround solution.
We are actively continuing to engage with Azure Support and moving along the escalation paths. Internally, we are verifying and testing the internal workaround solution in Staging environments.
Our infrastructure team is continuing to engage with Azure’s Technical Support Engineering while continuously escalating to work directly with a dedicated Azure HSM Engineering team. Progress was made on internal workaround solutions as we work through verifying the impacted underlying credentials and metadata.
Our infrastructure team is actively working alongside Azure’s Technical Support Engineering but continuously escalating to work directly with the dedicated Azure HSM Engineering team. Additionally, we have begun testing and debugging internal workaround solutions. We apologize for the continued trouble and will provide more information as we have it.
Our Infrastructure team continues to work alongside Azure’s Support and Engineering team to have the Production Azure HSM Key Vault return to a healthy state. Additionally, we have spun off parallel internal efforts to provide workaround solutions that will allow customers to reconnect to their warehouses and have workbooks recovered with limited functionality.
Through continued engineering engagement with Azure Support, we have learned that the deployment of the Production Azure HSM Key Vault has entered an inconsistent state. We are working with Azure Support to explore avenues for recovery of the keys in this managed HSM Key Vault, and this will require engaging Azure Engineering. Your patience is appreciated as our Infrastructure team liaises with Azure’s APAC team and remain committed to identifying mitigation measures on a rotational basis until this incident is resolved. Please stay tuned for the next update in 3 hours.
We're currently collaborating with Azure Support to address an issue related to establishing a secure connection for our key vault in the production environment. Initial assessments suggest it might be related to DNS caching on Azure's side. Azure's technical team is exploring possible solutions, including adjusting DNS caching settings. We appreciate your patience and will provide further updates as we make progress.
Engineering has verified a process in Staging where deleting the entire HSM Key Vault and restoring it from backup restores access to workbooks. They are next working on reproducing in Staging the "Unknown blocking process" issue that we encountered in Production. Once the "Unknown blocking process" issue has been reproduced in Staging, they will be able to confirm that this delete/restore of the Key Vault should resolve the issue in Production. Please stay with us for updates to follow.
Our engineering team remains committed to addressing the current issue and is tirelessly working to resolve it. We have a fix applied in our Staging environment which has been verified and works. However, we are encountering an "Unknown blocking process" error that is preventing its application in the Production environment. This is being investigated with Azure Cloud Support and Engineering. We anticipate that once this blocking process is removed, we should be able to run our fix in Production.
Engineering continues to thoroughly investigate this issue around the clock. They attempted a solution to the issue in Staging and verified that this worked in the Staging environment. However, this attempted solution does not apply to the Production environment, due to a conflict with a Cloud Provider Background process. We have escalated this issue with Azure escalation management to secure assistance in resolving this conflict. We'll keep you informed about our progress.
Our engineers have identified the root cause for workbooks not loading and are developing mitigation measures to circumvent it. The Workbooks' loading issue was caused by Azure Key Vault becoming inaccessible, affecting our encryption/decryption service. Customer metadata remains secure, and workbook access will be restored when we regain access to Azure Key Vault. We're actively collaborating with Azure support to address the problem. We'll provide another update within 60 minutes.
Current Status: We are investigating an issue causing workbooks to not load for all customers on Azure-hosted orgs. We are treating this as our highest priority. Our apologies for any inconvenience caused. Impacted Cloud Regions & Services: Sigma users hosted on Azure User Experience: Workbooks not loading for all customers on Azure-hosted clouds Incident Start Time: Approximately 19:50 UTC Sep 6, 2023
Report: "Unable to login and load workbooks for Sigma AWS US"
Last updateThe mitigation measures have proven effective, and the issue is resolved. We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs. Impacted Cloud Regions & Services: Sigma users hosted on AWS US Incident Start Time: Approximately 17:45 UTC September 14, 2023 Incident End Time: Approximately 18:15 UTC September 14, 2023
Engineering has identified the root cause and implemented mitigation measures. We have confirmed that users are able to login to the app and load Workbooks again. We are continuing to closely monitor. Impacted Cloud Regions & Services: Sigma users hosted on AWS US Incident Start Time: Approximately 17:50 UTC September 14, 2023
We are investigating an issue with Sigma AWS US users unable to login to the app. We are treating this as our highest priority. Our apologies for any inconvenience caused. Impacted Cloud Regions & Services: Sigma users hosted on AWS US Incident Start Time: Approximately 17:50 UTC September 14, 2023
Report: "Snowflake Tables Showing No data within Sigma but have a non-zero Row Count in Summary and Sigma SQL returns rows in Snowflake."
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Post Mortem for Aug 29, 2023: Tables sourced from Snowflake appeared empty, displaying “No Data” message.](https://community.sigmacomputing.com/t/post-mortem-for-aug-29-2023-tables-sourced-from-snowflake-appeared-empty-displaying-no-data-message/2578)
The Snowflake rollback has proven effective and the issue is resolved. Impacted Customers are no longer seeing the issue. Incident Start Time: Approximately 3:40 UTC Aug 29, 2023 Incident End Time: Approximately 03:26 UTC Aug 31, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Snowflake has rolled back the changes for Impacted Customers and we have heard back that the issue is now resolved for our Impacted Customers. We are continuing to monitor the situation.
Snowflake has acknowledged the issue and has rolled back the server changes for one of the affected customers. We're awaiting the rollback for the other affected customers. We'll keep you updated on our progress.
We're in direct collaboration with Snowflake to address this critical issue. Additionally, we're conducting thorough investigations on our end as we haven't ruled out the possibility of this being an issue within our code. Rest assured, we're leaving no stone unturned as we explore both avenues for a swift resolution. Your patience is greatly appreciated.
Engineering is actively working on incorporating the most recent Snowflake driver code into Sigma, aiming for it to handle the variation in payload format we are observing for impacted customers.
Engineering has found what appear to be incomplete payloads being returned from the Snowflake data warehouse for the impacted customers. We believe this may be a result of a change that might have happened in the Snowflake environment and continue to work with Snowflake to root cause and identify a fix.
Our engineering team is actively investigating the Snowflake driver interface, focusing on analyzing Snowflake responses and checking driver version compatibility. Thank you for your patience and understanding as we work through this matter. We are committed to swiftly resolving this issue and will keep you informed about the progress. If you have any immediate concerns or questions, please feel free to reach out to our support team.
Engineering is continuing to investigate this issue, which we have isolated to the fetching of results from Snowflake. We are engaging with Snowflake Engineering to look into possible changes which might explain why results are not fetching as expected.
We are currently investigating an issue that’s occurring for a small number of Snowflake customers where Tables are showing No Data but have the non-zero row count in the Summary. This issue is being treated with the highest priority and we will update the status accordingly. User Experience: Workbook elements are showing No Data and their source connection table is showing No Data but has row count in the Summary. Sigma SQL however returns results when run directly within Snowflake. Impacted Cloud Regions & Services: Sigma organizations hosted across Clouds. Incident Start Time: Approximately 3:40 UTC Aug 29, 2023
Report: "Errors in exports and existing workbooks on Sigma Azure"
Last update[08/24/2023 Incident](https://community.sigmacomputing.com/t/postmortem-for-august-24-2023-errors-in-exports-and-existing-workbooks-on-sigma-azure/2564)
Current Status: The mitigation measures have proven effective and the issue is resolved. Impacted workbooks have been restored. Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: This errors for new workbooks has already been resolved and impacted workbooks are now restored. Incident Start Time: Approximately 12:53 UTC Aug 24, 2023 Incident End Time: Approximately 01:30 UTC Aug 25, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring closely. We are still actively working on fixing impacted workbooks. Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: We expect the rate of errors for new workbooks to decrease. Workbooks already impacted are still being recovered. Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. We are in the process of fixing impacted workbooks. Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: We expect the rate of errors for new workbooks to decrease. Workbooks already impacted are still being recovered. Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
Current Status: Our engineers have identified a problem with our workbook retention mechanism. This is being fixed now and we're looking at recovery of inaccessible workbooks Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: Errors when opening existing workbooks and on scheduled exports Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
We are currently investigating an issue with opening existing workbooks and scheduled exports for customers on Sigma Azure. Users will still be able to create new workbooks. This issue is being treated with the highest priority and we will update the status accordingly. Impacted Cloud Regions & Services: Sigma organizations hosted on Azure User Experience: Errors when opening existing workbooks and on scheduled exports Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
Report: "Infinite Loading Page on Sigma AWS US-West-2"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for June 7, 2023 Infinite Loading Page on Sigma AWS US-West-2](https://community.sigmacomputing.com/t/postmortem-for-june-7-2023-infinite-loading-page-on-sigma-aws-us-west-2/2506)
Backdated Incident: Sigma AWS users were experiencing an infinite loading page. This led to long wait times for logging into Sigma, and opening or editing workbooks. Our engineers rapidly identified and fixed the root cause of this. Impacted Cloud Regions & Services: Sigma users hosted on AWS US-West-2, all Connection types. Incident Start Time: Approximately 19:19 UTC Jun 7, 2023 Incident End Time: Approximately 19:23 UTC Jun 7, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "CloudFront outage US-EAST-1"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for July 18, 2023 incident: Sigma outage for users using AWS/CloudFront US-EAST-1](https://community.sigmacomputing.com/t/postmortem-for-july-18-2023-incident-sigma-outage-for-users-using-aws-cloudfront-us-east-1/2497)
Backdated incident: CloudFront outage US-EAST-1 Impacted Cloud Regions & Services: Sigma users accessing the application from the Eastern US were impacted, receiving 421 error pages when attempting to log in or access workbooks. Scheduled exports and API services were unaffected. Incident Start Time: July 18, 2023 16:37AM UTC Incident End Time: July 18, 2023 17:13AM UTC We have performed a root cause analysis (RCA) and have published the findings in a postmortem note on the Sigma Community site.
Report: "Databricks "execute statement request" errors"
Last updateBackdated Incident: Databricks users were seeing a high rate of warehouse query errors with the error message "execute statement request". Impacted Cloud Regions & Services: Only Sigma users using Databricks CDWs were impacted User Experience: We expect the rate of errors to decrease for all Sigma users using Databricks. Incident Start Time: Approximately 18:00 UTC July 27, 2023 Incident End Time: Approximately 16:25 UTC July 31, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "Scheduled exports using image (PNG) files erroring"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for July 27, 2023 - Image \(PNG\) Exports Erroring on Sigma AWS, GCP, and Azure](https://community.sigmacomputing.com/t/postmortem-for-july-27-2023-image-png-exports-erroring-on-sigma-aws-gcp-and-azure/2488/1)
The fix has resolved the root cause for the scheduled export error increase. You can resend any impacted exports. Our apologies for any inconvenience this caused. Incident End Time: Approximately 13:30 UTC July 27, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
A fix has been implemented and we have seen the rate of errors decrease substantially. We are continuing to closely monitor scheduled export activity.
We are continuing to diagnose this issue; currently scheduled export errors appear to be occurring intermittently.
Sigma Computing is currently addressing a Severity 0 incident with Scheduled Exports that are failing. This is currently affecting all scheduled export types.
Report: "Errors when drilling down on visualizations"
Last updateUsers with Sigma organizations hosted on all cloud providers (AWS and GCP) experienced errors when using drill downs on charts. The visualizations would return an error message: "We've encountered an error!". We identified a recent codebase change as the root cause and a fix for it was deployed.
Report: "Workbooks with Input Tables failing to load and new Input Tables unable to be created"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for June 16, 2023 Workbooks with Input Tables failing to load Incident - Sigma AWS & Sigma GCP](https://community.sigmacomputing.com/t/postmortem-for-june-16-2023-workbooks-with-input-tables-failing-to-load-incident-sigma-aws-sigma-gcp/2408)
Backdated Incident: Sigma users were unable to create new Input Tables and were unable to load existing workbooks if an Input Table was present in them. They encountered an error page stating: “cannot return null on non-nullable field ApiError.code”. Impacted Cloud Regions & Services: All Sigma workbook users on all cloud regions who were using the Input Tables feature. Our engineers rapidly identified and fixed the root cause of this issue. Incident Start Time: Jun 16, 2023 20:34 UTC Incident End Time: Jun 16, 2023 21:05: UTC We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in the coming days.
Report: "Workbook interactions yield error page with message: Unknown type "SymlinkTargetInfoInode""
Last updateBackdated Incident: All Sigma users encountered error pages when interacting with workbooks. Impacted Cloud Regions & Services: All Sigma workbook users encountered an error page on any interaction, with the error 'Unknown type "SymlinkTargetInfoInode". Did you mean "SymlinkTargetInfo"'. This issue did not impact API users. Our engineers rapidly identified and fixed the root cause of this issue. Incident Start Time: Jun 20, 2023 17:18:18 UTC Incident End Time: Jun 20, 2023 17:28:27 UTC We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "Degraded Performance on Sigma AWS - Connection Indexing"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [**Postmortem for May 30, 2023 Incident -Connection Indexing bug - Sigma AWS**](https://community.sigmacomputing.com/t/postmortem-for-may-30-2023-incident-connection-indexing-bug-sigma-aws/2375)
Current Status: The mitigation measures have proven effective, and the issue is resolved. Impacted Cloud Regions & Services: Sigma users hosted on AWS US, all connection types User Experience: As connection indexing has been reenabled, there should no longer be any issue with slow navigation or search of warehouse schema. Incident Start Time: June 1st 2023, Approximately 13:00 UTC Incident End Time: June 2nd, 2023, Approximately 06:40 UTC We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: Sigma users hosted on AWS US, all connection types User Experience: As connection indexing has been reenabled, there should no longer be any issue with slow navigation or search of warehouse schema. Incident Start Time: June 1st 2023, Approximately 13:00 UTC Incident End Time: June 2nd, 2023, Approximately 06:40 UTC
Current Status: In an effort to resolve the issue, we plan to restart connection indexing services tonight, June 1, between 11PM and midnight PDT. The restart event is expected to complete in under 2 minutes; during this time, workbook loads may fail. Impacted Cloud Regions & Services: Sigma users hosted on AWS US, all connection types User Experience: Up to 2 minute disruption in workbook loads during 11PM-midnight PDT maintenance window. Incident Start Time: June 1st 2023, Approximately 13:00 UTC
We are continuing to work on a fix for this issue.
To mitigate system slowness while we fix the root cause, we temporarily disabled automated connection indexing. Newly-created connections may be slow to navigate and search may not return them. Existing connections may need to have their databases/schemas/tables manually synced in Sigma if changes are made in the warehouse. Impacted Cloud Regions & Services: Sigma users hosted on AWS US, all connection types
Report: "Severity 0 Incident: Workbook Export Failures"
Last updateThe postmortem can be located here: [https://community.sigmacomputing.com/t/postmortem-for-may-16-2023-send-now-and-page-download-bug-sigma-aws-gcp/2356](https://community.sigmacomputing.com/t/postmortem-for-may-16-2023-send-now-and-page-download-bug-sigma-aws-gcp/2356)
Sigma Computing is currently addressing a Severity 0 incident: Users trying to export via Slack scheduled Exports, "Send Now", or download Workbook PDF will find a "Corrupt UUID" error and be unable to export their workbooks.
Report: "Timeouts across Sigma AWS"
Last updateBackdated Incident: Sigma AWS users were experiencing query timeouts and high latency. This caused additional errors for materializations and exports. Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: Sigma users hosted on AWS US, all Connection types User Experience: We expect the rate of errors to decrease for all Sigma users hosted on AWS. Incident Start Time: Approximately 13:00 UTC May 30, 2023 Incident End Time: Approximately 20:40 UTC May 31, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Report: "Errors when attempting operations such as creating, sharing and publishing Sigma documents"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for May 2nd, 2023 Creating, Sharing, Publishing Documents Incident - Sigma AWS & Sigma GCP ](https://community.sigmacomputing.com/t/postmortem-for-may-2nd-2023-creating-sharing-publishing-documents-incident-sigma-aws-sigma-gcp/2312)
Current Status: Our engineers have resolved the issue and we no longer see errors occurring for these operations Impacted Cloud Regions & Services: Sigma organizations hosted on AWS US and GCP, all Connection types User Experience: Users would have experienced errors when attempting operations such as creating, publishing and sharing Sigma documents like Workbooks and Datasets. These errors should no longer be occuring We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs Incident Start Time: Approximately 14:05 UTC May 2, 2023 Incident End Time: Approximately 14:55 UTC May 2, 2023
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. We expect the rate of errors to decline substantially Impacted Cloud Regions & Services: Sigma organizations hosted on AWS US and GCP, all Connection types User Experience: Users would have experienced errors when attempting operations such as creating, publishing and sharing Sigma documents like Workbooks and Datasets. These errors should decline substantially now Incident Start Time: Approximately 14:05 UTC May 2, 2023 Incident End Time: Approximately 14:55 UTC May 2, 2023
Current Status: We are investigating an issue causing errors when users attempt operations such as: creating, publishing and sharing Sigma documents like Workbooks and Datasets. We are treating this as our highest priority. Our apologies for any inconvenience caused. Impacted Cloud Regions & Services: Sigma organizations hosted on AWS US and GCP, all Connection types User Experience: Errors when attempting operations such as creating, publishing and sharing Sigma documents like Workbooks and Datasets Incident Start Time: Approximately 14:05 UTC May 2, 2023
Report: "Application Server Outage"
Last updateOn May 1, 2023, there was a brief period of time when Sigma instances were not loading any data (96% of requests to our application server failed). The root cause was quickly identified and mitigations were subsequently implemented to resolve the issue. A post-mortem with the root cause analysis will be published in the Community forum later. Approximate Start Time of Incident: 19:06 UTC on May 1, 2023 Approximate End Time of Incident: 19:19 UTC on May 1, 2023
Report: "Materialization & Scheduled Exports Degraded Performance"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem For April 20, 2023 Materialization and Scheduled Export Incident - Sigma AWS & Sigma GCP](https://community.sigmacomputing.com/t/postmortem-for-april-20-2023-materialization-and-scheduled-export-incident-sigma-aws-sigma-gcp/2297)
Current Status: We have implemented mitigation measures and are monitoring their effect closely. The occurrence of errors has declined substantially. Apologies for any inconvenience caused. Impacted Cloud Regions & Services: AWS & GCP User Experience: We expect that the rate of failures for materializations and scheduled exports to be substantially decreased. Users should not need to manually sync their databases/schemas/tables anymore at this time, changes done in the warehouse should automatically show in Sigma again. Navigating newly-created connections should also be functional. Incident Start Time: Approximately 11:50 UTC April 20, 2023 Incident End Time: Approximately 12:30am UTC April 21, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Current Status: We have implemented mitigation measures and are monitoring their effect closely. The occurrence of errors has declined substantially. As a consequence of this mitigation, users may need to manually sync their databases/schemas/tables if changes done in the warehouse are not showing in Sigma. Additionally, navigating newly-created connections may be slow and their search functionality will not work. Apologies for any inconvenience caused. Impacted Cloud Regions & Services: AWS & GCP User Experience: Users would have experienced sporadic failures for materializations and scheduled exports. Users may need to manually sync their databases/schemas/tables if changes done in the warehouse are not showing in Sigma. Additionally, navigating newly-created connections may be slow and their search functionality will not work. Incident Start Time: Approximately 11:50am UTC April 20, 2023
Current Status: We have implemented mitigation measures and are monitoring their effect closely. The occurrence of errors has declined substantially. Impacted Cloud Regions & Services: AWS & GCP User Experience: Users would have experienced sporadic failures for materializations and scheduled exports. Incident Start Time: Approximately 11:50am UTC April 20, 2023
Current Status: We are investigating technical issues leading to degraded performance for both materializations and scheduled exports. This is currently our highest priority. Apologies for any inconvenience caused. Impacted Cloud Regions & Services: AWS & GCP User Experience: Users may experience sporadic failures for materializations and scheduled exports now and in the past Incident Start Time: Approximately 11:50am UTC April 20, 2023
Report: "Materialization and CSV upload failures"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for April 17, 2023 Materialization and CSV Upload incident - Sigma AWS & Sigma GCP](https://community.sigmacomputing.com/t/postmortem-for-april-17-2023-materialization-and-csv-upload-incident-sigma-aws-sigma-gcp/2296)
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are seeing materializations and CSV uploads are operational now. Impacted Cloud Regions & Services: All Connection types, AWS and GCP deployments. User Experience: We expect users should no longer see materialization and CSV upload failures due to this incident. Incident Start Time: Approximately 13:00 UTC April 17, 2023 Incident End Time: Approximately 17:30 UTC April 19, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: All Connection types, AWS and GCP deployments. User Experience: We expect the rate of errors to decrease for all materializations and CSV uploads. Incident Start Time: Approximately 13:00 UTC April 17, 2023 Incident End Time: Approximately 17:30 UTC April 19, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: All Connection types, AWS and GCP deployments User Experience: We expect the rate of errors to decrease for all materializations and CSV uploads Incident Start Time: Approximately 13:00 UTC April 17, 2023 Incident End Time: Approximately 17:30 UTC April 19, 2023
Current Status: We are investigating an issue causing failures on write-back queries for materializations and CSV uploads. We are treating this as our highest priority and we expect this issue to be transitory. Our apologies for any inconvenience caused. Impacted Cloud Regions & Services: All Connection types, AWS and GCP deployments User Experience: Users are experiencing failures on materializations and CSV uploads Incident Start Time: Approximately 13:00 UTC April 17, 2023
Report: "Errors when duplicating or 'Save As' Workbooks that have Input Tables in them"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for March 21, 2023 Incident - Sigma AWS & Sigma GCP](https://community.sigmacomputing.com/t/postmortem-for-march-21-2023-incident-sigma-aws-sigma-gcp/2231?u=erin)
Users with Sigma organizations hosted on all cloud providers (AWS and GCP) experienced errors when duplicating a Workbook that had an Input Table in it, or clicking 'Save As' on an Exploration that had an Input Table in it. The Input Table would fail to create and rather render with the error message: "Request Timed Out: timed out waiting for datasheet to exist". We identified a recent codebase change as the root cause and a fix for it was deployed. Incident start time: 17:20 UTC March 20, 2023 Incident end time: 22:50 UTC March 21, 2023
Report: "GCP Queries Failing"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for February 17th Incident - Sigma GCP](https://community.sigmacomputing.com/t/postmortem-for-february-17th-incident-sigma-gcp/2200)
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Impacted Cloud Regions & Services: Sigma users hosted on GCP User Experience: Affected users describing performance as having returned to normal. Incident Start Time: 2023, February 16th ~21:48 PST Incident End Time: 2023, February 16th ~22:35 PST We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
We have applied a fix and are currently monitoring to ensure full functionality has been restored.
We're currently investigating reports of errors from some of our users on GCP. This is being handled as our highest priority issue and we expect to resolve it shortly. Impacted Cloud Regions & Services: Sigma users hosted on GCP. User Experience: Affected users are experiencing unusual volume of errors.
Report: "BigQuery performance degradation"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for January 27, 2023 Big Query incident](https://community.sigmacomputing.com/t/postmortem-for-january-27-2023-big-query-incident-sigma-aws-sigma-gcp/2129)
We have identified the root cause and resolved the issue causing performance degradation on BigQuery connections.
Monitoring: mitigations have been deployed and we are seeing improved performance in BigQuery connections.
We are continuing to investigate this issue.
Users with BigQuery Connections are experiencing long query times and timeouts. We are aware of the issue and actively working to mitigate it.
Report: "Snowflake incident impacting warehouses hosted in AWS - US West (Oregon) region also causing errors in Sigma Usage tables"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for January 26, 2023 incident - Sigma AWS & Sigma GCP](https://community.sigmacomputing.com/t/postmortem-for-january-26-2023-incident-sigma-aws-sigma-gcp/2128)
The direct impact and side-effects from the earlier Snowflake incident have now all been resolved.
The errors that were being encountered when trying to load Usage tables in the Administration section are now no longer occurring. Snowflake Incident: some Snowflake users are reporting long query times and timeouts as still occurring, but these may be isolated incidents. We are continuing to investigate these.
Snowflake Incident: Snowflake has declared the incident as being resolved on their end, however side-effects from it are still being experienced and we are actively investigating these: All Sigma users may still encounter errors when trying to load Usage tables in the Administration section. Some Snowflake users are reporting long query times and timeouts as still occurring.
Snowflake Incident: Users who have their Snowflake warehouses hosted in the AWS-West (Oregon) region may be unable to execute queries due to an issue in Snowflake. We are closely monitoring this. You can subscribe to Snowflake's status page for updates here: https://status.snowflake.com/incidents/xr2j1b6hf46c As a side-effect of this, all Sigma users may encounter errors when trying to load Usage tables in the Administration section.
Report: "Increased error rates on Sigma AWS"
Last updateSigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for January 17, 2023 incident - Sigma AWS](https://community.sigmacomputing.com/t/postmortem-for-january-17-2023-incident-sigma-aws/2106)
Current status: Our engineers have implemented mitigation measures to circumvent the issue. We have closely monitored their effect and confirmed with affected users that the issue has been resolved. Customer experience: We expect Sigma users hosted on AWS to no longer encounter a high rate of errors when performing regular operations. Incident start time: Approximately 19:45 UTC January 17, 2023 Incident end time: Approximately 21:25 UTC January 17, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive an email notification when that occurs.
Current status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. Customer experience: We expect the rate of errors to decrease for all Sigma users hosted on AWS. Incident start time: Approximately 19:45 UTC January 17, 2023 Incident end time: Approximately 21:25 UTC January 17, 2023
Current status: Our engineers have identified the root cause for the issue and are developing mitigation measures to circumvent it. Customer experience: Users in Sigma organizations hosted on AWS are experiencing increased error rates when performing regular operations. Incident start time: Approximately 19:45 UTC January 17, 2023
Users in Sigma organizations hosted on AWS are experiencing increased error rates when performing regular operations. We are treating this as our highest priority issue and our engineers are actively investigating. We expect this issue to be transitory. Our apologies for any inconvenience caused.
Report: "Organizations using OAuth authentication experiencing high rate of errors"
Last updateFor organizations that have OAuth authentication enabled, OAuth tokens need to be passed to the warehouse as part of their query authorization flow. A code change exposed a bug in our backend that handles OAuth token transit between our microservices. As a result, OAuth tokens were not being sent to the warehouse correctly. To mitigate this, we rolled back the offending code change and subsequently resolved the root cause of the bug itself. Incident start time: 16:44 UTC January 16, 2023 Incident end time: 20:19 UTC January 16, 2023
Report: "Increased error rates on Sigma AWS"
Last updateFrom approximately 18:20UTC to 18:40UTC users in Sigma organizations hosted on AWS would have experienced increased error rates when performing regular operations. This was caused by a change in our codebase that we were able to rapidly identify and roll back. Our apologies for any inconvenience this transitory issue caused.
Report: "Increased error rates on Sigma GCP"
Last updateSigma Engineering has completed the postmortem of this service incident. A Root Cause Analysis \(RCA\) is available on the Sigma Computing Community site: [Postmortem for December 2nd, 2022 incident - Sigma GCP](https://community.sigmacomputing.com/t/postmortem-for-december-2nd-2022-incident-sigma-gcp/2032)
Impact Statement: Between 09:30 UTC and 19:00 UTC, Sigma users hosted on GCP may have experienced sporadic errors for some of their queries. Preliminary Root Cause: Under investigation. Next Steps: We have seen the rate of these errors decline to zero as a result of our mitigation measures. We will continue to investigate to establish the full root cause and prevent future occurrences. A detailed postmortem will be published.
We have implemented mitigation measures and seen the rate of these errors decline to zero as a result. We will continue closely monitoring our system's health as our engineers root cause and implement a permanent fix. Scheduled materializations have now been re-enabled and should run as scheduled.
We have implemented mitigation measures and seen the rate of these errors decline to zero as a result. We will continue closely monitoring our system's health as our engineers root cause and implement a permanent fix. Note that all scheduled materialization runs are still temporarily disabled for the time being.
We are continuing to investigate the root cause for this issue. Users on Sigma GCP will still experience sporadic errors for some of their queries. As part of our mitigation effort, we have temporarily disabled all scheduled materialization runs.
Users on Sigma GCP may experience increased error rates affecting the ability to view data. We are currently investigating.
Report: "Live Support Chat Currently Inaccessible"
Last updateThis incident has been resolved.
We are experiencing issues with out 3rd party chat client. We have reached out to them to resolve the issue.
Report: "Partial Outage with Snowflake affecting some Sigma organizations"
Last updateAccording to the Snowflake status page (https://status.snowflake.com/incidents/yh1n0ly69chm) a fix has been implemented.
Based on the information at https://status.snowflake.com/incidents/yh1n0ly69chm, Snowflake is aware of this issue. This will affect users attempting to access Snowflake data in the AWS - US East (N. Virginia) region. Sigma is currently monitoring the situation for any changes.
Report: "Scheduled exports delayed"
Last updateImpact Statement: Between 21:15 UTC and 22:25 UTC, Sigma users may have experienced delays to their scheduled exports being sent. Preliminary Root Cause: We identified a compatibility issue between our services which resulted in the delayed exports. Next Steps: All delayed exports have been sent. We will continue to investigate to establish the full root cause and prevent future occurrences. A detailed postmortem will be published.
Impact Statement: Between Between 21:15 UTC and 22:25 UTC, all Sigma users may have experienced delays to their scheduled exports being sent. Mitigation: The fix for this has been deployed and delayed exports have been sent. Users may still experience some delay in receiving scheduled exports as our system processes the queue. Our apologies again for any inconvenience this may have caused.
Incident Start Time: November 16, 2022 21:15 UTC User Experience: Presently all types of scheduled exports (email, Slack, Google Sheets etc.) are delayed due to a technical issue. Sigma Engineering has identified the root cause and a fix is being deployed. Once the fix is live, we expect that all the delayed exports will be sent. Our apologies for any inconvenience this may cause. Impacted Clouds: AWS and GCP Current Status: Investigating
Report: "Scheduled exports delay"
Last updateThis incident has been resolved. All schedule exports are being sent successfully with no delay and the backlog has been cleared.
The issue has been identified and mitigated. Users may experience some delay in receiving scheduled exports as we process the queue of scheduled items.
At the moment all scheduled exports (email, slack, etc) are delayed. Sigma Engineering is investigating.
Report: "Errors opening workbooks, worksheets and dashboards for some Sigma GCP customers"
Last updateSigma Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Sigma Computing Community site: [https://community.sigmacomputing.com/t/post-mortem-on-september-20th-2022-incident-sigma-gcp/1877](https://community.sigmacomputing.com/t/post-mortem-on-september-20th-2022-incident-sigma-gcp/1877)
This incident has been resolved. A full post-mortem will be published on our Status Page in the next 48 hours.
The service in GCP has been restored, with the exception of the document search (which our team is now focusing on).
The outage is partially resolved, and our engineering team is working to fully restore the service.
No additional news to share at this time, but we’re focused on getting things back to normal as quickly as we can.
We are continuing to work on a fix for this issue.
No additional news to share at this time, but we’re focused on getting things back to normal as quickly as we can.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate the issue. The area of impact has now been limited only to legacy Sigma Worksheets and Dashboards.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Workbook publish and share action failures"
Last updateThis incident has been resolved.
We have identified the issue and a fix is being implemented.
We are continuing to investigate this issue and can confirm that the same issue is also impacting Datasets.
Users hosted on Sigma GCP are at the moment experiencing failures to share and publish Workbook changes. Our team is actively looking into the issue.
Report: "Degraded performance on Sigma AWS"
Last updateSigma Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis \(RCA\) is available on the Sigma Computing Community site: [https://community.sigmacomputing.com/t/post-mortem-on-service-degradation-for-sigma-aws/1779](https://community.sigmacomputing.com/t/post-mortem-on-service-degradation-for-sigma-aws/1779)
This incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are now implementing a fix.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
Users hosted on Sigma AWS are reporting again today longer than usual load times. We are currently investigating.