Historical record of incidents for Stitch
Report: "Extraction issues"
Last updateThis issue has been resolved. We apologize for the inconvenience and appreciate your patience.
The issue has been fixed and the team is monitoring the results.
The issue has been identified, and our team is actively working on mitigation and implementing a fix.
Our team is continuing to diligently work on resolving the issue. We will keep you posted on any further updates.
We're investigating an issue affecting Extractions in Stitch Cloud and are diligently working to resolve the issue as soon as possible. We'll provide new information as it becomes available while the issue is being addressed. We apologize for any inconvenience and appreciate your patience.
Report: "Extraction issues"
Last updateThis issue has been resolved. We apologize for the inconvenience and appreciate your patience.
The issue has been fixed and the team is monitoring the results.
The issue has been identified, and our team is actively working on mitigation and implementing a fix.
Our team is continuing to diligently work on resolving the issue. We will keep you posted on any further updates.
We're investigating an issue affecting Extractions in Stitch Cloud and are diligently working to resolve the issue as soon as possible. We'll provide new information as it becomes available while the issue is being addressed. We apologize for any inconvenience and appreciate your patience.
Report: "Linkedin Ads Integration - Extraction Job Failure"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring.
We are aware of an issue affecting our Linkedin Ads integration and are investigating the issue.
Report: "Facebook Ads Integration - Extraction Job Failures"
Last updateOur Facebook Ads integration was experiencing extraction job failures due to an issue with our Facebook app. The issue has been resolved and we have returned to normal operations.
Report: "Snowflake Loading Issues"
Last updateThis incident has been resolved.
A fix has been implemented to resolve the data duplication issue on some Snowflake customer environments. Impacted customers may experience load performance lag temporarily during this time. We are continuing to monitor the situation closely.
We have identified an issue where some Stitch Snowflake loading processors are working on the same sets of data at the same time, which is causing duplicate data to be loaded on some customer Snowflake environments. We are working to address the issue to stop the data duplication as soon possible.
We are currently investigating an issue causing duplicate records on some Snowflake destinations.
Report: "Web App Outage"
Last updateThe issue has been resolved and the Web App should now be responsive again. Data pipelines have not been impacted.
We have identified the issue and are actively working on deploying the solution
Report: "Redshift loader performance issues"
Last updateThe issue has been resolved.
A fix has been implemented and we are monitoring internal statistics to ensure proper performance of Redshift loaders.
The issue has been identified and we are in the process of deploying a fix.
We are currently investigating a degradation of performance in the Redshift loader.
Report: "Latency issue with Stitch App"
Last updateThe issue seems to have resolved
We are currently facing latency issue with our app. In the process of investigating the issue
Report: "Shopify API Errors"
Last updateThis incident has been resolved
A work around has been provided for customers
We have identified the issue and waiting for partner to rectify/resolve the issue
Our Shopify integration is experiencing elevated error rates reaching the Shopify API. Our team is troubleshooting the issue and reaching out to Shopify for more information and to resolve the error as soon as possible
Report: "Web application instability"
Last updateThe root cause has been resolved and web application availability has remained stable during monitoring. Integrations encountered no disruption to extraction or loading processes, and no data has been lost as a result of this incident.
Our team has identified the cause of the degraded availability and have implemented a workaround while we address the root cause.
Our team is investigating an issue affecting availability of the web application. Note that this does not affect any extraction or loading processes.
Report: "Shopify Integration"
Last updateThe issue is resolved with Shopify API connectivity
Our Shopify integration is experiencing elevated error rates reaching the Shopify API. Our team is troubleshooting the issue and reaching out to Shopify for more information and to resolve the error as soon as possible.
Report: "Single-Sign-On Outage"
Last updateThe incident has been resolved
We're currently investigating an issue affecting our Single Sign On login flow. Users may be unable to log into their Stitch accounts if Single Sign On has been enabled for their workspace as we continue to investigate. If account access is needed during this time period, Stitch Admin users can continue to log in with basic authentication.
Report: "User Interface Outage"
Last updateThe user-interface has remained available and stable throughout our monitoring period.
The user interface availability has been restored and we are continuing to monitor for stability.
We've identified and are working to resolve a networking change that has affected the front-end availability.
Our team is currently investigating an issue that is preventing users from logging into and / or using the user-interface. We will provide updates as they become available.
Report: "Loading delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Some load jobs are experiencing abnormal latency and taking longer than expected to complete. We are actively investigating this issue for a prompt resolution
Report: "Shopify integration setup error"
Last updateCustomers are now able to create new Shopify integrations successfully with our updated setup process. All existing Shopify connections remain unaffected and will continue to replicate data without issue.
Our team is working on an updated setup process for the Shopify integration. New Shopify connections are not available at this time and all existing Shopify connections will continuing to replicate data without issue.
Report: "User login issue"
Last updateClients should be able to log in at this time.
Some users may face extended browser load times on various pages of the application as service levels return to normal. We are continuing to monitor impact as mitigating solutions are propagated.
Some users may face issues when logging in. The issue has been identified and solutions are being worked on.
Report: "User login issues"
Last updateThere should no longer be issues with logging into the app.
The issue has been identified and fixes have been implemented. There may still be lingering issues but we are actively monitoring.
We have identified an issue where users are unable to log into their accounts. We are currently investigating.
Report: "User login issue"
Last updateThe issue with logging into app.stitchdata.com has been resolved. Users should now be able to log into their accounts. This incident solely affected logins to the Stitch web application; all other functions remained operational including extractions and loads.
Access has been restored to users who were experiencing difficulty logging in. Root cause has been identified and will be resolved shortly.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We have identified an issue where users are unable to log into their accounts. We are currently investigating.
Report: "Extraction Delays"
Last updateExtraction jobs have consistently begun at their normal schedules during our monitoring window. No data has been lost as a result of this incident.
A fix has been applied and we are currently monitoring extraction jobs as they return to their normal schedules.
Some extraction jobs are failing to start at the scheduled time. We have identified the issue and are working to implement a fix.
Report: "Degraded service in AWS us-east-1 affecting both extractions and loads"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Some extraction jobs not starting"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Some extraction jobs are failing to start at the scheduled time. We are investigating the issue.
Report: "500 Status Errors"
Last updateThis issue has been resolved. No data has been lost as a result of this issue and extractions that encountered errors can now be retried if they have not already.
An issue affecting a subset of integrations has been identified and a fix has been released. Subsequent extraction jobs should result in successful posts to Stitch and our R&D team is continuing to monitor current behavior.
We are currently investigating an issue affecting extractions and our import API while persisting data to Stitch which results in status 500 errors.
Report: "Degraded Performance In App"
Last updateThis incident has been resolved. App loading times are back to normal and our third-party service provider has resolved the issue.
App loading times have returned to normal and we are monitoring requests sent to our third-party service provider.
App loading times are currently being affected by an issue impacting a third-party service provider.
We are currently investigating increased latency in app loads. Extractions and load jobs are running as expected.
Report: "Extraction and Loading Delays"
Last updateThis incident has been resolved.
Extraction and loading systems are both exhibiting normal performance and availability. We are continuing to monitor while our cloud provider works toward full recovery.
We are seeing improved availability in our extraction job systems, however loading systems are still degraded. We continue to work with our cloud service provider to resolve the issue.
Our cloud provider has identified a root cause for this issue and is working toward resolution.
We have identified service outages with our cloud service provider as the root cause of our degraded availability. We are working with them to resolve the issue.
We are investigating elevated error rates with our extraction and loading processes.
Report: "Missing Data in BigQuery Destinations"
Last updateThis incident has been resolved.
We have fixed the issue and are monitoring the rescheduled loads.
We have identified an issue affecting data loading into BigQuery destinations between 17:38:41 and 21:37:45 UTC, customers will notice missing data during that time. Our team is working to reschedule missing loads so no data is missed.
Report: "BigQuery loading delays"
Last updateThis incident has been resolved.
We are experiencing delays of up to 2 hours loading data into BigQuery destinations.
Report: "Redshift destination loading errors"
Last updateThis incident has been resolved.
Load times and errors have returned to normal for most customers. AWS is reporting that S3 and Redshift error rates are improving. We are continuing to monitor.
We are monitoring the issue.
AWS is reporting degraded performance for AWS S3 and AWS Redshift in the us-east-1 region. We are experiencing elevated error rates while loading data into Redshift warehouses for some customers.
Report: "Webhooks errors"
Last updateThe issue has been resolved.
We are experiencing elevated error rates in our webhooks receiver API. We have identified the root cause and are releasing a fix.
Report: "App Page Loading and Job Scheduling"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
Our team has confirmed this issue is related to a cache repopulation process imposing heavy load on the system. We are working to implement a mitigation.
The issue has been identified and a fix is being implemented.
We are investigating an issue that is causing page load failures and job scheduling delays.
Report: "Service Degradation"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Users are experiencing slow page load times due to a software release that increased the load on an internal database. Our team is working to resolve the issue.
Report: "Data Extraction Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are experiencing an outage with Stitch data extraction processing due to an outage with our underlying infrastructure provider. We are working with the provider to identify and resolve the issue.
Report: "Google Analytics Extraction Errors"
Last updateSuccess rates for Google Analytics extractions and configurations have returned to normal and remained stable.
We've released an update to our Google Analytics integration to mitigate failure scenarios when retrieving metric and dimension compatibility information from Google. Attempts to set up new Google Analytics integrations and extractions from existing Google Analytics integrations are no longer impacted by this issue, and we're continuing to monitor success rates.
Our R&D team has identified that an asset retrieved from Google's API that is used for metric and dimension identification and compatibility has been resulting in a 404 response page. They're currently evaluating potential local resolutions until a time at which API calls to Google return the asset successfully. Until resolved, this error will prevent the creation of new Google Analytics integrations and extraction for existing Google Analytics integrations. Configurations and stored bookmarks for existing Google Analytics integrations are unaffected.
Report: "Issue Navigating to Add Integration and Add Destination Pages"
Last updateThe Add Integration page and Add Destination page were affected by an issue impacting a third-party service provider that resulted in an error indicating, "Something Went Wrong" and prevented the creation of new integrations or an account's destination. This issue has been resolved by the service provider and ability to access these pages has been restored.
Report: "Web Application Degradation"
Last updateAll application and API functionality have remained stable during the monitoring window.
Our R&D team has deployed a fix and is monitoring services to verify that error rates remain at normal levels.
Our R&D team has identified the root cause and are working to implement a fix.
Our R&D team is investigating reports of errors displaying connection information within the Stitch user interface and via API. They are working to resolve the occurrences to restore normal functionality.
Report: "Web Application Degradation"
Last updateApplication functionality including user interface access, extraction schedules and performance, loading performance, and API availability have remained stable during the monitoring window.
Our R&D team has deployed a fix and is monitoring services to verify that error rates remain at normal levels.
Our R&D team has identified a potential root cause and are working to implement a fix.
Our R&D team is continuing to investigate the cause and scope of this issue. Thus far we have identified it as impacting the user interface, the scheduling and performance of extraction jobs, the performance of loading jobs, and API availability.
Our R&D team is investigating reports of delays while loading the Stitch application and errors while the application fetches account information, which appear to stem from a spike in internal database errors. They are working to resolve the occurrences to restore normal functionality.
Report: "Issue displaying integrations' Settings pages"
Last updateApplication functionality has remained stable since the fix was deployed earlier today.
A fix has been deployed, and the ability to access the Settings page for affected integrations has been restored.
Our team had isolated the cause of this UI issue and is currently testing a fix.
Our team is investigating an issue affecting the display of the Settings page for various integration in our web user interface. This does not affect existing configurations for integrations or replication.
Report: "Internal Errors Affecting Extraction Jobs and Authentication"
Last updateThe rate of 401 errors has returned to and remained at normal levels, and subsequent extraction jobs and authentication attempts have completed successfully as expected.
We've corrected an issue stemming from a recent token change that had disrupted communication with some aspects of our infrastructure. Subsequent extraction jobs and authentication attempts should complete successfully.
We're currently investigating an issue causing 401 errors to be returned by Stitch during processes like extraction jobs and the re-authentication of new or existing integrations.
Report: "Xero Integration Extraction Schedules Lagging"
Last updateExtractions have resumed their normal schedules. No data was lost, and subsequent extractions will resume from their bookmarks.
The root cause has been identified as a side effect of the Xero authentication upgrade and extractions are resuming their normal schedules.
We're currently investigating an issue affecting extraction jobs for Xero integrations occurring according to their specified replication frequencies and anchor times. Extraction jobs will be delayed as a result.
Report: "Degraded Availability & Service"
Last updateFunctionality has remained consistent and without issue since our billing service's performance was restored.
Our billing service has released a fix for this issue and we're continuing to monitor availability to take mitigating steps as necessary.
Our billing service is encountering a partial outage, affecting Stitch account access and functionality. We're monitoring the situation closely to evaluate any necessary action to expedite availability.
Report: "Chat / Email Support Temporarily Unavailable"
Last updateOur support team is now available as our chat and email service provider has completed their maintenance.
Our support team is temporarily unavailable as our chat and email service provider performs emergency maintenance.
Report: "Transient Redshift Loading Errors"
Last updateRedshift loading functionality has remained at normal levels, and there have been no recurrences of this error since the fix was deployed.
Our team deployed a fix for this issue circa 19:50 UTC, and Redshift loading functionality has remained at normal levels since then. We're continuing to monitor loading behaviors at this time.
Our team has identified an issue stemming from logic used to handle Primary Key fields that are nested maps, which increased the workload on loading services. We're currently working to implement a fix for this behavior.
Our engineering team is currently investigating transient loading errors affecting a subset of accounts working with Redshift destinations. A small percentage of loading processes for this destination may experience intermittent loading errors.
Report: "Facebook Ads Authentication Issue"
Last updateFacebook Ads integrations have returned to normal functionality.
Facebook's team is reporting a partial outage that results in the behavior exhibited by our integrations that can be viewed here: https://developers.facebook.com/status/issues/470651240279373/ They've deployed a fix and our team is continuing to monitor the situation as extractions resume normal functionality.
Users are reporting an error from Facebook Ads during authentication processes that indicate the source ad account cannot be located, which is preventing extraction from occurring. Our engineering team is investigating this behavior.
Report: "Legacy Database Integration Extraction Issues"
Last updateThis issue was resolved circa 06:42 UTC, and since then, extraction jobs for legacy database integrations have been scheduled and are running successfully.
Our engineering team is working to resolve an outage affecting a subset of legacy database integrations that is preventing their extraction jobs from beginning. In the event that this data is critical for your team, consider using the current version of the integration as additional segments of our infrastructure are unaffected.
Report: "Abnormal Extraction Behavior"
Last updateIntegrations that were previously affected have continued to run and complete successfully. No data was lost as an effect of this incident.
The fix deployed by our team appears to have corrected internal server errors that were occurring, where subsequent extraction jobs have continued to begin and complete successfully. We're continuing to monitor affected source integrations.
A subset of source integrations impacted by the previous connection issue are experiencing long running extractions or are not showing progress within extraction jobs. Our team is deploying an update to correct this behavior.
Report: "Connection Information Errors"
Last updateConnections have remained accessible and stable since this fix was deployed.
A fix has been deployed and application access is returning to normal. We're continuing to monitor behavior.
An update to an internal database has caused server errors while accessing information about connections in Stitch accounts. Our engineering team has identified the issue and is working to address it now.
Report: "Integration Scheduling Issues"
Last updateExtraction jobs for previously affected integrations have returned to, and are maintaining, normal execution patterns according to their schedules.
A fix has been implemented and we're continuing to monitor the recovery of extraction jobs for previously affected integrations.
Our engineering team has identified an issue in an internal database that they're working to resolve currently, and the number of affected integrations is beginning to decrease. No data will be lost as a result of this behavior, and extraction jobs will begin to resume their regular schedules.
We're investigating an issue that's preventing extraction jobs from beginning for legacy SaaS integrations (excluding any database integrations, Singer-based integrations, or webhook integrations).
Report: "BigQuery Service Unavailable Errors"
Last updateAll recent loading attempts for BigQuery destinations that were previously impacted by these status 503 errors have shown to complete successfully across our infrastructure.
The number of loading processes affected by 503 errors from Google has greatly subsided, and we are continuing to monitor the frequency of their occurrence.
Google's API is currently returning 503 errors indicating that their service is unavailable during attempts to load data to numerous BigQuery destinations and affecting loading processes for these accounts. We're monitoring Google's status page for updates on this issue. Note that no data will be lost as a result of this behavior.
Report: "UI Redirection Issue"
Last updateEarlier today our team released change that included a bug that would incorrectly considered some requests unauthenticated that should have been authenticated successfully. We've deployed a fix for this behavior, and our team is following up to analyze the impact of the bug and take steps to ensure it does not occur again.
Our team has identified an issue causing users to be redirected while working with Stitch through our user-interface, and we're working to deploy an update to correct this behavior. This issue does not affect any replication processes for existing integrations.
Report: "Web Application Delays"
Last updateApplication functionality including web access, extraction schedules and log access, and API functionality have remained stable during the monitoring window.
Our engineering team is currently monitoring the situation as error rates have returned to normal.
Our engineering team has identified a spike in internal errors and are working to resolve the occurrences to restore normal functionality.
There have been reports of delays while loading the Stitch application. Our team is actively investigating this issue.
Report: "Shopify Customers Error"
Last updateThis incident has been resolved.
A fix has been implemented by Shopify and we are monitoring the results.
We are continuing to work on a fix for this issue.
Update from Shopify: We’ve identified the source of the problem, and there will be a fix pushed out for it shortly.
We are experiencing elevated failure rates replicating data from the Shopify Customers API endpoint. We are working with the Shopify team to resolve the issue.
Report: "Application Latency"
Last updateApplication functionality, extraction schedules, and API functionality have remained stable at normal levels during monitoring.
Systems are returning to normal levels and our team is continuing to monitor the recovery. Application functionality has been restored, and subsequent extraction schedules will be adhered to. Extraction jobs scheduled during the latency period will be replayed where possible, though this may delay replication for integrations with low replication frequencies. For integrations with low replication frequencies that have not occurred at their typical interval, users can Run Extraction Now to manually trigger an extraction job.
Our team has identified an issue that is causing application latency and may cause some integrations to deviate from their typical extraction schedules. Calls against the Connect API may see increased timeouts. We're actively working to resolve this issue and will provide additional updates when available.