Historical record of incidents for Builder.io
Report: "Upstream issue with our Cloud provider"
Last updateWe are currently monitoring an upstream issue with Google Cloud Platform which is impacting broad availability of Builder services
Report: "Projects: Service disruption"
Last updateSome users are facing issues launching and editing Projects. We are actively investigating a fix.
Report: "Projects is experiencing sporadic failures for starting servers"
Last updateResolved - This issue has been resolved, and all Projects functionality has been restored.
Builder has identified an issue causing sporadic failures in starting servers for Projects. Our team is currently working on rolling out a fix for this issue.
Report: "Projects is experiencing slowness"
Last updateWe have identified that the earlier slowness in the Projects feature was due to general internet network conditions outside our control. While the issue may still impact some users intermittently, there is no action we can take at this time. We have added a notice in the app to inform users if delays are detected. We will continue to monitor the situation.
The Projects feature is currently experiencing slower speeds due to an issue with an upstream provider. We are currently working with our provider to resolve the issue.
Report: "Projects is experiencing slowness"
Last updateThe Projects feature is currently experiencing slower speeds due to an issue with an upstream provider. We are currently working with our provider to resolve the issue.
Report: "Partial Outage due to dependant service"
Last updateThis incident has been resolved.
We are experiencing partial outage on the webapp side due to our feature flag provider being down
Report: "Partial Outage due to dependant service"
Last updateWe are experiencing partial outage on the webapp side due to our feature flag provider being down
Report: "Projects is experiencing sporadic failures for starting servers"
Last updateThis issue has been resolved, and all Projects functionality has been restored.
A fix has been implemented for the issue causing sporadic failures in starting servers for Projects, and we are currently monitoring it. Machines created or used in the last hour will likely still experience problems, but all machines newer than 1 hour old should function as intended.
Builder has identified an issue causing sporadic failures in starting servers for Projects. Our team is currently working on rolling out a fix for this issue.
Report: "Projects is experiencing sporadic failures for starting servers"
Last updateBuilder has identified an issue causing sporadic failures in starting servers for Projects. Our team is currently working on rolling out a fix for this issue.
Report: "Projects branch creation down"
Last updateThe underlying cause of the issue affecting Projects branch creation has been successfully addressed. All functionality has been restored and is operational at this time.
A fix has been implemented and we are currently monitoring. Branch creation should function as intended.
There is currently an issue with an external provider that prevents users from creating any new Projects branches. We are working with our provider to bring this functionality back online.
Report: "Projects branch creation down"
Last updateThere is currently an issue with an external provider that prevents users from creating any new Projects branches. We are working with our provider to bring this functionality back online.
Report: "Outbound emails not being sent"
Last updateThis incident has been resolved.
Due to an integration issue with our email provider, outbound emails from Builder.io are not being sent currently. That includes user invite emails as well as form submission emails. We are working with our provider to bring this functionality back online.
Report: "Outbound emails not being sent"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
Due to an integration issue with our email provider, outbound emails from Builder.io are not being sent currently. That includes user invite emails as well as form submission emails. We are working with our provider to bring this functionality back online.
Report: "Outbound emails not being sent"
Last updateWe are continuing to work on a fix for this issue.
Due to an integration issue with our email provider, outbound emails from Builder.io are not being sent currently. That includes user invite emails as well as form submission emails. We are working with our provider to bring this functionality back online.
Report: "WebApp Outage"
Last updateThis incident has been resolved.
We are experiencing an issue in the webapp. Details to be followed soon
Report: "WebApp Outage"
Last updateWe are experiencing an issue in the webapp. Details to be followed soon
Report: "Content API Partial outage"
Last updateThis incident has been resolved.
We are noticing high latency and higher than normal error rates with our underlying caching infrastructure. We are currently investigating the root cause
Report: "Content API partial outage"
Last updateThe incident is resolved and services should be to normal.
All Content API services should be recovered. We are still actively monitoring the situation and will update once we are satisfied that services are fully up and running.
Following some corrective actions, our instances should start to recover.
We are continuing to investigate this issue.
We are noticing high latency and higher than normal error rates with our underlying caching infrastructure
Report: "High error rate on content API"
Last updateThe underlying infrastructure issue is now resolved; all service availability should be resumed. Further details will be provided at a later stage.
An underlying infrastructure was having issues. We have rolled out a fix to change it and are monitoring the status. Content APIs should now be back to normal, other APIs are being evaluated
We have noticed a higher than normal error rate on the content api, along with increased latencies. We are currently investigating the issue.
Report: "Failure to finish Figma Imports"
Last updateFigma Imports were unable to complete due to an issue processing images inside of the Figma plugin. The plugin was rolled back.
Report: "GraphQL API partial outage"
Last updateWhen the GraphQL API was deployed to release a performance improvement in model field selection, a regression was introduced for customers who only selected the content model from the model, which is a primary usecase of our Gatsby integration. This resulted in the API returning results with an empty content field. The GraphQL API was rolled back to fix the issue.
Report: "Partial outage of APIs"
Last updateThe source of the issue has been identified and the problem is now resolved. All systems are back to normal.
We are currently experiencing errors across our API due to an issue in one of our infrastructure components.
Report: "Content API p99 latency regression"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Content API partial outage"
Last updateBetween 6:45 a.m. GMT and 7:00 a.m. GMT, some customers might have experienced higher-than-normal error rates with our content API responses. The root cause is still being investigated; however, services are being carefully brought back to normal.
Report: "Content API partial outage"
Last updateBetween 9:40 p.m. GMT and 10:15 p.m. GMT, some customers might have experienced higher-than-normal error rates with our content API responses. The root cause is still being investigated; however, services are being carefully brought back to normal.
Report: "Content API partial outage"
Last updateBetween 8:50 a.m. GMT and 9:10 a.m. GMT, some customers might have experienced higher-than-normal error rates with our content API responses. This issue has been resolved, and services are back to normal.
Report: "Content API partial outage"
Last updateBetween 12:30 PM PST and 1:20 PM PST, we experienced connection issues with our MongoDB servers, which affected our content API responses. This issue has been resolved, and services have been restored.
Report: "Shopify App Publishing Issues"
Last updateWe have found the root issue and applied a fix.
We are aware of and currently investigating an issue affecting Builder.io Shopify App users that prevents them from being able to publish updates to their stores.
Report: "Firestore Authentication Issue"
Last updateThe incident has been resolved by Google Firebase. More information here: https://status.firebase.google.com/incidents/JH8hiSsfnSZHMdZqFTaC
We are continuing to investigate this issue.
We are aware of and are investigating an issue with a third party provider (Google Firebase) that may cause problems for some users when using the Builder web application. We do not yet have an ETA for resolution. More information can be found here: https://status.firebase.google.com/incidents/JH8hiSsfnSZHMdZqFTaC
Report: "Content API targeting parameters"
Last updateAround 2:30am PDT we rolled out an update to our content API. That deploy contained a regression that affected certain requests to our content API that contained targeting parameters. This could result in content being returned that did not match the specific query. Once the regression was recognized, we rolled back the update to fix the issue at 10:08am PDT.
Report: "Content API Issues"
Last updateThis incident has been resolved.
We are aware of possible performance degradation of the Builder.io Content API and are currently investigating and some customers might be experiencing higher than expected response times.
Report: "AWS Cloudfront Operational Issue"
Last updateBetween 9:37 AM and 10:13 AM PDT, AWS Cloudfront experienced elevated error rates for request serviced by the CloudFront Origin Shield and Regional Edge Cache in the US-EAST-1 region. The issue has been resolved and service is operating normally. https://status.aws.amazon.com
Report: "Degraded Content API Performance"
Last updateOne of our cloud infrastructure providers had an urgent maintenance incident that affected our platform. This caused some requests to our content API to run slower than normal. We have taken measures to ensure any future issues with the cloud infrastructure will not degrade Builder's API performance.