Historical record of incidents for Glitch
Report: "Network Degradation"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We're currently investigating reports of broad downtime around our project hosting, editor, and API systems.
Report: "System Migration: help.glitch.com"
Last updateThe new platform is online and serivce is restored. The team will be updating embedded links within content and other polish work on the site.
The vendor migration for the help.glitch.com documentation and helpdesk system is currently in progress. During this time, access to documentation and the ability to create tickets through the user interface may be temporarily unavailable. Rest assured, all Glitch production systems remain unaffected, and you can still reach us via direct email at support@glitch.com.
Report: "Ongoing network maintenance"
Last updateThis incident has been resolved.
Glitch has completed the migration, and systems appear to be operating normally. If you're experiencing continued issues with your projects, please email support@glitch.com. We remain monitoring to ensure that the updates have been completed successfully.
We have made some networking changes, and projects are running as expected. A remaining side effect is that original Glitch custom domains are not routing to projects, including glitch.new functionality. We hope to restore that functionality in the next few days.
In response to ongoing malicious activity targeting our platform, we are making urgent changes to our networking configuration that may temporarily impact some users' ability to access projects hosted on glitch.me subdomains. We have no evidence that any of our systems or your user data have been compromised. There should be no impact on access to glitch.com, the support forum, or our help docs during this maintenance. We apologize for the short notice and will continue to do everything we can to make sure your projects stay available.
Report: "Experiencing project downtime"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We're seeing many projects consistently returning to health status. Still, we will keep the status page as "Identified" out of an abundance of caution as we continue to stay in an active mitigation posture.
A subset of projects are restored to serving traffic, but mitigation work is ongoing.
The issue has been identified and a fix is being implemented.
A fix has been deployed and we are seeing projects coming back online. We will continue to monitor results.
We are currently investigating why projects are experiencing downtime.
Report: "Some projects experience slow start and placement times"
Last updateThis incident has been resolved.
We are currently investigating an issue that is causing some projects to have slow start times.
Report: "Version of Python 3 on containers downgraded unexpectedly"
Last updateThis incident has been resolved, all Python projects should be running on their intended version. We apologize for the trouble and thank you for your patience. As always, if you continue to run into issues, let us know by filing a ticket at help.glitch.com.
We've deployed a fix and are starting to see some projects get the expected, correct version of Python. We are still monitoring the situation, and it can take up to 12 hours for all containers to get the update. We appreciate your patience as this fix rolls out.
We are still testing a fix to the Python degradation issue and will post an update when we plan to deploy any changes.
We’ve identified the issue causing the degradation of the version of Python on containers. We need to do more testing of our solution to ensure we do not cause more issues, and we anticipate that fix going out tomorrow (Tuesday 3/25). We’ll keep you updated and appreciate the patience of our Python developers.
We're currently investigating an issue where the version of Python on containers had downgraded to 3.5. This only impacts projects running with Python instead of Node.
Report: "Continued intermittent issues loading projects"
Last updateThis incident has resolved and we appreciate your patience as we worked on making project loading stable again.
We've identified the issue causing projects to not be connecting. We're still doing a bit of work to fix things, so there may be continued degraded performance for some users.
Hey folks this is a continuation of last night's incident, which we prematurely resolved. Project hosting *is* much more stable than it was, but we're still seeing intermittent issues in the fleet and we continue to work to stabilize Glitch.
Report: "Issue with projects not loading"
Last updateThis incident has been resolved. If you have issues with your project loading, let us know by filing a Glitch Request in the Help Center at help.glitch.com.
We've identified the issue, and your projects should start coming back online. We're still monitoring the issue.
We are currently investigating an issue that's causing project hosting to not be working.
Report: "Some projects experiencing extra long start/waking times"
Last updateAfter monitoring for a day, we're considering this incident resolved. If you run into further issues, let us know by filing a ticket in our help center at https://help.glitch.com
The issue and impact has been mitigated and we're monitoring project start times. If you run into further issues, let us know by filing a ticket in our help center at https://help.glitch.com
We're currently investigating an issue that's causing projects that are starting up to take an extra long time to do so.
Report: "Scheduled maintenance today 12:30-3:30pm EST - may lead to issues with uploading assets"
Last updateWe've completed the work and are resolving today's scheduled maintenance. If you see any issues, contact support at help.glitch.com.
The engineering team will be making some infrastructure changes that may affect uploading assets in the Glitch editor. If you find yourself unable to upload assets via the Glitch editor, please contact support describing your issue and include your operating system/version and web browser/version. Assets uploaded prior to the maintenance window will not be affected.
Report: "Puppeteer usage to be temporarily blocked while we manage an incident"
Last updateWe have removed our temporary ban on the Puppeteer npm dependency. Through mitigating the incident that got us here, we realized that having Chromium on containers has been more trouble than it's worth due to its abuse, which negatively impacts the service availability for the rest of the Glitch community. In short, the vast majority of Chromium usage on Glitch is not by legitimate projects, and the result of those projects running is a worse experience for everybody else. Therefore, we will be removing Chromium from containers on Glitch on Monday, February 12 (one week from today). This unfortunately means that while Puppeteer will be initially installable, its need for Chromium on the file system will mean that it won’t be useful on Glitch. We know there are some apps that will be negatively impacted by this change, and we encourage you to share your feedback with us if your apps are affected, so we can help you find an alternate solution. If your project has been suspended for using Puppeteer and you have not yet reached out, file a ticket at help.glitch.com with the project name and we will unsuspend it for you.
We're investigating an incident involving malicious use of the puppeteer package, it may lead to projects running puppeteer being suspended.
Report: "Issues routing requests to projects"
Last updateOur mitigations continue to hold and we believe this incident has been resolved. If you continue to have issues connecting to your project, let support know at https://help.glitch.com
We're pretty confident the fleet is recovering - it will take a bit longer for all projects to start back up. We'll continue to monitor for additional instability.
We are continuing to work on a fix for this issue.
At roughly 20:00 UTC (3:00 PM US Eastern) Glitch started having difficulty routing requests to projects. We believe we've identified the issue and have a potential resolution in place; the project hosting fleet will take some time to get back to normal but you should see signs of improvement. Our apologies for our delayed notification.
Report: "Some projects not connecting in the editor or waking up."
Last updateThis incident has been resolved. If you continue to have issues connecting to your project, let support know at https://help.glitch.com
A fix has been implemented and we are monitoring progress on projects coming back on.
We are aware of and investigating an issue where some projects are not connecting in the editor or waking up.
Report: "Degraded performance of projects"
Last updateThis incident has been resolved.
A fix has been implemented and we are actively monitoring status to ensure that all issues have been addressed.
The issue has been identified, and work on a fix is in progress.
We are continuing to investigate this issue.
We are currently investigating an issue involving projects loading slowly or not at all.
Report: "Projects with custom domains are not loading"
Last updateThis incident has been resolved.
The service we use for custom domains had issues and the team responsible has deployed a fix. Custom domains should be back up and running now. If not, contact support@glitch.com.
We have received reports that project custom domains are not working. The team is investigating the issue. In the meantime, the glitch.me domains still work.
Report: "Problems loading projects in the editor"
Last updateThis incident has been resolved.
We're saw an issue where the editor wasn't connecting to projects correctly, affecting the ability to edit projects. We believe it's been resolved and we're currently monitoring it.
Report: "Some projects suspended in error"
Last updateThis incident has been resolved. If your project is still suspended and you believe it's in error, contact support at help.glitch.com with the project name in question.
A fix has been implemented and projects should be unsuspended and starting up again. We are still monitoring to make sure that's the case.
We are working on a fix to an issue that lead to some projects being suspended in error.
Report: "Some experiencing inability to log in and slow loading times"
Last updateThis incident has been resolved.
We are currently investigating reports of issues with logging in via Google or magic link. We've seen some success with users clearing browser cache and trying again, or waiting several seconds after submitting credentials if there is a blank page.
Report: "API outage causing login and editing errors"
Last updateThis incident has been resolved.
API response has been restored and access to editing and login should be functional. We’re continuing to monitor the fix as it deploys.
The issue has been identified and a fix is being deployed.
An API failure is causing users to be unable to log in or access the editor. We are working to identify the cause.
Report: "New custom domain SSL certs not resolving"
Last updateThe third-party service we were using to provide this feature has been shut down and will remain so indefinitely. Until the Glitch team can rebuild this functionality, we will be temporarily removing the “Add Custom Domain” feature. Read more in our full update here: https://help.glitch.com/kb/article/9-adding-a-custom-domain/
We've identified an issue causing newly-added custom domains to not have SSL certs set. The team is looking into it.
Report: "Degraded performance of projects"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified an issue with the platform that may cause project slowness
Report: "API outage causing project downtime or slowness"
Last updateThis incident has been resolved. If you're experience downtime still, contact Glitch Support at help.glitch.com with the project name.
We're still monitoring projects as they come back. Some projects may seem slow as they come back up.
A fix has been deployed and we are monitoring results.
We're experiencing an outage with our API and have identified the issue but some will experience downtime or slowness with projects. The team is working on a fix.
Report: "Editor not loading"
Last updateThis incident has been resolved. If your editor does not load after a browser refresh, contact our Help center at help.glitch.com.
We're investigating an issue causing the editor to not load. This does not impact hosting of projects.
Report: "Project hosting issues"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We have identified the issue that we believe was causing the project hosting issues, and taken measures to address the cause. We'll continue to monitor projects closely, and will mark this incident as resolved when we're confident that the issue has been fully addressed.
Glitch is currently experiencing an issue with project hosting and availability for some projects. We're currently investigating and will update with more information as we learn more.
Report: "Ongoing project hosting issues"
Last updateHosting has stabilized and returned to normal levels.
We've identified the issue and have implemented fixes that should stabilize the situation. We'll continue to monitor servers closely, and mark this incident as resolved when we're confident the fix has fully resolved the problems.
We are currently experiencing an issue with project hosting / availability. We're investigating now and will update with more information as we learn more.
Report: "Project hosting down"
Last updateProjects were improperly returning 50X errors. This incident has been resolved. If you encounter issues, you can contact support at help.glitch.com.
Report: "Logging in serving incorrect information"
Last updateWe have resolved the issue and logging in should should work again. If not, clear your browser cache and try again. Sorry for the trouble!
We are currently investigating an issue where the Community site and logging in is serving from a staging environment instead of production.
Report: "Glitch API down"
Last updateThis incident has been resolved. If you encounter issues, you can contact support at help.glitch.com.
The API is back up, but we're continue to monitor and anticipate some slowness in connections.
We are currently investigating why our API is down, causing editor and login issues.
Report: "Custom domain connecting issues"
Last updateThis incident is resolved.
A fix has been made and we are seeing custom domains connecting again as we monitor the results of the fix.
We have new reports of custom domains not connecting again and are currently investigating the issue. Only custom domains are impacted; apps can still be accessed at their glitch.me domain.
Report: "Custom domain connecting issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring results. Custom domains should be connecting at this time. If you continue to see an issue, please contact Support via our Help Center: https://help.glitch.com
We are currently investigating why some custom domains are not connecting, or connecting intermittently.
Report: "Projects showing connecting error"
Last updateWe have resolved the issue and projects are connecting again. Thanks for your patience!
A fix is out and we are monitoring results.
We've identified the issue and are working on a fix now.
We are currently investigating why some projects are showing a connecting error in the project editor. We will post additional updates here as we work to fix this.
Report: "Slow project requests and start times"
Last updateThis incident has been resolved. If you continue to experience issues with your projects, email the project name to support@glitch.com and we can help you out!
A fix has been deployed and we are monitoring the results.
The issue has been identified and a fix is being deployed.
We are still investigating this issue, and apologize for any inconvenience this is causing project creators.
We are still investigating this issue.
We are continuing to investigate this issue and are adding the Editor experience to where some users are experiencing issues.
We are currently investigating an issue where some users are experiencing slow project start times and requests.
Report: "Projects not loading"
Last updateThis incident has been resolved. Projects should be loading as expected. If you continue to see an issue with your projects, please send us a support request here: https://help.glitch.com/new/
We are receiving reports that some apps are still not loading. Our team is looking into this now and we will post an update here once it is resolved.
We are continuing to monitor for any further issues.
A fix is out and we are monitoring the results. Apps should be able to run now, and projects should be connecting in the project editor. If you are still seeing an issue with your apps, please send us a support request here: https://help.glitch.com/new/
The issue has been identified and we are working on a fix now. Thanks for your patience!
We are currently investigating an issue that is preventing projects from loading. During this time, apps may not be running, and projects may not be connecting in the project editor.
Report: "Some projects suspended in error"
Last updateProjects suspended in error should be running again. If your project is still not running, please send the name of the project and issue to support@glitch.com.
We are aware that some Glitch projects have been suspended in error and are working to unsuspend them. We apologize for the trouble.
Report: "Custom domains are not connecting."
Last updateThis incident has been resolved. Custom domains are now loading for Glitch projects.
Our custom domain partner believes this is a network issue and they are looking into it now.
We are investigating an issue that is preventing projects from loading at their custom domain. Please note that visiting the glitch.me URL for these projects is still working at this time.
Report: "Glitch account login & signup down"
Last updateThe issue that was preventing folks from logging in or creating an account on glitch.com has now been fixed. You should now be able to login again. If you find that you are still unable to login or create an account, please go to our Help Center at https://help.glitch.com/home/ and click Contact Support.
A fix is out and we are monitoring the results.
We are currently investigating an issue that causes a blank, white screen to appear after attempting to login to Glitch or sign up for a Glitch account.
Report: "Login & Signup not working with Google or GitHub option"
Last updateThis incident has been resolved.
A fix is out and we are monitoring the results. This means that login/signup should work now. Please give it a try! If you still can't login or signup, please go to our Help Center and send us a support request. https://help.glitch.com
We are continuing to investigate this issue.
We have received reports that the Email with Magic link option is not working as well. Our team has added this to their investigation.
We are currently investigating an issue that is causing the screen to go blank after attempting to Login or Signup for Glitch while using the Google/GitHub and Magic Link options.
Report: "Projects with custom domains returning a 403 error"
Last updateThis incident has been resolved.
A fix has been implemented and now we're monitoring to see if this eliminates the 403 errors on custom domains.
Users are still experiencing issues with custom domains resolving, which we are still investigating and apologize for the inconvenience. Projects do work at their glitch.me domain.
We are currently investigating an issue where custom domains are returning 403 errors. Projects are still working at their glitch.me domains.
Report: "The project editor is not connecting when working on a static app."
Last updateThis incident has been resolved.
The project editor should be connecting for static apps now. We are keeping an eye on this and will move this incident to Resolved shortly.
We are rolling out a fix for this issue now. During this time, the live sites will continue to serve - but you may see a connecting error when opening the project editor. We estimate that this fix may take up to two hours to complete as we apply it to our containers. We will update this page once the issue is resolved.
We are working to resolve an issue that is causing static apps and other apps that don't use a .env file to stop connecting.
Report: "Projects returning Consul error"
Last updateThis incident has been resolved.
A fix is out and we are monitoring the results. Projects should not be returning Consul errors moving forward.
We are currently investigating an issue that is causing some projects to return a Consul error. Updates will be provided here as we work to resolve this issue.
Report: "Project hosting is down"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
Hello, project hosting is down at the moment. During this time, projects and their live sites may not load as expected. We are investigating the issue.
Report: "Projects are not serving requests"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
API stability is returning. We are now gradually allowing projects to start up. During this time, requests to the editor or lives sites may be be slower than usual.
Apps are receiving traffic again. We are now working to make sure all projects have the ability to start up as expected.
We are continuing to work on a fix for this issue.
The project dashboard may be working intermittently at this time. We are working to restore it.
Part of our infrastructure that is used for routing traffic to projects appears to be unhealthy. We are experimenting with provisioning fresh infrastructure to see if that resolves the issue.
Just wanted to drop a note here to let folks know that we are still investigating this issue. Thanks for all your support and patience!
We are continuing to investigate this issue.
We are continuing to investigate this issue.
At this time, projects may not be visible when going to the live site. We are currently investigating this issue.
Report: "glitch.com is showing a 502 Bad Gateway Error"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Custom domains that are connected to Glitch projects are not responding."
Last updateThe issue has now been fixed. Custom domains that are used with Glitch projects should begin connecting again. If you still have issues with custom domains, please send a message to support@glitch.com and we will help you out!
We are currently investigating this issue. During this time, it may not be possible to use or update a custom domain with a Glitch project.
Report: "Custom domains being used with Glitch projects are not connecting."
Last updateThis incident has been resolved.
Custom domains should be working now. If you have a custom domain that is still not responding, please send a message to support@glitch.com.
We are currently investigating an issue that is causing custom domains that are associated with Glitch projects to not work.
Report: "Some projects having editor connection issues"
Last updateThis incident has been resolved. If you continue to encounter this issue, contact support@glitch.com with the name of the project.
We are currently investigating an issue where some projects are having issues connecting in the editor.
Report: "Glitch.com is timing out"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "glitch.com is timing out"
Last updateThis incident has been resolved.
glitch.com should now be loading. during this time we will continue to closely monitor site performance.
We are currently investigating this issue. During this time, you may see a timeout/504 error when going to glitch.com.
Report: "glitch.com website is down"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The glitch.com website is working again. We are continuing to investigate the cause.
Project hosting does not appear to be affected, but the community site, glitch.com is still down.
The glitch.com website and project hosting are not working at this time. We are investigating.
Report: "Projects failing at proxy level."
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We are currently investigating an issue that is causing projects to fail. During this time, projects may throw the following error "TypeError: Cannot read property".