Historical record of incidents for Marvel App
Report: "Design Tool storage intermittent failures"
Last updateOur infrastructure has been upgraded and monitoring has ensured that we are experiencing no more intermittent errors on our design tool storage systems.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring results.
We are continuing to investigate this issue.
Our design tool storage system is experiencing intermittent failures that we are currently investigating.
Report: "Outage of backend system supporting Marvel"
Last updateDue to a problem with a subsystem responsible for our processing queues we were unable to serve the application. We have failed over to an alternate system and all systems are now operational.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Infrastructure Outage"
Last updateMarvel is operational again. We will continue to monitor the situation.
Traffic beginning to hit Marvel's database again. We are monitoring the situation.
Confirmed issue with Google's infrastructure.
We are currently investigating this issue. Google's infrastructure is potentially having issues with networking and load balancing affecting our service.
Report: "DNS Failure"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Parts of the service are not responding"
Last updateThe incident has been resolved.
We are currently investigating an issue where prototypes and some parts of the system are not responding.
Report: "Marvel web app is experiencing intermittent outages"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Infrastructure Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Marvel backend update failure"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue where a code deploy has disabled our frontend site from functioning.
Report: "Partial outage due to cache server issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Design API backend Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Our provider has identified the source of the issue and is attempting a fix. New designs are now available to be created again, with updates and document fetches also now fully available. We will continue to monitor the issue.
We are continuing to investigate an issue with a third party data provider which is impacting on new design documents being created. Document retrieval and some updates are still working. We are working with the provider to identify the source of the issue and solve it as soon as possible.
We are currently investigating this issue.
Report: "CDN hosting service experiencing issues"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
Our CDN provider appears to be addressing the issue and we are seeing some of the service reappear in different locations.
The outage is affecting specific locations where Fastly is down. We are still able to host to a number of countries globally.
We are currently investigating an issue with our edge content network having a failure. Our service is currently unavailable at this time
Report: "Issue with application servers: http 503"
Last updateIncident resolved
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating the issue and will update you as soon as we can.
Report: "Unsplash API integration failure"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue with our Unsplash Image API integration which is failing to load images added to a design in our design tool.
Report: "Issue serving Design tool, Sketch and Handoff"
Last updateFixed, but we will continue to monitor.
We are continuing to work on a fix for this issue.
We are currently fixing an issue that affects the Design Tool, Sketch and Handoff. These features are currently offline.
Report: "Global CDN disruption"
Last updateThis incident has been resolved.
Fastly has a fix out and they are monitoring.
Our CDN provider Fastly is experiencing some issues https://status.fastly.com/incidents/1rbr326kjsmt
Report: "Degraded operations due to Google service outage."
Last updateThis incident was Google related and appears resolved their end.
Google appear to have stabilised their performance issues and the service appears to be back to normal operation, we will continue monitoring to ensure everything is working correctly.
We are currently investigating this issue.
Report: "Backend sync queues degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Backend failure on hosting service"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
The issue with the blog not loading is localised so far to locations in the UK, specifically London. We are working with our hosting provider to solve the issue.
We are currently investigating this issue.
Report: "Images not loading for some customers - Image Hosting Provider"
Last updateAn SSL network error prevented certain customers from being able to see images on the platform. This issue was affecting customers utilising internal network proxies which prevented the images from loading due to an SSL Protocol error. All of our images are hosted via our CDN system Fast.ly. We altered the Fastly system to force TLS and HSTS security headers for all clients requesting image URLs and it solved the issue. We are still investigating the root cause of why this suddenly occurred as no configuration was changed or altered in the Fast.ly hosting system. We will be working to determine further with Fastly and our customers to determine the specific conditions leading to the error occurring.
Report: "Design API Node Failure"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Design Tool API Failure"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Issues for users logging in and managing their projects."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
An unplanned feature deployment scheduled incorrectly and is being updated.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Thumbnail Generation failure"
Last updateDue to a certificate chain authority update our services have an expired Root certificate authority. The root certificate was due to be expired today the 30th of May 2020 and thus caused requests to fail from internal services. [https://www.xolphin.com/support/Rootcertificates/Phasing\_out\_Addtrust\_External\_CA\_Root\_certificate](https://www.xolphin.com/support/Rootcertificates/Phasing_out_Addtrust_External_CA_Root_certificate)
Due to an error with our backend process for outputting thumbnails for the Design Tool. Thumbnails have failed to update from 11:52PM (GMT) to 13:00PM (GMT). The issue has been resolved and content is being reprocessed to output any updated thumbnail.
Report: "Planned Maintenance"
Last updateWe've finished our upgrade work and released a major new update to Marvel
Our planned maintenance has been extended until 6pm GMT due to an extended DB migration.
We are currently undergoing maintenance
We're currently updating the platform with new features. This will last for the following times throughout Saturday 18th January, 7am - 6pm GMT | 2am - 1pm EST | 11am - 10pm GST. Apologies for any disruption caused, see you in a jiffy!
Report: "Design tool image sync issues"
Last updateThis incident has been resolved.
The issue has been resolved
We're investigating difficulties syncing images from the design tool back to Marvel. The design tool itself is not affected.
Report: "iOS and Android app builds failing"
Last updateApp builds for Android are working fully and due to an alteration of Apple's iOS build protocols we will be altering our iOS build process with a further update in the next few weeks.
We are monitoring our fix for Android app builds. There are still ongoing issues with iOS builds
We have identified and are deploying a fix for the issue with Android app builds not working. You will soon be able to build APK files again.
Our feature for building and downloading iOS IPA files and Android APK files is currently experiencing an issue with our hosting provider. Due to a hardware failure. We are investigating and attempting to determine a fix for the problem.
Report: "Slow loading times in London"
Last updateNo sign of issues for the last 60+ minutes
Error rates have returned to normal but we're keeping an eye on the situation
Our CDN provider fastly is having issues in London. We're experiencing slow loading and timeouts across the platform for users in the affected region.
Report: "Design Tool Image Rendering"
Last updateThis incident has been resolved.
We've implemented a fix and are monitoring the situation.
We're seeing elevated error rates for rendered Design Tool images. The team is investigating the issue. All design data remains safe.
Report: "Design syncs processing slowly"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We have identified the source of the delay in our queueing system and patched it. Queues are now operating as normal.
The design syncing system has suffered from degraded performance when processing designs. The queuing system that handles processing design files is being investigated.
Report: "Internal Database Issues"
Last updateWe had an internal issue with our database, which has now been resolved.
We've implemented a fix and are monitoring the situation
We're currently having a few issues with our internal database, we're working hard to resolve and will update as soon as we can.
Report: "Design Tool versioning issues"
Last updateAn internal service outage led to some versioning issues on some user's designs.
Report: "Loss of Service"
Last updateAn internal service outage caused the service to return 502s for some of our Users.
Report: "Loss of Service"
Last updateToo many requests to our DB, meant a momentary loss of service for our Users
Report: "Google Cloud SQL Planned Maintenance"
Last updateGoogle Cloud SQL planned maintenance caused a small outage to our service
Report: "Design Tool image serving issue"
Last updateOur Design tool image rendering service failed to serve some User's designs
Report: "Design Tool image serving issue"
Last updateOur Design Tool rendering service was unable to create images for designs.
Report: "Design Tool sync issues"
Last updateOur Design Tool API experienced an outage causing some users design work to fail in saving.
Report: "Network issues in some EU Locations"
Last updateAn issue with our DNS service provider caused outages in the EU for some customers.
Report: "Cloudflare DNS Issue."
Last updateCloudflare had an issue with their DNS provisioning which was causing problems in certain global regions.