Historical record of incidents for Shorthand
Report: "Media uploads failing"
Last updateThis incident has been resolved.
This incident has been resolved.
The source of the failures has been identified as a partial outage with a third-party service provider. They've reported that a fix has been implemented and that results are being monitored. Image uploads are currently working in Shorthand web app but we'll continue to monitor and provide updates.
We are investigating media upload failures from the Shorthand web app.
Report: "Media uploads failing"
Last updateThis incident has been resolved.
This incident has been resolved.
The source of the failures has been identified as a partial outage with a third-party service provider. They've reported that a fix has been implemented and that results are being monitored. Image uploads are currently working in Shorthand web app but we'll continue to monitor and provide updates.
We are investigating media upload failures from the Shorthand web app.
Report: "Inaccessible pages from HTTPS certificate renewal failure"
Last updateOn Friday, September 6th from about 17:00 to 18:43 Pacific/UTC-07:00, pages published with Shorthand hosting were inaccessible due to an HTTPS certificate renewal failure. The issue was resolved at 18:43, at which point all affected pages became fully accessible again.
Report: "The Shorthand app is not able to send email at this time"
Last updateThis incident has now been resolved, and emails, including account password resets, notifications, and workspace invitations, are now being processed and delivered.
Our email service provider is currently being affected by an outage. Emails sent from the Shorthand application, including account password resets, notifications, and workspace invitations, are unavailable.
Report: "Service outage"
Last updateWe are pleased to report that the database locking issue which led to increased CPU usage on our RDS system has been fully resolved. Our technical team has carefully monitored the system following the implementation of the fix and confirmed that normal operations have resumed without further complications
We are continuing to monitor for any further issues.
We are pleased to inform you that our technical team has successfully implemented a fix for the issue identified earlier. Currently, we are in the process of monitoring the system's performance to ensure stability and functionality are fully restored.
We've implemented a fix to some database connectivity related errors causing the outage and are monitoring the solution.
We are investigating a general service outage of the Shorthand web app.
Report: "Issue Detected: Export Analytics Feature Not Working"
Last updateAfter monitoring, this incident is resolved.
A fix has been implemented and we are monitoring the update.
We have detected an issue with the export analytics feature and are currently working to find a fix.
Report: "Difficulty selecting media from Unsplash in Shorthand Editor"
Last updateThe issue affecting media selection from our Unsplash integration in the Shorthand Editor has been successfully resolved. We apologise for any inconvenience this may have caused.
We are aware of an ongoing issue where users are experiencing difficulty selecting media from our Unsplash integration in the Shorthand Editor. As a temporary workaround, you can download Unsplash images directly from the Unsplash website and then easily upload them to the Shorthand Editor. We are working to resolve this issue as quickly as possible. We will provide an update with an ETA once we have more information.
Report: "Uploading media within a "Text" section is failing"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We continue to attentively monitor this situation and we will post a further update soon.
We are currently aware of an ongoing issue where users experience difficulty when attempting to upload media within a "Text" section. For those affected, we've identified a temporary workaround: Instead of the usual method, try copying the media with the keyboard shortcut "ctrl + c" (or "cmd + c" for Mac users) and then pasting it into the editor using "ctrl + v" (or "cmd + v" for Mac users). Our team is actively working on a permanent resolution. Please note that this particular issue only affects stories created before today. New stories produced today onward should not encounter this problem. We genuinely apologize for any inconvenience this may have caused and appreciate your patience as we work towards a solution. Thank you for your understanding.
Report: "Background Settings Reverting to Defaults Unexpectedly"
Last updateThis incident has been resolved.
We continue to attentively monitor this situation and we will post a further update soon.
The issue has been identified and a fix is being implemented.
We're aware of an ongoing issue where the Background Media settings appear to be reverting to defaults, particularly when set to text only. This means that sections initially set with solid colour backgrounds might unexpectedly display an uploaded image/video or the default placeholder image. We have identified a temporary workaround that can alleviate the issue. You can reset the background media options for the affected sections back to "text only" or by deleting the background media on Grid Sections. This should restore your intended solid colour backgrounds and prevent unwanted images or videos from appearing. Our team is actively investigating this matter and working on a resolution. It's worth noting that this problem only impacts stories that were made prior to today, and any new stories are not affected. We understand the inconvenience this might cause and sincerely apologize for disrupting your workflow.
Report: "Issue with our upstream provider"
Last updateThe problem with one of our upstream service providers has been resolved, and transactional emails have now been sent.
Issue with our upstream provider Our upstream provider, Campaign Monitor, is currently experiencing an issue that is affecting our ability to send transaction emails, ie password reset, sharing a story via email. See https://status.campaignmonitor.com/ for more details. We will resolve this incident once we have received confirmation the issue has been resolved from our upstream provider.
Report: "Issue with upstream provider preventing images from being uploaded"
Last updateThis incident has been resolved.
Our upstream provider is currently experiencing asset delivery and transformation issues. We apologize for any inconvenience caused
Report: "Video Uploads: Service Disruption"
Last updateThis incident has been resolved.
We are currently experiencing an issue with uploading videos into stories. Our team is aware of the issue and is working on a fix. We apologize for any inconvenience this may cause.
Report: "Issue with upstream provider preventing images from being uploaded"
Last updateOur upstream provider is currently experiencing asset delivery and transformation issues, which have caused a 19-minute outage. We apologize for any inconvenience caused.
Report: "Issue with upstream provider preventing images from being uploaded"
Last updateThis incident has been resolved.
A fix appears to have been implemented by the upstream provider and the results are being monitored but image upload performance may still be degraded.
Report: "Increase in CPU IO"
Last updateAn Increase in CPU IO Resolved - We resolved the root issue and the site performance returned to normal. Aug 7, 11:55 UTC Monitoring - We've identified the issue. During this time the site is slow to respond. Aug 7, 11:30 UTC Investigating - There was an increase in disk I/O that is affecting the performance problem with the application. Aug 7, 11:25 UTC
Report: "We are investigating a high load on our database cluster; this may impact your ability to access Shorthand."
Last updateThis incident has been resolved and we're continuing to investigate the root cause of the issue.
You may see a banner that shows this error: "Shorthand is currently unavailable. Please try again in a few minutes". We are currently investigating the issue and will add more information shortly.
Report: "Partial outage of Shorthand"
Last updateOur engineering team have identified the cause of the high load and have now resolved the issue. We will continue to monitor this issue for any further issues.
We are investigating a high load on our database cluster; this may impact your ability to access Shorthand. You may see a banner that shows this error: "Shorthand is currently unavailable. Please try again in a few minutes". We are currently investigating the issue and will add more information shortly.
Report: "Issue with our upstream provider"
Last updateThe problem with one of our upstream service providers has been resolved, and the publishing configuration is now operational.
Issue with our upstream provider AWS have reported increased error rates and latencies for certain services within the US-East-1 region. Until these issues are resolved, Shorthand users attempting to create a custom domain or custom subdomain Shorthand hosting may be affected. We would recommend waiting until the affected AWS services are returned to normal operation before attempting to update any publishing configurations. We will resolve this incident once we have received confirmation the issue has been resolved from our upstream provider.
Report: "Temporary service interruption caused by failed database migration"
Last updateAn unexpected lock was caused on a critical table during a product release for an updated set of features. The release was aborted and the upgrade transaction reversed, restoring Shorthand to normal operation. We will investigate and possibly schedule a service outage in order to release the Shorthand product upgrade again
Report: "Partial outage of Shorthand app due to themes issue (Fixed now)"
Last updateA broken base theme was deployed at 7:45pm UTC that affected the ability to use the editor and previews. This was realised 15 mins later. A fix was deployed at 8:45pm UTC.
Report: "A few users experiencing Shorthand is currently unavailable. Please try again in a few minutes."
Last updateA web application firewall rule was wrongly detecting some normal user editor actions and blocking them in some rare cases in certain story content. The rule was fixed at 11:33 pm Tuesday 24th January UTC time. We are updating our internal firewall procedures to prevent a similar incident from occurring in the future.
We are continuing to work on a fix for this issue.
We've found an issue related to our firewall and have put in a fix and are monitoring the solution.
A few users are experiencing a banner error showing "Shorthand is currently unavailable. Please try again in a few minutes". We are currently investigating the issue.
Report: "Publishing and story preview intermittent partial page load failure"
Last updateAll services are now back to full operation, including the slower to resolve Shorthand Analytics service.
All affected AWS underlying infrastructure is back to normal, except for Shorthand Analytics which relies on an AWS service still being fully restored (API Gateway). Story publishing, previewing and all other core functionality is back to normal. We will continue to monitor service availability until AWS has fully closed the incident.
AWS has reported elevated error rates in their EC2 APIs resulting in some Shorthand service issues in publishing and previewing stories. Retrying publishing or reloading previews works intermittently. We are monitoring the situation and working with AWS to address the problem.