Historical record of incidents for Semaphore CI
Report: "Delays in job processing on Semaphore 2.0"
Last updateWe are currently investigating this issue.
Report: "Partial Account Access Issue: UI Message"
Last updateSome users may see incorrect account or billing messages in the UI. CI pipelines might be affected. We're investigating and will provide updates shortly.
Report: "Delays in job processing on Semaphore"
Last updateThe issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateWe are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Slow job processing on e2 and f1 machine types"
Last updateThis incident has been resolved.
We have deployed a fix for f1 machines, and we are monitoring the results.
A fix was deployed for e2 machines. We are continuing work on f1 machines.
Job delays on the Ubuntu Noble (24.04) image.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Partial API outage"
Last updateWe experienced an outage affecting a subset of API endpoints due to a misconfiguration in API request routing. The issue persisted for approximately 2 hours and 30 minutes and was fully resolved at 16:30 UTC.
Report: "Slow job processing on e2 and f1 machine types"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Issues with Chrome and Chromedriver versions in Semaphore"
Last updateThe rollback of the Ubuntu image has been completed and the issue has been resolved.
Due to several issues reported related to the latest Chrome versions, we are in the process of a rollback of the Ubuntu image update.
The issue has been identified and a fix is being implemented.
Report: "Delays in job processing on Semaphore"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Issues with Semaphore login"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Issues with GitHub Status checks on PR builds"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Elevated error rates on Semaphore"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Initialization jobs failing on Semaphore"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Issues with E2 machines availability"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Delays in job processing on Semaphore"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Delays in Job Processing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Elevated error rates on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Elevated error rates on the “Plans & Billing” page and Delays in loading pages"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are finalizing a fix for this issue. Thank you for your patience.
We are continuing to work on a fix for this issue. Thank you for your patience.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue. Thank you for your patience.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Slower job processing on e2-standard agent types"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Delays in Job Processing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Elevated error rates on the "Plans & Billing" page"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Elevated API error responses"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Elevated API error responses"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Delays in Job Processing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Elevated error rates on Semaphore"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Degraded performance of the Flaky Tests feature"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Degraded performance of the Flaky Tests feature"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Job processing issues"
Last updateWorkflows were unable to start due to machine unavailability. The issue was resolved 25 minutes later
Report: "Linux Arm-based Build Server Issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue involving a Linux arm-based build server. Running jobs were also impacted by this issue and may have been interrupted.
Report: "Issues with sem debug and sem attach"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Elevated Level of Errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We're experiencing an elevated level of errors and are currently looking into the issue.
Report: "Elevated Errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We're experiencing an elevated level of errors and are currently looking into the issue.
Report: "Elevated error rates on Semaphore"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Semaphore UI Issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Elevated error rates on Semaphore"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We're experiencing an elevated level of connection errors and are currently looking into the issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "User Interface Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently experiencing an issue with our user interface that may affect your experience. Our team is aware of this problem and is actively working to resolve it as quickly as possible.
Report: "Workflow issues"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Delays in job processing"
Last updateThis incident has been resolved.
There is an ongoing issue with job processing. Jobs may take additional time to start. The SRE team is investigating this issue.
Report: "Trouble with loading logs for finished jobs"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "BitBucket integration not operational"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and we are working on a fix.
Report: "Elevated error rates on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Job Processing Delay"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
At the moment, there should be no delays in the job-starting process. However, the job logs might still not be accessible for some users. We are continuing to work on a fix for this issue.
We are continuing to work on a fix for the issue. The job logs still might not be accessible for some users. Jobs might experience some delays to start.
We are continuing to work on a fix. The job logs page still might not be accessible for some users. Jobs might experience some delays to start.
We are continuing to work on a fix. The job logs page might not be accessible for some users in the meantime. Jobs might experience some delays to start.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Jobs are not starting in Docker Containers"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Elevated error rates and delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Delays in job processing on Semaphore 2.0"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Job processing is still experiencing degraded performance. We are continuing to investigate it.
We are currently investigating this issue.