Historical record of incidents for Compass
Report: "Customers may experience delays or failures receiving emails"
Last updateWe were experiencing cases of degraded performance for outgoing emails from Confluence, Jira Work Management, Jira Service Management, Jira, Opsgenie, Trello, Atlassian Bitbucket, Guard, Jira Align, Jira Product Discovery, Atlas, Compass, and Loom Cloud customers. The system is recovering and mail is being processed normally as of 16:45 UTC. We will continue to monitor system performance and will provide more details within the next hour.
Report: "Both Compass UI and APIs are not responding"
Last updateWe have verified that the incident has been resolved.
The issue has been identified and a fix is being implemented. We have put some mitigation mechanisms in place and our systems are starting to recover.
We are currently investigating an issue impacting both the Compass UI and APIs. We are investigating this issue and will share an update once we have additional information.
Report: "Both Compass UI and APIs are not responding"
Last updateWe are currently investigating an issue impacting both the Compass UI and APIs. We are investigating this issue and will share an update once we have additional information.
Report: "Issues with data pipelines for customers using GitHub for Jira Connector."
Last updateExisting events are continuing to be processed. Any updates to existing entities such as pull requests will also correct and update the data here as required. If there are any concerns about impact to your site with this connector please reach out to support and the team will address your concerns as soon as possible.
Processing of previously impacted events is now underway. An update will be provided when these operations have been completed and historical processing has been completed.
Our team has identified the root cause of this issue and have reverted the change that was causing these data processing errors. All new updates from GitHub should now be flowing correctly. We are actively working to address a potential data gap for previously impacted events, and will provide an update when available.
We are aware of an issue where users of GitHub for Jira connector may be missing pull requests and experiencing other data pipeline issues. Our team is looking into this with urgency and an update will be provided when available.
Report: "Issues with data pipelines for customers using GitHub for Jira Connector."
Last updateWe are aware of an issue where users of GitHub for Jira connector may be missing pull requests and experiencing other data pipeline issues. Our team is looking into this with urgency and an update will be provided when available.
Report: "New components not visible when performing searches"
Last updateThis incident has been resolved.
We've implemented a fix for this issue and are seeing systems recover. We will keep monitoring to make sure we don't see a recurrence of this issue.
We're seeing a recurrence of the earlier issue around components not showing up in our search indexing. We're working on restoring the service.
Report: "New components not visible when performing searches"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're investigating an issue around new components not being visible in search for a period of time after creation.
Report: "Management of Apps is partially unavailable on different platforms"
Last updateBetween 10:55 AM UTC to 13:15 PM UTC on April 28, 2025, we experienced partial unavailability in managing Forge apps; Such as lifecycle developer console usage, and the creation and deployment of Forge apps for Atlassian Partners, Jira Work Management, Jira, Atlassian Developer, Jira Product Discovery, Compass, and Atlassian Migrations. We have deployed a fix to mitigate the issue and have verified that the services have recovered. The conditions that cause the bug have been addressed and we're actively working on a permanent fix. The issue has been resolved and the service is operating normally.
The issues causing the unavailability of related Forge apps capabilities on multiple Atlassian platforms have been resolved, and services are now operating normally for all impacted customers. We will continue to closely monitor for stability.
We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms, such as apps lifecycle (installation, removals), developer console usage (including tunneling), and the creation and deployment of Forge apps. We have identified the root cause and are in the process of mitigating the impact. We'll keep you posted with further updates. Next updates in 1 hour.
We are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms. Such as apps lifecycle (installation, removals), developer console usage (including tunnelling), and the creation and deployment of Forge apps. Our team is working diligently to resolve this issue and restore services quickly. We are in the process of identifying the root cause and implementing the necessary fixes. Next updates in 1 hour.
Report: "New components not visible when performing searches"
Last updateWe're investigating an issue around new components not being visible in search for a period of time after creation.
Report: "Management of Apps is partially unavailable on different platforms"
Last updateWe are currently experiencing partial unavailability in managing Forge apps on different Atlassian platforms. Such as apps lifecycle (installation, removals), developer console usage (including tunnelling), and the creation and deployment of Forge apps. Our team is working diligently to resolve this issue and restore services quickly. We are in the process of identifying the root cause and implementing the necessary fixes.Next updates in 1 hour.
Report: "[Scheduled Maintenance] Database Upgrade"
Last updateCompass will be conducting a required database upgrade that will impact the ability to perform any write operations to Compass for all users on April 13, 2025 between 5:00 UTC and 6:00 UTC.For more information about this maintenance, please refer to the community post - https://community.atlassian.com/forums/Compass-articles/Scheduled-maintenance-for-Compass/ba-p/2991224.
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Report: "Some users are experiencing errors while creating new Jira and Confluence sites"
Last updateOn March 14th, 2025, between 09:00 and 19:14 UTC, some customers experienced issues creating new sites. The errors occurring for some users during new site creation should now be resolved.
We are investigating cases of errors for some users creating new sites. We will provide more details within the next hour.
Report: "Issues with search within Confluence and Compass"
Last updateSearches within Confluence and Compass should now be operating as normal. This issue will now be resolved.
Issues causing error messages when users were attempting to search within Confluence and Compass should now be resolved. Our team is monitoring ongoing performance at this time.
We are aware that some users of Confluence and Compass are currently experiencing errors when attempting to search. Our team is investigating this issue with urgency and will provide an update when available.
Report: "Unable to load the Operations page"
Last updateThe issue has been resolved and the service is operating normally.
We are investigating reports of the Operations page not loading for Compass Cloud customers. There is no impact to the API. We have identified the issue and are in the process of fixing it.
Report: "Github integration not working"
Last updateCompass customers using Github may have experienced intermittent problems in receiving events or leveraging features such as "config-as-code", this was due to an outage on Github
Report: "Gitlab integration not working"
Last updateCompass customers using Gitlab may have experienced intermittent problems in receiving events or leveraging features such as "config-as-code", this was due to an outage on Gitlab.
Report: "Bitbucket integration is experiencing a partial outage related to an ongoing incident with Bitbucket Cloud"
Last updateWe have verified that the Bitbucket integration is now working as expected. Marking this incident as Resolved.
Bitbucket Cloud is experiencing a major outage: https://bitbucket.status.atlassian.com/ . Compass cannot setup new integrations with BBC and is not able to process updates. Events and metrics that have already been synced from Bitbucket to Compass are not impacted.
Report: "Gitlab integration not operational"
Last updateGitlab has mitigated the issue and we are seeing recovery and no errors during ingestion.
We're seeing issues with the Gitlab integration, given Gitlab's ongoing issues: https://status.gitlab.com/pages/incident/5b36dc6502d06804c08349f7/67645861f92e4e58064f7d32
We're seeing issues with the Gitlab integration, given Gitlab's ongoing issues: https://status.gitlab.com/pages/incident/5b36dc6502d06804c08349f7/67645861f92e4e58064f7d32
Report: "Issues accessing Compass"
Last updateThis incident has been resolved.
We're seeing recovery across all of our systems. The team is still working on ensuring that impact is fully mitigated.
We're seeing issues accessing the Compass application. The team is working on identifying the underlying cause.
Report: "Compass service inaccessible"
Last updateThis incident has been resolved.
We've identified the issue and reverted the faulty change. We're monitoring the change propagation and are seeing systems recover across the board.
We're seeing issues accessing Compass. The team has identified the issue and is currently rolling out a fix.
Report: "Compass Github app unavailable"
Last updateThis issue has now been fixed and the app is now available for ise.
We have identified the issue and are working on a fix at the moment.
We are currently investigating an issue that is causing the Github app to be unavailable for some tenants.
Report: "Compass services not accessible"
Last updateThis incident has been resolved.
We've identified the cause of the issue and are seeing service quality recover.
We are currently investigating this issue.
Report: "Github integration not operational"
Last updateThis incident has been resolved.
We're seeing issues with the Github integration, given Github's ongoing issues: https://www.githubstatus.com/incidents/kz4khcgdsfdv
Report: "Event ingestion delayed"
Last updateEvent ingestion and metric derivation are both working at normal capacity. This incident is resolved.
Event ingestion times are back to normal, and we're seeing the delay between event ingestion and metric derivation go down. We're still monitoring the fix and will update once things are back to nominal.
A fix has been implemented and we are monitoring the results.
The team has identified the cause for the delays and is rolling out a fix.
We're seeing event ingestion and metric derivation delayed. The team is working on resolving the issue.
Report: "Issues logging in to Compass"
Last updateBetween 03-07-2024 20:08 UTC to 03-07-2024 20:31 UTC, we experienced downtime for Compass. The issue has been resolved and the service is operating normally.
The team has implemented a mitigation and we're seeing service quality hold. We're continuing to look into the root cause of the issue. We've also been able to isolate the time of impact to a 20-minute timeframe between 8:08pm 03/07 UTC and 08:31pm 03/07 UTC.
We're seeing recovery but are still working on identifying the root cause of the issue.
We're seeing customers experience issues logging into Compass and using the service. The team has been notified and is investigating the cause.
Report: "The Developer Experience and Pull Requests dashboard (in the Teams page) include incorrect PR data"
Last updateThe incident has been resolved.
We've resolved the issue and are monitoring to verify changes.
The Developer Experience and Pull Requests dashboard in the Teams page include incorrect PR data from other repositories (in the same workspace) connected to the Compass instance. The root cause has been identified and a fix is in progress.
Report: "Users experiencing errors using when using the product"
Last updateThis incident has been resolved.
We've rolled back the change that was identified as having caused the issues and we're seeing systems recover. We will continue monitoring recovery and will post further updates as needed.
The issue has been identified and a fix is being implemented.
We're seeing errors around authentication across the platform. The team has been alerted and is working on identifying the underlying issue.
Report: "Navigating to Apps in Component Details is not working"
Last updateThis incident has been resolved.
We have released and verified the fix. Compass is operational again. Users are able to navigate to apps.
Users are not able to navigate to "apps" in the App menu on the Component details view. We are currently rolling out the fix. We will report back once the fix has been verified.
Report: "Admin Portal Feature Access Issue"
Last updateBetween 6:30 AM UTC to 9:50 AM UTC, we experienced failures in accessing some features from the Admin Portal. The issue has been resolved and the service is operating normally.
We are investigating an issue causing failures in accessing some features from the Admin Portal, which is impacting some of our Cloud customers. We have identified the root cause and anticipate recovery shortly.
Report: "Some paginated queries in Forge hosted storage kept repeating the last page"
Last updateWe have identified and resolved a problem with Forge hosted storage, where some paginated queries kept repeating the last page. The incident was detected by our internal monitoring and was resolved quickly after detection by reverting the deployment. Activating changes recently made to the query cursors for paginated queries introduced a bug that impacted some apps. A small number of requests were impacted over a 16-minute window, while the incident lasted. Timeline: - 25/Mar/24 10:08 p.m. UTC - Impact started, when the changes were deployed to production - 25/Mar/24 10:09 p.m. UTC - Incident was detected - 25/Mar/24 10:24 p.m. UTC - Incident was resolved and impact ended The impact of this incident has been completely mitigated and our monitoring tools confirm that query operations are back to the pre-incident behaviour. We have also resolved the underlying bug and deployed the fix to production, completely eliminating the cause of this incident. We apologise for any inconvenience this may have caused to our customers and our developer community.
Report: "Investigating new product purchasing"
Last updateBetween 28th Feb 2024 23:15 UTC to 29th Feb 2024 00:05 UTC, we experienced issue with new product purchasing for all products. All new sign up products have been successfully provision and confirmed issue has been resolved and the service is operating normally.
We are investigating an issue with new product purchasing that is impacting for all products. Customers adding new cloud products may have experienced a long waiting page or an error page after attempting to add a product. We have mitigated the root cause and are working to resolve impact for customers who attempted to add a product during the impact period. We will provide more details within the next hour.
Report: "Service Disruptions Affecting Atlassian Products"
Last update### **Summary** On February 14, 2024, between 20:05 UTC and 23:03 UTC, Atlassian customers on the following cloud products encountered a service disruption: Access, Atlas, Atlassian Analytics, Bitbucket, Compass, Confluence, Ecosystem apps, Jira Service Management, Jira Software, Jira Work Management, Jira Product Discovery, Opsgenie, StatusPage, and Trello. As part of a security and compliance uplift, we had scheduled the deletion of unused and legacy domain names used for internal service-to-service connections. Active domain names were incorrectly deleted during this event. This impacted all cloud customers across all regions. The issue was identified and resolved through the rollback of the faulty deployment to restore the domain names and Atlassian systems to a stable state. The time to resolution was two hours and 58 minutes. ### **IMPACT** External customers started reporting issues with Atlassian cloud products at 20:52 UTC. The impact of the failed change led to performance degradation or in some cases, complete service disruption. Symptoms experienced by end-users were unsuccessful page loads and/or failed interactions with our cloud products. ### **ROOT CAUSE** As part of a security and compliance uplift, we had scheduled the deletion of unused and legacy domain names that were being used for internal service-to-service connections. Active domain names were incorrectly deleted during this operation. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We know that outages impact your productivity. The detection was delayed because existing testing & monitoring focused on service health rather than the entire system’s availability. To prevent a recurrence of this type of incident, we are implementing the following improvement measures: * Canary checks to monitor the entire system availability. * Faster rollback procedures for this type of service impact. * Stricter change control procedures for infrastructure modifications. * Migration of all DNS records to centralised management and stricter access controls on modification to DNS records. We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability. Thanks, Atlassian Customer Support
We experienced increased errors on Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Align, Jira Product Discovery, Atlas, Compass, and Atlassian Analytics. The issue has been resolved and the services are operating normally.
We have identified the root cause of the Service Disruptions affecting all Atlassian products and have mitigated the problem. We are now monitoring this closely.
We have identified the root cause of the increased errors and have mitigated the problem. We continue to work on resolving the issue and monitoring this closely.
We are investigating reports of intermittent errors for all Cloud Customers across all Atlassian products. We will provide more details once we identify the root cause.
Report: "Newly created/updated Compass components are not visible while searching via the GraphQL API or UI"
Last updateThis incident has been resolved.
The issue has been resolved and impact has been mitigated.
This issue is under investigation.
Report: "User searches failing"
Last updateBetween 15:40 UTC to 15:57 UTC customers experienced intermittent failures when searching for users in Atlassian cloud services: Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, and Compass. The issue has been resolved and the service is operating normally.
We have mitigated the problem. We are now monitoring closely.
We are investigating an issue with our user search service that is impacting Atlassian cloud service: Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, and Compass. We will provide more details within the next hour.
Report: "Drop in the success rate of Forge hosted storage API calls"
Last updateWe identified a problem with the Forge hosted storage API calls, which resulted in a drop in invocation success rates in the developer console. The impact of this incident has been mitigated and our monitoring tools confirm that the success-rate is back to the pre-incident behaviour. It impacted 16 apps according to our logs, where these apps saw a reduced success rate of storage.get API calls, as listed in https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-basic. As part of Forge's preparation to support Data Residency, Forge hosted storage has been undergoing a platform and data migration for storing app data. As part of this migration we do comparison checks for data consistency between the old and new platform. The previous incident earlier, https://developer.status.atlassian.com/incidents/9q71ytpjhbtl, had put the data on the new platform out of sync and so comparisons of the data from the old and new platform started showing failures and the migration logic retries on failures to test for consistency issues. This retry behaviour increased latency of these requests which led to 16 apps receiving an increased number of 504 timeout errors. Checking synchronously was identified by the team as a bug and should have been async. Once the root cause was identified we moved our backing platform rollout to a previous stage. The rollout is split into several stages. The issues we were having were on our blocking stage where we make calls to both the old and new platform and wait for both to complete so we can test any performance issues in the new platform before using it as our source of truth. It was in this blocking stage where we had a bug that included waiting on comparisons when it should've been async. To recover, we reverted back to our shadow mode stage. In this stage, all operations to the new platform are asynchronous, including comparisons that were blocking in the other stage and resulted in timeout issues and 504 errors being sent to apps. This is the state that Forge hosted storage has been in for several months without any problems. Here is the timeline of the impact: - On 2024-02-05 at 06:42 PM UTC, impact started with comparisons start happening on out of sync data in blocking mode - On 2024-02-05 at 08:57 PM UTC, impact was detected to API by our monitoring systems - On 2024-02-05 at 11:34 PM UTC, rollout to new platform was reverted to known stable state and impact ended We will release a public incident review, PIR, here in the upcoming weeks for this and the incident that happened earlier, https://developer.status.atlassian.com/incidents/9q71ytpjhbtl. We will detail all that we can about what caused the issue, and what we are doing to prevent it from happening again. We apologise for any inconveniences this may have caused our customers and the developer community and committed to preventing further issues with our hosted storage capability.
Report: "Performance degradation for Forge app invocations"
Last updateAt around 4am UTC, about 40 percent of Forge app invocations experienced high latency, with a portion of the requests failing, during a 15 minute time window. The scaling of the instances was misconfigured following a new deployment of the service, which needed manual intervention which took a few minutes to resolve the issue. Timeline: - 2024-01-31 04:00 UTC: impact started - 2024-01-31 04:03 UTC: incident detected - 2024-01-31 04:15 UTC: the incident was resolved and the impact ended This issue is now resolved and Forge is fully operational. We apologize for any inconveniences this may have caused to our customers, partners, and our developer community.
Report: "Outage in Atlassian Intelligence functionality in multiple products"
Last updateBetween 23:45 UTC to 00:30 UTC, we experienced an outage in some Atlassian Intelligence features for Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, Atlas, and Compass. The issue has been resolved and the service is operating normally.
We have identified the root cause of the increased errors and have mitigated the problem. We are now monitoring closely.
We are investigating an issue with Atlassian Intelligence that is impacting some Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Product Discovery, Atlas, and Compass Cloud customers. We will provide more details within the next hour.
Report: "Intermittent errors when trying to access Compass"
Last updateBetween 11:00 UTC to 12:00 UTC, we experienced degradation in our services. This impacted our catalog, search and provisioning operations. The issue has been stabilized and services are operating normally.
Our services are stable now. We are now monitoring closely.
We are investigating reports of intermittent errors for some Compass Cloud customers. We will provide more details once we identify the root cause.
Report: "Atlassian's cross product user search service is currently degraded."
Last update### **SUMMARY** On Dec 18, 2023, between 12:29 p.m. and 3:35 p.m. UTC, Dec 18, 2023, Atlassian's cloud customers using Atlas, Bitbucket Cloud, Compass, Confluence Cloud, Jira Service Management, Jira Software, Jira Work Management, Jira Product Discovery products were unable to search for users or use the "@mention" functionality. Customers' user search results failed or were delayed as Atlassian's service returning user search results was degraded in several regions. The incident originated from a computationally intensive operation that was triggered multiple times in rapid succession, resulting in degraded performance of Atlassian's user search service across several regions. Notably, customers in the EU west region were most affected. The incident was detected within 2 minutes by automated monitoring, and our team promptly took action by recovering unhealthy systems and scaling up the service's infrastructure temporarily. The resolution process concluded in 3 hours and 06 minutes. ### **IMPACT** The overall impact was between Dec 18, 2023, between 12:29 p.m. UTC and Dec 18, 2023, 3:35 p.m. UTC. The Incident caused service disruption to cloud customers worldwide. Customers experienced delayed or failed user searches when using the following Atlassian cloud products: * Atlas * Bitbucket Cloud * Compass * Confluence Cloud * Jira Service Management * Jira Software * Jira Work Management * Jira Product Discovery ### **ROOT CAUSE** The incident stemmed from Atlassian's user search service receiving commands to process multiple computationally intensive operations in rapid succession. These operations were directed at the same customer data set, and therefore overloaded resources within a clustered database system, leading to memory exhaustion and subsequent unresponsiveness to user search requests. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** To prevent a recurrence of such incidents, we are implementing the following measures: * Implement a mechanism to queue computationally intensive operations in order to avoid overloading the resources within the systems and process them without impact on customer experience. * Fine-tune our clustered database settings to mitigate the impact of resource exhaustion on the overall system. We apologize to customers whose services were affected during this incident; we are taking immediate steps to improve the service’s resiliency. Thanks, Atlassian Customer Support
It has been resolved. Atlassian's cross product user search is working.
Atlassian's cross product user search service is currently healthy. Searches for users within Atlassian products are working as expected. We are in the process of investigating the root cause of this incident.
Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.
Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.
Atlassian's cross product user search service is recovering. Searches for users within Atlassian products are returning to normal.
We are investigating reports of intermittent errors for <SOME/ALL> Atlassian, Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Jira Align, Jira Product Discovery, Atlas, and Compass Cloud customers. We will provide more details once we identify the root cause.
Report: "Search indexing delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We're seeing degraded performance with our indexing pipeline which may result in components not showing up in search for several minutes after creation.
Report: "Gitlab Forge Integration Degraded"
Last updateGitlab has resumed processing all background jobs. The Gitlab Forge integration is now fully operational.
We have identified an issue with our Gitlab integration. Gitlab is currently experiencing delays in background processing that is impacting the Gitlab Forge integration.
Report: "Degraded performance for Forge integrations"
Last updateThis incident has been resolved.
We're seeing error rates subside and app behavior return to normal, but will continue monitoring impact.
We are continuing to work on a fix for this issue.
We're seeing an increase in errors in API calls to the Atlassian Forge platform across all our integrations. The team has identified the cause and is working on mitigating impact.
Report: "Atlassian Account login issues"
Last update### **SUMMARY** On Sep 13, 2023, between 12:00 PM UTC and 03: 30 PM UTC, some Atlassian users were unable to sign in to their accounts and use multiple Atlassian cloud products. The event was triggered by a misconfiguration of rate limits in an internal service which caused a cascading failure in sign-in and signup-related APIs. The incident was quickly detected by multiple automated monitoring systems. The incident was mitigated on Sep 13, 2023, 03: 30 PM UTC by the rollback of a feature and additional scaling of services which put Atlassian systems into a known good state. The total time to resolution was about 3 hours & 30 minutes. ### **IMPACT** The overall impact was between Sep 13, 2023, 12:00 PM UTC and Sep 13, 2023, 03: 30 PM UTC on multiple products. The Incident caused intermittent service disruption across all regions. Some users were unable to sign in for sessions. Other scenarios that temporarily failed were new user signups, profile retrieval, and password reset. During the incident we had a peak of 90% requests failing across authentication, user profile retrieval, and password reset use cases. ### **ROOT CAUSE** The issue was caused due to a misconfiguration of a rate limit in an internal core service. As a result, some sign-in requests over the limit received HTTP 429 errors. However, retry behavior for requests caused a multiplication of load which led to higher service degradation. As many internal services depend on each other, the call graph complexity led to a longer time to detect the actual faulty service. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** We are continuously improving our system's resiliency. We are prioritizing the following improvement actions to avoid repeating this type of incident: * Audit and improve service rate limits and client retry and backoff behavior. * Improve scale and load test automation for complex service interactions. * Audit cross-service dependencies and minimize them where possible related to sign-in flows. Due to the unavailability of sign-in, some customers were unable to create support tickets. We are making additional process improvements to: * Enable our unauthenticated support contact form and notify users that it should be used when standard channels are not available. * Create status page notifications more quickly and ensure that for severe incidents, notifications to all subscribers are enabled. We apologize to users who were impacted during this incident; we are taking immediate steps to improve the platform’s reliability and availability. Thanks, Atlassian Customer Support
Between 12:45 UTC to 15:30 UTC, we experienced login and signup issues for Atlassian Accounts. The issue has been resolved and the service is operating normally. We will publish a post-incident review with the details of the incident and the actions we are taking to prevent similar problem in the future.
We are no longer seeing occurrences of the Atlassian Accounts login errors, all clients should be able to successfully login now. We will continue to monitor.
We can see a reduction in the Atlassian Accounts login issues after the mitigation actions were taken. We are still monitoring closely and will continue to provide updates.
We have identified the root cause of the Atlassian Accounts login issues impacting Cloud Customers and have mitigated the problem. We are now monitoring this closely.
We are investigating an issue with Atlassian Accounts login that is impacting some Cloud customers. We will provide more details within the next hour.
Report: "Multiple product logins"
Last updateBetween 4:30AM UTC to 6:00AM UTC, we experienced issues for users attempting to login for Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Jira Product Discovery, Compass, and Atlassian Analytics. The issue has been resolved and the service is operating normally.
We are investigating reports of intermittent errors for login to Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Jira Product Discovery, Compass, and Atlassian Analytics Cloud customers. We will provide more details once we identify the root cause.
Report: "Atlassian services unable to scale"
Last updateWe have resolved a case of degraded performance for Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Atlassian Access, Jira Align, Jira Product Discovery, Atlas, and Compass Cloud customers. We will provide more details within the next hour.
Report: "Degraded performance for Github Forge App"
Last updateThis incident has been resolved.
We are noticing degraded performance for customers using Github Forge App. This is due to external dependency on Github's webhook delivery We will provide more details within the next hour.
Report: "Sign-ups, Product Activation, and Billing not working"
Last updateWe mitigated the issue with Sign-ups, Product Activation, and Billing, and the systems are back to BAU, and all functionality is restored.
We have identified the root cause of the Sign-ups, Product Activation, and Billing not working and have mitigated the problem. We are now monitoring closely.
We are investigating an issue with Sign-ups, Product Activation, and Billing that is impacting all of our Cloud Customers. We will provide more details within the next hour.
Report: "Github App installation unavailable"
Last updateThis incident has been resolved.
We have identified and fixed an issue with installing Compass Github App. The issue was related to a bug introduced in Github's OAuth API as documented here https://github.com/orgs/community/discussions/61291.
Due to a recent change, users are unable to complete the installation flow in the Compass Github App. All other functionality in the Compass Github App remains unaffected and is working as intended. The team has identified the issue and is working on deploying a fix.
Report: "Performance issues and outages with Cloud products"
Last update### **SUMMARY** We understand the importance of providing reliable and consistent service to our valued customers. On July 6, 2023, from 03:52 to 15:11 UTC, we experienced an issue with an upgraded version of a third-party tool that functions as our internal artifact management system. Despite our monitoring system identifying the incident within two minutes, this issue led to the degradation of the scaling capabilities of our internal hosting platform, resulting in service degradation or outages for customers of Atlassian cloud. In response to this situation, we are taking immediate measures to enhance the stability of our system and prevent similar issues from re-occurring. ### **IMPACT** This incident affected multiple regions and products due to the diminished scaling capabilities of our internal hosting platform. In most products and offerings, customers faced reduced functionality, slower response times, and limited access to specific features. ### **ROOT CAUSE** The root cause of the incident was the introduction of new functionality in a third-party tool that functions as our internal artifact management system. It led to an unexpected increase in the load on the primary database of the artifact system. Upon identifying and localizing the problem, we promptly adjusted the system configuration to regain stability. ### **REMEDIAL ACTIONS PLAN & NEXT STEPS** Over the next months, we will enact a temporary freeze on non-critical upgrades of the artifact management system, and we will focus our efforts on three high-priority initiatives: 1. **Enhancing system scaling:** We prioritized work ensuring that downtime in a critical infrastructure component does not affect the scaling of other components. We expect to complete this initiative within the next two months. 2. **Reducing interdependencies:** We are working to mitigate the risk of potential cascading failures by ensuring that significant system components are able to operate independently in the case of issues. Initiatives 1 and 2 are already in progress but have been given priority to be completed as soon as possible. 3. **Strengthening testing procedures:** Alongside these initiatives, we are addressing the need for even more stringent testing procedures than we already have in place to prevent potential issues in future updates. We are committed to collaborating closely with our technology partners to ensure the most optimal experience for our customers. We apologize for any inconvenience caused by this incident and appreciate your understanding. Our team is dedicated to continually improving our systems and processes to provide you with the exceptional service you deserve. Thank you for your continued support and trust in us. Sincerely, Atlassian Customer Support
We experienced performance issues and outages for several Atlassian Cloud Products. The issue has been resolved and the service is operating normally.
We have identified the root cause of an issue with an internal infrastructure component that has been impacting multiple Cloud products, including Jira Software, Jira Service Management and Confluence, and customers. This issue had lead to a performance impact and, in some cases, outages. We have implemented a fix to resolve the issue and recovery is in progress.
We are investigating an issue with an internal infrastructure component that is impacting multiple Cloud products, including Jira Software, Bitbucket, Jira Service Management and Confluence, and customers. These issues include performance impact and, in some cases, outages. Users may experience slow loading and uploading of attachments, login issues or inability for new customers to sign up. We have identified the root cause and are actively working on the service recovery.
Report: "Intermittent errors during login for some customers"
Last updateBetween 07:31 UTC to 12:32 UTC, we experienced errors during login for Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Product Discovery, Compass, and Atlassian Analytics. The issue has been resolved and the service is operating normally.
We have identified the root cause of the errors during login and have mitigated the problem. We are now monitoring closely.
We are investigating reports of errors during login that is impacting some Atlassian Support, Confluence, Jira Work Management, Jira Service Management, Jira Software, Opsgenie, Trello, Atlassian Bitbucket, Atlassian Access, Jira Product Discovery, Compass, and Atlassian Analytics. We have identified the root cause and expect recovery shortly.
Report: "Compass Bitbucket Event Integration Degraded"
Last updateBetween 17:30 to 22:20 UTC, we experienced issues with the Bitbucket App for Compass resulting in missing events and failed Config-As-Code syncs. The issue has been resolved and the service is operating normally.
We are investigating reports of intermittent errors for some Compass Cloud customers using the Bitbucket cloud event integration. We will provide more details once we identify the root cause.
Report: "Degraded performance impacting Search capabilities and signups"
Last updateBetween 01:38 UTC to 02:46 UTC, we experienced degraded performance of Confluence, Jira Work Management, Jira Service Management, Jira Software, Jira Product Discovery, and Compass impacting Search capabilities, creation of new product instances and joining existing ones. The issue has been resolved and the service is operating normally.
We are investigating reports of intermittent errors for some Confluence, Jira Work Management, Jira Service Management, Jira Software, Jira Product Discovery, and Compass Cloud customers impacting Search capabilities, creation of new product instances or joining existing ones. We will provide more details once we identify the root cause.
Report: "Intermittent Search and Page Load Failures"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently working to resolve intermittent search and page load failures. We have identified the root cause and are working on a fix.
Report: "Atlassian cloud product signup timeout"
Last updateBetween 08:20 UTC to 09:30 UTC, we experienced degraded performance on new account signups for Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Atlassian Access, Atlas, and Compass. The issue has been resolved and the service is operating normally.
We are investigating reports of intermittent errors for Confluence, Jira Work Management, Jira Service Management, Jira Software, Atlassian Bitbucket, Atlassian Access, Atlas, and Compass Cloud customers. We will provide more details once we identify the root cause.
Report: "Errors navigating products, logging in, and logging out"
Last updateBetween 03/Aug/22 15:20 UTC to 03/Aug/22 17:47 UTC, some customers experienced errors using Atlassian products, including errors while logging in or being forcibly logged out. The root cause was a DNS service deployment in US East region, which caused widespread DNS lookup errors for a variety of Atlassian services including authentication services. We have rolled back the change to mitigate the issue and have verified that the authentication services have recovered. The issue has been resolved and Atlassian services are operating normally.