codefortynine Incident History

This incident has been resolved.

Atlassian appears to have resolved the underlying issue before responding to our notification. We are still awaiting their response for confirmation and clarification.

We are currently experiencing delays in Jira events, which are causing the following apps to process changes more slowly than usual. We have notified Atlassian and are waiting for their response.

# Post Mortem ## Summary On July 07, 2026, from approximately 06:30 UTC until 14:00 UTC, External Data for Jira Fields stopped receiving work item events from Atlassian for a subset of customer installations. As a result, Dependant Field Configurations were no longer triggered when work items were created or updated. ## Impact **Affected component** External Data for Jira Fields - Dependant Field Configurations **Customer impact** Dependant field configurations were not executed after work item creation or updates, causing field values that rely on automatic updates to remain stale; this issue only affected installations that were in the Outdated permissions state during the rolling release. ## Timeline July 07, 2026, 06:30 UTC - A new version of the app was deployed with rolling release. July 07, 2026, 07:20 UTC - Canary monitoring reported failing dependant field updates. July 07, 2026, 08:00 UTC - Investigation started after logs showed that no work item events had been received since approximately 06:30 UTC, matching the start of the rolling release. July 07, 2026, 08:40 UTC - Incident classified as SEV-1 and Status Page incident created. July 07, 2026, 09:00 UTC - ECOHELP ticket opened with Atlassian July 07, 2026, 09:30 UTC - Decision made to deploy a new app version without the rolling release feature. July 07, 2026, 09:50 UTC - Deployment verified in staging and production rollout started. July 07, 2026, 11:00 UTC - Logs confirmed that work item events were being received again, while remaining installations continued upgrading. July 07, 2026, 15:00 UTC - Incident resolved and Status Page updated. ## Root Cause Analysis A new app version was deployed using Atlassian's rolling release mechanism. During the rollout, installations that had not yet completed the upgrade entered the Outdated permissions state because of a permission \(scope\) change introduced in the new version. Atlassian no longer delivered Dynamic Trigger work item events to these installations while they were in this state. Since Dependant Field Configurations rely entirely on these events, they were no longer executed for affected customers. ## Resolution and Recovery To restore functionality as quickly as possible, we created a new app version that reverted the changes introduced in the previous release. This version was deployed and a bulk upgrade was initiated to automatically migrate all affected installations to the new version. Throughout the recovery process, we closely monitored incoming work item events to verify that event delivery had resumed. Once the bulk upgrade progressed, work item events began arriving again, and Dependant Field Configurations resumed normal operation. ## Improvements Expand automated testing in the staging environment

This incident has been resolved.

The automatic migration has now completed successfully, and all app installations have been updated to the fixed version. We are continuing to monitor for any further issues.

We have deployed a fix and are automatically migrating all app installations to the updated version. Most instances have already been migrated, but it may take another 1–2 hours for the update to reach all remaining instances. As a workaround, you can manually update the app to the latest version to restore functionality immediately.

We are still working on a fix. As a quick workaround, you can manually update the app to the latest version.

The issue has been identified and we are working with Atlassian to resolve the issue.

We are currently investigating this issue.

This incident has been resolved.

The automatic migration has now completed successfully, and all app installations have been updated to the fixed version.We are continuing to monitor for any further issues.

We have deployed a fix and are automatically migrating all app installations to the updated version. Most instances have already been migrated, but it may take another 1–2 hours for the update to reach all remaining instances.As a workaround, you can manually update the app to the latest version to restore functionality immediately.

We are still working on a fix. As a quick workaround, you can manually update the app to the latest version.

The issue has been identified and we are working with Atlassian to resolve the issue.

We are currently investigating this issue.