Turbot Incident History

On July 20, between approximately 14:00 and 21:00 UTC, the scheduled process that refreshes temporary credentials for cloud connections stopped completing. As existing credentials expired (beginning around 15:00 UTC), live queries against affected connections, primarily AWS connections using cross-account roles, returned authentication errors such as ExpiredToken. Cached datatank and snapshot queries were not affected. Engineering identified and resolved the issue at approximately 21:00 UTC. Temporary credentials have been rotated across all affected connections, and live queries are working normally. A sweep is running to backfill any datatank syncs that were missed during the impact window, and we are continuing to monitor. We apologize for the disruption. A root cause analysis is underway, and we will follow up with affected customers as needed.

Monitoring indicates the the issue has been resolved.

We are currently investigating an issue impacting pipes ability to authenticate to AWS cloud providers.

This incident has been resolved.

We are continuing to monitor for any further issues.

A fix has been implemented and we are monitoring the results.

The issue has been identified and a fix is being implemented.

We are currently investigating this issue.