Sentry Incident History

The backlog has been cleared and ingestion is back to normal.

We have resolved the incident and are monitoring our ingestion backlog.

We have identified the issue and error and profile ingestion is back up in US. Expect delays in ingestion and alerting for errors and profiles.

We are experiencing issues with error and profile ingestion in our US region and are investigating the issue.

We've burned through our spans backlog. Latency across span consumers are normal.

We're still burning through our spans backlog

We've found and fixed the root cause and are burning through our spans backlog. Ingestion should be back to normal in around an hour.

We are still actively investigating this issue

We are still actively investigating this issue

We are actively investigating this issue

We have fully burned the backlog of EU error events ingested during the incident. Latency is normal across all event types.

We are still burning the backlog of EU region error events received during the incident. We are around halfway through the backlog.

The backlog on attachments in our EU region is fully burned, and latency is back to normal. While new error events are being processed with normal latency, we have a backlog of old error events that we are now burning through.

We have implemented a fix for our attachment ingestion issue in our EU region, and are monitoring while the backlog burns. We will update here when latency is back to normal.

We have implemented a fix for errors and feedback ingestion in our EU region, and latency for those pipelines has returned to normal. We are still working on fixing our attachments ingestion pipeline in EU.

We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are continuing to work on a fix.

We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are continuing to work on a fix.

We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are continuing to work on a fix.

We have identified the issue causing errors, attachments, and feedback events to have high ingest latency in our EU region and are working on a fix.

we are investigating an issue affecting errors ingestion in the DE/EU region

The webhook backlog is now resolved.

The webhook backlog is processing.

The webhook backlog is processing.

The webhook backlog is processing. Estimated recovery is now 2-2.5 hours.

The webhook backlog is processing.

The webhook backlog is processing.

The webhook backlog is being processed, ETA ~3 hours until fully caught up. Seer code reviews will run and linked Github issues and PRs will update incrementally.

The issue is mitigated and the webhook backlog is being processed. Seer code reviews will run and linked Github issues and PRs will update incrementally.

Feedback ingestion is back to realtime and the backlog has been processed.

We are continuing to monitor ingestion, and a fix is being deployed to unblock feedback ingestion.

We are processing errors in realtime, and error backlogs have been processed. We're continuing to monitor the situation before restoring feedback ingestion.

We are still working on recovery. New errors should be processed in real time, and we are working on recovering historical data. We are working to start allowing user feedback in DE.

We are still working on recovery. New errors should be processed in real time, and we are working on recovering historical data.

We have identified the problem and are working on recovery.

We continue to investigate the issue.

We are currently investigating the issue.

Service has been restored.

Service has been restored and we are actively monitoring the situation

We identified the issue and restoring service.

We are investigating ingestion delays.