Historical record of incidents for AppOptics
Report: "Some alerts are delayed"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Hourly summarizations are delayed."
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues. Next update in 3 hours.
A fix has been implemented and we are monitoring the results. Next update in 6 hours.
Report: "Increased error rate on Web and API"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Absent alerts are delayed"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Metric aggregation is delayed"
Last updateThis incident has been resolved.
The metric aggregation lag is reducing over time. The next update will be in 12 hours.
The metric aggregation lag is reducing over time. The next update will be in 12 hours.
The metric aggregation lag is reducing over time. The next update will be in 12 hours.
The metric aggregation lag is reducing over time. The next update will be in 12 hours.
The metric aggregation lag is reducing over time. The next update will be in 12 hours.
The metric aggregation lag is reducing over time. The next update will be in 12 hours.
The metric aggregation lag is reducing over time. The next update will be in 12 hours.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Major outage from 18:03 to 18:40 UTC"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Traces and the metrics generated by them are delayed"
Last updateThis incident has been resolved.
We are processing the backlog of traces.
We are currently investigating this issue.
Report: "Traces and the metrics generated by them are delayed"
Last updateThis incident has been resolved.
We are processing the backlog of traces.
We are currently investigating this issue.
Report: "Increased Error rates"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Increased error rates and delayed alerts for composite metrics"
Last updateThis incident has been resolved.
Starting at 09:00 UTC On Monday Composite Metrics had increased error rates and delays in processing alerts on Composite Metrics.
Report: "Increased error rates"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Increased error rates from 02:38 - 02:42 UTC"
Last updateIncreased error rates from the API and other services for several minutes.
Report: "Traces delayed and trace related metrics outage from 17:45 to 20:27 UTC on 27May2023"
Last updateSome traces submitted after 17:45 UTC were delayed. The backlog of traces was processed by 20:27.
Report: "Metrics from Azure Monitor may be delayed or absent"
Last updateMetrics sent via the Azure Monitor integration in AppOptics may be delayed or absent.
Report: "Issue with sending mail for account invites and password reset"
Last updateIssue started: 14:05 UTC Issue resolved: 14:54am UTC Issue with sending mail for account invites and password reset. Other mail notifications should have still worked.
Report: "Some metric streams are delayed"
Last updateThis incident has been resolved.
Some customer may have alerts that trigger in error as we catch up on a backlog of data.
Report: "Alert or Screenshot integrations with slack are not delivered"
Last updateWe are currently investigating this issue.
Report: "Tracing delay"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are seeing delay on Tracing, we are investigating.
Report: "Traces are delayed"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Report: "Alert processing errors"
Last updateThis incident has been resolved.
From 01:33 to 02:17 UTC Alerts and metric ingestion were disrupted. Metrics sent by some agents may have gaps during that time and Absent Alerts may have fired incorrectly.
We are currently investigating this issue.
Report: "Delayed historical summaries when viewing spans of time 3 days long or longer"
Last updateThis incident has been resolved.
We continue to work on processing the backlog of data into hourly historical summaries.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Delayed historical summaries when viewing spans of time 3 days long or longer"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Report: "Delayed metrics"
Last updateThis incident has been resolved.
We continue to work on the metric processing pipeline.
We continue to process the backlog. Most metrics and alerts should be back to normal behavior.
We continue to process the backlog. Most metrics and alerts should be back to normal behavior.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Increased Error rates"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Traces are delayed"
Last updateThis incident has been resolved.
We are continuing to process the backlog and expect processing to be complete in an hour.
We have started to process the backlog.
We are currently investigating this issue.
Report: "Delays in processing metrics"
Last updateThis incident has been resolved.
The backlog is still being processed. Traces are back to 100% sample rate.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue. The sample rates for traces have been reduced.
We are currently investigating this issue.
Report: "Metric ingestion is delayed"
Last updateThis incident has been resolved.
We are working on the backlog of ingested metrics.
The issue has been identified and a fix is being implemented.
Report: "Service Side Metric Aggregation is delayed"
Last updateThis incident has been resolved.
The backlog is being processed.
The issue has been identified and a fix is being implemented.
Report: "Alert processing"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Traces are delayed"
Last updateThis incident has been resolved.
Trace sample rates have returned to normal and we continue to monitor the situation.
The issue has been identified and we've reduced the sample rates by half while we are working on addressing it.
We are currently investigating this issue.
Report: "Delayed metrics from AWS Cloudwatch"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Report: "Delayed alert processing"
Last updateThis incident has been resolved.
Alert processing was delayed starting at 03:07 UTC.
Report: "Traces are delayed"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Trace collection is normal but processing of traces is delayed.
Report: "Trace sample rates reduced"
Last updateMaintenance is complete and no future maintenance is planned. The sample rates of traces are back at 100%.
Sample rates will continue at 30-50% while maintenance continues.
Sample rates are returning to normal and we continue to monitor.
The maintenance work continues and sample rates continue to be held to a low level.
Sample rates have been reduced while we perform maintenance.
Report: "APM Metrics and Traces are not being received."
Last updateThis incident has been resolved.
We are returning to normal sample rates and continue to monitor.
The backlog is being processed and we are preparing to return to normal sample rates.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Delay in web interface"
Last updateThe issue has been resolved
A fix has been implemented and we are continuing to monitor
The issue has been identified and a fix is being implemented
Actions taken in the web interface may be delayed
Report: "Alerts grouped by a tag will not trigger"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Alerts delayed"
Last updateThis incident has been resolved.
We have identified an issue which may cause alerts to be delayed and are working on a fix.
Report: "Web errors"
Last updateThis incident has been resolved.
We continue to monitor but do not expect any additional disruptions.
We are seeing an increase in HTTP 500 errors when loading dashboards or https://my.appoptics.com/
Report: "Traces, alerts, and metrics were delayed from 12:10PM UTC - 2:20PM UTC"
Last updateThis incident has been resolved.
Report: "Metrics and Alerts processing is delayed"
Last updateThe trace backlog has been processed.
We should finish processing the trace backlog in about an hour.
We expect to be done processing the backlog of traces in a few hours.
Alert processing is back to normal. There is a backlog of traces and we continue to monitor that as it is processed.
We are working on restoring full functionality to alerts and processing the backlog of traces.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Trace Details Delay"
Last updateThis incident has been resolved.
The backlog of trace details is still being processed and we expect to be caught up in an hour.
A fix has been implemented, trace details will continue to be delayed while the backlog is being processed.
We believe that the delay started around 15:45 UTC and is the result of issues in AWS: https://status.aws.amazon.com/
Trace details are delayed. The cause of the issue has been identified.
Report: "Alerts delayed"
Last updateA fix has been implemented, alerts are function properly.
Alerts are delayed and spurious absent alerts may be firing.
Report: "Traces delayed"
Last updateThe backlog of traces has been processed. This incident is resolved.
We expect to be done with processing the backlog of traces in an hour.
We are continuing to process the backlog of traces.
We are continuing to process the backlog of traces.
As the AWS S3 recovery improves (https://status.aws.amazon.com/) we continue to work through the backlog.
The issue has been identified.
Traces are currently delayed, the issue is being investigated.
Report: "https://my.appoptics.com inaccessible"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Partial web outage"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results. If you continue to see issues, please contact customer support.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Disruption of APM agent metrics and traces during 13:30-14:05 UTC"
Last updateThere was a disruption of the APM processing tier from 13:30 to 14:05 UTC
Report: "Delayed alerts for small number of metrics (23:30 UTC on 22Apr2021 to 21:15 UTC 23Apr2021)"
Last updateThis incident has been resolved.
A small number of alerts were delayed starting at 23:30 UTC on 22Apr2021. As of 21:15 UTC 23Apr2021 all delayed alerts have been delivered and alert processing has returned to normal.
Report: "Login failures for web site"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Report: "Increased Error rates when viewing data in a 3 day window or longer"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.