Historical record of incidents for Anodot Revenue
Report: "US Region - Errors in Processing Incoming Metrics - Data Loss with High Probability"
Last updateThis incident has been resolved.
We has an issue that caused missing a 4 hours of data. It may have caused many false No-Data alerts. A fix has been implemented and we are monitoring the results.
Report: "US Region - Errors in Processing Incoming Metrics - Data Loss with High Probability"
Last updateThis incident has been resolved.
We has an issue that caused missing a 4 hours of data. It may have caused many false No-Data alerts. A fix has been implemented and we are monitoring the results.
Report: "US Region - Errors in Processing Incoming Metrics - Data Loss with High Probability"
Last updateThis incident has been resolved.
Incident Resolved
A fix has been implemented and we are monitoring the results.
We're Investigating the issue
Report: "Elevated API Errors"
Last updateWe're experiencing an elevated level of data API errors and are currently looking into the issue.
Report: "UI intermittently unavailable in US zone"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "US Region - Anomaly Detection and Anomaly Alerts unstable"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
Anomaly detection and anomaly alerting in US region are currently unstable, we've identified the issue and are working to resolve it
Report: "US Region | Some Metrics Are Loosing Samples"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We're currently investigating an issue where small portion of metrics (the same constant metrics) are experiencing sample loss. As a result false-positive no-data alerts can be triggered and the metrics data shall not be shown in UI.
Report: "Some Alerts Went Missing During Last 24 Hours"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
During the last 24 hours some alerts may have been missed by the system, we're deploying a hotfix in an hour time
Report: "US Cluster | Degraded operation of multiple services, probability of minor data loss"
Last updateThis incident has been resolved.
We're experiencing connectivity issues since 5AM UTC time, this may result in degraded performance and reduced accuracy of alerts and composite calculations. There is also a minor chance of sample data loss. We have identified the problem and applied the necessary measures and are monitoring the system state.
Report: "BigQuery stream does not produce data"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Streams working with BigQuery does not retrieve new data, seems to be related to incompatible changes in BigQuery - Investigating. This is a cross region issue that impact all of our clusters.
Report: "US Region - Errors in Processing Incoming Metrics - Data Loss with High Probability"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Around 05:00 UTC we detected errors in samples receiving pipeline, issue is under investigation. Please be advised that at least some of the incoming metrics data was lost and false-positive alerts can be triggered
Report: "Mumbai site is not accessible"
Last updateThis incident has been resolved.
We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.
Report: "Alert Notifications via Slack Channels Are Not Being Sent"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We've identified the issue and working on a fix. Please note that alerts console is fully functional and no events are lost.
Report: "Processing of Anomaly Alerts is currently impared for customers of "in.anodot.com" cluster (Mumbai AZ)"
Last updateThe incident was resolved. Alerts dispatching got delayed but no data was lost.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Alert Processing Delays In US-EAST Region (app.anodot.com)"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
Alerts is now back to normal and we're investigating the root cause
Our alert data processing infrastructure is running behind which is causing delays in sending alerts. No data has been lost and the system should be caught up shortly.
Report: "Alerts dispatch through channels does not operate in Frankfurt, Mumbai, Sydney and Oregon data-centers"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and applied globally, additional monitoring has been applied and we'll be monitoring it over the next few days
Mitigation identified and is being rolled out to production clusters
We are working on a mitigation with an ETA of 1-2 hours.
Report: "US Availability Zone - Some Segment Collector samples are lost"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Report: "Alerts Management Page Shows No Alerts - Only Display Is Impacted, Alerts are Intact"
Last updateThis incident has been resolved.
The Alerts Management page does not show the configured alerts, this is UI-only issue, the configurations are intact and alerts are operational
Report: "Business Collectors - US Region - Creation of New Streams May Fail"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We're experiencing an issue with creation and initialization of new streams. The existing streams are operating correctly
Report: "Data collectors issue in data center us-east-1"
Last updateThis incident has been resolved.
We identified a lag in kinesis and mparticle streams, investigating.
Some customers may have received false system notification on missing data There was no data loss or any impact on the service.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Data collectors page not available in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "US AZ - Possible Sample Loss in Calculation of Composite Metrics"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Report: "Degraded performance of UI in US region"
Last updateThis incident has been resolved.
We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.
Report: "UI Slowness in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are experiencing high error rate with one of our backend data bases, alerts are expected to be delayed
We are continuing to investigate this issue.
We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.
Report: "Some Customers In US AZ May Experience Static Alert Processing Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Our static alert processing infrastructure is running behind which is causing delays in sending alerts for a small portion of our customers. No data has been lost and we're working to restore system to full performance.
Report: "The UI is non-responsive in US region"
Last updateThis incident has been resolved.
We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.
Report: "Alerts Notifications May Be Delayed for US-Cluster Based Customers"
Last updateThe incident was resolved
Currently only static and no-data alerts are affected
Since 03 January 2022 around 19:00 UTC alert notifications may be delayed
Report: "Failures in creating new alerts [Data Center - US]"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We see an increase of errors when creating new alerts in the US data center.
Report: "Security notification - CVE-2021-44228 (log4j)"
Last updateUpdate regarding log4j vulnerability CVE-2021-44228 Anodot products are not affected by this vulnerability
Report: "UI not accessible in Frankfurt DC"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.
Report: "Elevated API Errors"
Last updateWe're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Elevated API Errors in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Elevated API Errors in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
The system is back to normal and we are monitoring it.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Issue with alerts on composite metrics in US east"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are experiencing an issue affecting alerts on composite metrics. Alerts may not be sent during this issue for those metrics.
Report: "Elevated API Errors for Kinesis in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Elevated API Errors for Kinesis in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Elevated API Errors for Kinesis in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Login page security error - Datacenter US"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Google safe browsing marked app.anodot.com as a deceptive site. We are working with google support to resolve this issue.
Report: "Elevated API Errors in US data center"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Data Processing Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Our data processing infrastructure is running behind which is causing delays in showing graphs and sending alerts. No data has been lost and the system should be caught up shortly.
Report: "Major service outage int US availability zone"
Last updateThe incident was resolved
We're experiencing a major outage in US availability zone, samples are possibly lost and alerts are suspended
Report: "US region: Possible duplicate alerts or alerts on anomalies that were already closed in the past"
Last updateThere might be a momentarily spike of notifications about alerts that were already closed in the past, the issue was already resolved. The time of incident is around 13:30 UTC.
Report: "Application not responsive in US"
Last updateThis incident has been resolved.
Alerts are back, some customers may see a delay in alerts.
UI is back and we are monitoring the fix. Alert should be back soon as well.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.
Report: "Data Processing Delays and Minor Samples Loss"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
Our data processing infrastructure is running behind which is causing delays in showing graphs and sending alerts. Some samples may have been lost
Report: "Elevated API Errors"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Issue with anomalies"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are experiencing an issue affecting alerts on anomalies. Alerts may not be sent during this issue for those metrics.
Report: "Elevated Errors in the US east Datacenter"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
This incident has been resolved.
We are currently investigating this issue.
Report: "Issue with alerts on composite metrics"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are experiencing an issue affecting alerts on composite metrics in Mumbai data center. Alerts may not be sent during this issue for those metrics.
Report: "Data Processing Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
Our data processing infrastructure is running behind which is causing delays in showing graphs and sending alerts.
Report: "Elevated API Errors"
Last updateThis issue has been resolved! Who affected: Customers in the US data center who are using mparticle data streams
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Search Delays In Sydeny"
Last updateThis incident has been resolved.
We are experiencing issues with the application effecting the ability to visualize metrics and anomalies.