Red Canary

Is Red Canary Down Right Now? Check if there is a current outage ongoing.

Red Canary is currently Operational

Last checked from Red Canary's official status page

Historical record of incidents for Red Canary

Report: "Investigating issues with Google Cloud Platform and Palo Alto Cortex"

Last update
investigating

We are continuing to investigate this issue.

investigating

We are investigating delayed data from Palo Alto Cortex and Google Cloud Platform that may result in delayed detections until these issues are resolved. Additional status may be obtained from our partner's status pages: Palo Alto Cortex: https://rootly.com/external-status-pages/palo-alto-networks Google Cloud Platform: https://rootly.com/external-status-pages/palo-alto-networks

Report: "Increased number of failing system status checks"

Last update
investigating

We are currently investigating an increased number of failing system status checks across multiple customers.

Report: "Investigating issues with VMware Carbon Black Cloud"

Last update
investigating

We are investigating authentication issues with VMware Carbon Black (hosted by VMware Carbon Black). We have notified the VMware Carbon Black production engineering team and are working with them to ensure they resolve the issue promptly. Active Remediation response actions may be delayed.

Report: "Investigating issues with VMware Carbon Black Cloud"

Last update
investigating

We are investigating authentication issues with VMware Carbon Black (hosted by VMware Carbon Black). We have notified the VMware Carbon Black production engineering team and are working with them to ensure they resolve the issue promptly. Active Remediation response actions may be delayed.

Report: "Investigating issues with VMware Carbon Black Cloud"

Last update
Investigating

We are investigating authentication issues with VMware Carbon Black (hosted by VMware Carbon Black). We have notified the VMware Carbon Black production engineering team and are working with them to ensure they resolve the issue promptly.Active Remediation response actions may be delayed.

Report: "Ingestion Delays for SentinelOne"

Last update
resolved

This incident has been resolved.

monitoring

We are monitoring delayed data from the earlier SentinelOne outage. Detections may be delayed until the issue is resolved.

monitoring

We are monitoring a potential SentinelOne outage that is impacting telemetry ingestion for SentinelOne customers. Detections may be delayed until the issue is resolved.

Report: "Ingestion Delays for SentinelOne"

Last update
Resolved

This incident has been resolved.

Update

We are monitoring delayed data from the earlier SentinelOne outage. Detections may be delayed until the issue is resolved.

Monitoring

We are monitoring a potential SentinelOne outage that is impacting telemetry ingestion for SentinelOne customers. Detections may be delayed until the issue is resolved.

Report: "Delayed Detection for Identity and Cloud"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We have identified an issue that may lead to delayed detections for Identity and Cloud activity. We are currently remediating this issue and will provide updates as they become available.

Report: "Security Data Lake Ingestion Degradation"

Last update
resolved

This incident has been resolved.

identified

We have resolved the issue impacting data ingestion into the Security Data Lake. Data is being ingested as expected. Searches will only return data ingested prior to 7 AM MT on 4/21/2025.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating an issue impacting data ingestion into the Security Data Lake. There is no indication of data loss at this time. Searches will only return data ingested prior to 7 AM MT on 4/21/2025.

Report: "Delayed SentinelOne Telemetry Ingestion"

Last update
resolved

SentinelOne’s maintenance window is complete. We are ingesting telemetry from SentinelOne customers.

identified

We are currently monitoring a SentinelOne maintenance window which is impacting our ability to ingest telemetry for SentinelOne customers. Detection for these customers may be delayed until maintenance is complete.

identified

We are currently monitoring a SentinelOne maintenance window which is impacting our ability to ingest telemetry for SentinelOne customers. Detection for these customers may be delayed until maintenance is complete.

Report: "Ingestion Issues for Carbon Black Cloud Customers"

Last update
resolved

Broadcom's incident remains in a monitoring state and we have not observed any errors on our end over the past 24 hours.

monitoring

Broadcom's incident remains in a monitoring state and is being regularly checked for updates. Telemetry from Carbon Black Cloud is being successfully ingested and processed. We will provide additional updates when they are available.

monitoring

Broadcom's incident remains in a monitoring state and we have been successfully ingesting and processing telemetry from Carbon Black Cloud for the past 6 hours. We will continue to leave this incident open while we monitor Broadcom's status.

monitoring

Carbon Black Cloud has implemented a fix and they are monitoring the service for stability. Ingestion of Carbon Black Cloud telemetry is no longer delayed. We will leave this incident open until Broadcom resolves theirs.

monitoring

We are investigating ingestion issues with Carbon Black Cloud. We are monitoring the Carbon Black Cloud status page and will provide updates when they are available. Sensor telemetry, events, and detections may be delayed.

Report: "Alert Ingestion Delays Causing Failing Status Checks"

Last update
resolved

This incident has been resolved and all backlogged alerts processed.

monitoring

We have successfully deployed a fix for the alert ingestion issue and alerts are now being ingested. Detections may still be delayed while we process the backlog of failed alerts. In addition, status checks that appeared to fail during this incident will automatically resolve; no customer action is required. We will continue to monitor the situation and will close this incident once all backlogged alerts have been ingested.

monitoring

We have successfully deployed a fix for the alert ingestion issue and alerts are now being ingested. Detections may still be delayed while we process the backlog of failed alerts. In addition, status checks that appeared to fail during this incident will automatically resolve; no customer action is required. We will continue to monitor the situation and will close this incident once all backlogged alerts have been ingested.

identified

We have identified an issue causing some alerts to fail ingestion, which may result in failing status checks for certain customers. Our team is actively testing a fix and assessing its impact. No alert data has been lost and all failed alerts will be reprocessed once the fix is implemented. Please note that detections based on these alerts may be delayed until ingestion is fully restored. We appreciate your patience and will provide further updates as they become available.

Report: "Delayed Telemetry Processing for Multiple Telemetry Sources"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are continuing to investigate this issue.

investigating

We have identified an issue that is causing delayed telemetry processing for Carbon Black Cloud EDR, Microsoft Defender for Endpoint, Microsoft Office365, SentinelOne, CrowdStrike, TrendMicro, AWS, Microsoft Azure, Jamf, and Linux EDR. This delay in telemetry may cause potential delays in detecting threats based on this telemetry. We are investigating the cause of this issue currently and will provide updates as we know more and are able to take action to address the issue.

Report: "Tracking an external incident with Broadcom Carbon Black Cloud"

Last update
resolved

Broadcom's incident remains in a monitoring state and we have not observed any errors on our end over the past 24 hours.

monitoring

After a few false starts, Broadcom has shifted their incident to "Monitoring". We have been successfully ingesting and correlating alerts for the past 5 hours. We will leave this incident open until Broadcom resolves theirs.

identified

Broadcom Carbon Black continues to investigate this issue.

identified

We are tracking an outage with Broadcom Carbon Black Cloud (CbC) that is affecting alert ingest and correlation of alerts to endpoint data. https://status.broadcom.com/services/carbon-black/notices/kcjrp2fn7pniepsu-carbon-black-cloud-alerts-page-search-degraded-performance

Report: "Alert ingestion Issues for Microsoft Defender for Endpoint Customers"

Last update
resolved

This incident has been resolved.

investigating

We are continuing to monitor the status of this ingestion issue with Microsoft Graph V2 alerts. At this time there is no update.

investigating

We are investigating ingestion issues of alerts from Microsoft Graph V2. We have notified the Microsoft support team and are working with them to resolve the issue promptly. Process execution (EDR) telemetry is still being ingested and processed by Red Canary. Detection of threats from Microsoft Graph V2 alerts may be delayed.

Report: "Ingestion Delays for JAMF Telemetry"

Last update
resolved

We have confirmed that we are now ingesting telemetry from JAMF.

identified

We are currently monitoring an operational incident with Jamf (https://status.jamf.com/) that is impacting our ability to ingest telemetry for Jamf customers. Detection for these customers may be delayed until the issue is resolved. This incident will be updated as additional information becomes available from Jamf.

Report: "Detection delayed"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently experiencing processing delays, which may cause late detection notifications for some customers. We will provide updates as more information becomes available.

Report: "Web Portal / API request timeouts"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

investigating

We are investigating reports that Red Canary web portal and API requests are failing for some customers. We will update this page as we learn more.

Report: "Detection Delayed"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results. At this time detections may be delayed for some customers.

investigating

We are currently experiencing data processing delays, resulting in late detection notification to some customers. This incident will be updated as additional information becomes available.

Report: "Issues accessing the Red Canary portal"

Last update
resolved

This incident has been resolved.

identified

We are aware of an incident causing issues accessing the Red Canary web portal. We have identified the issue and are working on a fix. Data processing and detections are not affected at this time.

Report: "Web Portal / API request timeouts"

Last update
resolved

We continued to monitor the environment for 24 hours and are now confident the incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating reports that Red Canary web portal and API requests are failing for some customers. Detections may be delayed. We will update this page as we learn more.

Report: "Red Canary is not directly impacted by the CrowdStrike-caused outage"

Last update
resolved

This incident has been resolved.

monitoring

Red Canary is not directly impacted by the CrowdStrike-caused outage. We are monitoring customer environments and will receive telemetry from affected CrowdStrike endpoints as they come back online.

Report: "Detection delayed for Microsoft sources"

Last update
resolved

This incident has been resolved.

monitoring

Microsoft reports they have identified the issue and are actively applying mitigations. We are starting to see data flowing again. We will continue to monitor the situation and resolve this incident when we are confident services are fully restored.

investigating

We are currently experiencing data processing delays for Microsoft Defender for Endpoint and Microsoft Azure telemetry. This may result in late detection notifications to some customers. We are following a Microsoft incident as the likely cause. https://azure.status.microsoft/en-us/status This incident will be updated as additional information becomes available.

Report: "Reports of Endpoints Showing as Unmonitored"

Last update
resolved

The issue around endpoints incorrectly showing as unmonitored has been fixed and this incident is now resolved.

monitoring

A fix is being implemented and we are monitoring the results.

identified

We are currently investigating an issue where endpoints are showing as unmonitored. This is a display issue only. Telemetry ingestion, processing, analysis, and detection are all working as expected.

Report: "Detection delayed for Microsoft sources"

Last update
resolved

This incident has been resolved and all backlogged data has been processed.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are currently monitoring the results. No data has been lost and we are beginning to process the backlog at this time.

investigating

We are continuing to investigate this issue.

investigating

We are currently experiencing data processing delays for Microsoft Defender for Endpoint and Microsoft Azure telemetry. This may result in late detection notifications to some customers. This incident will be updated as additional information becomes available.

Report: "Delayed Ingestion"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results. At this time detections may be delayed for VMware Carbon Black Cloud.

identified

We have identified an issue with telemetry ingestion from VMware Carbon Black Cloud, Linux EDR, and Azure Cloud. A fix has been implemented. Detections may be delayed.

Report: "Disruption in Okta Telemetry Ingestion"

Last update
resolved

This incident has been resolved and Okta telemetry is again flowing as expected.

investigating

There is currently a disruption in our ability to ingest telemetry from Okta. We are investigating the cause of this disruption, but will not be able to ingest or investigate Okta telemetry or alerts until the issue is resolved. Updates will be provided as we look to implement a solution.

Report: "Delays Processing Telemetry"

Last update
resolved

We have identified the cause of the delays in processing telemetry and implemented a fix. This incident has been resolved.

identified

We have identified an issue that is causing delays in processing EDR telemetry. Users may experience delayed detections at this time. We will continue to investigate the cause of this delay and will post updates when a solution has been identified

Report: "Investigating delays with AWS Cloud Control Plane telemetry"

Last update
resolved

The issue has been identified and a fix has been implemented.

investigating

We are investigating ingestion issues with AWS Cloud Control Plane telemetry. Telemetry, events, and detections are delayed.

Report: "Investigating issues with CrowdStrike"

Last update
resolved

Monitoring has confirmed the system is operating normally and detections are no longer delayed.

investigating

We are investigating ingestion issues with CrowdStrike (hosted by CrowdStrike). We have notified the CrowdStrike production engineering team and are working with them to ensure they resolve the issue promptly. Sensor telemetry, events, and detections may be delayed in our Web Portal, APIs, and systems that leverage those APIs such as Canary Exporter. New sensor connections and installations may be impacted.

Report: "Red Canary Platform Processing Time"

Last update
resolved

This incident has been resolved.

investigating

We are currently working on a platform infrastructure issue impacting processing time. Detections could be delayed.

Report: "Analysis notes were not added to external alerts between 3/5 18:15 and 23:00 UTC"

Last update
resolved

Analysis notes were not added to external alerts processed by Red Canary for alerts processed on Tuesday March 5 between 18:15 and 23:00 UTC. The issue has been corrected and analysis notes are being successfully added.

Report: "Subset of Alert Sources Failing to Ingest"

Last update
resolved

This incident has been resolved and all alert source data is being ingested.

identified

The issue has been identified and a fix is being implemented.

Report: "Subset of Alert Sources Failing to Ingest"

Last update
resolved

This incident has been resolved and all alert source data is being ingested.

investigating

We are currently investigating reports of a subset of alert sources failing to ingest. We are actively investigating scope and cause.

Report: "Activity Feeds Showing Erroneous Information"

Last update
resolved

This incident has been resolved.

identified

We are aware of an issue and are working on a solution where Activity Feeds are erroneously displaying "Profile Published" for every published intelligence profile.

Report: "Issues with Multi-factor Authentication Setup"

Last update
resolved

This incident has been resolved.

identified

We have identified the root cause of the incident. A third party integration used to display TOPT QR codes is no longer working. We are actively working on a fix and will have an update posted here as soon as possible.

investigating

We are currently investigating reports of issues setting up multi-factor authentication for Red Canary portal users.

Report: "Subset of Alert Sources Failing to Ingest"

Last update
resolved

This incident has been resolved and all alert source data is being ingested.

monitoring

The issue has been identified and fixed. We are currently monitoring a backlog of alerts being ingested. No data has been lost.

investigating

We are currently investigating reports of a subset of alert sources failing to ingest. We are actively investigating scope and cause.

Report: "Ingestion Issues for VMware Carbon Black Cloud customers"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are investigating ingestion issues with VMware Carbon Black Cloud. We have notified the VMware Carbon Black Cloud support team and are working with them to resolve the issue promptly. Events and detections may be delayed.

Report: "Delay in processing identity telemetry"

Last update
resolved

The backlog has been processed and identity data is flowing normally.

monitoring

We are currently processing a backlog of identity telemetry. We have addressed the root cause of the spike in data and are monitoring the system. Some threats based on identity telemetry may be delayed.

Report: "Disruption in Data Ingestion from Cortex XDR"

Last update
resolved

This incident has been resolved.

investigating

We are currently investigating a disruption in data ingestion from Palo Alto Cortex XDR. Customers using Cortex with Red Canary will experience delays in telemetry and alert ingestion. We are prioritizing this issue and will provide updates until it is resolved.

Report: "Disruption in Phone-based Automate Actions and Notifications"

Last update
resolved

The incident has been resolved.

identified

We are continuing to work on a fix for this issue.

identified

We have identified the issue causing this outage and are in the process of implementing a fix. We will provide updates once we have confirmed resolution.

investigating

We are currently investigating reports of deliverability failures for phone/voice and SMS notifications from Red Canary. We will provide more details as soon as more information becomes available.

Report: "Investigating issues with Crowdstrike"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating ingestion issues with CrowdStrike (hosted by CrowdStrike). We have notified the CrowdStrike production engineering team and are working with them to ensure they resolve the issue promptly. Sensor telemetry, events, and detections may be delayed in our Web Portal, APIs, and systems that leverage those APIs such as Canary Exporter. New sensor connections and installations may be impacted.

Report: "Detection delayed for VMware Carbon Black Hosted EDR customers"

Last update
resolved

This incident has been resolved.

monitoring

AWS is continuing to make progress on this incident and is continuing to work through backlogged events. Red Canary continues with normal operations and will provide an update when new information is available.

monitoring

AWS is continuing to make progress on this incident. Red Canary continues to operate normal operations and will provide an update when new information is available.

monitoring

AWS is now reporting processing at full throughput, though the incident is ongoing. Red Canary continues to operate normal operations and will provide an update when new information is available.

monitoring

The AWS incident is ongoing and they are taking measures to mitigate the operational impact. We are experiencing no significant disruption at this time, and will continue to monitor the situation. This incident will be updated as additional information becomes available.

monitoring

Ingest, processing, and detections have returned to normal and are no longer delayed for VMware Carbon Black Hosted EDR customers. We are continuing to monitor the situation and the AWS incident.

monitoring

Ingest, processing, and detections have returned to normal and are no longer delayed for VMware Carbon Black Hosted EDR customers. We are continuing to monitor the situation and the AWS incident.

monitoring

Ingest, processing, and detections have returned to normal and are no longer delayed for VMware Carbon Black Hosted EDR customers. We are continuing to monitor the situation and the AWS incident.

monitoring

After the most recent AWS fix our ingest, processing, and detections have returned to normal and are no longer delayed for VMware Carbon Black Hosted EDR customers. We will continue to monitor the system and the AWS incident and update this page should anything change.

monitoring

AWS has implemented a fix and we are continuing to monitor the system to confirm it is returning to normal operations.

monitoring

AWS has implemented a fix and we are now starting to see queued telemetry. We are monitoring the system to confirm it is returning to normal operations.

identified

We are continuing to monitor the AWS incident affecting ingestion for VMWare Carbon Black Response customers. AWS is currently isolating the root cause of the issue, as well as taking mitigating actions. We will continue to monitor this situation and keep you updated as more information becomes available.

identified

At approximately 03:50 MST / 09:50 UTC, we became aware of an operational incident with Amazon Web Services (AWS) and subsequently determined it is affecting ingest for our VMware Carbon Black Hosted EDR customers. Detection for these customers may be delayed until the issue is resolved by AWS. No telemetry has been lost and we will process all queued telemetry automatically once the AWS service health improves. This incident will be updated as additional information becomes available from AWS.

Report: "Delayed Ingestion"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are continuing to work on a fix for this issue.

identified

We are continuing to work on a fix for this issue.

identified

We are continuing to work on a fix for this issue.

identified

We have identified the issue with Linux EDR and are working to implement a fix. Some detections may be delayed.

investigating

We are investigating ingestion issues with Linux EDR hosted by Red Canary. Detections may be delayed. We have notified the Linux EDR production engineering team and are working with them to ensure they resolve the issue promptly.

Report: "Monitoring Degraded Performance"

Last update
resolved

This incident has been resolved and all data is current.

monitoring

VMWare Carbon Black Cloud alert source correlation is in process and we are monitoring to its completion.

monitoring

All data has been ingested and is current. With the exception of VMware Carbon Black Cloud alert sources, all process correlation is current as well. We are continuing to monitor the progress of VMware Carbon Black Cloud correlation.

monitoring

All detection data is caught up and current. We are continuing to monitor the recovery of external alert data processing.

monitoring

We are continuing to monitor the recovery of this incident.

monitoring

We have identified the issue and are monitoring the recovery progress.

investigating

We are actively monitoring degraded performance issues. Some detections may be delayed.

Report: "Ingestion Issues for Microsoft Defender for Endpoint Customers"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

Telemetry is flowing normally again. We are monitoring the system while replaying any messages left in queue.

investigating

We are investigating ingestion issues with Microsoft Defender for Endpoint. Sensor telemetry, events, and detections may be delayed.

Report: "Investigating issues with Microsoft Defender"

Last update
resolved

Monitoring has confirmed the system is operating normally and detections are no longer delayed.

investigating

We are investigating ingestion issues with Microsoft Defender (hosted by Microsoft Defender). We have notified the Microsoft Defender production engineering team and are working with them to ensure they resolve the issue promptly. Sensor telemetry, events, and detections may be delayed in our Web Portal, APIs, and systems that leverage those APIs such as Canary Exporter. New sensor connections and installations may be impacted.

Report: "Monitoring degraded performance"

Last update
resolved

This incident has been resolved.

monitoring

We are actively monitoring degraded performance issues. Some detections may be delayed.

Report: "Ingestion Issues for SentinelOne Customers"

Last update
resolved

This incident has been resolved.

investigating

We are investigating ingestion issues with SentinelOne. We have notified the SentinelOne Support Team and are working with them to resolve the issue promptly. Sensor telemetry, events, and detections may be delayed.

Report: "Ingestion Issues for Microsoft Defender for Endpoint Customers"

Last update
resolved

Microsoft Defender has reported that the issue has been resolved. We have confirmed that telemetry flow has returned to normal.

investigating

We are continuing to work with Microsoft to investigate this issue.

investigating

We are investigating ingestion issues with Microsoft Defender for Endpoint (hosted by Microsoft Defender). We have notified the Microsoft Defender for Endpoint support team and are working with them to resolve the issue promptly. Sensor telemetry, events, and detections may be delayed.

Report: "Data processing backlog may contribute to detection delays"

Last update
resolved

This incident has been resolved.

monitoring

We are currently processing a burst of data that may contribute to detection delays for some customers. We are monitoring the health of the system and will resolve this incident when the increased volume had been processed.

Report: "License Page Error for a Subset of Customers"

Last update
resolved

This incident has been resolved.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating an issue that is causing an error for the licensing page for a subset of customers.

Report: ""Leads Investigated" Counts are Displaying Inaccurately"

Last update
resolved

This incident has been resolved.

identified

We are aware of and working to resolve an issue where the "Leads Investigated" counts in the "by the numbers" section of the Red Canary dashboard is displaying inaccurately. There is no data anomaly; this is a display issue only.

Report: "Investigating Data Ingestion issue affecting a Subset of SentinelOne Customers with Cloud Funnel V2"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

monitoring

We are observing that telemetry has been restored in some of the affected systems and are continuing to monitor.

investigating

We continue to investigate the issue with the SentinelOne team.

investigating

We are investigating ingestion issues with SentinelOne (hosted by SentinelOne). We have notified the SentinelOne production engineering team and are working with them to ensure they resolve the issue promptly. Sensor telemetry, events, and detections may be delayed in our Web Portal, APIs, and systems that leverage those APIs such as Canary Exporter. New sensor connections and installations may be impacted.

Report: "Investigating Issue with Status Checks Displaying Unhealthy in the Web Portal for CrowdStrike customers."

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

investigating

We are continuing to investigate the issue.

investigating

We are currently investigating issues with Status Checks displaying Unhealthy in the Web Portal for CrowdStrike customers.