Historical record of incidents for Huntress
Report: "ITDR Processing Delay"
Last updateWe are currently investigating this issue.
Report: "Database Maintenance"
Last updateThe scheduled maintenance has been completed.
Verification is currently underway for the maintenance items.
Scheduled maintenance is currently in progress. We will provide updates as necessary.
We will be undergoing scheduled maintenance during this time.
Report: "Backend Processing Latencies"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently observing increased latencies in backend data processing queues that may impact timely detections and reporting
Report: "Portal Web Errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "502 errors in production"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Portal Outage"
Last updateA system failover occurred at ~2:15pm ET resulting in a period of downtime. Systems are fully operational again.
Report: "Increased timeout rate"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Portal Degradation"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Report: "Portal Frontend Errors"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Report: "SIEM 500 Errors"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
Some users are getting 500 errors when checking pages within the Portal. We are actively investigating the cause.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Portal Timeouts"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Increased Error Rate"
Last updateThis incident has been resolved.
Errors have subsided. Engineers are investigating further.
We are currently investigating this issue.
Report: "Portal Degraded"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
The Portal is experiencing a higher-than-normal error rate. Staff are currently investigating.
Report: "Portal Degradation"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
A core upstream cloud provider is experiencing an outage impacting portal availability.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Huntress.io performance issues"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Increased Error Rate Due to Timeouts"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Report: "Increased Portal Latencies for Non-US Users"
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Portal UI Outage"
Last updateThis incident has been resolved.
We are currently investigating this issue.
Report: "Huntress Rio Agent"
Last updateThis incident has been resolved. Rollback was completed.
What is happening? Huntress is aware of an issue affecting the Rio agent, versions 0.5.11 & 0.5.12. The issue is causing slowness on RDS servers and some Box Drive interactions. Next steps: We'll be rolling these versions back to 0.5.7. This rollback is automatic and should be complete for everyone by end of day today. In parallel, we will continue to investigate the issue and look to provide a fix in a future release.
Report: "Huntress.io performance issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Huntress.io"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Portal 502s"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Portal Degradation"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "M365 Outage: May 2, 2024"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
Earlier today, Huntress experienced a partial outage of detection for Huntress MDR for Microsoft 365. What is the issue? This morning, Thursday, May 2, at approximately 11:21 AM ET a code change was deployed that broke ingestion of events into our detection engines. This resulted in a temporary outage in our detection capabilities that lasted until 1:33 PM ET. What is Huntress doing to rectify the situation? The root cause issue has been addressed and normal operation has been restored; we reversed the code change and new events are now being detected normally. During the outage period, some detections were missed. Our team is working to re-process the missed events to ensure that all detections are delivered. Further updates will be posted here. If you have additional questions about the activity, please contact Huntress customer support.
Report: "Huntress.io performance issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Huntress Rio Service "Stop Pending" state"
Last updateUPDATE: We have pushed the fix to the known impacted endpoints, and are continuing to finish the rest of the rollout of the update. We are going to resolve this issue since impacted endpoints have received the new update.
We are continuing to monitor for any further issues.
UPDATE: We have pushed the fix to the known impacted endpoints. We will continue to roll out the update to all remaining endpoints and monitor the progress. No action is needed on your end.
UPDATE: We have identified the root cause of the outage impacting Huntress Managed EDR and have begun pushing out an update to address the problem. No action is needed on your end. We are pushing the fix to known impacted endpoints first and will continue updating all endpoints on a rolling basis afterward. Please continue to follow this page for more updates. The issue was caused due to the Huntress Rio service (which gathers telemetry for our EDR process insights feature) being stuck in a “stop pending” state. The update being pushed will ensure that any hung service is killed and restarted on the new version.
Huntress is currently experiencing an outage that is impacting Huntress Managed EDR. Currently, a portion of our agent population is experiencing an outage where the Huntress SOC cannot receive some EDR data. The issue is due to the “Huntress Rio” service being stuck in a “stop pending” state. As a result, the agent is not collecting “process insights” data and the SOC cannot analyze related activity. Other detection capabilities are still functioning but this data source is an important one and we are taking this outage extremely seriously. The issue occurs on shutdown or restart of the service on systems with a lot of event log activity, primarily servers. We pushed out an agent version last night that attempted to resolve the problem, but was unsuccessful. We are currently preparing another version that will correct that, which we expect to send to the specific systems impacted by the end of the day today. We will post updates here as additional information is available.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
We are experiencing data processing delays which may cause slowness in the Dashboard.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
You may experience some slowness while we're running through a deployment.
Report: "Survey Delays"
Last updateThis incident has been resolved.
The Huntress Console is currently experiencing a backup in survey processing. Data you see on the dashboard could be slightly outdated.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We've identified a backend issue that is causing timeouts for some users. Try refreshing the page if this happens.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
Report: "Systems Down"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Access to huntress.io is limited and agent deployments using scripts cannot be performed at this time. Installing the Agent via the installer will still work as expected.
Report: "Degraded Performance and Delayed Processing"
Last updateThis incident has been resolved.
Our backend is experiencing processing delays. It may take longer than normal for new agents to call back and there are delays in running custom reports.
Report: "Huntress - Down"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
We've identified a backend issue that may cause timeouts or latency for some users. Once this is resolved we will post an updated status.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We've identified a backend issue that may cause timeouts or latency for some users. Once this is resolved we will post an updated status.
We've identified a backend issue that may cause timeouts or latency for some users. Once this is resolved we will post an updated status.
We've identified a backend issue that may cause timeouts or latency for some users. Once this is resolved we will post an updated status.
Report: "Huntress Incident"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are continuing to troubleshoot the issue.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified the issue and are working on resolving it.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "Degraded Performance"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Huntress Outage"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Huntress Portal down"
Last updateThis incident has been resolved.
The dashboard is back online and we are monitoring performance
We are currently investigating this issue.
Report: "Portal Incident"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Huntress Portal down"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Log in issues with session timeouts for SAT"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Operational issues reported by AWS"
Last updateThis incident has been resolved.
Things look to be operational at this time. We will continue to monitor.
The issue has been identified as an operational issues reported by AWS
The issue has been identified as an operational issues reported by AWS
We are currently investigating this issue.
Report: "Security Awareness Training Login issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Huntress.io latency"
Last updateThis incident has been resolved.
Some users may experience latency when accessing and using the Huntress.io. We are monitoring the issue, and will update this page once the issue is resolved.
Some users may experience latency when accessing and using the Huntress.io. We are investigating the issue, and will update this page once the issue is resolved.
Report: "incident-20231002"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.