Voyado Incident History

1. resolvedOct 21 · 10:06 UTC

   A fix was deployed and confirmed working. Service is restored.

Reported by Voyado on their status page.

1. postmortemOct 22 · 08:09 UTC

   Summary

   On October 19, between approximately 11:05 and 12:00 CEST, the Voyado Engage APIs experienced degraded performance and some API requests failed, this was due to an infrastructure issue affecting our cloud provider. The issue caused increased response times across our API services, which were further amplified by an internal traffic management mechanism. We identified the contributing factors and took action to restore normal performance.

   Customer impact

   During the incident, customers experienced slower response times when making requests to the Voyado Engage APIs. Some customers also experienced failed requests, with temporary errors (503). Performance was back to normal at about 12:00 CEST.

   Other platform functionality was confirmed not to be affected.

   Root cause

   The incident was initially triggered by an infrastructure issue at our cloud provider, which caused reduced performance across multiple services simultaneously. This triggered an internal traffic management mechanism designed to balance load and isolate unusually slow API traffic to protect overall platform performance. The mechanism did not manage the situation as intended which caused delays and some failed API requests for affected customers.

   Mitigation

   As part of our mitigation efforts, we increased the capacity available to handle the redirected API traffic. Alongside the recovery of the underlying cloud infrastructure, this helped restore normal API performance by approximately 12:00 CEST on October 19.

   We continued monitoring the platform to ensure stability before formally closing the incident on October 20.

   Next steps

   To reduce the risk of similar incidents, we are reviewing and taking appopriate actions of the following:

   • Review and enhance how our traffic management mechanism responds to platform-wide performance degradation.
   • Expand the capacity available to handle redirected API traffic to improve resilience during unexpected load.
   • Improve our ability to detect unusual traffic patterns and infrastructure-related performance issues sooner.

   We remain committed to improving the reliability and resilience of Voyado Engage and appreciate your patience and understanding. We apologize for the disruption this caused.

Reported by Voyado on their status page.

1. postmortemOct 9 · 12:18 UTC

   Date: October 7, 2026
   Duration: 14:13–15:32 CEST (approximately 1 hour and 20 minutes)
   Affected service: Engage Web Application

   Summary

   On October 7, 2026, between approximately 14:13 and 15:32 CEST, the Voyado Engage web application was unavailable to customers. During this period, users were unable to log in or access the web application.

   Our team identified the cause and restored service at 15:32 CEST. Following a period of monitoring, the incident was confirmed as resolved.

   Customer impact

   The incident affected access to the Engage web application for nearly all production environments. Users attempting to access the application experienced continuous loading, preventing them from logging in or using the interface.

   The partner portal remained accessible, and no data was lost during the incident.

   Root cause

   The incident was caused by a software update that introduced an issue with how the web application handles security tokens used during user sessions.

   Mitigation

   Once the issue was identified, our development team initiated a new deployment that removed the problematic change.

   In parallel, we restored a previously working version of the affected application components to accelerate recovery. This allowed functionality to return progressively while the updated deployment was completed.

   Full access was confirmed restored at approximately 15:32 CEST.

   Next steps

   To reduce the risk of similar incidents and improve our ability to recover quickly, we are focusing on the following improvements:

   • Faster recovery: Improve our deployment processes to enable quicker rollbacks to previously working application versions.
   • Improved monitoring: Introduce automated checks for login and application loading to help detect issues before customers are affected.
   • Enhanced testing: Strengthen testing procedures to better cover differences between production and testing environments.

   We sincerely apologize for the disruption this caused and appreciate your patience while we worked to restore the service.

Reported by Voyado on their status page.

1. postmortemOct 13 · 07:52 UTC

   Summary

   Between 5 October and 6 October 2026, processing of some events in Voyado Engage was delayed. The cause was an internal update to one of our background processes, which unexpectedly triggered a large amount of extra processing. The issue was limited to this process and has been corrected.

   Customer Impact

   Automations triggered by contact interactions ran later than normal. Webhooks for contact, consent and preferences changes were delayed as well.

   Root Cause and Mitigation

   A safeguard in the update did not work as intended, so existing data was processed again as if it were new. This happened on two occasions, 5 and 6 October. Our engineering teams stopped the process, cleared the extra processing and confirmed that everything was back to normal.

   Next Steps

   We are adding stronger safeguards and checks for this type of update, as well as earlier alerts so we can react faster if something similar happens. We apologize for the disruption and appreciate your patience.

Reported by Voyado on their status page.