Historical record of incidents for Optimove Service
Report: "Optimail visual editor template - Impacting saving, duplicating, or moving templates"
Last updateWe received reports of increased issues with some Optimal templates with the visual editor, and will continue investigating the issue. This does not impact existing campaign execution and only limits template edits and creation via the visual editor.
Report: "Login Issues for Some Users"
Last updateWe are currently investigating an issue where some users are unable to log in. Our team is actively working to identify the cause and implement a fix. We will share more updates as they become available. Thank you for your patience.
Report: "Batch Update Process - Delay in Campaign execution"
Last updateWe are investigating elevated errors in our batch update process. Specifically, we are facing an issue with one of our cloud-based data warehouses. This issue is affecting a significant number of clients across both the US and EU regions who are on the same data warehouse. As a result, scheduled campaign executions may experience delays during batch processing.
Report: "DNS Resolution Issue Affecting Multiple Services"
Last updateWe are continuing to investigate this issue.
We are investigating an increased error rate while accessing the Core Application and executing the Batch process.
Report: "US | Urgent maintenance | Batch process"
Last updateThe scheduled maintenance has been completed.
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Optimove infrastructure is being updated to enhance the reliability of the batch process.During this upgrade, some Optimove Batch services will have degraded performance for up to 60 minutes, leading to possible delays in completing the batch process.
Report: "EU - Delay in Campaign execution"
Last updateThis incident has been resolved.
We are monitoring a fix released to address elevated errors in our Scheduled Campaign Execution. Some tenants will experience delays in scheduled campaign execution as part of the batch process.
Report: "EU - Batch Update Process - Delay In Data Import Leading To Delays In Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The Fix is ready and is in testing stages - We expect a delivery ETA of 15 minutes that will allow our teams to restart the Batch process for impacted tenants.
We are still working on a fix. Additional updates within 20 minutes.
The issue has been identified and a fix is being implemented.
We are investigating elevated errors in our batch update process. Specifically, we are facing an issue with the Data importing service on the optimove side. This is causing delays in batch data processing. Possible impact: Some tenants will experience up to 2 hours delays in scheduled campaign execution as part of the batch process.
Report: "EU - Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We have identified elevated errors in our batch update process. Specifically, we are facing an issue with the Import and execution phase of the batch process. Possible impact: Some tenants will experience delays in scheduled campaign execution as part of the batch process. Workarounds: 1. Manually execute the most critical campaigns via the Optimove site with the latest available data Optimove has. 2. If you want to execute all the campaigns with the available data, please let us know, and we will run them all (reach out to Support@optimove.com).
Report: "US | Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and Batch processes are back in progress.
The issue has been identified, and we are implementing a fix to address the batch process. Once a fix is available, our teams will recover the impacted batch process
We are investigating elevated errors in our batch update process. Specifically, we are facing an issue with the cloud data import service. Possible impact: Some tenants will experience delays in scheduled campaign execution as part of the batch process.
Report: "Opti X - Smart Campaigns - High error rate and execution failures between April 11th - April 21th"
Last updateOpti-X Smart campaigns service encountered unexpected errors between April 11th 12:30PM UTC - April 21th 15:10 PM UTC. Note: Scheduled campaigns and normal triggered campaigns were not impacted - remained fully functional
Report: "Elevated errors experienced for campaign execution."
Last updateThe issue has been successfully resolved. However, we were unable to execute the campaign today for certain tenants who do not have the Force Run option enabled, and the scheduled time for the campaign has now passed. Should you wish to proceed with the campaign, please do not hesitate to initiate it at your convenience.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We have identified the issue impacting a specific subset of accounts and are presently engaged in recovery efforts.
We are currently examining the elevated errors in campaign execution across multiple accounts and are investigating the issues being experienced.
Report: "US - Batch Update Process Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating elevated errors in our batch update process processing. Specifically, we are facing an issue with the ETL service. Sites that have not yet started or completed the Batch process for today might be impacted in our US region. Impact: Impacted tenants will experience delays in scheduled campaign execution as part of the batch process. Delays are up to 120 minutes.
Report: "Some instances of Run Anyway Campaigns does not run on schedule."
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We have identified the root cause of the issue and have implemented an interim solution to minimize its impact. Our team is actively working on a permanent resolution, and we will provide further updates as progress continues. Thank you for your patience and understanding.
We are investigating instances where some Run Anyway campaigns have remained scheduled and require manual intervention. We appreciate your patience and understanding as we work to resolve this matter.
Report: "Optimove Site - Mission control - Future Run Anyway Campaigns in 'Running' status"
Last updateThis incident has been resolved.
A fix has been implemented, and the status of "Run Anyway" Campaigns are back to the expected "Scheduled" state. We are monitoring the results
We are investigating an issue affecting the Optimove site and UI. Future Run Anyway Campaigns in 'Running' status. This incident is limited to UI and site functionality Batch process, Campaign execution, and API access is fully functional
Report: "High error rate for our Optimail visual editor template issues which includes saving, duplicating, or moving templates."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified the root cause of the issue and are working on a fix. We apologize for any inconvenience this may have caused and will continue to update you on our status.
We received reports of increased issues with some Optimal templates with the visual editor and will continue investigating the issue.
Report: "EU - API and Batch Update Process - Delay in Campaign execution and 'POST' endpoints 503 errors"
Last updateThis incident has been resolved.
We identified and implemented a fix for elevated errors in our batch update process and API services. A fix has been implemented, and we are seeing the error rate drop as expected. We are still monitoring Batch process progress and API error rates. Possible impact: 1. Some tenants will experience delays in scheduled campaign execution as part of the batch process. 2. Some API endpoints returned 503 errors for "POST" calls
Report: "Delayed tracking of web and app activity, along with some experience issues related to the batch update process."
Last updateThis incident has been resolved.
We have identified the issue, implemented a fix, and are continuing to monitor the situation.
We are investigating elevated issues with our web and app tracking activity, as well as some batch update processes. Please note that some tenants will experience delays in scheduled campaign execution due to the batch process.
Report: "EU | Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
We believe that a resolution has been successfully implemented and are committed to closely monitoring the situation.
The issue was identified and a recovery process had already been started.
We are currently conducting an investigation into the recent increase in errors within our batch update process. As a result, some tenants may experience delays in the execution of scheduled campaigns that are part of this batch process.
Report: "US - Specifical channels - Google, Facebook, Braze - Delay in Campaign execution"
Last updateThis incident has been resolved.
We have resolved the issue, and it is presently under observation.
The issue has been identified, and our team is actively working to implement a solution.
We are investigating elevated errors in execution for a limited set of channels. Specifically, we are facing an issue with Facebook, Google, Braze, TikTok, AirShip, Snapchat
Report: "US - Batch Update Process And Real Time - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating elevated errors in our batch update processing and Real-Time Campaign isolation. Specifically, we are facing an issue with the target group isolation service. Possible impact: Some tenants will experience delays in scheduled and real-time campaign execution.
Report: "Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating elevated errors in our batch update process processing. Specifically, we are facing an issue with the data transfer service. Possible impact: Some tenants will experience delays in scheduled campaign execution as part of the batch process.
Report: "Optimove Site Notifications - Batch process update notifications not delivered"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating an issue with Batch process update notifications not being delivered. The notifications that are impacted include: Batch-Data Process Started - notifies the recipients when the Batch Data update started (on ETL start time). Batch-Data Process Completed - notifies the recipients when the batch-data update is completed successfully (on ETL completion time). Campaign Execution Started - notifies the recipients when the campaign execution process started. Campaign Execution Completed - Notifies the recipients when the campaign execution process is completed. Batch-Data Process Analytics Completed - Notifies the recipients when the entire process is completed. This only impacts site notifications and not campaign event notifications. More information about site notifications: https://academy.optimove.com/hc/en-us/articles/8662796504605-Site-System-Notifications
Report: "Optimove Site - UI - SIte unavailable"
Last updateOptimove site are unavailable. This incident is limited to UI and site functionality. Batch process, Campaign execution and API access are fully functional and unaffected.
Report: "EU | Batch Update Process - Delay in Campaign execution"
Last updateWe have identified and resolved an issue of elevated errors in the Batch Update Process. Possible impact: Some tenants experienced delays in scheduled campaign execution as part of the batch process.
Report: "EU | Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
We identified and are working to resolve elevated errors in our processing of the Batch Update Process Possible impact: Some tenants will experience delays in scheduled campaign execution as part of the batch process.
Report: "Elevated API Errors - Attribute updates via API calls - Update Customer Attributes"
Last updateThis incident has been resolved.
We're experiencing an elevated level of API errors for the specific end point "Update Customer Attributes" and are looking into the issue. "This attribute cannot be updated via this API. Use the next batch process..."
Report: "Optimove Internal Channels - Metrics Availability"
Last updateThe incident has been resolved and all metrics restored.
A fix has been implemented and we are monitoring the results.
We are currently experiencing elevated error rates in the Optimove Internal Channels Metrics Service (OptiMail, OptiMobile, Web-Push Notification, OptiText), leading to incomplete or missing data.
Report: "EU - Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are investigating elevated errors identified in our processing of the Batch Update Process leading to delays in campaign execution for some tenants.
Report: "Elevated API Errors - Resolved"
Last updateWe've experienced an elevated level of API errors and immediately rolled out a fix -outage time is evaluated at ~17 minutes
Report: "EU - Real Time Campaigns - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating elevated errors identified in our execution of Real-Time Campaigns, which are causing delays in campaign execution for some tenants.
Report: "Optimove Site - UI - Sign Out of the site after Export from Customer explorer"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
A fix is ready and going through the testing cycle to make sure the UI incident is resolved
We are continuing to work on a fix for this issue.
We identified the source of an issue affecting the Optimove site - exporting customer lists from the customer explorer leads to logging out of users. This incident is limited to UI and site functionality. The Batch process, Campaign execution, and API access are fully functional and unaffected.
We are investigating an issue affecting the Optimove site in exporting customer lists from the customer explorer. This incident is limited to UI and site functionality The Batch process, Campaign execution, and API access are fully functional and unaffected
Report: "Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
A fix is ready and being implemented with an ETA of 30 minutes for the next update
Update: the root cause was identified as a networking issue in a managed service used by Optimove batch process. We are working with our hosting provider to implement a solution. More updates will follow.
We are still working on a fix - no immediate ETA for resolution
The issue has been identified - we are working on a fix
We are investigating elevated errors identified in our processing of the Batch Update Process leading to delays in campaign execution for some tenants.
Report: "Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented, and batch updates are back on track. Some tenants might experience a delay of ~45 minutes in the completion of the Campaign Execution phase of the Batch update
We identified and are working to fix elevated errors identified in our processing of the Batch Update Process leading to delays in campaign execution for some tenants.
Report: "Campaign Event Notification Service - Delay in delivery of events to registered event listeners"
Last updateThis incident has been resolved.
We are investigating elevated errors in our processing of the campaign events notification (example: event type 1), which are leading to delays in campaign execution. This means delays in third-party API channels and conditional campaigns may have executed to no customers for conditional campaigns and third-party event subscription channels.
Report: "EU - Optimove Site - Unable to load some sites"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are investigating an issue affecting the Optimove site. This incident is limited to UI and site functionality Daily process, Campaign execution and API access is fully functional and unaffected
Report: "Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are investigating performance issues in the Batch Update Process, leading to delays in campaign execution for some tenants.
Report: "US | Track and Trigger | Server Side Events not received/digested by Optimove"
Last updateThe Optimove Us region (API4) suffered an incident in which real-time events were not received and digested, leading to failure in targeting and campaign execution. This incident is now fixed, an Optimove is again receiving and digesting events for real-time triggering Event start time: October 7th ~11AM UTC Event resolution time: October 8th ~7AM UTC
Report: "Some accounts are experiencing difficulties with Google Customer Match functionality"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified issues affecting Google Customer Match impacting a subset of accounts. Our technical teams are actively investigating the root cause of the problem and are working to implement a resolution as quickly as possible.
Report: "Optimove Site System Notifications Outage"
Last updateThis incident has been resolved.
We identified and are working to fix an outage with the Optimove Site System Notifications. The notifications impacted are: Batch-Data Process Started - notifies the recipients when the Batch Data update started (on ETL start time). Batch-Data Process Completed - notifies the recipients when the batch-data update is completed successfully (on ETL completion time). Campaign Execution Started - notifies the recipients when the campaign execution process started. Campaign Execution Completed - Notifies the recipients when the campaign execution process is completed. Batch-Data Process Analytics Completed - Notifies the recipients when the entire process is completed.
Report: "Optimove site - UI - Optimail Campaign Metrics Unavailable In The Site"
Last updateThe incident is resolved, and the metrics are being updated as expected. As part of the incident, some customers might identify a gap in metrics availability in the UI this is being restored and recovered with an expected ETA of Sep 26th to complete the restoration.
We are continuing to monitor for any further issues.
A fix has been applied to address the campaign metrics issue. We're closely monitoring the situation and working to promptly restore any missing data.
We are investigating an issue affecting the visibility of Optimail Metrics in the Optimove site. This incident is limited to UI and site functionality. Daily process, Campaign execution and API access is fully functional and unaffected.
Report: "Optimove Site - UI - Site does not load - authentication errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating an issue affecting the Optimove site. This incident is limited to UI and site functionality Daily process, Campaign execution and API access is fully functional and unaffected
Report: "Increased Errors in Campaign execution for Meta/Facebook & Google Customer Match"
Last updateThis incident has been resolved.
We identified elevated errors in our processing of the Batch Update Process, leading to failures in campaign execution for Facebook/Meta and Google customer match channel campaigns. A fix has already been implemented and we are monitoring the results Incident start time: 7:00 UTC
Report: "EU - Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating elevated errors identified in our processing of the Batch Update Process leading to delays in campaign execution for some tenants.
Report: "EU - Campaign Notification Service - Delay in Campaign execution events (event types)"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating elevated errors in our processing of the campaign notification events, which are leading to delays in campaign execution. This means delays in third-party API channels and conditional campaigns may have executed to no customers for conditional campaigns and third-party event subscription channels.
Report: "Real-Time Triggered Campaigns - Campaigns With Delayed Execution"
Last updateOptimove experienced an Outage in real-time-triggered campaigns configured with delayed execution were impacted. Start Time: 28/8 12:30 UTC End Time: 01/09 16:00 UTC During this time, real-time triggered campaigns with delayed execution were not triggered. This means that while events were executed, and the users did not receive the triggered campaigns.
Report: "US - Real Time Optimail Campaigns - Elevated execution Errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified, and we are working on a fix to improve stability
We're experiencing an elevated level of errors in real time (Trigger) Optimail campaigns and are currently looking into the issue. This is impacting a limited set of Trigger campaigns using Optimail that are not delivered to customers
Report: "EU - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
We are investigating increased error rates in our campaign services, leading to delays in campaign execution and Run Anyway campaigns.
Report: "Batch Update Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
We identified elevated errors in our processing of the Batch Data update, which delayed campaign execution for some tenants. Delays are up to 90 Minutes
Report: "Batch Data Process - Delay in Campaign execution"
Last updateThis incident has been resolved.
I am pleased to inform you that we have successfully identified and resolved the issue. We are currently monitoring the situation.
We are investigating elevated errors in our batch data processing that have led to delays in campaign execution for some tenants.
Report: "Optimail Execution Errors - Delay in Campaign Execution"
Last updateResolved incident: Optimail Campaigns were delayed in execution. This incident is still under investigation pending additional details.
Report: "Optimove Site - Error message when creating or editing a email template."
Last updateWe are pleased to inform you that the issue that caused an error and prevented users from editing or creating templates has been successfully addressed. Thank you for your patience and understanding.
A fix has been implemented and we are monitoring the results.
The issue has been identified, and the necessary fix is currently being implemented.
We are actively collaborating with our technical partner to address the email template error, and will promptly share updates as we work towards identifying the root cause of the issue.
We are currently investigating an error that occurs when creating or editing an email template.
Report: "EU - Daily Batch Update - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating elevated errors identified in our processing of the daily batch data leading to delays in campaign execution for some tenants.
Report: "Optimove Site - Increased error rate while loading the UI"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating an increased error rate while accessing the Core Application (Loading the Optimove site UI). This does not affect non-UI functions such as campaign execution.
Report: "EU - Daily batch data update - Delay in Campaign execution"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating elevated errors identified in our processing of the daily data leading to delays in campaign execution for some tenants.
Report: "EU - Track and Trigger - Processing Delays - Delay in Trigger Events"
Last updateThis incident has been resolved.
Our Track and Trigger Optimail processing is still running behind; this incident is resolved, delay durations are dropping significantly, and triggers are clearing from the queue.
Our Track and trigger processing infrastructure was running behind, which caused trigger times to be delayed by up to 24 hours. This incident is resolved, and we are monitoring the results. Events are triggering in the expected time frames.
Our Track and trigger processing infrastructure is running behind, which caused trigger times to be delayed by up to 24 hours.