Historical record of incidents for Kevel
Report: "Delayed Ad Publication"
Last updateA fix has been implemented and we are monitoring the results.
We identified an issue impacting ad publication starting 2025-06-08 16:00 UTC. An investigation is underway with our team. We'll keep you updated along the way.
Report: "Ad Balancing & Publication Delays"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We identified an issue with our ad publication & balancing systems starting 2025-04-25 18:29 UTC. This is being investigated with high priority. This impacts us-east only.
Report: "Ad Balancing & Publication Delays"
Last updateWe identified an issue with our ad publication & balancing systems starting 2025-04-15 14:00 UTC. This was resolved on 2025-04-16 15:50 UTC. Some customers may have experienced a delay in new ads serving, as well as a delay in ads restarting after a daily cap was met prior day.
Report: "Ad Balancing & Publication Delays"
Last updateWe identified an issue with our ad publication & balancing systems starting April 9 at 18:00 UTC. This was resolved on April 10 at 14:30 UTC. Some customers may have experienced a delay in new ads serving, as well as a delay in ads restarting after a daily cap was met prior day.
Report: "We are currently investigating an issue with Kevel ad serving systems"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified the issue and are working on a fix.
We identified an issue impacting ad serving in us-east-1, beginning at 13:10 UTC. An investigation is underway with our team. We'll keep you updated along the way.
Report: "Ad Balancing & Publication Delays"
Last updateWe identified an issue with our ad publication & balancing systems starting March 12 at 19:00 UTC. This was resolved at 19:40 UTC for 'us-east' ad servers, and 20:17 UTC for servers in other regions. Some customers may have experienced delay in new ads serving, as well as over or under delivery of existing ads during this time frame.
Report: "Increased Response Times for Events in EU Central 1 Region"
Last updateThis incident has been resolved.
We are currently investigating an increase in response times for Events (Impressions and Clicks) in EU Central 1 Region. An investigation is underway with our team. We'll keep you updated along the way.
Report: "Degraded Performance in UserDB US-East-1"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are currently investigating an issue that is causing degraded performance for UserDB endpoints in our US East Region. If a customer receives a 500 type error, it means that the attempted update to UserDB failed and will need to be retried.
Report: "Degraded Performance in UserDB US-East-1"
Last updateThis incident has been resolved.
A resolution has been implemented and performance has been restored. We are continuing to monitor. Please note, if a customer received 500 type error during the degraded performance time, it means that the attempted update to UserDB failed and will need to be retried.
We've identified a potential resolution and are in the process of implementing and monitoring.
We are currently investigating an issue that is causing degraded performance for UserDB endpoints in our US East Region. If a customer receives a 500 type error, it means that the attempted update to UserDB failed and will need to be retried.
Report: "User DB degraded performance in us-east-1"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue we have identified in our US East region which is causing high response times for UserDB endpoints.
Report: "We are currently investigating an issue with Kevel reporting systems."
Last updateWe resolved the issue of delayed Report processing. All affected reports have been processed. The incident is now resolved.
We identified an issue impacting Reporting. Reports are processing through the system. We'll keep you updated along the way.
Report: "We are currently investigating an issue with Kevel reporting systems."
Last updateThis incident has been resolved. Kevel Reports that used a "group by day" setting were ignoring the start/end dates chosen and returning all dates. Impacted reports still in a state of "Processing" were marked as failed. Timeline of the incident is noted below. Any reports generated during this window with "group by day" may need to be re-ran. Began: October 4, 2024 10:30 am UTC Ended: October 4, 2024 5:15 pm UTC
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We identified an issue impacting Reporting. An investigation is underway with our team. We'll keep you updated along the way.
Report: "Incorrect branding in some UI pages"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We have implemented a fix and are monitoring the application
The issue has been identified and a fix is being tested.
Unauthenticated self-serve users may see incorrect branding on [yoursubdomain].campaign-dashboard.com. We are currently investigating this issue.
Report: "Decision Request performance in eu-west-1"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We have identified an issue with our Decision Request engine in the eu-west-1 region which started at 8:25 UTC. This issue will be impacting responses from the API in that region. We have resolved the issue and are actively monitoring the results.
Report: "Reporting - Partial loss of functionality in reporting"
Last updateKevel Reporting system experienced a partial loss of functionality in reporting resulting in blank or stale campaign and flight auxiliary data. Performance metrics and entity ids were not impacted. Specifically, entity names, price, rate type and dates of campaigns and flights that were created or updated during the incident window would appear blank in reports generated during the same timeframe. This is now resolved. Newly created reports will contain correct and up to date campaign and flight auxiliary data. You may have to regenerate any reports that were created during the impacted timeframe. This issue started on 13 September 2024 10:00 AM UTC and was resolved 17 September 2024 11:00 AM UTC.
Report: "Reporting - delayed data ingestion"
Last updateThe ingest of historical data has now been complete, and we've been able to confirm data accuracy. This incident is now resolved.
The ingest of the historic data is progressing, and a final update will be provided once we have confirmed that all historic data has been ingested correctly.
Our systems are working on reprocessing the historic data
Our team is continuing to work on reprocessing the historic data
Our team is continuing to work on reprocessing the historic data
We've identified what is causing the reporting data to not ingest correctly, and we're currently working on re-processing the historic data. No data loss has occurred.
We're currently investigating a delay in reporting data ingestion that started at 8pm UTC, Saturday the 10th August.
Report: "Reporting Delays"
Last updateDelay in all events ingestion from 3rd July 6pm UTC to 4th July 10AM UTC (16 hours). All of the data has now been backfilled and bucketed to the correct time.
Report: "Real Time Reporting API Outage"
Last updateOur queue has caught up and this issue has been resolved. Ads were not properly removed after hitting their caps or paced properly from about 04:00 UTC to 23:30 UTC on May 1st. All ads should now be serving as expected.
A fix has been implemented and messages are now processing. We expect the system to take some time to process all message and remove all ads that have reached their caps. We will continue to monitor and update when complete.
We are continuing to work on a fix for the identified issue.
We are continuing to work on a fix for the identified issue.
We have identified a misconfiguration in our Real Time Reporting system that resulted in messages not being processed. Ads that reached their caps after 04:00 UTC were not properly removed from the ad servers, which may have led to over delivery. Ad pacing is also impacted. We have determined the cause of the misconfiguration and are working on a fix.
We are seeing issues with our Real Time Reporting API. This system powers the data within our UI, as well as ad pacing and delivery. We are investigating at this time.
Report: "Reporting data issue on Jan 17, 2024"
Last updateWe discovered a data ingestion issue for impressions and other events in our Reporting system for 1 hour period of time on January 17, 2024. This led to a small underreporting of impressions for that date. We have since backfilled the missing data.
Report: "Delayed ad publication in us-east-1"
Last updateThis incident has now been resolved and all systems are fully operational.
A fix has been implemented and we are monitoring the results.
We have identified an issue that is causing delayed ad publication in us-east-1 since 22:22 (UTC) on the 13th of February. We're in the process of restoring ad publication to standard operation.
Report: "Degraded Performance: Scheduled Reports"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue impacting scheduled reports from being sent to recipients.
Report: "Issue: Scheduled Report Sends"
Last updateThis incident has been resolved.
Root cause has been identified and a fix has been deployed. Any delayed scheduled reports should send shortly.
We are currently investigating an issue impacting scheduled reports from being sent to recipients.
Report: "Increased Decision API latency in us-east-1"
Last updateThis incident has been resolved.
We have identified an increase in latency from the Decision API in us-east-1 starting at 12:20 (UTC). The issue has now been resolved and we are actively monitoring the recovery.
Report: "Partial Outage: Reporting UI and API"
Last updateThis incident has been resolved.
We identified the cause of the delay; queued reports in the backlog are being processed.
We are continuing to investigate this issue.
Processing of scheduled and new reports is experiencing delays.
Report: "Reporting API & UI -- Generating reports taking longer than normal"
Last updateThis incident has been resolved.
Reporting services are fully operational now and we are monitoring the current status to ensure it is fully resolved
We are continuing to investigate the delays in our reporting system
We are continuing to investigate the delays in our reporting system
We are continuing to investigate the delays in our reporting system.
We are continuing to investigate the delays in our reporting system
Generating reports via the UI and API are both slower than normal. We are investigating the cause of this.
Report: "Reporting API + UI: Unable to generate reports"
Last updateThis incident has been resolved. If you have generated a report during the incident, you are advised to run the report again.
We have rectified the reporting functionality. Data for reports is still being resync and we are monitoring the progress and performance.
We have identified another issue preventing the generation of reports for API and UI users. We are currently investigating. Report processing for new and existing reports will be delayed.
We identified an issue caused by an automatic update from AWS. It has been rectified. We are monitoring as the system is working to run the outstanding reports.
We identified an issue preventing the generation of reports for API and UI users. The source of the issue has been identified, and we are working to restore functionality.
Report: "Outage: Reporting API + UI"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We identified an issue impacting the generation of reports via API and within the UI. An investigation is underway with our team. We'll keep you updated along the way.
Report: "Reporting API + UI: Delay in generating reports"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We identified an issue preventing the generation of reports for API and UI users. The source of the issue has been identified, and the system is working to resync data.
Report: "Increased error rate in us-east-1"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We have identified an issue causing an increased error rate in us-east-1
Report: "Server Error when trying to save new image ads"
Last updateWe identified an issue that caused a server error when trying to save new ads that contained an uploaded image. This issue began at 11:30 am ET and was resolved at 1:52 pm ET.
Report: "AWS Lambda Outage Impacting Kevel Management UI/API + Reporting"
Last updateThis incident has been resolved.
Amazon reported the outage as resolved; they are recovering and processing data. The Kevel Management UI, Management API, and Ad Serving are operational. Reporting remains in a degraded status while backlogged data is processed.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
Amazon reported an outage at 18:51 GMT affecting their Lambda service. Amazon has identified the root cause and is working on a fix. The following Kevel services will be affected: Reporting: Reports can be run at this time; however, reporting data may be delayed. Custom targeting: Flights that contain custom targeting can now be saved. Ad serving: While ad delivery isn't affected, ad balancing will be AWS updates can be accessed here:https://health.aws.amazon.com/health/status
Report: "Ad Balancing Delay Detected for some Customers"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to investigate this issue.
Some customers are experiencing greater-than-normal publication delays. The source of the delay has been identified, and the system is working to restore publication to normal levels.
Report: "Partial delivery outage (related to earlier outage)"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to actively investigate these issues. A small percentage of customers may be seeing delayed ad serving
The issue has been identified and a fix is being implemented.
While outages have been resolved, we are seeing some increased delivery times for a small portion of our customers. We are continuing to investigate this.
We have finished implementing a fix for the subset of customers who were experiencing ad serving issues. We are monitoring those changes now.
The fix is in the process of being implemented. We will provide further updates shortly.
We have identified delivery issues for a small portion of our customers related to our earlier incident. We are moving forward with a fix and will update when it has been fully implemented.
Report: "Delivery outage for some customers with custom certificates"
Last updateWe have implemented the fix and now all customers, including those with custom certs, are back to fully operational ad serving.
We have identified that certain customers that have custom certificates may still be experiencing delivery issues related to the previous incident. We are working on a fix now and will update when this has been implemented.
Report: "Partial delivery outage"
Last updateThe problem has been identified and a fix has been implemented. Ad serving should be fully operational for all customers.
We are continuing to investigate this issue.
Traffic from the affected servers has been rerouted and ad serving should be live again for affected customers. We are investigating the cause of the underlying issue.
We are currently investigating delivery issues for a small portion of our customers beginning at around 14:30 ET / 18:30 UTC
Report: "Brief Management API and UI outage"
Last updateThis incident has been resolved.
From approximately 14:03 to 14:18 UTC (10:03 to 10:18 ET), Kevel experienced an outage affecting the Management API and UI. The system subsequently recovered without data loss. We currently believe there was an issue with our database service provider and are escalating with them. We do not believe there is elevated risk of additional outages, but will continue to monitor closely. Ad serving was not impacted during this period.
Report: "Real-time Reporting timeouts"
Last updateWe have identified the problem and implemented a fix. Reporting data in now available again via the UI and API.
We are investigating timeouts in the real-time reporting system, including requests made both via UI and API
Report: "Delays in ad publication"
Last updateWe have verified the fix in place now corrects the publication issues seen and have repaired existing Ads in the system.
We have rolled out a fix for the publishing errors, and are continuing to monitor expected results.
We are investigating and error with newly created Flights not being published. Our team is actively investigating this issue now and are verifying an identified fix.
Report: "Delays in ad publication"
Last updateThis issue is now resolved.
A fix has been implemented and we are monitoring the results
We have identified an issue that is impacting ad publication. Our team is actively investigating this issue now.
Report: "Ad balancing delays (non us-east regions)"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
Our team is still working on a fix to resolve this issue.
We have identified an issue impacting ad balancing/pacing in non us-east Decision API regions. We are actively working on resolving this issue now. Balancing in US-East is unaffected by this issue.
Report: "Ad Balancing & Publiciation Delay Detected"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We have identified an issue with our ad publication & balancing systems (starting February 24 at 22:40 UTC). Some customers may experience delay in new ads serving, as well as over or under delivery of existing ads during this time frame. We are working to resolve this as soon as possible.
Report: "Delayed updates"
Last updateThis incident has been resolved and updates are again publishing promptly.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Some customers may notice a delay in updates propagating to their networks . Current ads are still serving as expected. We are working to identify a fix for this issue.
Report: "UI Access Issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The UI at app.kevel.co may currently be unavailable for customers. We are investigating the issue with urgency and trying to resolve the issue as soon as possible. Ad serving is unaffected.
Report: "Increased Decision API latency"
Last updateThis incident has been resolved.
We are continuing to closely monitor our systems but latency remains stable and within normal levels of operation.
We observed an additional network connectivity issue that led to increased latency in the Decision API. Customers may have observed an increase in timeouts between 14:29 - 14:39 UTC. Latency is now back to normal levels, but we are continuing to monitor our systems.
Report: "Increased Decision API latency"
Last updateWe observed a network connectivity issue that led to increased latency in the Decision API. Customers may have observed an increase in timeouts between 10:53 - 10:56 UTC and 11:04 - 11:10 UTC. This issue has now been resolved and we are continuing to monitor our systems.
Report: "Degraded Latency + Delayed Publication"
Last updateAll systems impacted are fully operational and rates are back to normal. This incident is fully resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We identified an issue resulting in degraded latency and delayed publication for some networks. An investigation is underway with our team. We'll keep you updated along the way.
Report: "Ad publication delays"
Last updateThis incident has now been resolved. All delayed ad publication messages have been processed successfully, and new publication messages should be processed as expected.
We have identified and resolved an issue causing a delay in ad publication (starting at 7 am UTC). We are now in the process of catching up on the ad publication queue, which we expect to be fully processed within the next 30 minutes.
Report: "Delayed ad publication (us-east)"
Last updateBetween 2022-11-29 21:30 and 2022-11-30 09:50 UTC, customers using Kevel's US-East region experienced delays in new ads being published and delays in changes to existing flight/ad targeting. All changes made during that time have now been applied, and we have resolved the underlying issue. Published ad delivery/serving was not impacted.
Report: "UI Access Issues"
Last updateThe incident has been resolved and access to the UI is available for all customers. Some customers may have seen publication delays between 11am and 12:05pm ET.
The UI outage has been resolved and we are monitoring the results. All customers should have access to the UI once again.
The UI at app.kevel.co may currently be unavailable for some customers, however others may be unaffected. We are investigating the issue and will look to identify and resolve the situation as soon as possible. Ad serving is unaffected.
The UI at app.kevel.co may currently be unavailable for customers. We are investigating the issue and will look to identify and resolve the situation as soon as possible. Ad serving is unaffected.
Report: "Publication delays"
Last updateThis incident has been resolved.
Our team is currently working through a backlog of the publishing queue.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
Some customers are experiencing greater-than-normal publication delays. The source of the delay has been identified, and the system is working to restore publication to normal levels.
Report: "Degraded Performance: Ad Serving"
Last updateThis incident has been resolved.
We are continuing to monitor the results.
A fix has been implemented and we are monitoring the results.
We have identified the cause of the issue. A fix is currently being implemented.
We're investigating an issue affecting ad serving (Decision API). While error rates and latency have decreased, we're still working to identify the cause.
We are currently investigating this issue.