ImageKit.io

Is ImageKit.io Down Right Now? Check if there is a current outage ongoing.

ImageKit.io is currently Operational

Last checked from ImageKit.io's official status page

Historical record of incidents for ImageKit.io

Report: "Dashboard and upload service down"

Last update
Resolved

This incident has been resolved.

Monitoring

The issue has been fixed and we are monitoring.

Investigating

We are currently investigating the issue.

Report: "Dashboard and upload service down"

Last update
resolved

This incident has been resolved.

monitoring

The issue has been fixed and we are monitoring.

investigating

We are currently investigating the issue.

Report: "Increased error rate on uncached requests [Mumbai region]"

Last update
resolved

Error rate increased for new transformations in Mumbai region. The issue started at 13:10 PM IST and lasted till 13:18 PM IST. During this time, the error rate for CDN miss requests increased. Issue was identified by our team and fixed.

Report: "Issues in accessing ImageKit Dashboard"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Degraded image transformation and upload service in Mumbai region"

Last update
resolved

This incident has been resolved.

investigating

We are continuing to investigate this issue.

investigating

Image Processing and Upload services were degraded for Mumai region.

Report: "Error in image processing in US West region"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are currently investigating this issue.

Report: "Intermittent issues with fresh request processing in US West Region"

Last update
resolved

Some media processing requests in US West (North California) processing region were ending in 5xx timeouts. This issue was intermittent and observed on a few requests. The issue is now fixed and more details on the issue and the fix will be updated.

Report: "Error in image processing in SGP region"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "Issue with dashboard login"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Degraded image transformation processing in US region, Dashboard & Upload unavailability"

Last update
resolved

Issue has been resolved now. Our team is monitoring the traffic.

investigating

US east region was unavailable from 2:45PM UTC to 3:50PM UTC. Upload and Dashboard services were unavailable. Image processing requests for US east region were redirected to EU region.

Report: "Degraded image transformation processing in EU region"

Last update
resolved

There was minor degradation in image processing for few mins in EU region. Issue was identified and fixed immediately. We are monitoring the traffic.

Report: "Increase in processing times for some requests for new requests in Mumbai region"

Last update
resolved

We observed a spike in response times for <2% of new or uncached image processing requests. The incident was from 1:30 PM to 1:50 PM IST. After that, the response times stabilized around their regular levels for all requests.

Report: "Elevated response times in Mumbai region for fresh image transformations"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

We are continuing to monitor for any further issues.

monitoring

The request times have been stable for the last 12 hours but our team is still monitoring this issue.

identified

The issue has been identified and a fix is being implemented.

Report: "Increased error rate on uncached requests [Mumbai region]"

Last update
resolved

Error rate increased for new transformations in Mumbai region. The issue started at 6:24 PM IST and lasted till 6:34 PM IST. During this time, the error rate for CDN miss requests increased. Issue was identified by our team and fixed.

Report: "Increased error rate on uncached requests [Mumbai region]"

Last update
postmortem

**Summary:**  On July 7th, during the interval of 11:34 AM IST to 12:20 AM IST, our ImageKit CDN endpoint in the Mumbai region encountered an elevated error rate of around 4 to 5 percent. **Root cause:**  The anomaly was traced back to a partial DNS outage within our server cluster. While our processing servers were working fine, our CoreDNS service was overwhelmed by an unusually high volume of DNS requests. This unprecedented flood of requests exceeded the capacity of the `conntrack` table, which led to dropped connections and requests not being sent for processing. **Resolution:** The issue was quickly resolved by scaling the CoreDNS service, which successfully distributed the load and mitigated the problem. We have initiated meticulous performance monitoring of our DNS and will implement modifications, if required, after a thorough validation. **Lessons learned** **What went well:** Our efficient monitoring systems detected the anomaly promptly before customers started facing the issue. Our team responded swiftly and systematically, ensuring minimal impact on our services. Despite the relatively higher error rate, 95% of total requests continued to be served with ultra-low latency. **What went wrong:** A more proactive approach towards DNS performance monitoring could have averted this issue. We're now incorporating DNS into our regular monitoring routine to forestall such instances.

resolved

Error rate increased for new transformations in Mumbai region. The issue started at 11:35 AM IST and lasted till 12:12 PM IST. During this time, the error rate for CDN miss requests increased. Issue was identified by our team and fixed. We are currently monitoring the systems and response times.

Report: "Increased error rate in serving requests in US region"

Last update
resolved

The issue started at 06:47 PM IST and lasted till 06:59 PM IST. During this time, the error rate for CDN miss requests increased. The issue has been resolved and we are continuously monitoring the systems.

Report: "Increased error rate in serving requests in EU region"

Last update
resolved

The issue started at 11:29 AM IST and lasted till 11:35 AM IST. During this time, the error rate for CDN miss requests increased.

Report: "Increase in upload error rate"

Last update
resolved

The issue has been resolved.

Report: "Increased error rate in serving requests"

Last update
resolved

Increase in error rates in servicing requests in Mumbai, Singapore, Sydney, and Frankfurt region.

Report: "Increased error rate in serving requests in Singapore region"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are continuing to work on a fix for this issue.

identified

We are continuing to work on a fix for this issue.

identified

The issue has been identified and a fix is being implemented.

Report: "Increased error rate in EU region"

Last update
resolved

Increased error rate for CDN miss requests in EU region for image transformation and upload API. Issue started at IST 2:36PM and lasted till IST 2:45PM.

Report: "Delay in cache purge for files"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Increased error rate on uncached requests [us-east-1]"

Last update
resolved

Error rate for uncached image requests increased between 09:53PM to 09:57 PM IST in us-east-1 region.

Report: "Error in serving requests in Mumbai region"

Last update
postmortem

Earlier today, we ran into issues with scaling our systems in the Mumbai region, primarily because of an edge case identified in AWS's instance allocation. This resulted in a high error rate for customers using the Mumbai processing region a couple of times. After identifying the issue, which lasted for about 16 minutes around 11:50AM IST and 8 minutes around 09:45AM IST, our team made some changes to get around this limitation and restore functionality in this region. The service is completely functional now with no degradation in response times either. Our team is also working on a long-term fix so that the corner case encountered today does not impact services in the future. We remain committed to ensuring an uptime as close to 100% as possible.

resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are currently investigating this issue.

Report: "Setting propagation delays across regions"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

Experiencing delays in propagating new setting updates and information related to new user accounts to our servers globally. No impact on traffic for existing customers. Users might face issues related to settings not getting used intermittently.

Report: "Increase in error rate for serving new, uncached images in SGP region"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are continuing to work on a fix for this issue.

identified

The issue has been identified and a fix is being implemented.

Report: "Increase in error rate for serving new, uncached images in SGP region"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "Increase in error rate for serving new, uncached images in SGP region"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "Increase in error rate for serving new, uncached images in US1 region"

Last update
resolved

The issue has been resolved.

Report: "Increase in error rate for serving new, uncached images in EU region"

Last update
resolved

The service has been restored now.

Report: "Increased error rate on uncached requests"

Last update
postmortem

At 12:20 PM IST, we started facing major outages across all our processing regions and servers. All uncached requests \(missed from CDN and our internal caches\) were not getting served during the impacted duration. The issue was partially resolved at 12:40 PM IST and completely resolved at 12:45 IST. The issue resulted from a configuration change on our servers that got picked up around 12:20 PM IST when the issue started happening. Once our team identified the error, the configuration change causing the outage was handled on our servers as a temporary measure, and the service was restored. Soon after, our technology team started making changes to ensure that such configuration changes do not impact the server uptime in the future. Our team will soon deploy the change across our systems to prevent this issue from recurring in the future. Meanwhile, the systems are completely operational, and our team is constantly monitoring the same.

resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been deployed in all regions and service is being monitored.

identified

We have identified the issue and a fix is being deployed.

Report: "Non-AWS custom CDN Users - Increase in error rate for serving new, uncached images in EU region"

Last update
resolved

Users using a non-AWS, custom CDN with their home region set as Europe (Frankfurt), would have faced an increased error rate between 0140 UTC to 0200 UTC.

Report: "Increase in error rate for serving new, uncached images in US1 region"

Last update
postmortem

There was a server migration and upgrade that happened in the US East region at about 13:40 IST. Though this was a routine activity with no foreseeable downtime, during this transition, because of an incorrect configuration on a few servers, the new servers were not able to connect to other services necessary for delivery of content. Therefore, uncached content requests going to these servers started ending in an error. ImageKit's team mitigated the issue by correcting the config on all the impacted servers. We are making changes to our internal processes to avoid such issues from recurring in the future.

resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "Increase in error rates and response times for new, uncached images in EU region"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

Report: "Increase in error rate for serving new, uncached images in US East region"

Last update
resolved

The incident has now been resolved.

Report: "Increase in response time and error rate for serving new, uncached images in US1 region"

Last update
resolved

The issue has been resolved.

Report: "Increase in error rate for serving new, uncached images in SGP region"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "[External CDNs] Increase in error rate for serving new, uncached images in EU region"

Last update
resolved

The incident impacted delivery rates for 8 mins and has now been resolved.

Report: "Increase in error rate for serving new, uncached images in Europe region"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "Increase in error rate for serving new, uncached images in EU region"

Last update
resolved

A higher rate of error was observed for a few requests for a duration of about 20 minutes around 1300GMT. The error rates are now normal and our team is working on updates to minimize impact in the future.

Report: "High connection times to customer origins in Europe region"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are continuing to work on a fix for this issue.

identified

It is taking longer than usual to access original images from customer origins for customers with Europe Frankfurt selected as processing region in ImageKit. Our team is working on fixing the issue.

Report: "Increase in error rate for external CDN customers in EU region"

Last update
resolved

Brief periods of high load time and timeouts were observed for external CDN customers in the EU region. The load times and error rates are normal now. Our team is monitoring the same for further action, if needed.

Report: "[External CDN] Increase in error rate for serving new, uncached images for external CDN customers in SGP region"

Last update
resolved

High error rates were observed for image delivery because of issues with one of the intermediate caching layers for customers using ImageKit with custom CDNs. The issue has been fixed and error rates are back to normal now

Report: "Increase in error rate for serving new, uncached images in US1 region"

Last update
resolved

A higher rate of error was observed for a few requests for a duration of about 15 minutes.

Report: "High error rate for customer using Singapore region"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

Report: "High error rate for customer using Frankfurt region"

Last update
resolved

The issue has been identified and a fix is being implemented.

Report: "Slow video processing times for new video URLs"

Last update
resolved

The incident occurred between 10:25 IST (GMT +530) to 13:45 IST (GMT +530) where video processing times for new optimizations and transformations were elevated. Customers would have experienced timeouts, and slow video delivery during this time. The root cause of the incident was identified at 11:00 IST (GMT +530) and a fix put in place later to bring down the processing and delivery times back to normal. Additional monitoring and failovers have been setup to mitigate the issue in the future.

Report: "High error rate for customer using a custom CDN in Europe region"

Last update
resolved

The incident occurred between 5:09AM - 5:39AM IST where because of temporary network connectivity issues within our network of servers, requests from external CDN customers using Frankfurt processing region, saw a high error rate and load times. The connectivity was restored and the traffic and error rates are now back to normal.

Report: "Increased error rate on fresh requests [video delivery]"

Last update
resolved

Between 6:53 AM to 7:31 AM, 4th Sep 2022 IST, the error rate on the video delivery service increased. The issue has been fixed, and we are monitoring the service continuously.

Report: "Increased error rate on uncached requests - EU region"

Last update
resolved

Error rate for uncached image requests increased between 17:25PM to 17:35 PM IST and 18:30PM to 18:35PM IST in Frankfurt region.

Report: "Increased error rate on uncached requests [Mumbai region]"

Last update
resolved

Error rates were increased in Mumbai region for a few customers multiple times during the day. Issues lasted for 5-8 minutes. Incident happened at 8:40AM, 2:06PM, 3:09PM. Our team has identified the issue, and there are no ongoing errors. However, a permanent fix is still under development and will be deployed soon. 13 Oct 2022 issue timings 9:42 AM, 2:04 PM