Embedly

Is Embedly Down Right Now? Check if there is a current outage ongoing.

Embedly is currently Operational

Last checked from Embedly's official status page

Historical record of incidents for Embedly

Report: "app.embed.ly downtime"

Last update
resolved

This incident has been resolved.

identified

The issue has been identified and a fix is being implemented.

Report: "X (FKA Twitter) incident"

Last update
resolved

This incident has been resolved.

identified

We've identified an issue impacting x, formerly known as twitter, embeds. A fix is being deployed shortly, and we'll continue to monitor.

Report: "app.embed.ly maintenance"

Last update
resolved

This incident has been resolved.

investigating

We are performing scheduled maintenance on app.embed.ly. Users won't be able to login for the maintenance period.

Report: "We are aware of a brief API outage lasting for about 4 minutes 15/11/2023, 1:51PM PST."

Last update
resolved

This incident has been resolved.

monitoring

The issue has been resolved, but we are watching closely.

Report: "Intermittent YouTube provider latency"

Last update
resolved

This incident has been resolved.

monitoring

Embed.ly is currently monitoring a situation where we can observe higher than usual intermittent latency coming from the YouTube upstreams which may impact Embed/Extract performance.

Report: "i-cdn.embed.ly, i.embed.ly image links"

Last update
resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating an issue related to 525 errors upstream from our Display image links.

Report: "Images Missing"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We are continuing to work on a fix for this issue.

identified

We've identified an issue with image fetching with our Cards or Extract product. We're working behind the scenes to fix this bug as soon as possible.

Report: "High Latency"

Last update
resolved

This incident has been resolved.

investigating

We are continuing to investigate this issue.

investigating

We are currently investigating this issue.

Report: "API/Cards affected by Cloudflare Outage"

Last update
resolved

This incident has been resolved.

monitoring

Cloudflare reported that the root cause has been addressed, and we're seeing recovery in API/Cards service health as well. We will keep monitoring.

identified

Cloudflare is experiencing downtime at the moment, affecting our ability to properly route requests to our API/Cards services. We are waiting on an update from Cloudflare (https://www.cloudflarestatus.com/) regarding expected resolution.

Report: "Twitter Metadata Availability"

Last update
postmortem

Twitter made a policy change to restrict the availability of tweet metadata, like thumbnail images, but most twitter functionality has been restored to the best of our ability.

resolved

This incident has been resolved.

identified

We're working on a fix for Twitter metadata which can be missing from Embed and Extract content. This is related to a change in Twitter policy, and we're working quickly to resolve this issue if possible.

Report: "5/15/20 - Service Degraded"

Last update
resolved

This incident has been resolved. If you have any further issues please contact us.

monitoring

We are continuing to monitor for any further issues.

monitoring

Core Embed/Extract functionality has been restored.

monitoring

A fix has been implemented and we are monitoring the results.

identified

We've begun to recover API functionality. You may encounter periods of degraded service performance as we move towards full recovery. Thank you for your continued patience as we work through this incident.

identified

We're working around the clock to resolve this service outage and will continue investing all of our available resources towards restoring normalcy over the weekend. Thank you for your patience as we work through this incident.

identified

We are actively working with AWS to resolve cloud infrastructure issues, which is still preventing us from reliably serving API (extract, embed) and Cards functionality.

identified

We are actively working with AWS to resolve cloud infrastructure issues, which is still preventing us from reliably serving API (extract, embed) and Cards functionality.

investigating

We are currently experiencing networking failures in our cloud infrastructure, severely degrading our ability to provide API and Cards services. More updates to follow.

Report: "YouTube Embeds"

Last update
resolved

We have recovered from an outage affecting YouTube embeds which we determined was caused by upstream rate-limiting from YouTube's servers. We now have a solution in place, but are continuing to monitor for any further issues. Please keep us posted at https://support.embed.ly about any further issues you may experience.

investigating

We are currently investigating an issue with YouTube embed functionality

Report: "Cloudflare System Status"

Last update
resolved

This incident has been resolved.

investigating

We are currently tracking a Cloudflare incident that may impact our API performance for some customers: https://www.cloudflarestatus.com/incidents/tx4pgxs6zxdr

Report: "Elevated response times"

Last update
resolved

A connectivity issue between backend components was causing elevated response times in the Display product for about an hour this morning and has been resolved.

Report: "Display API"

Last update
resolved

Everything is back to normal.

investigating

We are looking into an issue with 502s for uncached URLs to the Display API.

Report: "High latency"

Last update
resolved

This incident has been resolved.

monitoring

We are reviewing a spike in our data service that caused API latency. Response times are returning to normal, but we will continue to monitor.

investigating

We are experiencing high latency on the Embed, Extract APIs at this time. We are currently reviewing the issue and hope to back to normal soon.

Report: "CDN Outage"

Last update
resolved

A bad config on our CDN caused an outage from ~11:15PST to ~11:35 PST. This has been resolved.

Report: "Display Usage Reporting"

Last update
resolved

This incident has been resolved.

identified

Due to a problem with an upstream provider, display endpoint usage in dashboard is missing data. The problem is being corrected up stream and the data should be filled in over the next couple of days.

Report: "Bad Deploy"

Last update
resolved

This incident has been resolved.

monitoring

Rollback complete, everything is returning to normal.

identified

A bad deploy went out, bringing the API down. We are rolling back now.

Report: "Image Latency for Embedly Display"

Last update
resolved

This incident has been resolved.

monitoring

We're currently monitoring issues related to the DDoS attack affecting DNS providers in the US. This may cause higher than normal latency for image processing through i.embed.ly

Report: "app dashboard unavailable"

Last update
resolved

This incident has been resolved.

investigating

We're reviewing an issue with our app dashboard access is temporarily unavailable.

Report: "Increased response times"

Last update
resolved

Good to go.

monitoring

We are seeing response times going back to normal. We will continue monitoring.

investigating

We are bringing up more API worker boxes to help with response times.

Report: "API latency"

Last update
resolved

An API update has been pushed to fix this issue.

identified

We experienced some increased latency on API calls caused by a regression.

Report: "API increased response times"

Last update
resolved

This incident has been resolved.

monitoring

We found an issue with one of our fetcher boxes that was slowing responses down. We have removed and created a new box. Response should be returning to normal.

investigating

We are experiencing some performance issues with the API. Currently reviewing, expect slower response times.

Report: "Vimeo URLs intermittent"

Last update
resolved

Vimeo links are now working through the API. Any previously called links will take about 12 hours to refresh.

monitoring

Vimeo recently integrated some aggressive rate limiting for their oEmbed API and video detail pages. Embedly got caught up by their rate limiting and were unable to serve their embeds. Vimeo white listed Embedly, so things are returning back to normal. Due to Embedly caching, all URLs called in the last 12 hours will continue to error till they are cleared. For cards, this is 24 hours. You can manually clear Embedly's cache by adding force=true to your API request like so: http://embed.ly/docs/explore/oembed?url=http%3A%2F%2Fvimeo.com%2F18150336&force=true ( This should only be done on a manual basis) Please reach out to support@embed.ly if you have any questions. Shout out to Brad and Peter at Vimeo for getting us white listing.

investigating

We are working with Vimeo to resolve a rate-limiting issue that is causing intermittent failures for Vimeo URLs. This effects calls to the Embed, Extract API for Vimeo URLs. All Vimeo embeds previously called should work correctly.

Report: "Timeouts and Drop in Response times"

Last update
resolved

Response times are back to normal.

monitoring

We have replaced a bad cache box and will be monitoring performance. It should get better over time as the cache builds back up.

investigating

We are investigating an issue with our caching layer that is causing issues with load and timeouts.

Report: "High Latency"

Last update
resolved

Everything has returned to normal. 2 bad things happened at once: 1. One of services that checks URLs against the safe browse index died. This is generally not a big deal as it's meant to quietly rebuild itself while traffic keeps humming along. 2. We saw a drastic change in traffic patterns. This made the first problem manifest into a big deal. We have corrected the first problem and almost double capacity for the night.

investigating

Root cause found, latency is moving in the correct direction.

investigating

Still on it, it's the engineer's birthday too. Poor @anrope.

investigating

We are experiencing some increased latency, we are looking into this now.

Report: "API Latency above 2 seconds"

Last update
resolved

This incident has been resolved.

monitoring

We have isolated the traffic spike and responses should be returning to normal.

monitoring

We are bringing up more boxes to handle the load.

identified

We are seeing a large traffic spike of about 1000 req/sec.

investigating

We are looking into some latency issues, our reporting indicates latency is above 2 seconds. It's normally at 500 milliseconds.

Report: "Display Issue"

Last update
resolved

All good in the hood.

monitoring

The new box is deployed and latency is returning to normal. We have removed the worker box and will be investigating the root cause.

investigating

We are deploying a new display worker box to offload traffic. This should be up in next 10 minutes.

investigating

We are experiencing some issues with our Display Endpoint. Seeing 502 and 504 errors.

Report: "Bad build"

Last update
resolved

Good to go.

identified

We introduced a bad build that caused issues with our API worker boxes. New build in progress. ETA 1 minute.

Report: "High Latency"

Last update
resolved

Things have returned to normal. <3

monitoring

Things are still sluggish, but response times have fallen to acceptable levels. We are going to continue monitoring response times for awhile. The engineers believe the root cause in our deployment system, but I'm not sure I believe those guys.

identified

Things are starting to improve, still looking into the root cause.

investigating

We are noticing some high latency. The team is working on a fix now.

Report: "502 Bad Gateway"

Last update
resolved

This incident has been resolved.

monitoring

Everything seems back to normal, but still monitoring for now.

identified

Bad push, reverting and waiting for a fix to hit all the boxes.

investigating

We are seeing a lot of 502 errors, looking into it now.

Report: "APP Usage Data"

Last update
resolved

Usage data is now working on our dashboard.

investigating

Usage data is currently not appearing in your App Dashboard. We will be putting in a fix after the weekend to resolve this issue.

Report: "CDN IFrames are down."

Last update
resolved

Embeds should all be working correctly. We are continuing to monitor to ensure embeds will work.

monitoring

A fix has been implemented and we are monitoring the results.

identified

Continuing to work through our CDN to get everything back up and running.

identified

Our devops guy is playing with admin panels and configuring servers. I don't want to bug him, but we are working on it.

identified

We have put in a temporary fix for http sites, but it is still down for https. We hope to have https back soon.

identified

Our CDN provider terminated our account, they are currently working on re-activating that account now. Yes, we did pay our bill.

investigating

We are looking into this.