Historical record of incidents for VideoAsk
Report: "Issue with 'Login with Typeform' option to access VideoAsk"
Last updateThis incident has now been resolved. Apologies for the inconvenience this may have caused.
We are currently investigating an issue where creators cannot log into their VideoAsk accounts via the 'Login with Typeform' option. Apologies for the inconvenience this may be causing.
Report: "Issue with with uploading & processing videos"
Last updateThis incident has been resolved.
Our technical support team has identified that the problem currently occurs with the Streamed recording method. To resolve this issue promptly, we advise you to switch the recording method in your forms from Streamed to Uploaded. For guidance on switching recording methods, please refer to this article: https://www.videoask.com/help/video-audio/360044119891-what-s-the-difference-between-an-uploaded-recording-and-a-streamed-recording Our team will continue to investigate why the Streamed version is failing. Please check back for updates. Again we appreciate your patience and understanding during this time. Thank you for your continued support.
Our team is currently investigating an issue where new responses can't be submitted. We appreciate your patience and understanding during this time. Please check back for updates. Thank you for your continued support.
Report: "Users can not log into VideoAsk"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Typeform login"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented. In the meantime, you can use these workarounds to log into your VideoAsk account: - Option #1: Log in using a private/Incognito window in your browser. - Option #2: If you get stuck in an infinite login loop, open your browser's developer console. While you're stuck in the loop, clear your storage for the webpage. Here is a screenshot showing where to find this option in Chrome: https://drive.google.com/file/d/13oUiGTyY9yOARTI0POTqwtaaR89FcRt7/view?usp=sharing
We are currently investigating an issue where some creators cannot log into their VideoAsk accounts via Typeform login. As a temporary workaround, you should be able to log into VideoAsk via Typeform from a private/Incognito window in your browser.
We are currently investigating an issue where some creators cannot log into their VideoAsk accounts via Typeform login.
Report: "Videoasks using custom subdomains aren't loading"
Last updateWe are glad to inform you that a fix has been implemented and the incident resolved. Thank you
A fix has been implemented and we are monitoring the results. Thank you for your patience & understanding.
We have identified a service interruption impacting videoasks using custom subdomains. We are working on a resolution with highest priority. We're sorry for any inconvenience this is causing.
Report: "Responses are not appearing in creator inbox"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results. VideoAsk should start functioning normally in approximately 30 minutes. Responses and replies that were sent during the outage should appear in your inbox shortly.
We are currently investigating an issue where responses submitted by a respondent are not appearing in the creator's inbox. This outage is also impacting webhooks and email notifications.
Report: "Service distruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results. Apologies again for the inconvenience caused.
The issue has been identified and a fix is being implemented. Thank you again for your patience & understanding.
Our services are currently experiencing an interruption, and we are actively working to resolve the issue. We appreciate your patience and understanding during this time. Please check back for updates. Thank you for your continued support.
Report: "VideoAsk functionality error"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results. Thank you again for your patience.
The issue has been identified and we are implementing a fix. Thank you for your patience.
We are aware of an issue currently affecting the VideoAsk functionality. We are working on a fix. Apologies for the inconvenience.
Report: "VideoAsk Checkout page - degraded - Users are not able to pay/upgrade"
Last updateWe are glad to inform you that a fix has been implemented and the incident resolved. We will run a problem investigation to understand what happened and how to prevent it from happening again. Thank you for your patience.
We are continuing to work on a fix for this issue. Apologies again for any inconvenience this is causing.
We have identified a traffic interruption impacting our checkout page. Users paying or upgrading may be impacted. We are working on resolution with high priority. We are really sorry for any inconvenience this is causing. Next update in 20-30 minutes or as soon as it's available.
Report: "Issue with uploading, processing, and transcribing videos"
Last updateThis incident has been resolved.
We are continuing to work on a fix for this issue.
Degraded performance with some Amazon Web Service (AWS) tools is causing issues with uploading, processing, and transcribing videos in VideoAsk. Updates on this incident can be viewed through AWS directly here: https://health.aws.amazon.com/health/status
Report: "Partial service distruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We're experiencing some errors that are affecting the performance of our application. We have identified the issue and are actively investigating it to find a prompt resolution.
We are continuing to work on a fix for this issue.
We're experiencing some errors that are affecting the performance of our application. We have identified the issue and are actively investigating it to find a prompt resolution.
Report: "Videoask's are not currently loading"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Subscriptions Services - Unavailable"
Last updateThis incident has been resolved.
We have identified errors impacting our Subscriptions Services. Users trying to change their plans are impacted. We are working on a resolution with high priority. We plan to provide an update in 20-30 minutes or as soon as it's available.
Report: "Service Interruption"
Last updateAll services have been restored with logins and signups working as they should. The incident was caused by an expired SSL certificate failing to update due to an integration issue with our security supplier. To note, although login and signup services were interrupted during the incident, submissions were functioning throughout and no data was lost. We sincerely apologize for this outage and any inconvenience we may have caused. Our engineering teams will review the details and ensure we put measures in place to prevent this from happening again.
Report: "Delay with processing responses, transcripts, and email notifications"
Last updateThis incident has been resolved.
All responses submitted during this downtime should be processed within the next 15-20 minutes. New responses, transcripts, and email notifications should return to processing as normal at that point.
The issue has been identified and a fix is being implemented.
Our team is currently investigating an issue where new responses, transcripts, and email notifications are delayed.
Report: "Processing responses and email notifications"
Last updateThis incident has been resolved.
Issue has been resolved.
This delay in processing responses and email notifications was caused by an unusually high volume of text responses, which created a bottleneck in our systems. Response and email notification processing times should return to normal within several hours.
We are currently investigating an issue related to responses not appearing in the inbox and email notifications not being sent. Responses that are submitted during this outage should eventually arrive in your inbox. Responses appear to be delayed by 15-20 minutes. Email notifications appear to be delayed by approximately 30 minutes.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue related to responses not appearing in the inbox and email notifications not being sent.
Report: "Audio & Video Content Not Processing"
Last updateThis incident has been resolved.
This incident appears to be related to an Amazon Web Services (AWS) outage, which has now been resolved. Audio and video content that was stuck in processing during the outage should now start reprocessing. Affected audio and video content should become available soon.
We are currently investigating an issue where audio and video content is not being processed. Responses, creator replies, and respondent replies are affected. Text responses do not appear to be affected.
Report: "VideoAsk Website Unavailable"
Last updateVideoAsk website was unavailable, returning 403 Forbidden errors. This resulted in the creation & answering being down for all users.
Report: "Videoask might be slow or unreachable"
Last updateThis incident has been resolved.
A task in our database was taking longer than expected and slowing our services down. The task is now completed and Videoask should be available again.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Report: "Incident with Email & Dashboard notifications (webhooks also affected)"
Last updateThe 'workers' service is now fully operational and all notifications are coming through normally. We apologize for any inconveniences the delays have caused today.
We're starting to see the 'workers' service coming back to normal and users should see notifications coming through in real-time. We're continuing to monitor the situation.
We've identified an issue with our 'workers' service which fires off all relevant notifications when new responses are received. The 'workers' service has been reinitiated however we're still experiencing issues. You may see intermittent interactions come through.
We're reporting an issue across the platform which is affecting all notification systems. Data is being collected, however, dashboard, email notifications as well as webhooks are not being fired. (Data will be showing up in your individual inboxes if you select the 'all people' filter)
Report: "Elevated API Errors"
Last updateThis incident has been resolved.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Video pipeline affected"
Last updateA fix has been implemented and video transcoding is running normally once again. Some customers may still notice some videos failing to load, we will be re-processing them in due course so that they eventually end up in customer accounts. Our sincere apologies for the service disruption.
We're investigating an issue causing a loss of functionality on part of the VideoAsk service. Currently, video transcoding is severely impaired and video/audio responses will be severely delayed coming in. Creation of new videoasks videos is also affected.
Report: "High latency on VideoAsk API"
Last updateThe incident has been resolved. Apologies for any inconvenience caused.
We've identified that our API has been taking longer than usual to respond and we're currently looking into the issue.
Report: "Elevated API Errors"
Last updateThis incident has been resolved.
System is back to normal we still monitoring.
We're experiencing an elevated level of API errors and are currently looking into the issue.
Report: "Videoasks down"
Last updateThis incident has been resolved.
We're back up. Apologies for any trouble caused.
Investigating an issue on the platform which is not allowing VideoAsks to be viewed
Report: "Video processing affected due to Amazon AWS outage"
Last updateAmazon Aws have announced that the issue is now resolved. via https://status.aws.amazon.com/ 4:00 AM PST: We have restored all traffic to Kinesis Data Streams via all endpoints, and have resolved the error rates invoking CloudWatch APIs. All services are now operating normally. We have identified the root cause of the Kinesis Data Streams event, and have completed immediate actions to prevent recurrence.
We are continuing to monitor for any further issues.
Update on the AWS outage: VideoAsk is now able to process (transcode) video/audio through AWS, although high volume customers may experience some level of throttling. This throttling will also affect the speed at which we can process the backlogged queue of video answers and replies that are currently waiting to get transferred to accounts. As for new answers and replies, this seems to be operating normally, however, we will continue monitoring closely. Update from https://status.aws.amazon.com/ : Previous Update (6:23 PM PST): We’d like to provide an update on the issue affecting the Kinesis Data Streams API, and other dependent services, within the US-EAST-1 Region. We have now fully mitigated the impact to the subsystem within Kinesis that is responsible for the processing of incoming requests and are no longer seeing increased error rates or latencies. However, we are not yet taking the full traffic load and are working to relax request throttles on the service. Over the next few hours, we expect to relax these throttles to previous levels. We expect customers to begin seeing recovery as these throttles are relaxed over this timeframe.
We are continuing to await a resolution on the AWS outage. Rest assured, we have all the video response data that is continually being collected. We just can't transcode videos for return delivery. Video replies are also held up in the same queue at the moment. Here is an update from AWS: 10:47 AM PST: We continue to work towards recovery of the issue affecting the Kinesis Data Streams API in the US-EAST-1 Region. For Kinesis Data Streams, the issue is affecting the subsystem that is responsible for handling incoming requests. The team has identified the root cause and is working on resolving the issue affecting this subsystem. The issue also affects other services, or parts of these services, that utilize Kinesis Data Streams within their workflows. While features of multiple services are impacted, some services have seen broader impact and service-specific impact details are below. https://status.aws.amazon.com/
Updated information on the AWS outage: We're still waiting for AWS to bring back up the part of their service affecting VideoAsk. At the moment we are not able to transcode videos. We are however still collecting RAW video and will be able to reprocess (and send into your accounts) once the outage is over.
The incident has been identified and is unfortunately related to an outage on Amazon AWS. AWS have annouced that the issue has been escalated to the highest possible priority.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
Currently investigating an issue affecting the API and webhooks
Report: "Live recording on VideoAsk down"
Last updateThe incident is now resolved and we've just turned Live Recording back on for all videoasks that are set to use Live Recording. We still have a backlog of videos to re-process.
We have implemented a fix and are currently re-processing a number of videos that have got stuck in the transcoding pipeline.
We are continuing to work on a fix for this issue.
We're investigating an issue with Live Recording on VideoAsk. We are switching all videoasks to use Uploaded Recording whilst we fix the issue.
Report: "VideoAsk Custom Domains failing"
Last updateVideoasks on custom domains are back up. Apologies for the service interruption.
We are investigating an issue with custom domains on VideoAsk...
Report: "[Datadog] [Triggered] Videoask Elevated level of 5XX responses"
Last updateThis incident has been resolved.
@opsgenie-videoask `sum(last_5m):avg:aws.applicationelb.httpcode_target_5xx{env:videoask}.as_count() > 10` Metric value: 16.0 Priority : normal Last Updated : 2020-11-09 16:07:13 +0000
Report: "Service videoask-api has a high latency"
Last updateThis incident has been resolved.
High latency
Report: "Platform down"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue affecting the entire platform. App and forms are currently down.
Report: "Degraded performance"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We're back online but still investigating. Our sincere apologies for the disruption.
We are continuing to investigate this issue.
We are currently investigating significantly degraded performance across the platform.
Report: "Platform down"
Last updateWe have continued monitoring systems closely since end of downtime (10:14 UTC). All systems have been working well since.
We're back up. Our sincere apologies for any inconvenience caused.
We are currently investigating an issue affecting the entire platform. App and forms are currently down.
Report: "Apps not loading data"
Last updateWe've resolved the issue. Sorry for any trouble caused.
We are continuing to investigate this issue.
We're currently investigating an incident with our apps not loading data. VideoAsks are still live.
Report: "Platform degradation incident"
Last updateThis incident has been resolved.
We have identified the cause of today's issue as resulting from an outage with one of our providers, we're working hard on negating the impact of this on our side and are doing our very best to ensure you have access to the platform again soon. Again, we thank you for your patience and apologize for the inconvenience this is causing. Updates will follow once we know more.
We have now managed to mitigate the impact of the outage with our provider and are operational again. We are monitoring this until the provider can confirm the issue is resolved on their side and will update accordingly when that's confirmed. Thank you for your patience and support, have a fantastic day!
We are currently investigating an issue that affects some videoasks. Please accept our apologies for any inconvenience this has caused. We will keep you posted here while we work on getting this resolved.
Report: "Degraded performance"
Last updateWe are back, apologies for the inconvenience.
Report: "Degraded performance"
Last updateWe've resolved today's performance issues. Sorry for any inconvenience caused.
We're currently investigating an incident regarding degraded performance across the platform.
Report: "Partially degraded service"
Last updateDue to a bad code deploy the following issues we're reported on the platform and were fixed within 10 minutes: - Loading contacts in the videoask inbox. - Submitting contact info in a videoask (data was saved but the respondent would have seen an error screen). - Some webhooks didn’t trigger, which have now been post-processed.
Report: "Videoasks and app down"
Last updateThis incident has been resolved.
We are currently investigating an issue that has caused videoasks and our apps to be unavailable.