Historical record of incidents for Alchemer
Report: "500 errors in the US when accessing surveys or navigating tabs in the builder."
Last updateThis incident has been resolved
This issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Some respondents are unable to access surveys"
Last updateSome respondents are unable to access and complete surveys. We're aware of this and our engineering and operations teams are working urgently to resolve this. We will update you here with the latest information.
Report: "Survey Taking Errors for EU Accounts"
Last updateIntermittent errors are occurring for respondents taking surveys based on accounts on the EU service. Our engineers are investigating currently to find and fix the source of the problem. Please click the subscribe button to receive further updates as we work to resolve this issue as soon as possible.
Report: "Reports Loading Issue"
Last updateThis incident has been resolved.
We continue to see some impact from yesterday's incident caused by the AWS outage. Reports are not loading properly. Our Engineers are working with AWS to resolve the issue as quickly as possible.
Report: "Update - Alchemer is being impacted by the AWS outage."
Last updateThe incident caused by the AWS outage has been resolved. All systems have been restored.
The incident caused by the AWS outage has been resolved. All systems have been restored.
The incident caused by the AWS outage has been resolved. All systems have been restored.
Report: "Exports Performance Issue"
Last updateThis incident has been resolved.
We are continuing to investigate the issue. Thank you for your patience!
There are also delays in response processing but data is gathered safely. We have all-hands-on-deck investigating the root cause and we will post updates regularly. We apologize for the inconvenience!
We are continuing to investigate this issue.
We are currently experiencing performance issues on loading and downloading exports. We are investigating the issue to restore performance as rapidly as possible.
Report: "Survey taking affected on EU"
Last updateThis incident has been resolved.
Our Engineers are actively engaged on fixing the root cause in the shortest time. We sincerely apologize for the inconvenience!
We are continuing to work on a fix for this issue.
The issue has been identified and we are currently working towards implementing a fix.
We are currently investigating this issue.
Report: "Application accessibility - US"
Last updateThis incident has been resolved. Engineering cleared the issue with our domain provider and our application performance was restored. Thank you for your understanding while we worked on fixing this!
Alchemer is experiencing an outage related to an adverse action from our domain provider. We are working diligently to address the issue and will provide regular updates here. Impacts may include access to the application, survey taking or email communication with Alchemer. If you would like to speak to someone, please contact us at 1-800-609-6480.
Engineering is still engaged with the domain provider to work towards a solution. We apologize for the inconvenience!
Thank you for your patience so far! Our Engineers are still actively engaged with our Domain partner to resolve the issue in the shortest time.
We are continuing to work on a fix for the issue together with our Domain partner.
The issue has been identified and we are continuing to work with our Domain partner to resolve the issue.
We are continuing to work on a fix for this issue.
The issue has been identified and we are currently working with our Domain partner to resolve the issue.
Some customers are unable to access Alchemer.com. Survey taking is not affected and data continues to be collected. We’re aware of the issue and are working on it urgently. We’re really sorry to be holding you up today! Please know we are working hard to get everything up and running and we will update you here with the latest information.
Report: "Survey Taking Errors for EU Accounts"
Last updateSurvey taking and response processing were restored. Engineering continues monitoring our systems to ensure stability.
Intermittent errors are occurring for respondents taking surveys based on accounts on the EU service. Our engineers are investigating currently to find and fix the source of the problem. Please click the subscribe button to receive further updates as we work to resolve this issue as soon as possible.
Report: "Survey Taking Errors for EU Accounts"
Last updateEngineering identified the root cause and implemented a fix that allows Survey Taking to work as intended. Response Processing is also catching up and delayed responses should start coming in as soon as the web servers process them.
Intermittent errors are occurring for respondents taking surveys based on accounts on the EU service. Our Engineers are investigating currently to find and fix the source of the problem. Please click the subscribe button to receive further updates as we work to resolve this issue as soon as possible!
Report: "US and CA are experiencing intermittent delayed email sending."
Last updateThis incident has been resolved.
We are continuing to investigate this issue.
We are currently investigating this issue.
Report: "US/CA Survey Taking Degraded Performance"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
We are continuing to monitor for any further issues.
We are to work to resolve continued survey taking issues on the US service. More information to follow
We are continuing to monitor for any further issues.
We have been able to resolve issues impacting US-based survey taking and are continuing to monitor the situation to ensure full resolution.
Our teams are continuing to work to resolve the issues impacting US survey taking at this time. More information to follow.
We are continuing to work on a fix for this issue.
We have identified an issue causing survey taking for US and CA-based surveys to return error messages to respondents. Our team is working swiftly to restore normal service as fast as possible.
Report: "EU Survey Taking Temporarily Unavailable"
Last updateSurvey links on the EU service have been returned to standard performance. We are continuing to monitor the situation.
Our system administrators have been able to identify the source of the issue and stabilize the issue. Survey links are loading consistently without error message at this time and we are monitoring the situation.
We are aware of an issue causing survey links on the EU-service to consistently return error messages to respondents upon opening. Our system administrators are working swiftly to resolve the issue. More information to follow.
Report: "EU survey taking is currently unavailable."
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "US Application Degraded Performance"
Last updateThe application has been restored to standard functionality as of now. Our teams are continuing to monitor and investigate any isolated incidents that occur in order to ensure continued stability. Thank you for your patience.
Stability of the US application is continuing to return to standard performance. We are still engaged in troubleshooting isolated areas that remain more impacted that the majority of the service. Our teams continue to investigate issues being reported involving access to reports, email campaigns, and the API, but the broader impact is diminishing. More information to follow.
Our Technology team continues to work to resolve issues impacting application performance for our US-based service. We are seeing improvements occur within the last thirty minutes and expect efforts to continue improvements going forward to resolve issues impacting the application, API, and reporting availability.
Thank you for your patience. Our Technology team is aware and continues to work to improve our current application performance issues. We continue to see improvements and responses are being processed. Any new changes or information will be posted here immediately.
Our Technology team is aware and continues to work to improve our current application performance issues. We do expect to see performance improvements throughout this evening.
Report: "EU Application Login Issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are investigating issues with logging into the EU-based application. Our engineers are working as swiftly as possible to restore normal service. More information to follow.
Report: "Response Processing on US is delayed, causing newly gathered responses to not appear in app."
Last updateEngineering implemented a fix and Response Processing has been restored back to normal. New responses should be starting to display in-app shortly.
We are continuing to investigate this issue.
Response Processing Delays causing new responses not to show up in Individual Responses. Engineering is currently online investigating.
Report: "US - Degraded Response Processing"
Last updateThis incident has been resolved.
A fix has been implemented. We continue to monitor to ensure responses continue to process. Thank you for your patience.
We are continuing to monitor for any further issues.
We are currently deploying a resolution for degraded response processing that has been taking place recently. You may experience oddities such as seemingly missing responses and slowdowns of the application as we actively work on the issue. Thank you for your patience, we expect to have this fully resolved soon.
Report: "EU - Degraded Reporting and Exporting"
Last updateThis incident has been resolved.
A fix has been implemented, reports and exports are operational. We are monitoring to ensure continued stability to those components.
We have Site Reliability Engineers and Developers reviewing the issue at hand. We are seeing improvements with intermittent phases of the issue.
We are investigating an issue with reports and exports affecting some EU users.
Report: "EU - Degraded Reporting and Exporting"
Last updateThis incident has been resolved.
We are investigating an issue with reports and exports affecting some EU users.
Report: "US Application Degraded Performance"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results at this time to ensure platform stability.
Additional app slowness has been identified, a fix is being implemented.
A fix has been implemented and we are monitoring to confirm application stability.
We are currently investigating reports of US application slowness.
Report: "EU - Degraded Reporting"
Last updateThis incident has been resolved.
We are currently investigating an issue affecting a portion of EU users when viewing reports.
Report: "EU - Service Outage"
Last updateThis incident has been resolved.
A fix has been implemented. Our engineering team is monitoring results to ensure EU application access has been fully restored.
We are working with our database partner AWS to resolve the issue. At this time we are still investigating.
Site Reliability Engineers and Developers are online and investigating the issue. We apologize for the long delay and interruptions this is causing.
We are continuing to investigate this issue.
We are currently experiencing a service outage on our EU platform. Site Reliability Engineers are online investigating the issue at this time.
Report: "US - Degraded Performance"
Last updateThis incident has been resolved.
Users are seeing intermittent errors across their Alchemer Account. Survey taking is not affected and data continues to be collected. We’re aware of the issue and are working on it urgently. We’re really sorry to be holding you up today! Please know we are working hard to get everything up and running and we will update you here with the latest information.
Report: "EU - Degraded Performance"
Last updateThis incident has been resolved.
We are currently investigating intermittent issues while accessing parts of the application.
Report: "EU - Service Outage"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently experiencing a service outage affecting our EU Platform. Our engineers have been notified and are investigating the issue now.
Report: "US Application Login Issues"
Last updateThe incident has been resolved.
We are currently monitoring the application. Access should now be restored. If you run into additional errors, please reach out to support.
We are currently investigating reports that the US application login page is inaccessible.
Report: "Intermittent performance issues"
Last updateThis incident has been resolved.
Users are now able to access the application. We will continue to monitor application behavior.
We are continuing to investigate this issue.
Some users are experiencing intermittent US application access and error messages.
Report: "Some users are unable to access our application - CA"
Last updateThis incident has been resolved.
A fix has been implemented and application access has been restored.
The issue has been identified. We continue to see intermittent CA application access issues and are working on implementing a fix now.
We are continuing to investigate this issue.
Some customers are unable to gain access to their Alchemer accounts. Survey taking is not affected and data continues to be collected. We’re aware of the issue and are working on it urgently. We’re really sorry to be holding you up today! Please know we are working hard to get everything up and running and we will update you here with the latest information.
Report: "Survey Taking and Login Errors for EU Accounts"
Last updateThis incident has been resolved.
Intermittent errors are occurring for respondents taking surveys based on accounts on the EU service. Our engineers are investigating currently to find and fix the source of the problem. Please click the subscribe button to receive further updates as we work to resolve this issue as soon as possible.
Report: "US app - Intermittent login issues"
Last updateThis incident has been resolved.
Our team has been able to stabilize the impact of this issue and are continuing to monitor to ensure it is resolved.
We have identified an issue that is causing some users to receive an error message when logging into the US-based application. Our teams are working swiftly to resolve the issue. EU and Canada services, as well as survey taking overall, are not impacted at this time. More information to follow.
Report: "US Application Login Issues"
Last updateThis incident has been resolved.
We're currently investigating Login Issues affecting the US Application causing timeouts.
Report: "Intermittent Login Issues on US Service"
Last updateAttempts to log into the US-based application are intermittently presenting an error page or allowing access to the main dashboard but without presenting any content. Our Technology Team has been able to fully resolve this issue as of 11:15 AM EST.
Report: "Intermittent US Survey Taking Errors"
Last updateBetween 12 PM MST and 5:53 AM MST on Saturday, December 5th, US-based surveys experienced intermittent issues causing error messages to appear for respondents attempting to load a survey. The Alchemer team was able to resolve the issue. We apologize for the issue.
Report: "US Application Login Issues"
Last updateThis incident has been resolved. Our Engineers are actively reviewing any intermittent issues if they are still occuring.
Residual issues from a vendor-specific incident causing intermittent application login issues. We are not seeing the EU or CA application affected at this time. More information to follow.
Report: "US App Login Issues"
Last updateThis incident has been resolved.
Our vendor has been able to make improvements that is causing services to come back up to speed as of two minutes ago. We are continuing to monitor the situation to ensure consistent service.
We are continuing to work on a fix for this issue.
We are experiencing an issue with issues logging into the application due to a vendor-specific issue. Our vendor is experiencing issues that are being investigated to resolve the issue as swiftly as possible.
Report: "US Application Login Issues"
Last updateLogin access for US-based accounts has been restored. If you are still experiencing issues with logging into your US-based Alchemer account, please reach out to our Support Team for assistance.
Login access for affected accounts has been restored. We are continuing to monitor to confirm complete resolution to the issues causing error messages to appear when logging into US-based accounts.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue that is causing some users to receive an error message upon logging into the US application. We are not seeing the EU or CA application affected at this time. More information to follow.
Report: "CA Service Login Issue"
Last updateLogin access for CA-based accounts has been restored. If you are still experiencing issues with logging into your CA-based Alchemer account, please reach out to our Support Team for assistance.
Login access for affected accounts has been restored. We are continuing to monitor to confirm complete resolution to the issues causing error messages to appear when logging into CA-based accounts.
We have identified the source of the issue and working to bring login access on CA accounts up to speed as swiftly as possible. More information to follow.
We are currently investigating reports that access to logging into the CA-based service is returning error messages. We are not seeing survey taking or response gathering impacted at this time. More information to follow.
Report: "Slow Performance and Intermittent Error Messages on US App"
Last updateWhile the issue has subsided, our team is continuing to work internally to come to a root cause and work to resolve the issue going forward. We are continuing to monitor.
We have been able to stabilize the issue causing the error messages. Our teams are continuing to work to ensure stability.
We are currently working to resolve issues causing some users to receive an error message when attempting to log into the US application. More information to follow.
Report: "US and EU Application Access Issues"
Last updateWe have been able to restore normal access as of now. If you are continuing to see issues, please reach out to our Support team. Thank you for your patience.
The fix has been implemented and we are beginning to see normal performance for survey taking and app login returning to normal. We are continuing to monitor to ensure everything is back up to speed.
Our team has identified the source of the issue. We are working to implement a solution that will restore consistent access to logging into the US application shortly. We have found that survey taking on some US accounts is running slowly at the moment; the solution being implemented will restore normal speed there as well.
We are continuing to investigate this issue.
Our teams are investigating issues causing the US and EU-based services to present error messages upon logging in. We are working swiftly to restore access at this time. More information to follow.
Report: "US - Response Processing Delays"
Last updateThis incident has been resolved.
A fix has been implemented for current response processing delays on our US service. We are currently monitoring the issue to ensure it's resolution.
Report: "Canada-based accounts intermittently not loading"
Last updateWe are seeing proper access restored to CA-based accounts that were intermittently affected. If you are still experiencing this issue, please reach out to the Support team for further assistance.
A fix has been implemented and we are monitoring the results.
We have identified a fix to the issue and have implemented it to restore consistent access. More information to follow as testing continues.
We are investigating intermittent reports of login access to Canada-based resulting in a white screen or an error message. Our system administration team is working swiftly to identify and resolve the issue. US and EU based accounts are not affected, and we have not seen impact upon survey taking. More information to follow.
Report: "HTML Errors in Translated Surveys"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently looking into reports of HTML not rendering in non-default languages within translated surveys.
Report: "Response Processing Delays"
Last updateThis incident has been resolved.
Responses are being processed but remain delayed for some accounts. We are monitoring this. Please check back soon and reach out to Alchemer support if you are experiencing these delays or have any questions.
We are currently experiencing delayed response processing for some accounts. If you are experiencing this behavior or any other issues, please reach out to Support from your Need Help portal in the application. Rest assured that your data is being collected and you will soon see it processed into your account. Thank you for your patience.
Report: "US Application Login Issues"
Last updateThis incident has been resolved.
A fix has been implemented for the US service which is causing the issue to subside.
We are currently investigating an issue that is causing some users to receive an error message upon logging into the US application. We are not seeing the EU or CA application affected at this time, as well as survey taking and response gathering not being affected. More information to follow.
Report: "EU application is not available"
Last updateThis is now resolved. Please reach out to support@surveygizmo.com if you continue to experience issues.
A fix has been implemented and we are now monitoring. You can access the EU application now.
Access to the EU application is unavailable. Our teams are working now to resolve this. We'll post updates momentarily.
Report: "US app intermittent login issues"
Last updateThe fix for the EU service has been completed. Normal service has been restored. If you are still experiencing this issue as of now, please reach out to our Support team to discuss.
A fix has been implemented for the US service which is causing the issue to subside. The fix is being implemented for our EU service now as well, and we will be providing an update shortly.
Our team has identified the source of the issue and is working to make changes to resolve the issue shortly. We have also identified that the EU application appears to be affected as well.
We are currently investigating an issue that is causing some users to receive an error message upon logging into the US application. The option to "Reload Page" from this error page is intermittently allowing the user to proceed. We are not seeing the EU or CA application affected at this time, as well as survey taking and response gathering not being affected. More information to follow.
Report: "US Application Outage is Resolved. Responses continue to be processed."
Last updateAll queued Survey Response data has successfully been processed and the application remains stable. Monitoring will remain in progress.
Access to the SurveyGizmo application has been completely restored. Queued survey response data continues to process and is almost complete. We continue to monitor progress.
Access to the SurveyGizmo application has been restored. Queued survey response data continues to process and we are monitoring progress. Application load times may vary as this process completes.
Access to the SurveyGizmo application has been restored for all users at this time. Queued survey response data continues to process and we are monitoring progress. Application load times may vary as this process completes.
Access to the SurveyGizmo application is now being restored for all users. Queued survey response data has begun to process and will continue over the next few hours. Application load times may vary as this process completes.
We continue work to fully restore systems and process all queued response data. As we have further updates on the progress, we will update you here. Previous update: As many of you know, SurveyGizmo experienced an outage yesterday at 10 AM Mountain Time (MT) due to a corrupted database on failed hardware. We have restored service to most of our customers, but 6% (~850 customers) remain affected. We are working to safely and securely restore access and all data for these customers. While system access has been impacted, survey taking / collecting responses continues on active surveys. SurveyGizmo has a solid data backup from 8 AM MT (2 hours prior to the outage) with no risk of data loss. Once the system has been fully restored, queued survey responses will be processed through the present time. We will continue to provide hourly updates on our status page. We apologize for any inconvenience this has caused and we will do everything in our power to restore your confidence in our platform.
As many of you know, SurveyGizmo experienced an outage yesterday at 10 AM Mountain Time (MT) due to a corrupted database on failed hardware. We have restored service to most of our customers, but 6% (~850 customers) remain affected. We are working to safely and securely restore access and all data for these customers. While system access has been impacted, survey taking / collecting responses continues on active surveys. SurveyGizmo has a solid data backup from 8 AM MT (2 hours prior to the outage) with no risk of data loss. Once the system has been fully restored, queued survey responses will be processed through the present time. We will continue to provide hourly updates on our status page. We apologize for any inconvenience this has caused and we will do everything in our power to restore your confidence in our platform.
System Administration teams are continuing to progress through maintenance to resolve all instances of errors appearing in the application. Thank you for your patience while we complete work to restore service to all affected accounts.
System Administration teams are continuing to progress through maintenance to resolve all instances of errors appearing in the application. Thank you for your patience while we complete work to restore service to all affected accounts.
We have been able to restore access for a majority of affected accounts as of now and are seeing consistency in function there. We are continuing to work to mitigate continued issues for a portion of accounts experiencing slow load times on the main dashboard, including alert messages noting "Sorry! There was an error. Please try again later." Troubleshooting is continuing to restore full service as soon as possible. Thank you for your patience.
We are continuing to progress through maintenance and troubleshooting to return stability and consistent US app performance. Thank you for patience.
Most users are now able to log in successfully. There are still intermittent issues for some customers. Work continues to fully resolve this for all of our customers. Please reach out to support@surveygizmo.com if you have any questions. We are here to help!
The US application continues to be unavailable at this time. Users are unable to access the application. Survey taking is not impacted by this. We are working to resolve this.
Our team continues to work to restore access to the application with consistent fashion. Progress to resolution is being made and next steps for restoring consistent performance are being taken. EU and CA application are not affected at this time. Respondent experience for survey taking is not affected at this time. We are seeing delays in response processing which is causing data to be delayed in appearing in the application. Data is being gathered safely and securely. More information to follow.
The US application is still experiencing issues with access and producing error messages upon moving between pages. Our Administration team is performing maintenance on the corresponding systems to restore normal service. Thank you for your patience.
We are monitoring the resolution put in place to restore normal service. More information to follow. If you are still experiencing intermittent error messages in the application at this time, please reach out to our Support Team to discuss.
We are seeing performance improve at this time. The issue has been isolated to a portion of the service and we are continuing to mitigate the issue going forward. More information to follow. Thank you for your patience.
We are investigating recurring issues with the US application not loading. More information to follow shortly.
Report: "US Application Outage"
Last updateWe have been able to resolve and restore access to the US application as of now. Our teams are working to investigate any further issues. Should you still have issues with the application at this time, please reach our to our Support Team for assistance. Response processing is expected to catch back up to realtime progressively shortly.
We have been able to restore access to the US application at this time. We are monitoring for effect to ensure consistent access to the US application. Investigation is continuing.
We are continuing to investigate reports of delays in actions firing following responses being completed. We are seeing this due to a delay in response processing at this time.
We are investigating an issue causing the US-based application to be inaccessible to users. Survey taking is not affected at this time. EU and CA applications are not affected at this time. We are working swiftly to resolve the issue. More information to follow.
Report: "Intermittent Errors Upon Login - US App"
Last updateWe have been able to restore normal service consistently across the service as of now. Thank you for your patience while we worked to resolve this issue.
We are continuing to investigate the source of the issue. Instances of errors upon login and when navigating within the US app are on the decline.
We are investigating intermittent instances of error messages presenting upon logging into the US application. Survey taking is not affected at this time and is functioning as normal. The EU and CA applications both are not affected at this time. More information to follow.