Historical record of incidents for TrueContext
Report: "SSO Login Service Disruption"
Last updateThis incident has been resolved.
Our systems are reporting SSO login functionality has been restored. We will continue monitoring.
A downstream provider used for single sign-on (SSO) is currently experiencing a service disruption. This may impact access to corporate login functionality in TrueContext across the portal and mobile applications. We will continue to monitor the SSO provider's status and will provide an update as soon as possible. TrueContext log in service is not affected by this disruption.
A downstream provider used for single sign-on (SSO) is currently experiencing a service disruption. This may impact access to corporate login functionality in TrueContext across the portal and mobile applications. We will continue to monitor the SSO provider's status and will provide an update as soon as possible. TrueContext log in service is not affected by this disruption.
Report: "Microsoft Connection Issues Resolved - Click below"
Last updateThis incident has been resolved.
We have deployed a fix that restores connectivity to Microsoft services and are monitoring activity. You may need to reconnect Microsoft connections configured for your team. Instructions on reconnecting can be found in our documentation here: https://support.truecontext.com/hc/en-us/articles/4406637825300-How-to-Re-Connect-an-Expired-Connection
We are continuing to work on a fix for this issue.
We are aware of and investigating an issue with establishing connections to Microsoft services. Our engineering team is actively working to resolve the issue.
Report: "Document generation failures"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are experiencing an increased number of errors when generating documents. The issue has been identified, and we are actively applying a fix
Report: "Processing Delays"
Last updateDelays in processing have been resolved and all scheduled jobs impacted by the incident have been processed successfully.
A fix has been implemented and we are monitoring the results.
We have identified delays in processing scheduled exports, and are actively investigating. Messages are currently queued and will process once we resolve the issue.
Report: "TrueContext Service Disruption"
Last updateServices are completely operational and the issue is now resolved.
Services look to be recovered. We will monitor for a short time before marking the incident completely resolved.
We have identified the root cause of the issue and are actively working to restore the service.
We are currently experiencing a service disruption. Our engineering team is working to identify the root cause and implement a solution.
Report: "Document generation failures"
Last updateBetween 1:05 PM EST and 8:35 PM EST we experienced an elevated level of intermittent errors generating documents. The issue has been identified and fixed.
Report: "Document generation failures"
Last updateWe have implemented a fix and services are fully operational. We apologize for the inconvenience this may have caused.
We have identified the root cause and are in the process of recovery. Some rendering is working again and we will update again when we are fully operational.
We are experiencing an elevated level of errors generating documents. Our team is actively investigating the issue and working to restore services as quickly as possible.
Report: "TrueContext Service Disruption"
Last updateThis incident has been resolved. We experienced a service disruption beginning July 10, 11:59 AM EDT. During this period, users were unable to log into the web portal and processing submissions may have been delayed. A fix was implemented and the incident was resolved at 12:04 PM EDT.
Report: "Submission Processing Delays"
Last updateDelays in submission processing have been resolved and all submissions impacted by the incident have been processed successfully.
We have implemented a fix for submissions sent during the processing delay incident that remained in a failed state. These submissions are now being processed. We will provide an update when this operation completes.
We are aware of ongoing processing delays and errors with some submissions that were sent during our processing delay incident on Apr 8, 2024 between 12:51 PM and 7:13 PM EST. We are actively working on a resolution to process these items, and continue to investigate any other issues resulting from the processing delays.
Report: "Processing Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We are continuing to work on a resolution for processing delays. Thank you for your patience.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We have identified delays in data source fetches, destination processing, and scheduled jobs and are actively investigating. Messages are currently queued and will process once we resolve the issue.
Report: "Elevated API Errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are experiencing an elevated level of API errors and are currently looking into the issue.
Report: "TrueContext Service Disruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently experiencing a service disruption. Our engineering team is working to identify the root cause and implement a solution. We are observing issues when accessing newly created customer feedback forms.
Report: "TrueContext Service Disruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently experiencing a service disruption affecting Customer Feedback Forms. Our engineering team is working to identify the root cause and implement a solution.
Report: "Email deliverability Issues for some customers"
Last updateEmail Destination deliverability is completely operational again. Re-tries will continue automatically over time. If you would like the retry to happen sooner, please manually re-execute any failed email destinations.
Some customers are experiencing email destination failures. We have identified the issue and a fix is currently being rolled out. We should see recovery within 15 minutes and complete recovery within 1 hour. We will update with more information as we monitor the fix. All destinations will re-run automatically as the system recovers.
Report: "Support Portal Unavailable"
Last updateOur support portal is operational again
There is an ongoing issue that is preventing users from accessing the ProntoForms Support Portal (support.prontoforms.com). A fix is underway and the ETA for resolution is 12:00 PM EST.
Report: "Unexpected Service Disruption"
Last updateThe systematic recovery effort related to this incident has concluded. However, we are aware of isolated recovery issues affecting some customers that we are continuing to address. We encourage customers to review the support article linked below for details on the impact to their account. We will continue to respond with priority to questions or concerns related to customer activity on December 30, 2021, or related to the recovery. Please log a ticket, and our Support or Customer Success teams will assist. Thank you for your patience. For additional information, please visit the following Support article: https://support.prontoforms.com/hc/en-us/articles/4414456014228
Efforts to recover affected form submissions, form changes, and changes to the ProntoForms portal are on-going
The service has been restored and we are closely monitoring the system. Data recovery processes are on-going. Please visit this article on the ProntoForms Support site for additional information and updates (login required). https://support.prontoforms.com/hc/en-us/articles/4414456014228-Service-Outage-Details-for-Dec-30-2021
We are continuing to work on a fix to restore the service as quickly as possible.
We've identified the issue and are working on a fix.
We are currently investigating an issue that is causing widespread disruption. Our team is actively working towards a resolution as quickly as possible.
Report: "Microsoft OneDrive for Business Connection errors"
Last updateThis incident has been resolved.
The service operational again for 99% of customers. We will provide an update once all customers are operational.
Some Microsoft OneDrive for Business Connections are experiencing an elevated number of errors. We are actively looking into the issue.
Report: "Document Generation errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Our document generation service is experiencing an elevated number of errors. We are actively looking into the issue.
Report: "Pages may be slow loading"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The site is currently experiencing a higher than normal amount of load, and may be causing pages to be slow or unresponsive. We're investigating the cause and will provide an update as soon as possible.
Report: "Document Generation errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Our document generation service is experiencing an elevated number of errors. We are actively looking into the issue.
Report: "Pages may be slow loading"
Last updateThis incident has been resolved.
A fix has been implemented, we are monitoring the results.
The site is currently experiencing a higher than normal amount of load and maybe causing pages to be slow or unresponsive. We're investigating the cause and will provide an update as soon as possible.
Report: "Document Generation Issue Identified"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We've identified an issue with document generation and are now working to return the service to normal operation.
Report: "Service Disruption"
Last updateThis incident has been resolved.
We've corrected the cause of degraded performance and are now monitoring the service.
We are now experiencing a service degraded performance and continue to work as quickly as possible to address it.
We are now experiencing a service disruption and continue to work as quickly as possible to identify the root cause and restore service.
We are now experiencing a service disruption and continue to work as quickly as possible to identify the root cause and restore service.
The site is currently experiencing a higher than normal amount of load and maybe causing pages to be slow or unresponsive. We're investigating the cause and will provide an update as soon as possible.
Report: "OCR Keyboard Unavailable"
Last updateThis incident has been resolved.
A fix in our iOS app has been deployed, allowing us to re-enable OCR Keyboard. Users are encouraged to download the latest iOS app version (v14.7.2) if they experience issues
We are experiencing issues with our Mobile Client OCR Keyboard feature. Customers are currently unable to use this feature.
Report: "Pages may be slow loading"
Last updateThe site is currently experiencing a higher than normal amount of load, and may be causing pages to be slow or unresponsive.
Report: "Document generation failures"
Last updateThis incident has been resolved.
Our Document Generation service is operating normally and we are continuing to monitor the situation.
We are experiencing an elevated level of errors generating documents, and are looking into the issue.
Report: "Windows App 14.6.0.0+ Upgrade Issue"
Last updateThis incident has been resolved.
A fix is available in version 14.6.2.0 of the Windows App where the issue is resolved. Anyone running into the error can upgrade to 14.6.2.0 instead of performing the workaround previously described.
We have implemented a fix to the issue and it is currently being verified. Once complete, it will be available as a new version of the ProntoForms Windows app. We will provide an update once it is widely available.
After the upgrade to the ProntoForms Windows app v14.6.0.0+, customers may experience errors such as "Draft could not be loaded" or "The form is missing some dependencies. Please reconcile." while clicking on forms. In order to resolve these errors, please perform a Reset Forms action in App Settings > Advanced Settings > Reset forms to download all of the form resources from the ProntoForms server. We are currently working on a fix to avoid having to perform a Reset Forms action. We will provide an update once availabie.
Report: "Elevated Errors for Basic Analytics"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are experiencing an elevated level of errors with submission processing for Basic Analytics for answers from Repeatable Sections. Investigation is in progress and we will provide an update when we know more
Report: "Document generation failures"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are experiencing an elevated level of errors generating documents, and are looking into the issue.
Report: "ProntoForms Service Disruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently experiencing a service disruption. Our engineering team is working to identify the root cause and implement a solution.
Report: "Document generation failures"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We are experiencing an elevated level of errors generating documents, and are looking into the issue.
Report: "ProntoForms Service Disruption"
Last updateOur engineering team has identified the root cause and resolved an issue which resulted in a service disruption.
Report: "ProntoForms Service Disruption"
Last updateOur engineering team has identified the root cause and resolved an issue which resulted in a service disruption.
Report: "Data source and destination failures"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified that Google data sources and destinations are also affected. We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
Dropbox data sources and data destinations are experiencing an elevated level of errors and we're looking into the issue.
Report: "Data destination and data source failures"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
Google and Dropbox data sources and data destinations are experiencing an elevated level of errors and we're looking into the issue.
Report: "ProntoForms Service Disruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently experiencing a service disruption. Our engineering team is working to identify the root cause and implement a solution.
Report: "ProntoForms Service Disruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results
We are currently in the process of implementing a fix for this issue.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently experiencing a service disruption. Our engineering team is working to identify the root cause and implement a solution.
Report: "ProntoForms Admin Console Service Disruption"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently experiencing a service disruption with our admin console. Our engineering team is working to identify the root cause and implement a solution.
Report: "Customer Feedback Form submission errors"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
AWS are reporting services are now recovered and we are monitoring the results.
We are impacted by an outage in a downstream service within AWS and we are actively monitoring any updates with them.
We are currently experiencing a service disruption around submissions of Customer Feedback Forms. Our engineering team is working to identify the root cause and implement a solution.
Report: "Submission Processing Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified issues in processing submission data originating from some mobile devices when submitting dispatched forms, and are actively working on a fix. Affected submission data will be processed once we resolve the issue.
Report: "Data destination failures"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Email Destinations are experiencing an elevated level of errors and we are investigating the issue.
Report: "Document generation failures"
Last updateBeginning Friday, August 11th and 4:00 PM EST and ending Monday, August 14th 8:02 AM EST customers may have experienced elevated error rates in generating documents using custom designed document templates. Documents using our standard document templates were unaffected.
Report: "Email Delivery Issues"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and we are in the process of implementing a fix to restore email services.
ProntoForms is currently experiencing an issue with an upstream provider which is affecting our ability to send email and we are investigating the issue.
Report: "Data destination failures"
Last updateSharepoint List Data Destinations experienced an elevated level of errors beginning 9:51 AM EDT. This issue was resolved 11:51 AM EDT.
Report: "Support Portal Authentication Issues"
Last updateThis incident is now resolved.
We should be fully operational again. We will continue to monitor the situation for 15 minutes before marking the incident resolved. We apologize for the disruption and thank you for your patience.
We are seeing some successful logins to the ProntoForms Support Portal now. We will update again once we are completely operational again.
There is an ongoing issue that is preventing users from logging into the ProntoForms Support Portal (support.prontoforms.com). A fix is underway and the ETA for resolution is 3:30pm EDT.
Report: "Email Service Delays"
Last updateThe issue is now resolved.
The issue is now resolved. If you are still experiencing delivery issues with emails from Email Destinations, please check with your IT/MS365 administrator to ensure the emails are not being quarantined.
We identified mail delivery issues to email addresses hosted in Microsoft/Office 365 beginning approximately 7:00 AM EDT on Monday, October 16th. We applied a fix at 7:17 PM EDT on Monday, October 16th and are monitoring the results. Mail originating from the ProntoForms system may have been flagged and quarantined by customer Outlook environments. We speculate that an update to the default quarantine and spam policies resulted in our mail being erroneously categorized. Mail that has been delivered but flagged may be found within your quarantined messages. Releasing messages from quarantine will allow these messages to be delivered successfully, as well as subsequent messages from ProntoForms. Affected customers may contact their Microsoft/Outlook 365 administrator to assist with the process of manually releasing affected mail from quarantine. Alternatively, customers may choose to re-execute the submissions from affected Email Destinations during the times listed above.
Update: We have confirmed mail delivery issues to email addresses hosted in Microsoft/Office 365 beginning approximately 7:00 AM EDT on Monday, October 16th. Mail originating from the ProntoForms system may be flagged and quarantined by customer Outlook environments. We speculate that an update to the default quarantine and spam policies is resulting in our mail being erroneously categorized. Mail that has been delivered but flagged may be found within your quarantined messages. Releasing messages from quarantine will allow these messages to be delivered successfully, as well as subsequent messages from ProntoForms. While we continue to investigate these delivery issues, affected customers may contact their Microsoft/Outlook 365 administrator to assist with the process of manually releasing affected mail from quarantine.
We have identified delays with our downstream email service provider and are actively investigating. Emails are currently queued and will be sent in a timely manner once we resolve the issue.
Report: "Intermittent Errors in Data Sources"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are seeing errors in a subset of Document Generation capabilities as well.
We are seeing intermittent errors in Data Source fetching as well as syncing between our Cloud systems and our Mobile Applications. We are actively working on a resolution to these errors.
Report: "Processing Delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We have identified delays in submission processing, data source fetches, and scheduled exports, and are actively investigating. Messages are currently queued and will process once we resolve the issue.
Report: "Elevated error rates"
Last updateWe experienced an elevated error rate beginning November 6, 9:41 AM EST. During this time higher than normal error rates were encountered during submission processing and scheduled jobs, such as data source fetching and data exports. A fix was implemented and the incident was resolved at 10:35 AM EST.
Report: "ProntoForms Service Disruption"
Last updateThe incident has been resolved and all services are operational.
We have discovered the root cause and implemented a fix. We believe we are operational across all systems. We will actively monitor and update again in 5 minutes.
We are currently experiencing a service disruption. Our engineering team is working to identify the root cause and implement a solution.