Historical record of incidents for DataGrail
Report: "Access to DataGrail via Okta SSO Impacted"
Last updateThis incident has been resolved. All customers using Okta SSO to access DataGrail can do so successfully.
The issue has been identified and fix is being implemented.
We are currently investigating an issue where customers utilizing Okta SSO for access to DataGrail are unable to successfully login and see a 403 error message. We will update again when we know more.
Report: "Intake Form Submission Errors"
Last updateThis incident has been resolved and all users are able to submit Privacy Requests.
Users are experiencing an error when attempting to submit a Privacy Request. The issue has been identified and we are working on resolution
Report: "Trouble logging into DataGrail via Microsoft SSO"
Last updateThis incident has been resolved and users with Microsoft SSO can now login to DataGrail successfully.
We have identified an issue with logging into DataGrail via Microsoft SSO only. We are actively investigating and will post updates as we learn more.
Report: "Direct Contact Integrations Closing Requests without Completion after 14 Day Due Date"
Last updateThis incident has been resolved.
We have identified an issue with Direct Contact Integrations that have reached their 14-day expiration date and are set to "Contact My Team". Intended Behavior : At the 14 day expiration date, if the Direct Contact integration has not been completed by the designated email processor and is set to "Contact My Team", the Super Admin will get a notification to reach out to the designated email processor and the request will remain open until either the Request Processor/Super Admin manually stops processing OR the designated email processor for the Direct Contact integration completes the task. Current Behavior : At the 14 expiration date, if the Direct Contact integration has not been completed by the designated email processor and is set to "Contact My Team", the Super Admin is getting a notification still but the Direct Contact integration is marking itself as complete without the influence of the Request Processor/Super Admin or the task being actioned by the designated email processor, thus causing the request to close when all other integrations have completed. We are actively investigating and will post updates as we learn more.
Report: "Direct Contact Integrations Not Initiating Email Workflow"
Last updateThis issue has now been resolved with very minimal customer impact. We are working with those impacted and have rectified the issue. Please reach out to support@datagrail.io with any questions!
We are continuing to work on a fix for this issue.
We have identified an issue with Direct Contact integrations that were newly connected on or after 09/28, not initiating the email workflow of sending requests for completion by designated Email Processors. We are actively investigating and will post updates as we learn more.
Report: "Hybrid Integrations Not Sending Direct Contact Emails to Processors"
Last updateThis incident has been resolved.
We have identified an issue with Hybrid integrations not sending Direct Contact emails to Processors. This is the result of the new Authenticated DSR Handling feature release from 08/30 that has since been reverted. We have identified the root cause and are working to fix and rerelease the feature. In the meantime, we are also actively working on resending all Direct Contact emails that should have gone out within the last ~24hrs and were impacted. We will update once the issue is fully resolved.
Report: "DataGrail Account Invites resulting in 500 errors for Customers that utilize GoogleSSO or MicrosoftSSO"
Last updateThis incident has been fully resolved and the platform is functioning normally.
We have identified an issue with DataGrail Account Invites leading to 500 errors for Customers that utilize GoogleSSO or MicrosoftSSO to log into DataGrail. We have identified a fix and are currently validating this internally.
Report: "Data Subject Requesters Unable to Verify Email Address"
Last updateThis incident has been fully resolved and the platform is functioning normally.
We have resolved the issue with data subject requesters hitting an error when attempting to verify their email address and we are currently monitoring for any issues.
We have identified an issue with data subject requesters hitting an error when attempting to verify their email address. We are actively investigating and will post updates as we learn more.
Report: "Network Connectivity Issues"
Last updateThis incident has been fully resolved and the platform is functioning normally.
Users may experience intermittent issues connecting and logging into the DataGrail platform. DataGrail is working with our partners to resolve this issue.
Report: "Live Data Map: Multiple Inventory System Reports Not Showing"
Last updateThere is no data loss associated with this incident and all Multiple System Inventory Reports should be appearing as previously created. This incident existed for 28 minutes from 11:50am to 12:18pm PST on 12/1/21. This incident has been fully resolved and the platform is functioning normally.
Report: "Network Connectivity Issues"
Last updateThis incident has been fully resolved and the platform is functioning normally.
Issues with intermittent connectivity with the DataGrail platform have been resolved and we are currently monitoring this fix.
Users may experience intermittent issues connecting and logging into the DataGrail platform. DataGrail is working with our partners to resolve this issue.
Users may experience intermittent issues connecting and logging into the DataGrail platform. DataGrail is working with our partners to resolve this issue.
Report: "Issue with Direct Contact Deletions (Resolved)"
Last updateToday (8/25/2021) we identified and resolved an issue with Deletions in which privacy requests were automatically closed before a response was recorded from Direct Contact Integrations. We have reset all affected privacy requests back to a “Pending Delete” state. No data or requests were lost. Please review your queue and process requests as you would normally. If you have any questions please contact support@datagrail.io.
Report: "Service disruption during routine database upgrade"
Last updateOn Friday, June 11, 2021, DataGrail experienced a service disruption of privacy intake forms and application downtime lasting from 8:08PM PST until 8:43PM PST. This was caused by a routine upgrade to apply security and performance improvements to DataGrail’s core database infrastructure. DataGrail Engineering had previously performed similar upgrades across its test environments and had not observed a service interruption. The DataGrail team has identified several measures to prevent this failure from occurring in the future. These include implementing a two-phase migration process for database upgrades that will avoid downtime due to delayed maintenance as well as enhanced failure mode benchmarking in our test environments. Please contact [support@datagrail.io](mailto:support@datagrail.io) if you have any questions or concerns.
The incident has been resolved. Please contact support@datagrail.io if you have questions or would like a copy of our root cause analysis (once available).
The service disruption has been resolved and all systems are back online. We are actively monitoring.
A service disruption started around 8:11 PM PT during a routine database upgrade. We are actively resolving the issue and should be back up shortly. Updates to follow.
Report: "Privacy Intake Forms Loading Inconsistently"
Last updateDataGrail's Engineering Team were alerted by internal monitoring and proactively remedied a memory issue that was appearing sporadically across privacy intake forms, impacting the ability to load these forms.The issue was discovered at 11:50am PST and resolved at 12:05pm PST. This incident has been fully resolved and the platform is functioning normally.
Report: "Issue Detected with Smart Verification"
Last updateThis incident has been fully resolved and the platform is functioning normally.
We have resolved the issue with Smart Verification and we are currently monitoring for any issues. We have resent verification emails to all requesters that encountered an error when attempting to verify their privacy request.
We are continuing work on a fix for this and will post updates as we learn more.
We are actively working on a fix for this and will post updates as we learn more.
We have identified an issue with certain flows for our Smart Verification service. We will post updates as soon as more information is available.
Report: "Access Result Files being sent with partial information"
Last updateThis incident has been fully resolved and the platform is functioning normally.
We have resolved the issue with access result files being sent with partial information and we are currently monitoring. We have re-sent all affected access result emails to the appropriate consumers.
We have identified an issue with access result files being sent with partial information. We are actively investigating and will post updates as we learn more.
Report: "CAPTCHA Failing to Load On Intake Forms"
Last updateThis incident has been resolved and the platform is functioning normally.
DataGrail's Engineering team has resolved the issue with CAPTCHAs not loading and we are currently monitoring for any issues.
DataGrail's Engineering team has identified the cause of CAPTCHAs failing to load on customer intake forms. The team is currently working on resolving this issue. We will provide another update in 15 minutes.
Report: "Emails not sending"
Last updateWe have fixed this issue as of now. Emails should be sending as expected.
Impact has been resolved. Long term fix is ongoing