Historical record of incidents for Causaly
Report: "Disruption to backend service"
Last updateThis morning at about 07:30 GMT, one of our backend systems experienced a periodic outage that may have affected some client logins. The issue was resolved, and the service was restored within 25 minutes. The outage has been resolved, and Causaly services are fully restored. We appreciate your patience and apologize for any inconvenience.
Report: "Disruption To Service"
Last updateThis incident has been resolved.
We have migrated to a secondary Azure GPT endpoint and resumed service. Our teams will monitor the situation until it is fully resolved. Due to increased demand for Azure services we may see intermittent and temporary service interruption
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
Azure GPT is currently experiencing a service outage, which is impacting our Copilot service. Our teams are currently working to find a solution to restore availability as soon as possibility, and we will inform you as soon as this is complete.
Report: "Application Outage"
Last updateAs per the previous outage on the 20th September, we have again observed long-running queries reduce the availability of the system. We have increased capacity in the pool to restore stability in the platform. Fortunately during this incident the team have been able to pin-point the specific problem and will release a code change to prevent future occurrences of this problem. We will review the auto-scaling process to ensure that capacity is better managed in the future.
Report: "Application Outage"
Last updateLong running queries reduced the available capacity on the platform; customers would have been unable to login whilst the auto-scaling increased the application server pool and the platform stabilised.
Report: "Increased latency"
Last updateDuring the scheduled release of an update we inadvertently left the application pool with insufficient capacity for the level of usage we were experiencing at the time. This resulted in slow behaviour and partial query timeouts for a period of around 10 minutes whilst the code was deployed and the system capacity stabilised.
Report: "Issues With Multi-Hop"
Last updateWe are currently investigating elevate error rates and increased latency with our multi-hop feature.
Report: "Internal Health Check Reporting A False Positive"
Last updateOn Friday 21th April at approximately 12:05 (UTC +1) our internal health check reported an outage which was automatically added to our status page. As no functions were disrupted according to our tests, this incident was removed.
Report: "Deterioration of Performance Due To A Connectivity Issue"
Last updateA deterioration of performance due to a connectivity issue was detected today between 18:15 (UTC+1) and 18:45 (UTC+1) causing a part of the Causaly Cloud to response slowly for some users. The issue has now been resolved.
Report: "Internal Health Check Reporting A False Positive"
Last updateOn Wednesday 29th March at approximately 19:49 (UTC +1) our internal health check reported an outage which was automatically added to our status page. Causaly Cloud was operational and fully available throughout this time and more details are available regarding the underlying cause from Google (https://status.cloud.google.com/incidents/nieR2aLyg1rwFKq1aWZU#7Nro3snGH5YXCyUfQuKS). It was determined that high and variable network latency was affecting the connection between our monitoring platform and our customer platform and resulting in sporadic but invalid failing checks. These 'false positive' alerts did not impact the availability of our services.
Report: "Issues With Multi-Hop"
Last updateThe issue has been fixed and the application is running normally.
Monitoring in progress.
A fix has been implemented and we are monitoring the results.
We are currently investigating issues with our backend services related to multi-hop and delta analysis
Report: "SSL Certificate Issue"
Last updateThis incident has been resolved.
The new SSL certificate has been issue and has fixed the issue. Causaly Cloud is now running normally.
A new SSL certificate is being created and issue at the moment.
We are currently investigating an issue with out SSL certificate that is managed by Google Cloud Servcies
Report: "Internal Health Check Reporting A False Positive"
Last updateOn Tuesday 18 October at about 07:30 UTC our internal health check reported a false positive which was automatically added to the Causaly incidents page. As no functions were disrupted according to our tests, this incident was removed.
Report: "Login Issue"
Last updateAn issue was detected with the authorization component which meant some customers may have experienced a problem when logging into the Causaly application. This only lasted about 5 minutes as it was quickly resolved once detected. The Causaly application is now fully operational.
Report: "Login Issue"
Last updateCloudFlare have now resolved the underlying issue
We are continuing to monitor for any further issues.
Auth0 is now investigating the issue. https://status.auth0.com/incidents/808fhwf0jb0g The issue is related to a CloudFlare issue: https://www.cloudflarestatus.com/incidents/xvs51y9qs9dj
Our authorization service provider (Auth0) is currently having an outage/issue. This will affect users logging onto the Causaly platform (med.causaly.com).
Report: "Deterioration of Performance Due To Network Issue"
Last updateA deterioration of performance due to a network issue caused a part of the Causaly Cloud to response slowly. The issue has been identified and further analysis is underway.
Report: "Corporate Website SSL Certificate"
Last updateIssue is now resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue with our SSL certificate that is only affecting our corporate website, Causaly Cloud is fully operational.
Report: "Deterioration of Performance Due To Network Issue"
Last updateA deterioration of performance due to a network issue caused a part of the Causaly Cloud to response slowly. The issue has been identified and further analysis is underway.
Report: "Internal Database Issue"
Last updateBetween 18:00 (GMT) to 18:50 (GMT) on Thursday 17th Febrarury and between 14:50 (GMT) to 14:55 (GMT) one of the internal databases had a performance issue which causes a part of the Causaly Cloud which caused the search to repsonse slowly. The issue has been identified and a further analysis is needed to find the root cause.
Report: "Internal Database Issue"
Last updateBetween 12:35 (GMT) to 13:00 (GMT) one of the internal databases had an issue which causes a part of the Causaly Cloud not to respond - specifically multihop search and article paper display. The issue has been resolved and a further analysis is needed to find the root cause. This will be published once it has been identified.
Report: "Intermittent Platform Outages"
Last updateCertain network configurations of our Platform led to mismanagement of high traffic spikes and increased rate of aborted requests. The network configurations were updated and the issue is resolved.
From Wednesday 22nd December 2021 at about 20:00 (GMT) to Thursday 23rd December 2021 at about 06:30 (GMT) the Causaly Platform experienced intermittent outages which could have prevented users logging into and using the Causaly Platform. The issue was related to a service hosted and managed by Google Cloud Platform (GCP). The root cause is still being investigated and once it is known a postmortem of the incident will be published.
Report: "Login Issue"
Last updateAuth0 have identified that this is a result of a larger-scale incident with their cloud provider AWS. Users are able to log into the Causaly platform now.
We are currently monitoring the situation as it seems the logins to platform are working however Auth0 haven't confirmed the fix is finalized.
Auth0 is now investigating the issue. https://status.auth0.com/incidents/2mstm0ylgvfn
Our authorization service provider (Auth0) is currently having an outage/issue. This will affect users logging onto the Causaly platform (med.causaly.com).
Report: "Internal Health Check Reporting A False Positive"
Last updateOn Thursday 2nd December at about 09:18 UTC our internal health check reported a false positive which was automatically added to the Causaly incidents page. As no functions were disrupted according to our tests, this incident was removed.
Report: "Internal Health Check Reporting A False Positive"
Last updateResolved - On Wednesday 17th November at about 12:25 UTC our internal health check reported a false positive which was automatically added to the Causaly incidents page. As no functions were disrupted according to our tests, this incident was removed.
Report: "Google Cloud Platform Issue"
Last updateOn Tuesday 16th November 2021 from approximately 17:48 UTC until 18:12 UTC our cloud service provider Google Cloud Platform \(GCP\) had a network issue that affected all their customers. The impact to the Causaly Platform was users were not able to access the platform. If you wish to view the full details about the GCP incident, the information can be found on the following link: [https://status.cloud.google.com/incidents/6PM5mNd43NbMqjCZ5REh](https://status.cloud.google.com/incidents/6PM5mNd43NbMqjCZ5REh)
Google Cloud Platform seem have to fixed the issue - we will continue to monitor and provide further details once we have them.
The issue on Google Cloud Platform seems to be fixed - we are now monitoring.
The issue on Google Cloud Platform seems to be fixed - we are now monitoring.
We are currently aware of an issue affecting our services. We believe this is the same issue effecting many other sites and apps around the world, potentially related to Google Cloud Platform. We will update further, as soon as we have more information.
Report: "Internal Health Check Reporting A False Positive"
Last updateOn Tuesday 09th November at about 09:53 UTC our internal health check reported a false positive which was automatically added to the Causaly incidents page. As no functions were disrupted according to our tests, this incident was removed.
Report: "Login Issue"
Last updateOn Monday 6th September our authorization service provider (Auth0) had an outage roughly between 23:43 UTC to 23:59 UTC, which would have affected logging into the Causaly Platform. The issue has now been resolved and full details can be found on the following link: https://status.auth0.com/incidents/rl95ff4gjp7f
Report: "Login Issue"
Last updateOur authorization service provider (Auth0) had an outage roughly between 18:30 UTC to 21:05 UTC, which would have affected logging into the Causaly Platform. The issue has now been resolved and full details can be found on the following link: https://status.auth0.com/incidents/89d7218yhxhv
Report: "Login Issue"
Last updateOur authorization service provider (Auth0) had an outage roughly between 22:50 UTC to 23:20 UTC, which would have affected logging into the Causaly Platform. The issue has now been resolved and full details can be found on the following link: https://status.auth0.com/incidents/5j06wdcvzpn8
Report: "Login Issue"
Last updateOur authorization service provider (Auth0) had an outage roughly between 15:55 UTC to 16:05 UTC, which would have affected logging into the Causaly Platform. The issue has now been resolved and full details can be found on the following link: https://status.auth0.com/incidents/k8yb7w2sq96j
Report: "Internal Health Check Reporting A False Positive"
Last updateOn Monday 17th May at about 18:25 UTC our internal health check reported a false positive which was automatically added to the Causaly incidents page. From our investigation, we found that the a component in GCP that the health check uses had an issue: https://status.cloud.google.com/incidents/rf3TWPFD9xQxK7eXZX2K As no functions were disrupted according to our tests, this incident was removed.
Report: "Login Issue"
Last updateOur authorization service provider (Auth0) had an outage roughly between 00:32 UTC to 00:58 UTC, which would have affected logging into the Causaly Platform. The issue has now been resolved and full details can be found on the following link: https://status.auth0.com/incidents/9yydljbrq263
Report: "Auth0 Reporting A False Positive"
Last updateOn Tuesday 27th April at about 18:50 UTC Auth0 reported a false positive which was automatically added to the Causaly incidents page. As no functions were disrupted according to our tests, this incident was removed.
Report: "Login Issue"
Last updateOn Tuesday 20th April 2021 from approximately 15:36 UTC until 19:49 UTC our authorization service provider Auth0 had a DB failure that affected all their customers within the US-1 region. The impact to the Causaly Platform was users were not able to login and for users who were logged in, requests were slow and/or failing. We will discuss this incident in depth internally and develop a plan to avoid such lasting outages to our authentication service in the future. If you wish to view the full details about the Auth0 incident, the information can be found on the following link: [https://status.auth0.com/incidents/zvjzyc7912g5](https://status.auth0.com/incidents/zvjzyc7912g5)
The platform is now working normally.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
Auth0, our authorization service provider, is currently down, which affects logging into the Causaly Platform. Auth0 is currently investigating the issue
Report: "Internal Component Issue"
Last updateAn issue was detected with one of the internal components which may have resulted in some search queries not responding. This only lasted for about 10 minutes as it was quickly resolved once detected. The Causaly application is now fully operational.
Report: "Login Issue"
Last updateAn issue was detected with the authorization component which meant some customers may have experienced a problem when logging into the Causaly application. This only lasted about 5 minutes as it was quickly resolved once detected. The Causaly application is now fully operational.
Report: "Causaly Portal Is Down - https://med.causaly.com/"
Last updateThis incident has been resolved.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.