Gainsight

Is Gainsight Down Right Now? Check if there is a current outage ongoing.

Gainsight is currently Operational

Last checked from Gainsight's official status page

Historical record of incidents for Gainsight

Report: "CS - US1: Investigating Elevated Error Rates"

Last update
postmortem

**Incident**: 27th May, 2025: Some customers may have experienced elevated levels of latency or issues logging into our CS-US1 application. **Root Cause**: A previously undetected cache issue was discovered. This issue caused increased latency for some services.‌ **Recovery Action:** As alerts were received, we blocked queues to investigate. Once the source was identified, a fix was applied which restored performance and availability. Queues were unblocked once stability was confirmed. **Preventive Measures:** The mentioned fix has resolved this issue. We will continue to review and learn from issues like this in effort to provide the best possible customer experience. Please contact [support@gainsight.com](mailto:support@gainsight.com) if there are any questions.

resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating a sudden increase in error rates which may lead to degraded performance or service interruption.

Report: "CS - US1: Investigating Elevated Error Rates"

Last update
Postmortem
Resolved

This incident has been resolved.

Monitoring

A fix has been implemented and we are monitoring the results.

Investigating

We are investigating a sudden increase in error rates which may lead to degraded performance or service interruption.

Report: "CS - EU: Investigating Elevated Error Rates"

Last update
resolved

This incident has been resolved. Self healing measures were triggered to resolve this issue and we monitored for an extended time to ensure stability. We are making threshold adjustments to prevent similar issues moving forward. Please contact support@gainsight.com with any questions.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating a sudden increase in error rates which may lead to degraded performance or service interruption.

Report: "Gainsight CS - US1 - Latency"

Last update
resolved

Queues are back to normal levels.

monitoring

We have identified the issue causing delays and taken recovery actions. Queues are now processing as expected and backlog will clear in next couple of hours. Monitoring closely.

investigating

We are investigating reports of latency in processing queues of CS-US1 environment. We will post further updates as we have more details.

Report: "Restrospective: Issues with C360"

Last update
resolved

Customers may have have experienced issues accessing tabs in C360 today. We have identified and resolved the issue.

Report: "Identified rule failures"

Last update
resolved

The Salesforce issue has been resolved. Gainsight Administrators would have received details regarding any failed rules and can re-run where needed.

monitoring

We are still tracking the ongoing Salesforce incident and will update our status once resolved.

identified

Salesforce status now reports "Feature Degradation" which could still impact the experience for some Gainsight users. We will continue to monitor Salesforce and will update our status once resolved.

identified

We will continue to monitor Salesforce and will update status once resolved.

identified

We have identified that rule failures are occurring due to an ongoing Salesforce (SFDC) incident. Our team is actively monitoring the situation and will provide updates as more information becomes available.

Report: "Delays in Journey Orchestrator"

Last update
resolved

Queues have returned to expected levels.

monitoring

We have identified an issue with the Journey Orchestrator service. A fix is applied to resolve the issue and we are monitoring it closely. We anticipate a delay in the processing, which should catch up in a few hours.

Report: "Gainsight CS - US1 - Degraded Performance"

Last update
resolved

The issue is resolved. We will share RCA details as they become available.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are observing degraded performance in CS-US1.

Report: "Gainsight CS - US1 - Latency"

Last update
postmortem

**Incident:** 26th August, 2024: Some customers may have experienced elevated levels of latency while using our CS-US1 application. **Root Cause:** * There was an increased frequency of database calls tied to a recent hotfix. The increased traffic was intermittent but led to slower API response times and thus impacted customer experience from browser. * Customers in CS-US1 may have experienced the latency while browsing. Impact was relatively less for our CS-US2 and CS-EU regions. * Sanity testing in lower environments did not initially produce this issue which was only observed with production traffic in US1. ‌ **Recovery Action:** Once the source was identified, a subsequent fix was applied which restored performance to expected levels. **Preventive Measures:** The mentioned fix has resolved this issue. We will continue to review and learn from issues like this in effort to provide the best possible customer experience. Please contact [support@gainsight.com](mailto:support@gainsight.com) if there are any questions.

resolved

The issue is resolved. Systems are back to their normal performance level. We will share the RCA as soon as it is available.

monitoring

We are continuing to monitor for any further issues.

monitoring

We are keeping the incident open at this time to monitor for further issues.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are investigating reports of latency with loading pages in CS-US1. We will post further updates as we have more details.

Report: "Retrospective: CS - US1 Export Failures"

Last update
resolved

CS-US1: Some customers may have experienced issues while exporting from Dashboard and Success Snapshot from 13:00 to 18:00 UTC. We restarted related services which resolved this issue. Please email support@gainsight.com with any questions.

Report: "Gainsight CS - EU - Elevated errors in NXT Authentication"

Last update
postmortem

**Incident Summary for issue on 28 May 2024  \(External\)** **Gainsight CS - EU - Elevated errors in NXT Authentication** On **2024-05-28** between **07:31 and 08:45 UTC**, users of the Gainsight Application in the CS EU Cloud experienced intermittent application availability issues. The Gainsight UI was inaccessible for approximately 75 minutes during this window. **Root Cause :** Investigations have identified the following cause of the incident: * An infrastructure component, specifically the backend worker service \(Kubernetes Karpenter\), was upgraded to a newer  version to patch critical security and other updates. * This change had already been successfully executed in the STAGE and other PROD environments. * During the EU environment upgrade, all metadata configurations were transferred except for one critical rule. * The missing rule allowed for UDP communication to DNS Servers. * Due to the absence of this rule, DNS requests could not be resolved, causing microservices on newly provisioned worker nodes to fail. Microservices on older worker nodes were unaffected. * These failures resulted in a significant number of stale threads/connections in a short time frame, rendering the API Gateway unresponsive. * Updating the missing rule in the Network Security Group and reprovisioning the worker nodes resolved the issue. * Pending rule jobs were either skipped or resubmitted as necessary. ‌ **Recovery Action :** 1. Updated the missing UDP rule in the Network Security Group. 2. Restarted all affected services. ‌ **Preventive Measures:** 1. Ensure network rules consistency before and after any upgrade – this process has been initiated. 2. Schedule critical security updates and even low-risk infrastructure changes during non-peak hours, despite previous successes in other environments, to minimize impact.

resolved

This incident has been resolved.

monitoring

We are continuing to monitor for any further issues.

monitoring

Fix is implemented and all services are back to normal. The queues are also released and the jobs will catchup in next couple of hours. We are monitoring closely

identified

The issue has been identified and fix is being implemented.

investigating

Gainsight NXT application is still down. We are working with the upstream service provider. We will post updates as soon as they are available.

investigating

We are investigating errors while logging into the Gainsight NXT application. We will post updates as soon as they are available.

Report: "CS - US1, US2, and EU: We are investigating intermittent errors tied to Cockpit / CTA360."

Last update
resolved

The issue is now resolved and all systems are back to normal.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are investigating an increase in error rates for Cockpit / CTA360, which may lead to degraded performance or service interruption. This may impact performance for services tied to CTA including Timeline.

Report: "Gainsight CS - US1, US2 and EU - Investigating Elevated Error Rates in Rules and Data Designer"

Last update
resolved

The issue is now resolved and all systems are back to normal.

monitoring

A fix has been implemented and we are closely monitoring the results.

identified

We have identified the issue causing the failures and are working on a fix.

investigating

We are investigating a sudden increase in error rates of Rules and Data Designer, which may lead to degraded performance or service interruption.

Report: "Gainsight CS Slowness"

Last update
resolved

This incident is now resolved.

monitoring

We have identified the issue and implemented a fix. The pages are now loading normally. We are monitoring closely.

investigating

We are investigating reports of Gainsight pages taking a long time to load. We will update this page with more details as soon as we have further information.

Report: "CS - US1: Investigating Email Queue Delays"

Last update
resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are continuing to investigate this issue.

investigating

We are investigating email service delays in CS - US1.

Report: "CS - US1: Investigating Rules Queue Delays"

Last update
resolved

Queues have returned to expected levels.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are investigating delays related to our Rules Queue and will provide an update as we have more information.

Report: "CS - US1 Region - Rules failure for a few customers"

Last update
resolved

Issue is resolved.

monitoring

We have identified and fixed an issue that has caused a few 'Rules with scorecards action' failures for a small subset of customers. Impacted customers may review their Rules and rerun the failed ones. Please reach out to support@gainsight.com if you have any questions or need help in rerunning any failures in your org.

Report: "CS - US1 - Delays with Gainsight Assist messages in Timeline"

Last update
resolved

Backlog processing is now complete and all systems are back to normal.

monitoring

A fix has been implemented and we are monitoring the results as we work through the backlog. New messages are still processing and logging as expected. A small number of customers may experience delays for messages queued earlier in the day.

identified

New messages are now processing and logging as expected. Please expect delays with existing messages as we work on the fix.

investigating

We are investigating delays with Gainsight Assist messages displaying in Email to Timeline (E2T). This includes Chrome and Outlook plugin activities.

Report: "Gainsight CS - EU - Rules and Data Designer failure for a few customers"

Last update
postmortem

**Incident:** An isolated number of customers experienced degraded performance in CS-EU Rules and Data Designer on the 9th of January, 2024. **Root Cause:** It was determined that a configuration issue in an underlying service caused the impact for Rules and Data Designer services only. **Recovery Action:** Paused the affected services and corrected the configurations. Once unpaused, services were processing as expected. ‌ **Preventive Measures:** We have made adjustments to configuration and testing controls to prevent issues like this moving forward. Please email [support@gainsight.com](mailto:support@gainsight.com) with any questions.

resolved

This incident has been resolved.

monitoring

We have identified and fixed an issue that has caused a few Rules and Data Designer failures for a small subset of customers. Impacted customers would have received failure notification and can rerun the failed rules and workflows. Please reach out to support@gainsight.com if you have any questions or need help in rerunning any failures in your org.

Report: "CS - EU: Investigating Elevated Error Rates"

Last update
postmortem

**Incident:** An isolated number of customers experienced degraded performance in CS-EU Rules on the 3rd of January, 2024. This could have also intermittently impacted the ability to log into the application. **Root Cause:** This incident was result of an elevated number of API requests coming from a single microservice. The unexpected increase led to a build-up of connections, impacting performance on a subset of API servers. Rate limiting functionality was not configured as expected in this case. **Recovery Action:** Once the affected systems and related traffic were identified, Isolating and restarting effected API services resolved the issue immediately. ‌ **Preventive Measures:** We have corrected the rate limiter functionality for the microservice that caused this issue.

resolved

This incident has been resolved.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating a sudden increase in error rates which may lead to degraded performance or service interruption.

Report: "Gainsight CS - US2 - Rules Delays for a subset of customers"

Last update
resolved

Rules processing is back to its normal state.

monitoring

We observed delays in Gainsight CS US2 Rules processing for a subset of customers. The source of this delay has been identified and addressed. However it can take some time for the queues to catch up. The issue is now resolved and we will continue to monitor closely. Please reach out to support@gainsight.com as needed with any questions.

Report: "Gainsight CS - US1 - Degraded Performance"

Last update
postmortem

**Incident:** 12th February, 2024: Between the times of 6:30 PM and 7:00 PM UTC, some CS-US1 customers may have experienced intermittent delays in application load times. **Root Cause:** The issue was result of query enhancements introduced in the latest release. This caused unexpected API latency which affected multiple areas of CS-US1. Application resiliency measures were effective in maintaining availability but there were still periods of poor performance, especially between 6:48 PM and 7:00 PM UTC. **Recovery Action:** Once the source was identified, the related functionality was rolled back which restored performance to expected levels. A hotfix was applied on the 14th of February for final resolution. **Preventive Measures:** The mentioned fix has resolved this issue. We will continue to review and learn from issues like this in effort to provide the best possible customer experience. Please contact [support@gainsight.com](mailto:support@gainsight.com) if there are any questions.

resolved

This incident has been resolved. We will post RCA details as they become available.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are continuing to investigate this issue.

investigating

We are investigating reports of degraded performance in CS-US1.

Report: "Gainsight CS - US1 - Degraded Performance"

Last update
postmortem

**Incident:** 12th February, 2024: Between the times of 2:30 PM and 3:30 PM UTC, some CS-US1 customers may have experienced intermittent delays in application load times. **Root Cause:** The issue was result of query enhancements introduced in the latest release. This caused unexpected API latency which affected multiple areas of the CS-US1 application. **Recovery Action:** Once the source was identified, the related functionality was rolled back which restored performance to expected levels. ‌ **Preventive Measures:** This issue was previously undetected as it only occurred with very high concurrency. We are modifying testing methods and monitoring solutions to detect issues like this moving forward. We will continue to review and learn from issues like this in effort to provide the best possible customer experience. Please contact [support@gainsight.com](mailto:support@gainsight.com) if there are any questions.

resolved

Between the times of 2:30 and 3:30 PM UTC, some customers may have experienced intermittent delays in application load times. We have identified the affected nodes and have resolved the issue.

Report: "CS-US1: Delays in Rules Queue"

Last update
resolved

Queues have returned to expected levels.

monitoring

We are continuing to monitor for any further issues.

monitoring

We have observed delays in Rules Queues and we are monitoring closely.

Report: "CS - US1, US2, and EU: Investigating issues with GS Assist Chrome Plugin"

Last update
resolved

The enhanced GS Assist chrome plugin(version 3.2.3 ) was published and approved by Chrome web store. Please install latest version for better experience. Reach out to support@gainsight.com if you have any questions. Thank you for your patience.

identified

We are continuing to work on a fix for this issue.

identified

We are still working with Chrome Web Store Support to publish the enhanced Gainsight Assist Extension and will share details once they approve.

identified

The Gainsight team has submitted an enhancement for our Chrome plugin to the Chrome Web Store. However, the review and publication process may take up to 3 business days. We will update the status once the new version is published.

identified

The issue has been identified and a fix is being implemented.

investigating

We have received reports where the GS Assist Chrome Plugin for Gmail is failing to load for some customers, possibly due to interface changes in Gmail. We are investigating and will update when we have more information.

Report: "CS - EU - Investigating Delays"

Last update
resolved

Incident: Some EU customers experienced delays in Rules and Journey Orchestrator on the 8th of June, 2023. Root Cause: After noticing the delays, Engineers temporarily paused related queues to investigate. While troubleshooting, we discovered network connectivity issues tied to underlying hardware. Recovery Action : While still in a paused state, action was taken to assign affected resources to new hardware. Once back online with new hardware, jobs were processing as expected. ‌ Preventive Measures: Redundant architecture design and quick action helped prevent major impact. Still, we are inspecting monitoring, alerting, and automation to ensure optimal design. Additionally, we are confirming self-healing configurations with upstream service providers.

Report: "CS - EU - Delays observed in Connector and People Dataload jobs"

Last update
resolved

We have observed delay in Connector jobs and People Dataload jobs for a subset of customers. Our engineers have identified and fixed the issue. A few customers with Dataload Rule failures would have received failure notification and they can rerun the failed rules and workflows.

Report: "CS - EU: Investigating Elevated Error Rates"

Last update
resolved

We have identified the issue with an upstream service provider. New logins may experience slowness or errors, where existing sessions are not affected. We are worked with the service provider for a fix.

Report: "CS - US1, US2, and EU: Investigating Elevated Error Rates with Authentication"

Last update
resolved

This incident has been resolved.

monitoring

We have identified an issue with an upstream authentication service provider which may have caused errors for some users while accessing Gainsight CS applications. US1 and US2 applications are working as expected at this time but we are monitoring closely.

investigating

We are investigating intermittent errors tied to Gainsight CS authentication. Some users may experience issues with logging in. Users who are already logged into NXT tenants or using the SFDC version of Gainsight CS are not affected. Others may experience slowness while logging in. We will post updates as they are available.

Report: "CS - US1: Cloud Storage Service Interruption"

Last update
resolved

This incident has been resolved. Any failed rules related to this issue would have triggered notifications.

monitoring

Within the last 90 minutes, upstream cloud storage functionality was unavailable intermittently for DataOperations or Dataload jobs, which would have also impacted Rule report downloads for some customers. All functionality is back to normal.

Report: "Investigating Elevated Error Rates with Authentication in US1 and US2"

Last update
resolved

This incident has been resolved and authentication services are performing normally.

identified

We have identified an issue with our authentication service provider which may cause errors for some users while accessing the Gainsight CS application. Users who are already logged into NXT tenants or using the SFDC version of Gainsight CS are not affected. Others may experience slowness while logging in. We are closely monitoring and will post updates as they are available.

Report: "CS-US1 - Degraded performance in Timeline"

Last update
postmortem

**Incident:** Some customers may have experienced degraded performance in CS-US1 Timeline services between 15:44 and 15:48 UTC on the 28th of July. **Root Cause:** A memory issue on multiple web servers led to Timeline becoming unresponsive intermittently, while other servers in the pool continued to serve traffic. **Recovery Action:** Monitoring systems alerted engineers of a memory issue on Timeline web servers. Engineers quickly made adjustments and redeployed Timeline web servers as a temporary fix. ‌ **Preventive Measures:** A permanent hotfix will be deployed in our next release window.

resolved

Some customers may have experienced slowness or errors while trying to load Timeline in CS-US1. Engineers responded quickly to restore services. We will post root cause details as they become available.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are investigating performance issues related to Timeline and will update as we have more information.

Report: "CS-US1 - Investigating - Elevated error rates"

Last update
postmortem

**Incident:** Some customers may have experienced degraded performance while trying to load CS-US1 services between 18:25 and 18:38 UTC on the 18th of July. **Root Cause:** API Gateway instability due to network contention at the data layer was found to be the root cause. **Recovery Action:** Engineers performed a rolling restart of API services once this issue was detected. ‌ **Preventive Measures:** System configuration adjustments have been made to prevent these issues moving forward.

resolved

Some customers may have experienced slowness or errors while trying to load CS-US1 services between 18:25 and 18:38 UTC. Engineers responded quickly to restore impacted services. We will post root cause details as they become available.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are continuing to investigate this issue.

investigating

We are investigating elevated error rates which may result in slowness or availability issues for some customers. More details to follow.

Report: "Investigating Elevated Error Rates with Authentication"

Last update
postmortem

**Incident:** Between 16:14 and 17:16 UTC on the 13th of March, CS-US1 customers may have experienced errors while trying to log into the application. Users who were already logged into NXT tenants or using the SFDC version of Gainsight were not affected. Others may have experienced slowness while logging in. **Root Cause:** This was caused by a widespread network connectivity issue with an upstream Cloud Provider tied to Authentication Services used by our applications. **Recovery Action:** While the Service Provider evaluated manual failover, the network issue at Cloud Provider level was resolved before further action was required. ‌ **Preventive Measures:** The upstream Service Provider is reviewing their detection logic and has enhanced their monitoring mechanisms to identify issues like this moving forward.

resolved

This incident has been resolved. We will relay RCA details as they become available.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We have identified an issue with our authentication service provider which may cause errors for some users while accessing the Gainsight NXT application. Users who are already logged into NXT tenants or using the SFDC version of Gainsight are not affected. Others may experience slowness while logging in. We are closely monitoring and will post updates as they are available.

Report: "US1 - Connector Delays"

Last update
postmortem

**Incident:** Beginning around 12:00 UTC on the 13th of June, Engineers were alerted of elevated queue levels for Connector services in CS-US1. **Root Cause:** A leader node was found to have higher than usual disk activity which prevented optimal job execution for Connector services. **Recovery Action:** Engineers scaled the number of Connector instances to correct the issue temporarily. Additionally, Engineers skipped long-running and duplicate jobs to help recover. **Preventive Measures:** System configuration adjustments have been made to prevent these issues moving forward.

resolved

This incident has been resolved. A subset of customers faced connector queue delays during this incident window. We will add RCA details as they become available.

monitoring

A fix has been implemented and we are monitoring. The Connectors queue was blocked for analysis and troubleshooting during this incident. We have since unblocked, and any duplicate sync jobs were aborted with no data impact. Please expect delays while the queue clears.

identified

The issue has been identified and a fix is being implemented.

investigating

Beginning around 12:00 PM UTC today, we detected a delay in Connectors traffic and adjusted accordingly. As we still have queue delays, we are investigating further and will update as more information becomes available.

Report: "CS-US1 - Data Designer Jobs delayed"

Last update
resolved

We have identified and resolved the issue. We expect the pending jobs to complete in next 2 hours.

investigating

We are investigating reports of delays in Data Designer Jobs. We will post further updates as more information becomes available.

Report: "CS-US1 - Delays observed in Connector and People Dataload jobs"

Last update
resolved

All queue backlogs have cleared. A small subset of customers with Rules failures would have received failure notification and can rerun the failed rules and workflows.

monitoring

A fix has been implemented and we are monitoring. The Connectors queue was blocked for analysis and troubleshooting during this incident. We have since unblocked, and any duplicate sync jobs were aborted with no data impact. Please expect delays while the queue clears.

identified

We have observed delay in Connector jobs and People Dataload jobs for a subset of customers. Our engineers have identified the issue with an underlying infrastructure component. We are working actively to resolve the same. Please reach out to support@gainsight.com with any questions.

Report: "CS-US1 - Retrospective: Isolated Rule Delays"

Last update
resolved

We have received reports of delays in rules executions for a subset of customers. Initial impact started around 6:00 AM UTC on the 27th of July. The source of this delay has been identified and addressed. However it can take some time for the queues to catch up. The issue is now resolved and we will continue to monitor closely. Please reach out to support@gainsight.com as needed with any questions.

Report: "US1 Region - Queues blocked for investigation"

Last update
resolved

This incident has been resolved. Our Network Operations Center Team detected errors from backend systems and immediately blocked queues to prevent major impact. After careful analysis, we were able to resolve and unblock queues. Please reach out to support@gainsight.com with any questions.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

We are investigating an increase in backend exceptions and have temporarily blocked queues to prevent impact.

Report: "US1 Region - Rules and Data Designer failure for a few customers"

Last update
resolved

We have identified and fixed an issue that has caused a few Rules and Data Designer failures for a small subset of customers. Impacted customers would have received failure notification and can rerun the failed rules and workflows. Please reach out to support@gainsight.com if you have any questions or need help in rerunning any failures in your org.

Report: "US1 Region - Rules Workflow failure for few customers"

Last update
resolved

We have identified and fixed an issue that has caused a few 'Rule Workflows' failures for a small subset of customers. Impacted customers may review their Rules and and rerun the failed workflows. Please reach out to support@gainsight.com if you have any questions or need help in rerunning any failures in your org.

Report: "Retrospective: Sandbox Creation Failures"

Last update
postmortem

**Incident:** Attempts to clone or create new tenants were failing. This included customer sandboxes which would be cloned from production instances. **Root Cause :** We discovered that creation would fail at the instance configuration stage. After review with an upstream service provider, we learned there were unannounced changes made with this provider which affected API behavior and ultimately led to the failures. **How this was missed:** We were not made aware of the mentioned API changes until reviewing with the vendor. **Recovery Action :** Once the source of the issue was determined, we made adjustments to API calls which fixed the issue. ‌ **Preventive Measures:** We're working to improve communication with service providers to ensure awareness of pending changes before they happen.

resolved

This incident has been resolved. The sandbox creation issues were tied to API call failures with an upstream service provider. We will release more details as they become available.

monitoring

A fix has been implemented and we are monitoring the results.

investigating

We are continuing to investigate this issue.

investigating

Thank you for your patience as we continue to investigate.

investigating

Customers may experience issues while creating sandboxes from Sandbox Management. Sandbox requests are currently being queued while we investigate.

Report: "Retrospective: Isolated Rule Failures"

Last update
resolved

This incident has been resolved.

monitoring

We have received reports of Rules failing for a subset of customers. The status of Rules execution was spuriously marked as failed in-spite of a successful execution. The source of this anomaly has been identified and addressed. The issue is now resolved and we will continue to monitor closely. Please reach out to support@gainsight.com as needed with any questions.

Report: "US1 Region - Rules Engine and Data Processing Delays for few customers"

Last update
resolved

Queues have returned to expected levels. If you have questions, please open a ticket with us by sending a note to support@gainsight.com.

monitoring

We have identified delays in the Rules Engine and Data Processor Queue for some Customers. We have taken recovery action to help clear the backlog and are closely monitoring as the Queues continue to drain. We anticipate a delay of about 4 hours in the processing. If you have questions, please open a ticket with us by sending a note to support@gainsight.com.

Report: "Partial Impact of Gainsight Home"

Last update
postmortem

**Incident:** "My Portfolio" widgets failed to load for some users with specific filter configurations under Gainsight Home. Users may have experienced the issue between January 09, 2023 - 08:45 UTC and January 10, 2023 - 00:50 UTC. **Root Cause :** A production change was performed on the 9th of January to fix an existing issue. As a result, My Portfolio under Gainsight Home failed to load if new global filters were added. **How this was missed:** As the mentioned fix was applied, multiple rounds of testing were performed. Unfortunately, we did not have test automation set up for the impacted use case. **Recovery Action :** Once the source of the issue was determined, we rolled back the affected module to resolve. Functionality was restored immediately after rollback. We will be rolling out a new, permanent fix in the coming release while taking the affected filters into consideration. ‌ **Preventive Measures:** Review and fix the edge-case scenario. Review of Synthetic Monitoring measures.

resolved

We have deployed a fix to resolve this issue. Please contact support@gainsight.com if you have any questions.

investigating

Thanks for your patience as we continue to investigate this issue.

investigating

Some customers may be experiencing an error when on Gainsight Home. We are aware of the issue and acting to correct it. More information will be available here soon.

Report: "Intermittent failures in loading of Reports, Cockpit, Timeline and Dashboards"

Last update
resolved

Today 20 Dec 2022 after 12:50 UTC, a subset of customers would have experienced intermittent failures while loading of Reports, Cockpit, Timeline and Dashboards. Our team identified this issue and resolved it immediately. The issue was caused due to one single node going faulty during a routine scaleup process. We take availability and performance seriously and are constantly evolving our failure detection workflows to ensure that we mitigate such issues in the future. We apologize for the inconvenience this issue may have caused. Please reach out to support@gainsight.com if you have questions.

Report: "Investigating issues with loading reports"

Last update
resolved

This incident has been resolved. We will follow up with RCA details as they become available.

investigating

We are investigating intermittent issues with Reports impacting a subset of customers.

Report: "Rules Engine Processing Delays for few customers"

Last update
resolved

Queues are back to normal operation. If you have questions, please open a ticket with us by sending a note to support@gainsight.com.

monitoring

We have identified an issue causing processing delays in the Rules Engine Queue for some Customers. We have taken recovery action to help clear the backlog and are closely monitoring as the Queues continue to drain. We anticipate a delay of about 4 hours in the processing of rules. If you have questions, please open a ticket with us by sending a note to support@gainsight.com.

Report: "Rules Engine Processing Delays for few customers"

Last update
resolved

Queues are back to normal operation. If you have questions, please open a ticket with us by sending a note to support@gainsight.com.

monitoring

Our rule processing infrastructure is running behind which is causing delays in the processing rules. No data has been lost and we are closely monitoring.

Report: "Retrospective : Issues While Loading Dashboards"

Last update
resolved

On 2022-09-14 between 11:28 & 12:01 UTC, loading of Dashboards failed Intermittently for about 28 minutes for a subset of customers. Root Cause: A critical security patch was rolled out which caused an unexpected race condition. While our monitor health checks were successful, not all components were in a healthy state. This caused intermittent connectivity for users connecting to the impacted host. Recovery Action: The affected host was removed from the server fleet. We then reviewed logs and confirmed status of the microservice. Preventive Measures: As our upstream service provider deploys patches on start-up, we have disabled this feature so we can analyze and implement an approach that fits our environment. We have also adjusted monitor health checks to catch issues like this moving forward.

Report: "Rule Processing Delays"

Last update
resolved

The fix is working as expected. Please expect rule delays over the next hour as we recover.

monitoring

A fix has been implemented and we are monitoring the results.

identified

The issue has been identified and a fix is being implemented.

investigating

Our rule processing infrastructure is running behind which is causing delays in the processing rules. No data has been lost and we are investigating.