Historical record of incidents for Splunk Cloud Platform
Report: "GCP Products Service Issue | CINC-61229 Partial Outage"
Last updateWe are continuing to work on a fix for this issue.
We are investigating a reported issue for all stacks hosted on GCP. As a result, the Splunk Cloud Platform environment may be inaccessible for customers. Please refer to the current live updates from GCP here - https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW Our teams are monitoring the status of the outage. Your patience is greatly appreciated and we will provide more updates upon resolution.
Report: "<SOAR Playbooks can't be saved> | <CINC-61144> (Major Outage)"
Last updateSplunk is aware of an issue currently affecting SOAR customers. As a result of the issue, users will not be able to save playbooks after making edits. Error message: "Unexpected keyword argument 'code_names' in function call" We understand the impact this has on your operations and are working to resolve the issue as quickly as possible. We will provide more information when it becomes available. No actions are needed from customers at this time. Thank you for your patience and understanding.
Report: "<SOAR Playbooks being disabled> | <CINC-61075> (Degraded Performance)"
Last updateThis fix will be deployed to all cloud stacks on Thursday, June 5, 2025, between 1:00 AM and 3:00 AM local stack time. On-premises customers will receive the updated release for download on June 5, 2025 (PDT). If you wish to apply the workaround before the scheduled fix is deployed, you can do so by following the steps below: - Open the affected playbook in the Visual Playbook Editor (VPE). - Manually review and adjust the join settings (check/uncheck as needed). - Save the playbook to apply the changes. Teams will now begin their root cause analysis. If you believe that you are still impacted by this issue please <a href="https://www.splunk.com/en_us/about-splunk/contact-us.html#customer-support">contact customer support</a> and reference CINC-61075.
Splunk has identified an issue in SOAR version 6.4.1 where Playbooks may appear to load correctly but failing in backend execution. A fix is in progress, and no immediate action is required from customers. As a workaround, to mitigate the impact: - Open the playbook in the Visual Playbook Editor (VPE) - Manually review and adjust the join settings (check/uncheck as needed) - Save the playbook to apply the changes We appreciate your patience and will share further updates as the fix is implemented.
Hello, Splunk Cloud customers using the SOAR platform may be impacted by Playbooks appearing to load but backend logic failing. We have identified the source of the issue in version 6.4.1 and are currently working on implementing a fix. At this time, there is no further action needed from customers. There is an active workaround to mitigate impact: - Open the playbook in the Visual Playbook Editor (VPE). - Manually review and adjust the join settings (check/uncheck as needed). - Save the playbook to apply the changes Your patience is greatly appreciated and we will provide more updates as we implement the fix.
Report: "<SOAR Playbooks being disabled> | <CINC-61075> (Degraded Performance)"
Last updateHello, Splunk Cloud customers using the SOAR platform may be impacted by Playbooks appearing to load but backend logic failing. We have identified the source of the issue in version 6.4.1 and are currently working on implementing a fix. At this time, there is no further action needed from customers.There is an active workaround to mitigate impact:- Open the playbook in the Visual Playbook Editor (VPE).- Manually review and adjust the join settings (check/uncheck as needed).- Save the playbook to apply the changesYour patience is greatly appreciated and we will provide more updates as we implement the fix.
Report: "Splunkbase is Down | CINC-60875"
Last updateFix has been implemented and Splunkbase is operational now along with the monitoring. This incident is now resolved. Thank you
Splunk is aware of an issue currently affecting Splunkbase (splunkbase.splunk.com). As a result of the issue, users are unable to access Splunkbase and are receiving the following error: "The site cant be reached". Users are currently not able to down load apps or do app installs in Cloud through the UI. We understand the impact this has on your operations and are working to resolve the issue as quickly as possible. We will provide more information in 1 hour or when more information becomes available. Thank you for your patience and understanding.
Report: "Splunkbase is Down | CINC-60875"
Last updateSplunk is aware of an issue currently affecting Splunkbase (splunkbase.splunk.com). As a result of the issue, users are unable to access Splunkbase and are receiving the following error: "The site cant be reached". Users are currently not able to down load apps or do app installs in Cloud through the UI.We understand the impact this has on your operations and are working to resolve the issue as quickly as possible. We will provide more information in 1 hour or when more information becomes available.Thank you for your patience and understanding.
Report: "Error messages on CMC Maintenance Windows dashboard | CINC-60549"
Last updateThe fix has been implemented, and the issue is now resolved If you believe that you are still impacted by this issue please <a href="https://www.splunk.com/en_us/about-splunk/contact-us.html#customer-support">contact customer support</a> and reference CINC-60549.
503 errors being returned for ACS calls to list maintenance schedules for several customers. Error message "multiple concurrent requests can cause the service to become unavailable. Retry your request at a later time" were observed on CMC Maintenance Windows dashboard. We have identified the source of the issue and are currently working on implementing a fix.
Report: "Error messages on CMC Maintenance Windows dashboard | CINC-60549"
Last update503 errors being returned for ACS calls to list maintenance schedules for several customers. Error message "multiple concurrent requests can cause the service to become unavailable. Retry your request at a later time" were observed on CMC Maintenance Windows dashboard.We have identified the source of the issue and are currently working on implementing a fix.
Report: "CMC Apps Data Migration Impact on Custom Dashboards | CINC-60310"
Last updateOn March 27th, Splunk updated the CMC app to the latest version. Some customers may be depending on indexes within this app for their custom dashboards and reports, which may no longer function due to the index name change. For more information and guidance on updating your searches, please visit https://splunk.my.site.com/customer/s/article/CMC-new-summary-index.
After March 27, 2025, a modification was enacted in the CMC applications (monitoring and summarizer) that led to the migration of summary events collected by saved searches from the index:summary to the index:_cmc_summary. As a result, dashboards and reports created by customers that depend on events from index:summary are currently non-functional. Our teams are actively working on deploying a patch to restore the functionality of these dashboards and reports; however, there may appear to be a data gap from the March 27 release date through to the time the patch is applied. To regain access to data from this period, customers are required to update their customer created searches and knowledge objects by replacing the phrase (index=summary) with (index=summary OR index=_cmc_summary). This update ensures compatibility with both current and future releases. ⚠ Note: To avoid any data gaps in your dashboards after March 27th, we recommend updating your dashboards and reports. This will ensure consistent data reporting.
Report: "CMC Apps Data Migration Impact on Custom Dashboards | CINC-60310"
Last updateAfter March 27, 2025, a modification was enacted in the CMC applications (monitoring and summarizer) that led to the migration of summary events collected by saved searches from the index:summary to the index:_cmc_summary. As a result, dashboards and reports created by customers that depend on events from index:summary are currently non-functional. Our teams are actively working on deploying a patch to restore the functionality of these dashboards and reports; however, there may appear to be a data gap from the March 27 release date through to the time the patch is applied. To regain access to data from this period, customers are required to update their customer created searches and knowledge objects by replacing the phrase (index=summary) with (index=summary OR index=_cmc_summary). This update ensures compatibility with both current and future releases. ⚠ Note: To avoid any data gaps in your dashboards after March 27th, we recommend updating your dashboards and reports. This will ensure consistent data reporting.
Report: "Incomplete Search Results | CINC-58535 Partial Outage"
Last updateAt 5.45 PM PT, a fix has been identified with our third-party vendor and has been implemented successfully to mitigate the issue.
A fix has been identified with our third-party vendor and has been implemented successfully to mitigate the issue. We are currently monitoring. Thank you for your patience as we work to resolve this issue.
We are continuing to investigate this issue with our third-party vendor. Further updates will be provided as we gather more information. Thank you for your patience as we work to resolve this issue.
We are continuing to investigate this issue with our third-party vendor. Further updates will be provided as we gather more information. Thank you for your patience as we work to resolve this issue.
We are continuing to investigate this issue.
We are continuing to investigate this issue.
We are investigating a reported issue indicating an issue with search for some customers on AWS region US-WEST-2. As a result, the Splunk Cloud Platform environment may experience issues completing searches for customers in the specified region(s).
Report: "Customers unable to install/upload apps | CINC-58054 (Major Outage)"
Last updateWe identified an issue impacting all Splunk Cloud customers regarding the upload and installation of apps due to the following error: "Unable to login with given credentials.". Splunkbase installs were not affected during this impact period. Summary: The issue has been fixed by correcting an incomplete SSL certificate chain for api.splunk.com. Details: During a recent update to the cert, the full chain was not included, causing errors for customers when uploading apps. We manually re-imported the certificate with the complete chain, resolving the error. Internal tests verified that the fix restored normal functionality.
Report: "ACS (Admin Config Service) Down | CINC-57633 (Degraded Performance)"
Last updateThis issue has been resolved. If you believe that you are still impacted by this issue please contact customer support United States and Canada (1.855) SPLUNK.S or (1.855) 775.8657 and reference CINC-57633.
We are investigating a reported issue indicating failures with ACS (Admin Config Service) requests, impacting all Splunk Cloud Platform customers. As a result, customers may experience failures in accessing the self service operations via ACS with an error message "service temporarily unavailable, please try again later". We understand the impact this may have on your operations and we apologize for any inconvenience.
Report: "Splunkbase - App Version Information and Download Links Unavailable | CINC-57529 (Partial Outage)"
Last updateSplunkbase (splunkbase.splunk.com) service is restored . App download links and App versions are presented in the site and function as expected. Thank you for your patience and understanding.
A fix has been implemented for Splunkbase (splunkbase.splunk.com) and the service is now considered restored . App download links and App versions are presented in the site and function as expected. Monitoring is in place to ensure proper function. Thank you for your continued patience as the internal technical teams work towards a full resolution.
We have identified a workaround for users to retrieve app versions and download links. Users can access the classic version of Splunkbase by visiting classic.splunkbase.splunk.com instead of splunkbase.splunk.com to retrieve app information. Thank you for your continued patience as we work towards a full resolution. We will keep you updated with any further developments.
We are investigating a widespread issue affecting Splunkbase (splunkbase.splunk.com). As a result of the issue, users are unable to view version information or download apps from SplunkBase user interface. This issue was detected at approximately 15:00 UTC on August 5th, 2024 and is under active investigation by our internal teams. We understand the impact this has on your operations and are working diligently to resolve the issue as quickly as possible. We will provide updates as more information becomes available. Thank you for your patience and understanding.
Report: "Victoria Experience | CINC-57465 (Degraded Performance)"
Last updateThis issue has been resolved. Teams will now begin their root cause analysis. If you believe that you are still impacted by this issue please <a href="https://www.splunk.com/en_us/about-splunk/contact-us.html#customer-support">contact customer support</a> and reference CINC-57465.
We have implemented a fix for this issue. During this time we are monitoring the results to confirm the resolution and will continue to provide any additional updates once available.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently implementing a fix. Your patience is greatly appreciated and we will provide more updates as we continue to implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently working on implementing a fix. Your patience is greatly appreciated and we will provide more updates as we implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently working on implementing a fix. Your patience is greatly appreciated and we will provide more updates as we implement the fix.
We have identified the source of the issue for Victoria Experience customers and are currently working on implementing a fix. Your patience is greatly appreciated and we will provide more updates as we implement the fix.
We are seeing an issue impacting the Indexer tier for customers on Victoria Experience. As a result of the issue, your indexers may experience a restart every 72 hours. We will provide an update as soon as possible.
Report: "Incident Review | CINC-56605 Major Outage"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
We are continuing to work on a fix for this issue.
We are continuing to make progress towards resolution and will provide more updates as we implement the fix.
We have identified the source of the issue and are currently working on implementing a fix within the next 2 hours. Impacted customers may continue to see error: “Unknown error: Failed to fetch from KV Store” occurring on the Investigations tab of the Enterprise Security application. Within the next 2 hours, customers may also see improvement in their environment. Your patience is greatly appreciated and we will provide more updates as we implement the fix.
We are investigating an issue where the error “Unknown error: Failed to fetch from KV Store” is occurring on the Investigations tab of the Enterprise Security app for several Splunk cloud platform customers. We will provide an update within the next 30 minutes.
Report: "Delayed allow-list changes | CINC-55837 (Degraded Performance)"
Last updateThis issue has been resolved by the provider. If you believe that you are still impacted by this issue please contact customer support United States and Canada (1.855) SPLUNK.S or (1.855) 775.8657 and reference CINC-55837.
We are investigating a reported issue indicating an AWS outage in US-East-1. As a result, customers may experience delays in allow-list changes, including those initiated through Admin Configuration Service (ACS), being processed. We are monitoring for provider updates via https://health.aws.amazon.com/health/status. Your patience is greatly appreciated and we will provide more updates upon resolution.
Report: "Searches impacted | CINC - 55235 ( Degraded Performance )"
Last updateWe have updated automation to apply the fix upon new instance creation eliminating the impact of this incident.
We have implemented a fix for this issue and all Cx stacks have been repaired. Remaining are Single Instance stacks are expected to complete 2/17/24 PDT. During this time we are monitoring the results to confirm the resolution and will continue to provide any additional updates once available.
We have implemented a fix for this issue. During this time we are monitoring the results to confirm the resolution and will continue to provide any additional updates once available.
Our technical resources has identified the issue. The hot-fix has been applied on the fleet. Please be aware, once the issue will be confirmed as resolved the Root Cause Messaging will be shared accordingly.
Splunk has identified the issue causing memory growth on the indexers. Splunk is taking steps to remediate over the next 3 hours.
We are investigating a potential issue where Splunk instances are experiencing out-of-memory events, causing searches to fail or take longer to complete on multiple Indexers and search heads that may impact several Splunk cloud platform customers. Our teams are working to resolve this issue. Your patience is greatly appreciated and we will provide more updates upon resolution.
Report: "Searches and indexing impacted | CINC - 54771 ( Degraded Performance )"
Last updateThis issue has been resolved. Please <a href="https://www.splunk.com/en_us/about-splunk/contact-us.html#customer-support">contact customer support</a> for any additional support you may need and reference CINC-54771. Thank you for your patience during this outage.
We have identified an incident with customers in selected US-East region who may be experiencing Search and Indexing performance degradation. The remediation path has been identified and the support teams are working towards recovery.
Report: "Customer’s Search Heads may not be able to resolve DNS address for new Indexers | CINC-54644 (Degraded Search Performance)"
Last updateThis issue has been resolved. Teams will now begin their root cause analysis. If you believe that you are still impacted by this issue please <a href="https://www.splunk.com/en_us/about-splunk/contact-us.html#customer-support">contact customer support</a> and reference CINC-54644.
We are continuing to monitor for any further issues.
We have implemented a fix for this issue. During this time we are monitoring the results to confirm the resolution and will continue to provide any additional updates once available.
We have identified an issue that may degrade search performance beginning 02:00 AM UTC January, 5th 2024. Issue is actively being mitigated and monitored. Impacted customers may see search error and/or warning messages, such as: - “Error resolving: Name or service not known Socket error: Cannot resolve hostname” - “Unknown error for indexer: [...]. Search Results might be incomplete.” Your patience is greatly appreciated and we will provide more updates as we implement the fix.
Report: "ACS (Admin Config Service) errors | CINC-53834 Major Outage"
Last updateThis issue has been resolved. Teams will now begin their root cause analysis. If you believe that you are still impacted by this issue please contact customer support United States and Canada (1.855) SPLUNK.S or (1.855) 775.8657) and reference CINC-53834.
We have identified the source of the issue and are currently working on implementing a fix. Impacted customers will continue to experience failures in allow-list changes being processed. Your patience is greatly appreciated and we will provide more updates as we implement the fix.
We are investigating a reported issue indicating failures with ACS (Admin Config Service) requests, impacting all Splunk Cloud Platform customers. As a result, customers may experience failures in allow-list changes being processed. We understand the impact this may have on your operations and apologize for any inconvenience.
Report: "Delayed allow-list changes | CINC-51960 (Degraded Performance)"
Last updateThis issue has been resolved by the provider. If you believe that you are still impacted by this issue please contact customer support United States and Canada (1.855) SPLUNK.S or (1.855) 775.8657 and reference CINC-51960.
The provider has implemented the fix. During this time we are monitoring the results to confirm resolution and will continue to provide any additional updates once available.
AWS teams continuing to work on a fix for this issue.
AWS teams continue to work on a fix for this issue.
We are investigating a reported issue indicating an AWS outage in US-East-1. As a result, customers may experience delays in allow-list changes being processed. Our teams are working with the provider to resolve this issue. Your patience is greatly appreciated and we will provide more updates upon resolution.