Is Lucidworks Platform Down Right Now? Check if there is a current outage ongoing.

Due to a widespread outage with our public cloud provider, the SaaS Platform user console is currently unavailable. This incident is affecting Lucidworks customers' ability to use LWAI, as well as Connected Search's ability to serve queries. We are actively working with our provider to reach a resolution as soon as possible, and will provide regular updates here.

## Summary On May 26, 2025, at approximately 17:50 UTC, clients began experiencing errors when attempting to access various parts of the Lucidworks platform, including LWAI and Connected Search. The incident was not immediately detected by our internal monitoring and alerting system. It was identified only after user reports and internal investigation, delaying the time to resolution. A recent change to the TLS certificate configuration used by these services was reverted at 20:04 UTC, resolving the issue. ## Root Cause The Lucidworks Platform utilizes several domain name spaces \(zones\) to provide its services. Critically, the `*.applications.lucidworks.com` zone houses addressable endpoints for all Applications powered by the Platform, including LWAI Applications and their associated Integrations \(which are used by Managed Fusion for intra-Platform communication\). During a routine update intended to replace an expiring certificate for `*.datasources.lucidworks.com `– an unrelated zone – the engineering team inadvertently deployed this certificate to several additional subdomains, including `*.applications.lucidworks.com`. Because the newly deployed certificate was only valid for `*.datasources.lucidworks.com`, services such as `*.applications.lucidworks.com` began serving an invalid certificate for their domain. This misalignment led to backend connection failures and disrupted user access to key parts of the Lucidworks Platform, including LWAI Applications and their associated Integrations, as well as Connected Search functionality. Lucidworks Engineering corrected the issue by rolling back the incorrect certificate and redeploying the correct certificates to all affected subdomains. ## Lucidworks Actions Lucidworks will take the following actions as a result of this incident: * **Automate Certificate Renewal**: We will implement an automated certificate renewal and deployment pipeline to remove manual steps and reduce the risk of human error. * **Enhanced Synthetic Testing:** We will expand synthetic monitoring coverage to include all Lucidworks subdomains, with continuous validation of TLS certificate alignment and service accessibility. While such monitoring is already in place, this incident has revealed that it is not currently extensive enough to capture all potential failure scenarios, so we are actively working to enhance it to ensure it will do so in the future. ## Recommended Client Actions There are no recommended client actions as a result of this incident.

We’ve confirmed that all Lucidworks Platform functionality related to this incident has been fully restored. We will share more details in a postmortem update as soon as our analysis is complete.

We have identified a misconfiguration related to the TLS certificate configured for the applications.lucidworks.com domain. This configuration has been reverted and the SSL handshake errors we had been seeing have stopped. We are monitoring to confirm stability.

The specific errors we’re seeing are “SSL handshake fails for connections to *.applications.lucidworks.com.” Our team is pursuing all avenues of investigation with a focus on TLS certificate configurations for endpoints in the applications.lucidworks.com DNS domain.

We have been alerted to Lucidworks SaaS Platform errors being returned in response to ingestion and query API calls for Connected Search clients, as well as for LWAI calls coming via Managed Fusion Integrations. The Platform UI continues to function, and configuration changes can still be made. The Lucidworks team is investigating.

We’ve confirmed that all Lucidworks Platform functionality related to this incident has been fully restored. We will share more details in a postmortem update as soon as our analysis is complete.

We have identified a misconfiguration related to the TLS certificate configured for the applications.lucidworks.com domain. This configuration has been reverted and the SSL handshake errors we had been seeing have stopped. We are monitoring to confirm stability.

The specific errors we’re seeing are “SSL handshake fails for connections to *.applications.lucidworks.com.” Our team is pursuing all avenues of investigation with a focus on TLS certificate configurations for endpoints in the applications.lucidworks.com DNS domain.

We have been alerted to Lucidworks SaaS Platform errors being returned in response to ingestion and query API calls for Connected Search clients, as well as for LWAI calls coming via Managed Fusion Integrations. The Platform UI continues to function, and configuration changes can still be made. The Lucidworks team is investigating.

## Summary On April 10, 2025, starting at approximately 15:43 UTC, users of the Lucidworks Platform UI at [platform.lucidworks.com](http://platform.lucidworks.com) may have experienced issues accessing or altering their Lucidworks Platform product configurations. This was due to an error introduced during the deployment of a new service as part of a forthcoming product rollout. The issue was identified and resolved by 16:19 UTC.  ## Root Cause The incident occurred when a new service was deployed with an incorrect network configuration. The Lucidworks Platform uses a global load-balancing and routing system to direct traffic to the various services it provides. When introducing a new route to accommodate a newly-developed service, a misconfiguration of that routing layer inadvertently resulted in UI traffic being directed to this new service, which affected the reachability of [platform.lucidworks.com](http://platform.lucidworks.com) for all users. Our engineering team identified the problem and resolved it by reverting the routing configuration to the previous stable state. Our internal investigation revealed that the configuration error stemmed from incomplete internal documentation as well as a need for more rigorous processes for deploying new services. We are taking steps to address these internal factors to prevent similar issues in the future, as described below. Detection of this incident could have been more timely.  Our postmortem analysis revealed a misconfigured alert that was incorrectly getting set to a lower severity than is appropriate for this level of impact. This misconfiguration has been corrected. ## Lucidworks Actions Lucidworks is committed to providing a stable and reliable platform experience for all users. As a result of this incident, we are taking the following actions: * We have updated our internal documentation and tooling to make it clearer to our engineers how to introduce a new service onto the Platform without affecting any others. * We are implementing stricter controls and validation processes for new service deployments, to ensure they are first rolled out in our dev environment for functional validation, and to confirm their introduction does not affect any existing services, prior to deploying them into production. * We are also enhancing the way we manage routing configurations on the Platform in order to make all routing rules more strict, to ensure it will not be possible for traffic intended for one service to be inadvertently routed to a different service in the future. * As mentioned above, we have fixed a misconfigured alert to appropriately notify our teams with the highest severity, should something similar to this incident happen again in the future. ## Recommended Client Actions There are no recommended client actions as a result of this incident.  If you have any follow-up questions or concerns, please submit a request with [Lucidworks Support](https://support.lucidworks.com/).

A change related to enabling a new product feature was rolled out that had the unintended effect of making our Platform UI unavailable, which was not caught in our development and testing environment. That change has been rolled back, and the UI is once again available. No other product functionality was impacted by this incident. We will share more details in a postmortem update as soon as our analysis is complete.

The Lucidworks Platform UI at platform.lucidworks.com is currently unavailable. This has no effect on product functionality other than users' ability to change configurations. Our teams are actively working to resolve the issue.

# Summary Beginning at approximately 10:17am PDT and lasting until service restoration at approximately 11:52am, the Lucidworks Signals Beacon on client sites was not functional due to an error in retrieving its configuration from the Lucidworks Platform. During this time, some signals from new site visits were not collected. # Root Cause A database change that was intended to only be applied in our development environment for testing purposes was inadvertently applied to the production database, which resulted in the creation of new, empty tables for storing Beacon configurations and authorization information. For the duration of this incident, when the Beacon initialized and attempted to retrieve its configuration from the Lucidworks Platform at page load time, the call failed with an HTTP 401 Unauthorized response, and the Beacon would silently fail to load. This behavior, while not site-impacting in any way, does result in the inability to send user behavior signals to Lucidworks Analytics, and so for the duration of this incident some analytics data for affected clients will be incomplete. The database change was rolled back, which allowed the Platform to once again successfully return Beacon configurations, which had persisted in the previously-used table throughout the duration of the incident.  No configuration data was lost, and Signals Beacon functionality has been fully restored. # Lucidworks Actions We will update our Beacon and Signals APIs to return HTTP 500 errors when the configuration cannot be loaded, in order to generate a higher-severity alert for our teams so that we can more urgently react to similar issues in the future. We will also enhance the Beacon to cache the most recently retrieved configuration in local browser storage so that there is a fallback in addition to a potential backoff strategy to minimize throwing errors. # Recommended Client Actions There are no recommended client actions as a result of this event. Affected customers with additional questions or concerns may contact us via[ Lucidworks Support](https://support.lucidworks.com/).

The fix has been verified and we've confirmed that full functionality has been restored. No Signals Beacon configurations were lost during this incident.

We identified the source of the error as being a database configuration update that was inadvertently applied to our production site, rather than in our development environment as intended. We have rolled back that change, and the Beacon is able to successfully load its configuration again. We're continuing to monitor to ensure there are no ongoing effects of this incident.

The Lucidworks Signals Beacon is currently unable to load its configuration due to a server-side error on the Lucidworks Platform. As a result, while existing site visits will continue to send signals to Lucidworks Analytics, new page loads will not be able to initialize the Beacon and some signals will not be collected. We are working to restore full functionality as soon as possible, and will provide timely updates as this incident is addressed.

A configuration change in our [ArgoCD](https://argo-cd.readthedocs.io/en/stable/#what-is-argo-cd) continuous deployment platform caused nondeterministic behavior when applied to our Kubernetes [ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/#what-is-ingress) controllers.  Though this change was successfully applied in our development environment, when the same configuration was deployed to the production [Springboard UI](https://platform.lucidworks.com/), it put the cluster into an inaccessible state.  We manually intervened to roll back the change as quickly as possible, but found that ArgoCD was still intermittently mis-applying the configuration in this environment.  Further troubleshooting efforts identified a stale cache as the culprit; once we forced a hard reset of ArgoCD’s cache, the correct configuration was applied, and the configuration UI returned to a fully functional state. No Connected Search indexing or query functionality was affected during this incident; the extent of the impact was limited to the web-based configuration interface, and no data loss occurred.

We have confirmed that reverting our earlier change has corrected the root cause of this incident, and platform.lucidworks.com is once again fully functional.

The platform.lucidworks.com interface is once again available and functional. No indexing or search query outage was experienced during this incident. We're continuing to monitor to ensure there are no lingering issues.

We have identified the source of the issue and have reverted the recent change that caused it. The fix is actively rolling out. Some users will see recovery and be able to log in to platform.lucidworks.com already, while for others it may not show full recovery until the rollout has fully completed.

The configuration UI at platform.lucidworks.com is currently unavailable and returning errors when users attempt to access it. Lucidworks engineers are actively investigating the problem. This incident does NOT affect any indexing or search functionality; it is only preventing configuration changes to existing Connected Search applications. We will post updates as soon as we have more details to share.

Customers affected by this incident may request a detailed Event Analysis Report via their TAM or through Lucidworks Support.

Our upstream vendor has deployed a fix for the bug that was impacting some Connected Search customers, and Lucidworks has verified that the issue is now fully resolved.

Our upstream vendor has identified the cause of the bug and is working on an urgent fix. In the meantime, we are keeping our mitigation in place and continuing to monitor to ensure continued operations and functionality of Connected Search for all customers. We will provide further updates as soon as we have them.

We have confirmed the underlying cause of this issue, and it has been mitigated at this time. A particularly resource-intensive query was found to be impacting the Connected Search backend for all customers by triggering a bug in the underlying data store service. Disabling this query has returned service to normal, and all Connected Search and Springboard UI functionality should be fully restored at this time. Lucidworks will follow up on a root-cause analysis with the upstream vendor in question, and will update our own findings here as we have more details to share.

We continue to work to fully resolve the source of the query timeouts, while minimizing the impact as much as possible. We believe we have identified the root cause and are now pursuing corrective actions. We'll continue to provide updates as soon as more information is available.

We have identified the source of intermittent query failures and occasional timeouts, and are working with an upstream vendor to resolve them. We do not have an ETA for full resolution at this time, but believe that our initial troubleshooting has already lessened both the frequency and severity of these query issues. We will continue to provide updates as the situation develops.

Some Connected Search requests to the Query API are intermittently timing out, which results in them returning either empty or inconsistent results. This issue is affecting all Connected Search customers, as well as the Experience Optimizer section of the Springboard UI (platform.lucidworks.com). Lucidworks engineers are investigating the cause of this issue and updates will provided here as soon as we have them.

Connected Search customers who were affected by this incident may request a detailed Event Analysis via Lucidworks Support.

An internal routing component of Connected Search had gotten into a stuck state where it could not validate a newly-rotated TLS certificate used to route traffic to the search backend. By restarting this component, we were able to mitigate the issue. We have updated our tooling to force a restart of this component every time the TLS certificate is rotated, in order to avoid a recurrence.

We have mitigated the issue, and all search functionality (including both API-based search requests, as well as the ability to use Experience Optimizer in our platform UI) has been restored.

The Lucidworks Connected Search platform is experiencing an issue with routing traffic that is resulting in failed search query requests. This issue is also affecting the ability to use Experience Optimizer from within the platform web interface.