Historical record of incidents for Lexer
Report: "File Upload API and Secure File Upload Outage"
Last updatePartial service outage affecting file uploads 28 May 2025 08:00 AEST – 3 June 2025 11:00 AEST Between 08:00 AEST on 28 May 2025 and 11:00 AEST on 3 June 2025, Lexer experienced an outage affecting two upload methods: the File Upload API and the Secure File Upload (drag and drop) feature. During this period, all file upload attempts via these channels failed. If you attempted to upload files between 28 May and 3 June via the affected methods, please reupload them to ensure they are processed. The issue was resolved as of 11:00 AEST on 3 June 2025. File uploads are now functioning as expected. We apologise for the disruption and appreciate your understanding.
Report: "File Upload API and Secure File Upload Outage"
Last updatePartial service outage affecting file uploads28 May 2025 08:00 AEST – 3 June 2025 11:00 AESTBetween 08:00 AEST on 28 May 2025 and 11:00 AEST on 3 June 2025, Lexer experienced an outage affecting two upload methods: the File Upload API and the Secure File Upload (drag and drop) feature. During this period, all file upload attempts via these channels failed. If you attempted to upload files between 28 May and 3 June via the affected methods, please reupload them to ensure they are processed.The issue was resolved as of 11:00 AEST on 3 June 2025. File uploads are now functioning as expected.We apologise for the disruption and appreciate your understanding.
Report: "Hub.lexer.io performance degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We are experiencing a high rate of search timeout errors when loading conversations in Listen and Respond. Our team is continuing to investigate the root cause.
We are continuing to investigate this issue.
We're investigating reports of degraded performance for hub.lexer.io.
Report: "Track metrics failing"
Last updateThis incident has been resolved. All dashboards are now successfully collecting daily metrics. Most dashboards will display metric gaps between July 16 and July 25 during this outage period. Unfortunately we are unable to replay these missed metrics.
A fix has been implemented and we are monitoring results.
We've identified an issue causing track metric failures since 16 July. Our team has identified the root cause and we're working on a resolution.
Report: "Email processing delay in Respond"
Last updateOur email service provider has fixed the root cause and resolved the incident.
A fix has been implemented and we are monitoring the results.
We've identified the issue with our email service provider and we're working on a resolution.
We're investigating reports of delayed emails in Respond.
Report: "Activation performance degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating a problem with the activation attributes.
Report: "Performance Degraded"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We're currently investigating degraded performance in the Hub.
Report: "Hub performance degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're investigating reports of degraded performance for hub.lexer.io.
Report: "Conversation processing delays"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're investigating conversation processing delays for integrated Facebook and Email accounts.
Report: "Facebook message publishing errors"
Last updateThis incident has been resolved.
Meta has indicated they are recovering from disruptions and we're observing a significantly lower rate of errors. We're continuing to monitor results.
Meta (Facebook) has acknowledged the issue and has assigned an engineering team to resolve the issue as quickly as possible. See more information here: https://metastatus.com/messenger.
We believe the root cause to be an issue with the Facebook API as this is impacting other platforms. We have raised this issue with Facebook and we're awaiting a response or resolution.
We're currently investigating a high rate of errors for publishing replies to Facebook messages.
Report: "Issues with Facebook Integrations: Displaying as disconnected"
Last updateThis incident has been resolved.
A fix has been implemented and we're monitoring results. Facebook pages integrations are now accurately displaying their connection status.
All Facebook pages integrations are displaying as 'disconnected,' while Instagram and Facebook ads integrations remain unaffected. Users are encountering difficulties when attempting re-integration, as they encounter errors in the process. It's crucial for users to refrain from re-integration attempts at this time.
Report: "Meta experiencing API issues"
Last updateThis incident has been resolved.
We have not observed the error for over 24 hours with reports from clients that Facebook content is being consumed as normal. Although it is unconfirmed by Meta, we believe they have fixed it.
Meta is experiencing an issue with their API that is impacting Respond users ability to reply to Facebook content.
Report: "Search Performance Degraded"
Last updateThis incident has been resolved.
We have identified the root cause for the issue and a fix is currently being deployed.
We're currently investigating degraded performance to search.
Report: "Data Processing Delays - AWS network connectivity"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Our data processing infrastructure is running behind which is causing inaccuracies in the reporting tools. No data has been lost and the system should be caught up shortly. AWS status page is being monitored closely for updates - https://health.aws.amazon.com/health/status
Report: "Lexer Forms are not displaying properly"
Last updateThis incident has been resolved.
Report: "Lexer Forms are not displaying properly"
Last updateWe're experiencing issue on selected Lexer forms are not loading properly and are currently looking into the issue.
Report: "Facebook page integration errors"
Last updateThis incident has been resolved.
While Meta has not issued an official resolution, we have not encountered any errors in the last 24 hours. We are continuing to monitor the situation.
We're continuing to observe publishing errors and we've informed Meta with details to facilitate a resolution.
Facebook is currently experiencing issues with publishing replies to comments. Although Facebook says that your message has failed to publish, in some cases we can see that comments have in fact been created on your page. We recommend checking facebook.com if you encounter publishing errors for comments while this issue remains active. If your reply has been published successfully, we recommend adding an internal comment in Lexer and closing the comment. We're working with Meta to ensure this can be resolved as soon as possible.
Report: "Yotpo API error"
Last updateThe missing data backload is now complete, and this incident is resolved.
Yotpo have identified the issue, and have applied a fix. We've now begun backloading the data that was missed during the time Yotpo was offline, and will confirm once this is complete.
Yotpo are currently experiencing issues with their APIs, meaning Yotpo integration data for all clients is not up to date. We're working with Yotpo to resolve this, and they're actively investigating. There is currently no ETA on resolution but we'll provide updates from Yotpo as soon as we receive them.
Report: "Shopify integration service disruption"
Last updateAll missing data has been loaded.
A solution has been deployed. The missing Shopify data is being processed and scheduled updates will resume within the next 24 hours. Users can monitor their dataset updates via the Dataset feature. Anyone with Admin permission is able to access Dataset. Read more here: https://learn.lexer.io/docs/dataset-management
We are currently experiencing issues with our Shopify integration resulting in failures in processing data. Our team is working to implement a solution and backfill missing data.
Report: "Hub performance degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We will be undergoing scheduled database maintenance this evening to resolve this issue.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently investigating this issue.
We are currently investigating this issue.
Report: "Hub performance degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating this issue.
Report: "Attribute searches in Segment are degraded"
Last updateThis incident has been resolved.
We are continuing to monitor for any further issues.
A fix has been implemented and we are monitoring the results.
We're aware of an issue with Attribute searches in Segment, and we're currently investigating.
Report: "Segment search performance degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We are currently seeing some issues in running searches in Segment, and we're looking into the cause.
Report: "Hub performance issues"
Last updateThis incident has been resolved.
We're currently experiencing some performance issues in the Hub. We've identified the problem, and are implementing a fix.
Report: "Respond Automation issue"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are continuing to investigate this issue.
We're investigating an issue with Respond Automation rules, which is impacting the ability to correctly assign objects in Respond queues.
Report: "Respond and Listen returning errors"
Last updateThis incident has been resolved. We have identified performance degradation in one of our database systems and have redeployed the system onto new hardware with additional resources available to it. Our monitoring now indicates that normal service has been restored. We will be continuing our investigation into the performance degradation to mitigate the impact of future incidents.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating reports of errors when loading the Respond inbox and Listen results.
Report: "Mailchimp Integration experiencing errors"
Last updateThis issue is now resolved. We are no longer seeing errors from Mailchimp integrations.
We have stopped receiving errors from the Mailchimp API, and both Activations and Integrations are succeeding once more. We will continue to monitor this, for the time being, and are confident this was a temporary issue.
We are continuing to investigate this issue, we have now found that this is impacting some Activations to Mailchimp also.
We are currently investigating an issue related to errors being received when collecting data from Mailchimp. We are in contact with Mailchimp to resolve this issue.
Report: "Facebook posts and comments failing to load"
Last updateThis incident has been resolved.
A fix has been implemented and Facebook posts and comments are loading into your inbox.
We have identified an issue related to Facebook Posts and Facebook Comments not loading into your account and we are working on a fix. Facebook posts stopped loading approximately 10am Wednesday 5th May. Once fixed, missed content will be loaded into your account.
Report: "Monitoring conversation performance degradation"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating slow performance for Respond Inbox and Activity searches.
Report: "Forms returning errors when opened"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating reports of Lexer Forms returning errors when opened.
Report: "Profile search errors in APAC region"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are observing an increased rate of errors when searching customer profiles for APAC customers. We are currently investigating this issue.
Report: "Hub connection errors to clients.lexer.io"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Respond inbox and context slowness"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We're investigating reports of slow response times for inbox and conversation context.
Report: "Hub connection errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating reports of users unable to connect to the Hub at clients.lexer.io.
Report: "Social content delayed in Engage and Listen"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating reports of delayed social content in Engage and Listen.
Report: "Facebook returning errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
We have identified an issue with status checks for Facebook content.
Report: "Lexer Hub Errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue causing errors and increased loading times for the users on clients.lexer.io.
Report: "Identity search degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
Identity search for customers in the APAC region is currently unavailable. This issue impacts Identify, Activate, and Engage. We are continuing to investigate.
We are currently investigating this issue.
Report: "Problems connecting to Lexer Hub"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are currently investigating an issue with connecting to the Lexer Hub.
Report: "Form submissions degraded"
Last updateThis incident has been resolved.
Our team are in the process of loading form submissions (ID Verification, NPS etc) from the last hour. These will begin to appear in your inbox shortly.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating an issue related to form submissions not appearing in the inbox.
Report: "Errors responding on Facebook"
Last updateThis incident has been resolved.
Facebook message replies are beginning to send successfully. We will continue to monitor and update until resolved.
Facebook appear to be experiencing issues that are preventing direct message replies. Responding in Lexer and native Facebook Inbox are both impacted. We will update here when Facebook have resolved this issue.
We are currently investigating a high rate of errors when replying on Facebook.
Report: "User login degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We are continuing to work on a fix for this issue.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Conversation search results degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We've identified an issue with search results in Engage and Listen taking longer than usual to return. Our team are working to resolve immediately.
Report: "Facebook activations experiencing errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are currently investigating this issue.
Report: "Facebook replies experiencing errors"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
The issue has been identified and a fix is being implemented.
We are investigating an issue related to sending replies on Facebook. We'll update as we know more.
Report: "Facebook activations temporarily unavailable"
Last updateThis incident has been resolved.
A fix has been deployed and we are monitoring to ensure resolution.
Root cause has been identified and we are working to resolve.
We've identified an issue with Facebook audiences sending from Activate. Our team are working to resolve immediately.
Report: "Conversation search results degraded"
Last updateThis incident has been resolved.
A fix has been implemented and we are monitoring the results.
We've identified an issue with search results in Engage and Listen taking longer than usual to return. Our team are working to resolve immediately.
Report: "Conversation searches not returning"
Last updateThis incident has been resolved.
Issue appears to be fixed. Our team are monitoring to ensure resolution.
We are continuing to investigate this issue.
We're currently investigating reports of Engage Inbox and Listen Search results not returning. Our team are looking into this issue and we'll update when we know more.
Report: "Listen & Engage Search Outage"
Last updateListen and Engage experienced temporary search errors on Sunday morning. This resulted in users being unable to query content in the Engage inbox and return results in Listen and Command. The issue was resolved at 1:50am AEDT with all functionality restored. No content was missed during the outage period.
Report: "Identify Search Degraded"
Last updateThis incident has been resolved.
Issue has been resolved. Our team are monitoring to ensure resolution.
The issue has been identified and our team are working to resolve.
We are investigating an issue causing Identify searches to return with delay.
Report: "Facebook Content Delayed"
Last updateThe incident has been resolved. Facebook content sent to your accounts during this period is now available in the inbox.
We have identified the cause of the issue and are working to resolve.
We're investigating reports of delayed Facebook content for integrated pages.