Molequle Incident History

1. postmortemSep 30 · 15:00 UTC

   On August 28, 2026, between approximately 05:33 and 07:24 CEST, the Molequle Portal and Sales Intelligence applications were unavailable. The disruption followed a scheduled overnight infrastructure upgrade. During this maintenance, application instances were restarted but did not return to a fully healthy state. Our automated health checks didn’t detect this condition, so the affected instances were not recovered automatically. Our team identified the issue and restored full service by 07:24 CEST, and confirmed stability by 07:29 CEST. No data was lost or exposed, and no other Molequle services were affected. To prevent this from happening again, we are improving our application health checks so unhealthy instances are detected and recovered automatically, and we are extending our monitoring and logging to speed up detection and diagnosis. We apologize for the inconvenience.

Reported by Molequle on their status page.

1. postmortemOct 1 · 14:17 UTC

   Molequle experienced significant delays in data processing, affecting request response times and the availability of synchronized data. Our investigation identified an issue in workload distribution and processing capacity with database contention contributing to reduced throughput. As processing fell behind incoming work, a backlog accumulated and increases the recovery time. Incoming work is distributed through a queue to processing workers. The affected configuration focused queued work in a limited set of active workers, restricting the system's ability to use additional capacity effectively. Increasing the number of available workers did not necessarily increase processing throughput and work continued to accumulate despite available capacity. Database contention added pressure to this processing path. Concurrent operations involving overlapping records could wait for locks or require retries, occupying workers for longer and reducing capacity available to other requests. Since queue consumption was also coupled to request execution, delayed processing waits could interrupt consumption and trigger reassignment of queued work. This interaction further disrupted processing. Recovery addressed the idle-worker issue to restore stable processing and increased effective processing capacity to improve throughput. Once processing recovered, the backlog still needed to be cleared alongside incoming requests. Restoring throughput and completing pending work were therefore separate recovery steps which explains why delays continued beyond the initial remediation. Our published incident updates on August 26 recorded investigation at 08:35 CEST, identification at 15:41 and implementation of a recovery fix at 20:43. Monitoring continued while the backlog was processed and the incident was marked resolved at 23:30 CEST. These timestamps describe the published investigation and recovery milestones. Subsequent engineering work introduced consistent database write ordering to reduce conflicts between concurrent operations and improved request queue cleanup. These changes address specific sources of contention and strengthen request handling alongside the measures taken to restore stable processing. The incident is resolved. We apologize for the disruption and the delays.

Reported by Molequle on their status page.