Blip Incident History

1. resolvedOct 15 · 21:29 UTC

   The issue affecting bot operations has been resolved. The failure, which originated at the external provider of the database service used by the platform, affected the message flow of bots, including message reception, processing, and response delivery, message routing, and conversation history. We worked together with the provider on the fix, restarted some services, and validated the environment, which remains under monitoring. We ask customers to validate that their operations are working normally and, if any behavior persists, to contact support with the bot, the affected contact, and the date and time of the occurrence.

Reported by Blip on their status page.

1. resolvedSep 29 · 14:04 UTC

   We identified a service outage affecting Blip Desk operations. This resulted in customer service difficulties: messages sent by customers failed to appear on the agent's screen, while responses sent by agents neither reached the customer nor were saved to the history; additionally, contact details and Desk history older than 90 days were not displayed. Actions taken: Mapping of error/warning spikes on the server (approximately 60,000 records/min) related to processing failures and "Destination not found" errors. Analysis of infrastructure, WebSocket connections, pod integrity, and Starter component configurations. Controlled shutdown of Starter instances to isolate concurrency issues regarding message consumption from Confluent topics. Restoration and validation of normal message flow for Desk and WhatsApp. Restart and final real-time monitoring tests to confirm environment stability.

Reported by Blip on their status page.

1. ResolvedSep 29 · 03:00 UTC

   We identified a service outage affecting Blip Desk operations.This resulted in customer service difficulties: messages sent by customers failed to appear on the agent's screen, while responses sent by agents neither reached the customer nor were saved to the history; additionally, contact details and Desk history older than 90 days were not displayed.Actions taken: Mapping of error/warning spikes on the server (approximately 60,000 records/min) related to processing failures and "Destination not found" errors. Analysis of infrastructure, WebSocket connections, pod integrity, and Starter component configurations. Controlled shutdown of Starter instances to isolate concurrency issues regarding message consumption from Confluent topics. Restoration and validation of normal message flow for Desk and WhatsApp. Restart and final real-time monitoring tests to confirm environment stability.

Reported by Blip on their status page.