Statusfield LogoStatusfield
Signadot status
Signadot logo and current status indicator

Signadot Incident History

Operational

signadot.com ↗3 top-level components

checked Oct 26, 2026 5:38 AM UTC · Signadot's official status page

Signadot is up and running.

Signadot is currently operational with all systems functioning normally.

All components operational

Be the first to know

Statusfield watches Signadot alongside the rest of your stack and tells you when its status changes. Email, Slack, Microsoft Teams, PagerDuty, and webhooks.

30-day free trial — no card needed. First-time customers; continued monitoring requires a paid subscription.

Signadot incident history

1 report published by Signadot in the last 30 days

September 24, 2026

Issues with Tunnel Server

Last update

  1. postmortemOct 16 · 19:55 UTC

    Postmortem: Cluster connectivity outage, September 24, 2026

    Duration: 21:43 to 23:15 UTC (about 92 minutes) Status page: Incident report

    Summary

    On September 24, 2026, clusters connected to Signadot lost their connection to the Signadot control plane for about 92 minutes, and sandbox creation failed during that window. A control plane release that day included an upgrade to Go 1.27. A behavior change in Go 1.27's HTTP client caused an internal health check to leak connections until the service that maintains cluster connections ran out of resources. We restored service by rolling back the release, and have since shipped a fix.

    We're sorry for the disruption this caused.

    What happened

    For over a year, an internal health check had opened a connection on each run and closed it without reading the full response. Under earlier Go versions, that closed the connection. Go 1.27 instead keeps the connection open for reuse, and since the health check never reused it, every run leaked one.

    The leak began when the release rolled out and grew steadily until the affected service ran out of resources and clusters lost connectivity.

    Timeline (UTC, September 24)

    TimeEvent
    20:03Control plane release rolls out. The leak begins.
    21:43Clusters lose connectivity.
    23:04Release rolled back.
    23:09Root cause identified.
    23:15Connectivity restored.

    A fix was merged on September 25 and shipped in a follow-up release.

    What we found

    • A toolchain upgrade changed the behavior of unchanged code. Our pre-release testing didn't run at production scale, so the leak grew too slowly there to notice.
    • A missing resource limit turned a leak into an outage. With a limit in place, the leak would have caused a quick restart instead.
    • We needed better health signals. Alerting on resource usage and cluster connections would have flagged the leak well before it became an outage.
    • The same pattern existed elsewhere, including in software that runs in customer clusters. None of it had shipped with Go 1.27, so it was never affected, and it is now fixed.

    What we've done

    • Fixed the health check, with a regression test that fails on Go 1.27 without the fix.
    • Found and fixed the same pattern across our other components before any affected release shipped.
    • Added a resource limit so a similar leak is contained to a quick restart.
    • Re-released the control plane with the fix and monitored it closely.

    For a more detailed postmortem, please contact us.

Reported by Signadot on their status page.