Login taking longer than usual
Last updatepostmortemSep 1 · 09:07 UTC
What happened
Between the evening of 30 August and mid-morning on 31 August, some customers experienced slower than normal login times. Logins completed, but response times were degraded during this period.
Why it happened
An internal configuration item was not renewed on schedule, which caused a background processing job to fall behind. Clearing the resulting backlog created a short load spike that extended the degraded performance before service returned to normal.
What we have done
Restored normal processing and cleared the backlog.
Automated the renewal of this class of configuration so it no longer depends on a manual step.
Added earlier alerting so the condition is detected before it reaches customers.
Added pacing to backlog processing so recovery cannot itself degrade performance.
Timeline (EEST)
30 Aug, evening: first signs of degraded background processing
31 Aug, 09:03: corrective change applied
31 Aug, 09:40: background processing resumes
31 Aug, 10:30: login performance back to normal
Reported by Sympa on their status page.