LiveKit incident

Elevated API latency affecting Real Time Communication services in London Region

Resolved after 7 hours 45 minutes, affecting United Kingdom - Real Time Communication, on January 23, 2026.

postmortem Jan 27, 2026, 09:35 PM UTC

Our cloud provider’s managed Kubernetes control plane experienced an issue that caused connectivity to cluster nodes to be disrupted, dropping some of the workloads in the cluster. Due to a bug in our auto-draining logic, detection of the issue was delayed. We have since fixed the alerting to ensure faster detection and response if a similar situation occur should occur in the future.

Started
Jan 23, 2026, 02:39 PM UTC
Resolved
Jan 23, 2026, 10:24 PM UTC
Duration
7h 45m
Detected by Pingoru
Jan 23, 2026, 02:39 PM UTC

Affected components

United Kingdom - Real Time Communication

Update timeline

  1. investigating Jan 23, 2026, 02:39 PM UTC

    We are currently investigating reports of elevated API latency in the UK region.

  2. identified Jan 23, 2026, 02:50 PM UTC

    We are routing traffic away from the London cluster to mitigate impact.

  3. identified Jan 23, 2026, 04:21 PM UTC

    We are continuing to investigate the root cause of the performance degradation of the London cluster. We will continue to share updates as we learn more.

  4. monitoring Jan 23, 2026, 06:38 PM UTC

    All non-pinned traffic has been healthy since we began routing traffic through the nearest healthy nodes at 14:50 UTC. Our team is still working to bring the London cluster back to full health.

  5. resolved Jan 23, 2026, 10:24 PM UTC

    We brought the London cluster back to full health at 21:40 UTC. This incident is now resolved. We will publish a postmortem outlining the root cause and the steps we will take to ameliorate this issue in the future as soon as possible.

  6. postmortem Jan 27, 2026, 09:35 PM UTC

    Our cloud provider’s managed Kubernetes control plane experienced an issue that caused connectivity to cluster nodes to be disrupted, dropping some of the workloads in the cluster. Due to a bug in our auto-draining logic, detection of the issue was delayed. We have since fixed the alerting to ensure faster detection and response if a similar situation occur should occur in the future.

Your product runs on more than LiveKit

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →