LiveKit incident

LiveKit Cloud Dashboard Down

Resolved after 9 minutes, affecting Cloud Dashboard (cloud.livekit.io), on April 3, 2026.

postmortem Apr 03, 2026, 11:00 PM UTC

**Root Cause** A gap in our DNS update process resulted in a misconfiguration of the DNS for [livekit.io](http://livekit.io), causing several A records to be overwritten. This resulted in an incorrect IP address being returned for [livekit.io](http://livekit.io). All real-time services - calls, RTC, and hosted agents - were unaffected as they operate on separate domains. We do have DNS monitoring in place, but it was not configured to page on-call. As a result, the issue was identified by LiveKit engineering after a short delay. **Timeline** * **2026-04-03 17:46 UTC**: DNS configuration change applied; resolution begins failing for [livekit.io](http://livekit.io). * **2026-04-03 17:52 UTC**: Issue identified by LiveKit. * **2026-04-03 18:15 UTC**: Root cause identified and fix applied. * **2026-04-03 18:18 UTC**: Fix fully propagated; services restored. Total duration: ~32 minutes. **Mitigations** * Enforce tighter automated guardrails around DNS updates. * Enable paging on existing DNS health checks so any future issues are caught even sooner.

Started
Apr 03, 2026, 05:13 PM UTC
Resolved
Apr 03, 2026, 05:23 PM UTC
Duration
9m
Detected by Pingoru
Apr 03, 2026, 05:13 PM UTC

Affected components

Cloud Dashboard (cloud.livekit.io)

Update timeline

  1. investigating Apr 03, 2026, 05:13 PM UTC

    We are currently investigating this issue and will update as soon as we know more.

  2. resolved Apr 03, 2026, 05:23 PM UTC

    A fix has been implemented and we are monitoring the results. All users should now be able to access the Cloud Dashboard.

  3. postmortem Apr 03, 2026, 11:00 PM UTC

    **Root Cause** A gap in our DNS update process resulted in a misconfiguration of the DNS for [livekit.io](http://livekit.io), causing several A records to be overwritten. This resulted in an incorrect IP address being returned for [livekit.io](http://livekit.io). All real-time services - calls, RTC, and hosted agents - were unaffected as they operate on separate domains. We do have DNS monitoring in place, but it was not configured to page on-call. As a result, the issue was identified by LiveKit engineering after a short delay. **Timeline** * **2026-04-03 17:46 UTC**: DNS configuration change applied; resolution begins failing for [livekit.io](http://livekit.io). * **2026-04-03 17:52 UTC**: Issue identified by LiveKit. * **2026-04-03 18:15 UTC**: Root cause identified and fix applied. * **2026-04-03 18:18 UTC**: Fix fully propagated; services restored. Total duration: ~32 minutes. **Mitigations** * Enforce tighter automated guardrails around DNS updates. * Enable paging on existing DNS health checks so any future issues are caught even sooner.

Your product runs on more than LiveKit

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →