Census incident

Sync Completion Rate Slowed

Resolved after 26 minutes, affecting Sync Engine, on November 15, 2024.

monitoring Nov 15, 2024, 04:24 AM UTC

We have rolled back a feature that we believe caused the slowdown and added extra capacity to compensate for the slowdown. Sync completion rates have returned to normal levels and any slowdown or queueing should be resolved in the next 30 minutes.

Started
Nov 15, 2024, 04:06 AM UTC
Resolved
Nov 15, 2024, 04:33 AM UTC
Duration
26m
Detected by Pingoru
Nov 15, 2024, 04:06 AM UTC

Affected components

Sync Engine

Update timeline

  1. investigating Nov 15, 2024, 04:06 AM UTC

    Syncs are completing at slower than usual rates and we are investigating the issue

  2. identified Nov 15, 2024, 04:16 AM UTC

    We have identified stuck sync workers and we are replacing these containers with new capacity

  3. monitoring Nov 15, 2024, 04:24 AM UTC

    We have rolled back a feature that we believe caused the slowdown and added extra capacity to compensate for the slowdown. Sync completion rates have returned to normal levels and any slowdown or queueing should be resolved in the next 30 minutes.

  4. resolved Nov 15, 2024, 04:33 AM UTC

    This incident has been resolved.

Your product runs on more than Census

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →