Delays in background jobs

Update timeline

1. investigating Aug 10, 2026, 09:23 PM UTC

   We are investigating some delays in background jobs, including things like transcript processing and webhooks. We will provide updates here and all jobs should eventually process

2. resolved Aug 11, 2026, 12:07 AM UTC

   Job delays have returned to normal.

3. resolved Aug 11, 2026, 05:33 PM UTC

   We are seeing intermittent delays with queues today again and are investigating. We'll provide updates as we have them.

4. identified Aug 11, 2026, 05:48 PM UTC

   We continue to see intermittent delays in background jobs.

5. monitoring Aug 11, 2026, 07:13 PM UTC

   We have identified the root cause of the processing delays and have released a fix. We are processing background jobs that were delayed and expect them to catch up as the day progresses.

6. resolved Aug 11, 2026, 08:17 PM UTC

   Thank you for your patience. All queues are back to normal. The issue appears to have been related to underlying hardware in one of our AWS cluster and was resolved via cycling out all the hardware. In the process of trying to manually keep queue volumes low and verify different potential solutions, some invalid webhooks were sent out (e.g webhooks for data that does not exist) and which are not able to be actioned on. These did not replace any valid webhooks, and all valid webhooks have been sent.