Omni Analytics incident

Intermittent page load failures

Resolved after 6 hours 7 minutes, affecting EastUsa, on May 15, 2025.

resolved May 15, 2025, 11:43 PM UTC

This incident has been resolved. A resource intensive process was initiated during a busy period that overwhelmed a subset of our internal resources and resulted in intermittent failure to complete requests. Manual scaling of component-specific resources was needed in addition to our automated scaling to recover. We have implemented some immediate efficiency improvements and are working on implementing additional larger improvements in the coming days.

Started
May 15, 2025, 05:36 PM UTC
Resolved
May 15, 2025, 11:43 PM UTC
Duration
6h 7m
Detected by Pingoru
May 15, 2025, 05:36 PM UTC

Affected components

EastUsa

Update timeline

  1. investigating May 15, 2025, 05:36 PM UTC

    We detected a spike in internal errors affecting some hosts. Behavior is intermittent and refreshing the page should result in successful load.

  2. identified May 15, 2025, 06:06 PM UTC

    The issue has been identified and temporary mitigation put in place. We have not observed internal errors being surfaced in the last 20 minutes. We are continuing to monitor while we work on a longer term fix.

  3. identified May 15, 2025, 08:23 PM UTC

    We are continuing to work on a long term fix and monitoring to ensure our mitigation is holding.

  4. resolved May 15, 2025, 11:43 PM UTC

    This incident has been resolved. A resource intensive process was initiated during a busy period that overwhelmed a subset of our internal resources and resulted in intermittent failure to complete requests. Manual scaling of component-specific resources was needed in addition to our automated scaling to recover. We have implemented some immediate efficiency improvements and are working on implementing additional larger improvements in the coming days.

Your product runs on more than Omni Analytics

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →