Omni Analytics incident

Slow response times in US Region

Resolved after 5 hours 30 minutes, affecting EastUsa, on April 29, 2025.

resolved Apr 29, 2025, 10:10 PM UTC

A model validation related process triggered repeated calls to an inefficient library function that exhausted memory in a subset of our US East infrastructure. A fix has been deployed that removes calls to the inefficient library function and we have validated it resolves the memory exhaustion.

Started
Apr 29, 2025, 04:39 PM UTC
Resolved
Apr 29, 2025, 10:10 PM UTC
Duration
5h 30m
Detected by Pingoru
Apr 29, 2025, 04:39 PM UTC

Affected components

EastUsa

Update timeline

  1. investigating Apr 29, 2025, 04:39 PM UTC

    Users are experiencing slow responses in the US region, our team is investigating.

  2. monitoring Apr 29, 2025, 05:35 PM UTC

    We have mitigated the issue and are monitoring the system.

  3. monitoring Apr 29, 2025, 07:57 PM UTC

    We have identified the root cause of the issue and are implementing a fix.

  4. resolved Apr 29, 2025, 10:10 PM UTC

    A model validation related process triggered repeated calls to an inefficient library function that exhausted memory in a subset of our US East infrastructure. A fix has been deployed that removes calls to the inefficient library function and we have validated it resolves the memory exhaustion.

Your product runs on more than Omni Analytics

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →