Resolved after 1 hour 18 minutes, affecting EastUsa, on September 8, 2025.
We are now considering this incident resolved. A recent code change generated unexpectedly large database queries that saturated CPU resources, leading to query timeouts and subsequent connection failures. We initiated a rapid rollback once the cause was identified. Since mitigating we have reverted the change and are working to prevent similar issues in the future.
Affected components
Update timeline
- investigating Sep 08, 2025, 05:57 PM UTC
Our monitoring picked up a spike in errors and we have begun getting reports from users. The team is investigating on highest priority
- identified Sep 08, 2025, 06:02 PM UTC
We have identified the source of the issue and are in the process of rapidly reverting a change to resolve.
- identified Sep 08, 2025, 06:29 PM UTC
We have completed rolling out the mitigation for this incident and are now in the monitoring phase.
- resolved Sep 08, 2025, 07:15 PM UTC
We are now considering this incident resolved. A recent code change generated unexpectedly large database queries that saturated CPU resources, leading to query timeouts and subsequent connection failures. We initiated a rapid rollback once the cause was identified. Since mitigating we have reverted the change and are working to prevent similar issues in the future.
Your product runs on more than Omni Analytics
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.