Is Omni Analytics down?

We're investigating slow page loading in Omni's US East region. You may notice pages taking longer than usual to open as you move around Omni. Our monitoring confirms the slowdown, and we're working to identify the cause.

Page loading times in US East have returned to normal, and we're monitoring to make sure the improvement holds. Our investigation points to a spikes of permission checks when people opened or edited dashboards, causing pages to load more slowly. We're working on a permanent fix to prevent those repeated checks.

The issue is now resolved, we fixed a bottleneck where permission checks in large workbooks could slow down page loads for other users.

We’re monitoring an issue where Microsoft’s Azure AI service is returning server errors to Omni. In Omni users may observe Blobby stays on “Thinking” or fails to answer questions for some customers hosted in Azure.

Microsoft has marked this incident as resolved

We are currently experiencing degraded performance and intermittent outages on the Omni application. The team is actively investigating.

The application has stabilized and we have not seen slowness or outages in the last 15 minutes, and we have also identified a candidate root cause and are currently working to mitigate.

We’ve identified an issue in the service that powers workbooks and dashboards. It caused some requests to get stuck, making those features slow or unavailable. Service has fully stabilized, and we’re monitoring it while we work on a fix, expected to land this afternoon.

The issue is now fully resolved and mitigated, we've deployed a fix to stabilize simultaneous requests that could block one another, which caused Omni to become unresponsive earlier.

We are investigating reports of Omni failing to load, we will provide updates as soon as possible.

We have rolled back our latest deploy and things are starting to recover. We will continue to monitor

Omni remains stable after the rollback, this issue has been resolved.

We are currently investigating reports of intermittent delayed responses from OpenAI models in Omni. Our model provider AWS Bedrock is reporting increased error rates on OpenAI models at this time. We'll provide more information as soon as we have an update.

Our model provider, AWS Bedrock, is currently reporting higher error rates for OpenAI models. Some AI requests may fail or stop partway through. We're monitoring and will update you when the issue is resolved on the provider's end.

Our model provider, AWS Bedrock, has reported the incident as resolved. Our testing confirms that OpenAI models are available again and AI requests are completing normally.