Grafana Outage History
Grafana is up right nowPingoru has recorded 195 incidents for Grafana: 195 in the last 12 months and 70 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 23 | 367 of 720 (51%) |
| May 2026 | 13 | 65 of 744 (9%) |
| June 2026 | 23 | 391 of 720 (54%) |
| July 2026 | 39 | 292 of 744 (39%) |
| August 2026 | 14 | 18 of 744 (2%) |
| September 2026 | 23 | 85 of 720 (12%) |
| October 2026 | 26 | 366 of 593 (62%) |
Incidents, newest first
Major June 29, 2026 Confluent API Outage 1h 49m Integrations
Started Jun 29, 2026, 12:37 PM UTC· resolved Jun 29, 2026, 02:27 PM UTC
- investigating Jun 29, 2026, 12:37 PM UTC
We are investigating an issue affecting Confluent metrics ingestion across all regions. Due to an elevated error rate on the Confluent side, some metrics may not be ingested, resulting in potential data loss. We are actively investigating the issue and will provide updates as more information becomes available.
- resolved Jun 29, 2026, 02:27 PM UTC
This incident has been resolved.
Minor June 29, 2026 Mimir read errors and high latency in prod-eu-west-0 20h 43m GCP Belgium - prod-eu-west-0: Querying, GCP Belgium - prod-eu-west-0: Ingestion
Started Jun 29, 2026, 11:37 AM UTC· resolved Jun 30, 2026, 08:20 AM UTC
- investigating Jun 29, 2026, 11:37 AM UTC
We are currently investigating an issue with Mimir in prod-eu-west-0 we are seeing read errors and high latency. This incident is currently ongoing. The errors are recovering but we are currently looking into the route cause of this.
- investigating Jun 29, 2026, 01:52 PM UTC
The errors are recovering, and we are still looking into the root cause of this.
- investigating Jun 29, 2026, 03:37 PM UTC
The errors and latency have now recovered, we continue investigating the root cause.
- identified Jun 29, 2026, 05:47 PM UTC
We've identified a possible cause, and a mitigation is in place to prevent further occurrences.
- resolved Jun 30, 2026, 08:20 AM UTC
Since the mitigation has been applied, we have not seen the errors return. At this point, we are considering the incident resolved.
Minor June 29, 2026 Rule evaluation error on cluster prod-gb-south-0 2h 38m GCP UK - prod-gb-south-0: Rule Evaluation
Started Jun 29, 2026, 10:34 AM UTC· resolved Jun 29, 2026, 01:12 PM UTC
- investigating Jun 29, 2026, 10:34 AM UTC
We are currently investigating Rule Evaluation errors on the cluster prod-gb-south-0 which is leading to error codes showing within the stacks. We are looking into the issue and will update accordingly.
- monitoring Jun 29, 2026, 11:14 AM UTC
A fix has been applied and we are currently monitoring results.
- resolved Jun 29, 2026, 01:12 PM UTC
This incident has been resolved.
Minor June 26, 2026 K6 - Test run metrics processing is delayed 1d 13h Cloud Test Runs, Result Analysis
Started Jun 26, 2026, 01:08 PM UTC· resolved Jun 28, 2026, 02:35 AM UTC
- investigating Jun 26, 2026, 01:08 PM UTC
We’re currently investigating an issue causing metrics not to appear during test runs. . Our team is actively working to identify the cause. Thank you for your patience.
- investigating Jun 26, 2026, 01:19 PM UTC
A small update: The issue is isolated to new test runs, and users can go see the metrics of all the previous test runs
- investigating Jun 26, 2026, 02:09 PM UTC
We are experiencing intermittent delays with secondary metrics processing for k6 Cloud test runs due to heavy load. We don't expect any data loss or impact on user runs, but results may take longer time to appear in UI.
- investigating Jun 26, 2026, 04:33 PM UTC
Update: Changed incident title to "Test run metrics processing is delayed" We have found the issue and are working on deploying the fix.
- investigating Jun 26, 2026, 11:30 PM UTC
We've improved the metric ingestion delay time and are working on additional fixes to bring it down to expected range. Customers can currently expect a delay of 2 to 5 minutes before their test run metrics show up (after starting a test).
- resolved Jun 28, 2026, 02:35 AM UTC
This incident has been resolved.
Notice June 20, 2026 Elevated Logs Query Usage — Unexpected Billing Impact –
Started Jun 20, 2026, 08:00 PM UTC· resolved Jun 20, 2026, 08:00 PM UTC
- resolved Jun 22, 2026, 10:10 PM UTC
Starting June 20, 2026 at approximately 20:00 UTC, some Grafana Cloud customers experienced unexpectedly elevated logs query usage. The issue persisted until it was resolved on June 22, 2026 at approximately 16:00 UTC. Our engineering team identified and mitigated the issue. Systems have since stabilized and are operating normally. Grafana Labs is reviewing affected accounts for appropriate remediation.
Major June 19, 2026 Rule Evaluation Outage in prod-us-central-0 2h 40m GCP US Central - prod-us-central-0: Rule Evaluation
Started Jun 19, 2026, 04:39 PM UTC· resolved Jun 19, 2026, 07:19 PM UTC
- monitoring Jun 19, 2026, 04:39 PM UTC
We had an outage affecting rule evaluations between 15:16-15:59 UTC in the prod-us-central-0 region. Our team quickly identified the issue and has since mitigated. The engineering team is monitoring.
- monitoring Jun 19, 2026, 05:46 PM UTC
We’re continuing to track progress post-mitigation. While we don’t have new information to share yet, our team remains actively engaged.
- resolved Jun 19, 2026, 07:19 PM UTC
This incident has been resolved. Thank you for your patience.
Major June 18, 2026 Issues with actions in the Grafana IRM mobile app 1h 48m Incident Management and Response (IRM)
Started Jun 18, 2026, 05:01 PM UTC· resolved Jun 18, 2026, 06:50 PM UTC
- identified Jun 18, 2026, 05:01 PM UTC
We're noticing an uptick in users being unable to respond to actions on the mobile app (acknowledging and silencing alerts, for example). Users working in the web UI should not be affected. Ingestion and notification delivery are working as expected. We have a fix in place and are in the process of deploying.
- monitoring Jun 18, 2026, 06:09 PM UTC
We've verified a fix in our staging environment to restore functionality to the mobile app. The fix is currently being deployed to production. Thanks for your patience as we continue to roll this out and monitor the resolution.
- resolved Jun 18, 2026, 06:50 PM UTC
This incident has been resolved. Thank you for your patience.
Major June 18, 2026 Potential Issues Loading Grafana for Users in India 5d 21h AWS Australia - prod-au-southeast-2, AWS Brazil - prod-sa-east-1, AWS Canada - prod-ca-east-0, AWS Germany - prod-eu-west-2 and 27 more
Started Jun 18, 2026, 03:18 PM UTC· resolved Jun 24, 2026, 12:46 PM UTC
- investigating Jun 18, 2026, 03:18 PM UTC
Due to the linked GCP outage below, users located in India may have trouble loading parts of Grafana. https://status.cloud.google.com/incidents/5fGQt4VbkDnr3Yp8PXPr Impacted users may receive error messages such as "Error Loading" or "Failed to load Assets". To be clear, it does not matter the region the stack is located, but the geography where the user is physically in. We are currently investigating this issue from our end, and will provide updates as they are available.
- monitoring Jun 18, 2026, 04:57 PM UTC
Due to the linked GCP outage below, users located in India may have trouble loading parts of Grafana. https://status.cloud.google.com/incidents/5fGQt4VbkDnr3Yp8PXPr Impacted users may receive intermittent error messages such as "Error Loading" or "Failed to load Assets". To be clear, it does not matter the region the stack is located, but the geography where the user is physically in. We continue to work with our CSP on this investigation.
- monitoring Jun 19, 2026, 02:33 AM UTC
Due to the linked GCP outage below, users located in India may have trouble loading parts of Grafana. https://status.cloud.google.com/incidents/5fGQt4VbkDnr3Yp8PXPr We are continuing to work with our CSP on this investigation. Impacted users may receive intermittent error messages such as "Error Loading" or "Failed to load Assets". To be clear, it does not matter the region the stack is located, but the geography where the user is physically in.
- monitoring Jun 19, 2026, 07:07 PM UTC
We're actively monitoring this issue and working with our 3rd party provider. The next update will be sent on Monday unless there's new information to share.
- monitoring Jun 22, 2026, 05:16 PM UTC
We’ve verified and begun to implement a fix that will improve loading errors. We are continuing to roll this out to all regions and monitor for efficacy.
- monitoring Jun 23, 2026, 09:14 AM UTC
We have deployed additional mitigations that should help with remaining errors. We are continuing to monitor error rates.
- monitoring Jun 23, 2026, 01:15 PM UTC
We are continuing to monitor for further issues.
- monitoring Jun 23, 2026, 05:54 PM UTC
Error rates have remained near zero, and we continue to monitor.
- resolved Jun 24, 2026, 12:46 PM UTC
This incident has been resolved. Error rates continued to remain near 0 and operations are performing as expected.
Critical June 18, 2026 Degraded k6 cloud UI performance 4h 56m AWS Australia - prod-au-southeast-1: API, GCk6 App, AWS Australia - prod-au-southeast-1: Public Probes, Cloud Test Runs and 55 more
Started Jun 18, 2026, 11:25 AM UTC· resolved Jun 18, 2026, 04:21 PM UTC
- investigating Jun 18, 2026, 11:25 AM UTC
We’re currently investigating an issue resulting in degraded k6 cloud UI performance and API response time. Our team is actively working to rectify this issue.
- investigating Jun 18, 2026, 11:45 AM UTC
We are continuing to investigate this issue.
- monitoring Jun 18, 2026, 01:15 PM UTC
The root cause of the issue has been identified and a fix has been successfully deployed. We are observing widespread improvements across all systems. Our team is currently monitoring the environment to ensure performance remains stable.
- monitoring Jun 18, 2026, 02:04 PM UTC
We are continuing to monitor for any further issues.
- resolved Jun 18, 2026, 04:21 PM UTC
This incident has been resolved. Thank you for your patience.
Minor June 18, 2026 Frontend Observability - Suspected commit feature not working as expected 1h 24m AWS Germany - prod-eu-west-2, AWS US East (OH) - prod-us-east-0, AWS US West - prod-us-west-0
Started Jun 18, 2026, 08:48 AM UTC· resolved Jun 18, 2026, 10:13 AM UTC
- investigating Jun 18, 2026, 08:48 AM UTC
We’re currently investigating an issue affecting Frontend Observability product. The "Suspected commit" feature is not currently working as expected. Ingestion and querying is unaffected by this. Our team has identified the cause and is actively working on a fix. Thank you for your patience.
- resolved Jun 18, 2026, 10:13 AM UTC
A fix has been deployed and the issue after monitoring as been fixed.
Major June 17, 2026 Loki data source-managed alert rules not visible in the Grafana Cloud Alerting UI 17h 51m AWS Australia - prod-au-southeast-1: Alertmanager and Rules Configuration API, AWS Brazil - prod-sa-east-1: Alertmanager and Rules Configuration API, AWS Canada - prod-ca-east-0: Alertmanager and Rules Configuration API, AWS Germany - prod-eu-west-2: Alertmanager and Rules Configuration API and 16 more
Started Jun 17, 2026, 08:17 PM UTC· resolved Jun 18, 2026, 02:08 PM UTC
- identified Jun 17, 2026, 08:17 PM UTC
We have identified an issue where alert rules and alerts managed directly in a Loki data source (data source-managed alerting) are not displayed in the Grafana Cloud Alerting UI. Rules created via Prometheus/Mimir data sources and Grafana-managed alert rules are not affected. Impact is limited to visibility and management in the UI. Affected alert rules continue to evaluate and send notifications normally — there is no impact to alert delivery. Workaround: Loki alert rules can still be viewed and managed directly through the Loki ruler API (for example, using cortextool against /loki/api/v1/rules). A fix has been identified and is in progress. We will provide a further update once it has been rolled out.
- identified Jun 17, 2026, 08:55 PM UTC
Our Engineering Team has implemented a fix which is now being rolled out. We will continue to monitor the situation and update as soon as we have more information.
- identified Jun 17, 2026, 09:43 PM UTC
Deployment of the fix is still in progress. We are continuing to monitor the rollout and validate recovery across affected systems. We will share further updates as they become available.
- identified Jun 17, 2026, 11:22 PM UTC
We are continuing to deploy the fix and monitor recovery efforts. As part of the rollout, we identified an issue that required adjustments to our deployment plan, which has extended the timeline for mitigation. Work remains actively underway, and we will share additional updates as progress continues.
- monitoring Jun 18, 2026, 08:04 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jun 18, 2026, 02:08 PM UTC
This incident has been resolved.
Notice June 12, 2026 Brief Loki Prod-012-eu-west-2 Disruption –
Started Jun 12, 2026, 06:30 PM UTC· resolved Jun 12, 2026, 06:30 PM UTC
- resolved Jun 12, 2026, 09:54 PM UTC
Our team had discovered a read issue around 19:35-20:08 UTC. Impact at the time would have provided errors similar to context deadline exceeded (DatasourceError response). This has since been resolved, and should not have caused any data loss, only a short query disruption.
Major June 10, 2026 Grafana Dashboards page not displaying when set to ‘View by Folders’ 18h 46m AWS Australia - prod-au-southeast-2, AWS Brazil - prod-sa-east-1, AWS Canada - prod-ca-east-0, AWS Germany - prod-eu-west-2 and 27 more
Started Jun 10, 2026, 10:49 AM UTC· resolved Jun 11, 2026, 05:36 AM UTC
- investigating Jun 10, 2026, 10:49 AM UTC
We’re currently investigating an issue affecting The Grafana Dashboards page. When set to view by folders, is currently experiencing an issue where no dashboards are shown. Our team is working on fixing the problem. In the meantime, switching to ‘View as list’ allows access to dashboards as usual”.
- resolved Jun 11, 2026, 05:36 AM UTC
This incident has been resolved.
Major June 9, 2026 Investigating Issues with Data Source-Managed Alerting 14h AWS Australia - prod-au-southeast-1: Alertmanager and Rules Configuration API, AWS Australia - prod-au-southeast-1: Alertmanager, AWS Brazil - prod-sa-east-1: Alertmanager and Rules Configuration API, AWS Brazil - prod-sa-east-1: Alertmanager and 36 more
Started Jun 09, 2026, 11:11 PM UTC· resolved Jun 10, 2026, 01:11 PM UTC
- investigating Jun 09, 2026, 11:11 PM UTC
We are currently investigating an issue affecting data source-managed alerting management functionality in Grafana Cloud. Customers may experience problems viewing, creating, updating, or managing alerts through Grafana when using data source-managed alerting. This issue is limited to alert management functionality within Grafana. Alert evaluation and backend alerting services continue to operate normally. Direct alerting APIs for Mimir and Loki remain fully operational and are unaffected. Grafana-managed alerting is not impacted. We identified this issue at approximately 20:45 UTC and are actively working on a resolution. We will provide additional updates as more information becomes available. Workaround: Customers can continue to use the direct Mimir and Loki alerting APIs while we work to restore normal functionality.
- monitoring Jun 10, 2026, 10:57 AM UTC
Our team has implemented a fix and we are currently monitoring the results of this.
- resolved Jun 10, 2026, 01:11 PM UTC
We continue to observe a continued period of recovery. At this time, we are considering this issue resolved. No further updates.
Minor June 8, 2026 IRM Degraded Performance 9h 44m GCP Belgium - prod-eu-west-0: Alertmanager and Rules Configuration API, GCP US Central - prod-us-central-0: Alertmanager and Rules Configuration API
Started Jun 08, 2026, 10:45 AM UTC· resolved Jun 08, 2026, 08:30 PM UTC
- investigating Jun 08, 2026, 10:45 AM UTC
We are experiencing access issues in IRM as there are elevated 500 API responses in prod-us-central-0.
- investigating Jun 08, 2026, 10:47 AM UTC
We are continuing to investigate this issue.
- identified Jun 08, 2026, 11:15 AM UTC
The issue has been identified and a fix is being implemented.
- identified Jun 08, 2026, 12:26 PM UTC
We are continuing to work on a fix for this issue.
- identified Jun 08, 2026, 01:14 PM UTC
The degraded performance is about labels, and we have seen this degradation in more regions.
- identified Jun 08, 2026, 03:21 PM UTC
We are continuing to work on a fix for this. To further clarify, this issue is not about accessing IRM or alert ingestion/notification/delivery, but rather with handling labels.
- monitoring Jun 08, 2026, 05:11 PM UTC
We've released a fix to the IRM app that should restore service for affected customers with issues related to labels. Thanks for your patience while investigating. We're continuing to monitor as we confirm the resolution in place.
- resolved Jun 08, 2026, 08:30 PM UTC
This incident has been resolved.
Major June 7, 2026 Brief Rule Evaluation Failures in prod-eu-west-3 5d 18h Azure Netherlands - prod-eu-west-3: Ingestion
Started Jun 07, 2026, 01:39 AM UTC· resolved Jun 12, 2026, 07:52 PM UTC
- investigating Jun 07, 2026, 01:39 AM UTC
From 00:20:00 to 00:27:00 and again 00:32:00 to 00:38:00 there were brief spikes in rule evaluation failures. Engineers are investigating.
- investigating Jun 07, 2026, 02:46 AM UTC
Intermittent spikes in rule evaluations continuing.
- investigating Jun 07, 2026, 03:50 AM UTC
We are continuing to investigate this issue.
- investigating Jun 07, 2026, 06:00 AM UTC
We’re making ongoing progress on the investigation alongside our upstream provider.
- monitoring Jun 07, 2026, 11:00 AM UTC
The incident has been mitigated, and services are operating normally. We are currently monitor the service to ensure full stability.
- monitoring Jun 08, 2026, 09:36 PM UTC
The incident has been mitigated, and services are operating normally. We continue to monitor the service to ensure full stability.
- resolved Jun 12, 2026, 07:52 PM UTC
This incident has been resolved. Thank you for your patience.
Critical June 5, 2026 Permissions Issues with IRM 5h 24m Incident Management and Response (IRM)
Started Jun 05, 2026, 04:11 PM UTC· resolved Jun 05, 2026, 09:35 PM UTC
- investigating Jun 05, 2026, 04:11 PM UTC
We are currently investigating an issue impacting permissions for IRM. As a result, users are not currently getting paged. We will provide updates as they become available.
- investigating Jun 05, 2026, 04:52 PM UTC
Our engineering team is still investigating this issue. We do not have any new information to share at this time, but will continue to provide timely updates.
- identified Jun 05, 2026, 05:14 PM UTC
We've identified an issue with RBAC syncing for a small number of customers and are working on a fix to restore permission services for those affected.
- monitoring Jun 05, 2026, 06:00 PM UTC
A fix is being deployed now, and we are monitoring the progress.
- monitoring Jun 05, 2026, 06:40 PM UTC
We've identified an earlier regression in one of our recent code changes that was affecting resolution of our previous fix. We're deploying this change now and applying a hot fix in the interim to restore access quickly
- monitoring Jun 05, 2026, 07:34 PM UTC
A fix has been released to prod and rolling out across the fleet for IRM, restoring access to affected customers. Thanks for your patience through this work. We're continuing to monitor to confirm we've returned to a steady state.
- monitoring Jun 05, 2026, 08:10 PM UTC
Continuing to monitor progress. Most customers affected should have all services restored, with a few remaining customers receiving updates as the rollout finishes out. Thanks again for your patience.
- resolved Jun 05, 2026, 09:35 PM UTC
This incident has been resolved.
Major June 5, 2026 Silences not Working as Expected 58m AWS Australia - prod-au-southeast-1: Alertmanager, AWS Brazil - prod-sa-east-1: Alertmanager, AWS Canada - prod-ca-east-0: Alertmanager, AWS Germany - prod-eu-west-2: Alertmanager and 14 more
Started Jun 05, 2026, 02:02 PM UTC· resolved Jun 05, 2026, 03:01 PM UTC
- identified Jun 05, 2026, 02:02 PM UTC
We have identified an issue causing Silences to not work as expected in the Cloud (Mimir) Alertmanager. Grafana Alertmanager is working ok, this is only affecting Data source-managed alerts.
- resolved Jun 05, 2026, 03:01 PM UTC
This incident has been resolved.
Major June 4, 2026 Grafana Assistant Skills Page Blank 1h 44m AWS Indonesia - prod-ap-southeast-2, AWS Brazil - prod-sa-east-1, AWS Canada - prod-ca-east-0, AWS Germany - prod-eu-west-2 and 27 more
Started Jun 04, 2026, 04:44 PM UTC· resolved Jun 04, 2026, 06:28 PM UTC
- investigating Jun 04, 2026, 04:44 PM UTC
We are currently investigating an issue affecting the Skills page of Grafana Assistant. Impacted deployments will encounter a blank screen when attempting to access this page. At this time, we have observed partial impact in the us-east-0 and us-central-0 regions, and will provide an update here if the scope of impact expands.
- investigating Jun 04, 2026, 04:56 PM UTC
We are continuing to investigate this issue.
- identified Jun 04, 2026, 05:00 PM UTC
The issue has been identified, and we are working on a fix.
- resolved Jun 04, 2026, 06:28 PM UTC
This incident has been resolved.
Minor June 3, 2026 K6 Test Runs Degraded Performance 9h 7m Cloud Test Runs, Cloud Output Test Runs
Started Jun 03, 2026, 08:40 PM UTC· resolved Jun 04, 2026, 05:47 AM UTC
- investigating Jun 03, 2026, 08:40 PM UTC
We are currently investigating an issue causing k6 test runs to take longer than expected to complete, or to time out within Grafana Cloud.
- monitoring Jun 03, 2026, 09:23 PM UTC
We have applied a fix, and are monitoring the results.
- resolved Jun 04, 2026, 05:47 AM UTC
This incident has been resolved.
Major June 3, 2026 Synthetic Scripted/Browser checks failure 6h 44m AWS Australia - prod-au-southeast-1: API, AWS Australia - prod-au-southeast-1: Public Probes, AWS Brazil - prod-sa-east-1: API, AWS Brazil - prod-sa-east-1: Public Probes and 52 more
Started Jun 03, 2026, 11:38 AM UTC· resolved Jun 03, 2026, 06:22 PM UTC
- investigating Jun 03, 2026, 11:38 AM UTC
We’re currently investigating an issue affecting Synthetic Monitoring where updates for Scripted/Browser checks might fail. Our team is actively working to identify the cause. Thank you for your patience.
- identified Jun 03, 2026, 05:07 PM UTC
We are in the process of deploying a fix for this issue.
- resolved Jun 03, 2026, 06:22 PM UTC
This incident has been resolved.
Minor June 2, 2026 tempo prod-25 write-path-down 8h 31m AWS UK - prod-gb-south-1
Started Jun 02, 2026, 11:15 PM UTC· resolved Jun 03, 2026, 07:46 AM UTC
- identified Jun 02, 2026, 11:15 PM UTC
we had an outage in tempo-prod-25/24 for the above times Looks like prod-25 was affected from 21:20 to 22:40 and prod-24 from 22:32 to 22:40 UTC
- resolved Jun 03, 2026, 07:46 AM UTC
This incident has been resolved.
Minor June 1, 2026 Alert manager unavailable in prod-us-central-0 32m GCP US Central - prod-us-central-0: Alertmanager and Rules Configuration API, GCP US Central - prod-us-central-0: Alertmanager
Started Jun 01, 2026, 07:53 PM UTC· resolved Jun 01, 2026, 08:26 PM UTC
- identified Jun 01, 2026, 07:53 PM UTC
Starting at 18:30 UTC, we noticed alert manager unavailability limited to prod-us-central-0 which affects grafana-managed and datasource-managed alerting, causing disruption to updating alertmanager config and limited disruption to alert sending. We have identified the cause and are in the process of remediation.
- monitoring Jun 01, 2026, 08:02 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jun 01, 2026, 08:26 PM UTC
This incident has been resolved.
Major May 29, 2026 Grafana Loki Log Query Issues 1h 56m AWS Australia - prod-au-southeast-1, AWS Brazil - prod-sa-east-1, AWS Canada - prod-ca-east-0, AWS Germany - prod-eu-west-2 and 12 more
Started May 29, 2026, 09:03 AM UTC· resolved May 29, 2026, 10:59 AM UTC
- investigating May 29, 2026, 09:03 AM UTC
We’re currently investigating an issue affecting Loki queries in Grafana. We have had reports from customers showing the logs are not loading or showing missing logs. Our team is actively working to identify the cause. Thank you for your patience.
- monitoring May 29, 2026, 10:28 AM UTC
We have identified the cause of this incident and a fix has been applied. Normal functions are returning. We are currently monitoring the recovery process.
- resolved May 29, 2026, 10:59 AM UTC
This incident has been resolved.
Major May 27, 2026 Prometheus Datasource Errors/Outage in prod-us-east-0 2h 36m AWS US East - prod-us-east-0: Querying
Started May 27, 2026, 08:22 PM UTC· resolved May 27, 2026, 10:59 PM UTC
- investigating May 27, 2026, 08:22 PM UTC
We’re currently investigating an issue affecting Prometheus datasources causing 500 internal or Unexpected errors. Our team is actively working to identify the cause. Thank you for your patience.
- investigating May 27, 2026, 09:36 PM UTC
We continue to investigate an issue affecting Prometheus datasources causing intermittent timeouts and unexpected errors, primarily impacting alert rule evaluations. Our team is actively working to identify the cause. Thank you for your patience.
- investigating May 27, 2026, 09:57 PM UTC
We are seeing recovery across affected Prometheus datasources, and error rates have significantly improved. The service is recovering without any required customer action, and our team continues to monitor stability while we investigate the underlying cause. We’ll provide another update as we learn more.
- resolved May 27, 2026, 10:59 PM UTC
This incident has been resolved. Thank you for your patience.