Grafana Outage History
Grafana is up right nowPingoru has recorded 195 incidents for Grafana: 195 in the last 12 months and 70 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 23 | 367 of 720 (51%) |
| May 2026 | 13 | 65 of 744 (9%) |
| June 2026 | 23 | 391 of 720 (54%) |
| July 2026 | 39 | 292 of 744 (39%) |
| August 2026 | 14 | 18 of 744 (2%) |
| September 2026 | 23 | 85 of 720 (12%) |
| October 2026 | 26 | 366 of 592 (62%) |
Incidents, newest first
Minor August 27, 2026 Elevated error rates affecting metrics writes in prod-us-central-0 56m GCP US Central - prod-us-central-0: Ingestion
Started Aug 27, 2026, 12:09 PM UTC· resolved Aug 27, 2026, 01:06 PM UTC
- investigating Aug 27, 2026, 12:09 PM UTC
We are investigating elevated error rates affecting metrics writes in prod-us-central-0. Customers in this region may see failed or delayed metrics ingestion, gaps in recent data on dashboards and queries, and alert rules that depend on the affected series may not evaluate as expected.
- monitoring Aug 27, 2026, 12:37 PM UTC
A fix has been applied and metrics write error rates in prod-us-central-0 have returned to normal levels. Metrics ingestion is operating as expected. We are continuing to monitor the affected systems to confirm full recovery before marking this incident resolved.
- resolved Aug 27, 2026, 01:06 PM UTC
This incident has been resolved.
Notice August 27, 2026 Mimir Writes Incident in prod-us-central-0 15m
Started Aug 27, 2026, 01:05 AM UTC· resolved Aug 27, 2026, 01:20 AM UTC
- resolved Aug 27, 2026, 01:54 AM UTC
Mimir writes in the prod-us-central-0 region were failing majorly for approximately 15 minutes from 1:05 to 1:20 UTC. The errors have currently stopped.
Minor August 25, 2026 Incident Management unavailable in US Central 23m Incident Management and Response (IRM)
Started Aug 25, 2026, 10:39 AM UTC· resolved Aug 25, 2026, 11:02 AM UTC
- investigating Aug 25, 2026, 10:39 AM UTC
We are investigating an outage affecting Grafana Incident in the US Central region (prod-us-central-0). Customers are unable to create, view, or query Incidents, and requests to the Incident public API are timing out. All customers in this region are affected. Grafana OnCall is not impacted — alert ingestion, alert group creation, and notification delivery are all operating normally.
- monitoring Aug 25, 2026, 10:49 AM UTC
A fix has been applied and Incident Management in US Central is recovering. The Incident public API is responding normally, and customers can create, view, and query Incidents again. We are continuing to monitor the recovery to confirm the service is fully stable.
- monitoring Aug 25, 2026, 11:01 AM UTC
We are continuing to monitor for any further issues.
- resolved Aug 25, 2026, 11:02 AM UTC
This incident has been resolved. Incident Management in US Central is operating as normal. Customers can create, view, and query Incidents, and the Incident public API is fully available. Grafana OnCall was not affected at any point during this incident.
Critical August 18, 2026 Cloud Logs read path outage on eu-west-2 –
Started Aug 18, 2026, 10:00 PM UTC· resolved Aug 18, 2026, 10:00 PM UTC
- resolved Aug 18, 2026, 10:53 PM UTC
We observed an issue which impacted the following service: reads of Cloud Logs in the eu-west-2 region. Affected services would include Explore, dashboards, alert evaluation when querying Logs. The time of impact lasted from approximately 22h15 and 22h23 UTC, on loki-prod-012. This incident has since been resolved.
Critical August 13, 2026 K6 Test Outage 1h 22m Cloud Test Runs, Cloud Output Test Runs
Started Aug 13, 2026, 03:25 PM UTC· resolved Aug 13, 2026, 04:47 PM UTC
- monitoring Aug 13, 2026, 03:25 PM UTC
Telemetry push APIs went down at 15:06 UTC, causing tests to lose telemetry data, like metrics, logs, traces, and browser screenshots. This issue has now been mitigated, and we are monitoring to determine root cause as well as full resolution.
- resolved Aug 13, 2026, 04:47 PM UTC
This incident has been resolved.
Minor August 10, 2026 Metrics: Elevated Error Rates Reads/Writes 55m
Started Aug 10, 2026, 12:20 PM UTC· resolved Aug 10, 2026, 01:15 PM UTC
- resolved Aug 10, 2026, 01:30 PM UTC
Between 12:20 and 13:15 UTC, we experienced elevated error rates for metrics reads and writes due to a networking issue. Impact was limited to a subset of deployments in the prod-us-central-0 region. A fix has been applied and error rates have since recovered.
Minor August 8, 2026 Alerting expressions pipeline failing when recovery settings – AWS Australia - prod-au-southeast-1: Rule Evaluation, AWS Australia - prod-au-southeast-1: Alertmanager and Rules Configuration API, AWS Australia - prod-au-southeast-1: Alertmanager, AWS Brazil - prod-sa-east-1: Rule Evaluation and 57 more
Started Aug 08, 2026, 07:48 PM UTC· resolved Aug 08, 2026, 07:48 PM UTC
- resolved Aug 08, 2026, 07:48 PM UTC
Due to a software bug the evaluation of some alert rules (primarily ones that have recovery threshold setting) were failing to be evaluated starting 14:00 UTC to 19:30 UTC today.
Notice August 5, 2026 Some Cloud Test Runs Terminated –
Started Aug 05, 2026, 05:57 PM UTC· resolved Aug 05, 2026, 05:57 PM UTC
- resolved Aug 05, 2026, 05:57 PM UTC
Between approximately 13:40 and 15:05 UTC today, a subset of cloud test runs were unexpectedly terminated and marked Aborted (by system). This affected runs that were in progress during three short windows in that period; test results and metrics for completed runs were not impacted. We have identified the cause and no further occurrences have been observed since 15:05 UTC. A fix is in place. The platform is currently operating normally and test runs are executing as expected
Minor August 5, 2026 o11y requests too large for nods in the prod-eu-west-2 region. 3m AWS Germany - prod-eu-west-2: Rule Evaluation, AWS Germany - prod-eu-west-2: Alertmanager and Rules Configuration API, AWS Germany - prod-eu-west-2: Alertmanager
Started Aug 05, 2026, 11:41 AM UTC· resolved Aug 05, 2026, 11:44 AM UTC
- investigating Aug 05, 2026, 11:41 AM UTC
We are currently investigating an issue with critical o11y requests being too large for nods in the prod-eu-west-2 region. This is leading to issues in scaling up services, and may may lead increased errors in requests.
- resolved Aug 05, 2026, 11:44 AM UTC
This incident has been resolved.
Critical August 4, 2026 Some Grafana Instances Unavailable 6h 20m GCP US Central - prod-us-central-0, GCP US Central - prod-us-central-3, GCP US Central - prod-us-central-4
Started Aug 04, 2026, 07:11 PM UTC· resolved Aug 05, 2026, 01:32 AM UTC
- investigating Aug 04, 2026, 07:11 PM UTC
We are currently investigating an issue that causing some Grafana deployments in prod-us-central-0 and prod-us-central-3 to become unavailable.
- investigating Aug 04, 2026, 08:12 PM UTC
We are continuing to investigate this issue.
- investigating Aug 04, 2026, 10:58 PM UTC
We have identified the cause: our cloud provider has exhausted compute capacity in the affected region. We are working directly with them and are moving affected workloads to alternative capacity to restore service.
- monitoring Aug 05, 2026, 12:54 AM UTC
We have rolled out a change to the affected regions to restore services.
- resolved Aug 05, 2026, 01:32 AM UTC
We are observing a continued period of stability and at this point, we are marking the incident resolved.
Minor August 4, 2026 Logs latency increase within prod-eu-west-3 3h 14m Azure Netherlands - prod-eu-west-3
Started Aug 04, 2026, 10:34 AM UTC· resolved Aug 04, 2026, 01:49 PM UTC
- investigating Aug 04, 2026, 10:34 AM UTC
We have identified minor latency increases in write endpoints within the prod-eu-west-3. Our team is currently monitoring and investigating this.
- investigating Aug 04, 2026, 12:46 PM UTC
There's still a minor latency increase in this cell however we have identified that it is improving although not back to normal as of yet. We will continue to look into this.
- resolved Aug 04, 2026, 01:49 PM UTC
This incident has been resolved.
Minor August 1, 2026 K6 - Cloud test-run issues 1h 5m Cloud Test Runs
Started Aug 01, 2026, 07:25 AM UTC· resolved Aug 01, 2026, 08:31 AM UTC
- investigating Aug 01, 2026, 07:25 AM UTC
We are currently investigating an issue which is resulting in some test runs to abort and metrics data to be incomplete for a subset of tests
- monitoring Aug 01, 2026, 08:06 AM UTC
A critical piece of infrastructure behaved poorly after a reboot. It has now been properly recovered
- resolved Aug 01, 2026, 08:31 AM UTC
This incident has been resolved.
Notice July 31, 2026 Partial Read Outage for Loki in prod-us-east-4 –
Started Jul 31, 2026, 07:30 PM UTC· resolved Jul 31, 2026, 07:30 PM UTC
- resolved Jul 31, 2026, 08:33 PM UTC
We are investigating a partial read outage affecting Loki in prod-us-east-4. Between 19:41 UTC and 19:50 UTC, a significant portion of read queries may have failed or returned errors. The issue has been identified and service has been restored. We are continuing to investigate the underlying cause and will provide additional information as it becomes available.
Minor July 31, 2026 Degraded Performance: Stack Provisioning Failures within certain reigons (PDC Setup) 1h 22m AWS Germany - prod-eu-west-2, AWS US East (OH) - prod-us-east-0, GCP US Central - prod-us-central-0, GCP US East - prod-us-east-1
Started Jul 31, 2026, 10:51 AM UTC· resolved Jul 31, 2026, 12:13 PM UTC
- investigating Jul 31, 2026, 10:51 AM UTC
We are currently investigating an issue impacting stack provisioning. Attempting to set up PDC on newly created stacks will currently fail across several regions. We are currently looking into the cause of this.
- resolved Jul 31, 2026, 12:13 PM UTC
He have identified the cause and applied a fix for this issue and all effected stacks within the affected regions are not working as expected.
Major July 30, 2026 PDC Authentication Issues 2h 54m AWS Germany - prod-eu-west-2, AWS US East - prod-us-east-0, GCP US Central - prod-us-central-0, GCP US East - prod-us-east-1
Started Jul 30, 2026, 02:45 PM UTC· resolved Jul 30, 2026, 05:40 PM UTC
- investigating Jul 30, 2026, 02:45 PM UTC
We are currently investigating an issue impacting PDC authentication. We will provide additional updates as they become available.
- investigating Jul 30, 2026, 02:51 PM UTC
We are continuing to investigate this issue.
- investigating Jul 30, 2026, 03:11 PM UTC
We are continuing to investigate this issue.
- monitoring Jul 30, 2026, 03:56 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 30, 2026, 05:40 PM UTC
This incident has been resolved.
Major July 30, 2026 Issues with Billing/Usage Dashboard Metrics and Panels. 2h 1m AWS Australia - prod-ap-southeast-2: Querying, AWS Australia - prod-ap-southeast-2: Ingestion, AWS Brazil - prod-sa-east-1: Querying, AWS Brazil - prod-sa-east-1: Ingestion and 46 more
Started Jul 30, 2026, 01:15 PM UTC· resolved Jul 30, 2026, 03:17 PM UTC
- investigating Jul 30, 2026, 01:15 PM UTC
We are investigating an outage for the Billing / Usage dashboard metrics and panels. This appears to be partially affecting organizations using Grafana Cloud. We are working on identifying and resolving the issue
- identified Jul 30, 2026, 01:50 PM UTC
We have identified the issue, and are working on deploying a fix.
- monitoring Jul 30, 2026, 02:34 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 30, 2026, 03:17 PM UTC
This incident has been resolved.
Minor July 29, 2026 IRM Performance Degradation in EU Region 2h 52m Incident Management and Response (IRM)
Started Jul 29, 2026, 09:46 PM UTC· resolved Jul 30, 2026, 12:39 AM UTC
- investigating Jul 29, 2026, 09:46 PM UTC
We are currently investigating an issue affecting Grafana IRM in our EU region. Users may experience a degraded or unresponsive IRM UI, reduced availability of the public API, and delays in notifications. Our engineering team is actively investigating the issue and working to restore normal service. We will provide another update as more information becomes available.
- identified Jul 29, 2026, 11:07 PM UTC
We have identified the underlying issue affecting Grafana IRM in our EU region and are actively working to restore normal service. Users may continue to experience a degraded or unresponsive IRM UI, reduced availability of the public API, and delays in notifications while mitigation efforts are underway. Our engineering team is working to restore full functionality as quickly as possible. We will provide another update as more information becomes available.
- resolved Jul 30, 2026, 12:39 AM UTC
services have been restored, the team is still investigating what the overall impact in regards to data is.
Minor July 29, 2026 Grafana Cloud non-billing usage metrics gaps in select regions –
Started Jul 29, 2026, 06:49 PM UTC· resolved Jul 29, 2026, 06:49 PM UTC
- resolved Jul 29, 2026, 06:49 PM UTC
Impact period: April 1 – July 29, 2026 Summary: During this period, some customers in a subset of regions may have experienced gaps in select ruler/recording rule metrics within their Grafana Cloud instances. Not all customers or environments in the listed regions were affected. Affected regions: A subset of environments across the following regions may have been impacted: ap-south-1 au-southeast-1 ca-east-0 eu-central-0 eu-west-2 eu-west-7 us-west-0 Affected values: grafanacloud_instance_queries_per_second grafanacloud_instance_rule_config_last_reload_successful grafanacloud_instance_rule_evaluations_total:rate5m grafanacloud_instance_rule_evaluation_failures_total:rate5m grafanacloud_instance_rule_group_interval_seconds grafanacloud_instance_rule_group_last_duration_seconds grafanacloud_instance_rule_group_iterations_total:rate5m grafanacloud_instance_rule_group_iterations_missed_total:rate5m grafanacloud_instance_rule_group_last_evaluation_timestamp_seconds grafanacloud_instance_rule_group_rules grafanacloud_instance_ruler_queries_failed_total:rate5m grafanacloud_instance_ruler_queries_zero_fetched_series_total:rate5m grafanacloud_instance_ruler_notifications_sent_total:rate5m grafanacloud_instance_ruler_notifications_errors_total:rate5m grafanacloud_instance_ruler_notifications_queue_capacity grafanacloud_instance_ruler_notifications_queue_length grafanacloud_instance_ruler_notifications_latency_seconds:99quantile grafanacloud_instance_ruler_notifications_latency_seconds:50quantile Current status: This issue has been resolved. No further action is required from customers. If you continue to notice gaps in the metrics described above, please reach out to support and reference this incident.
Major July 28, 2026 Partial OTLP Write Outage in prod-us-east-3 3m AWS US East - prod-us-east-3
Started Jul 28, 2026, 07:40 PM UTC· resolved Jul 28, 2026, 07:43 PM UTC
- investigating Jul 28, 2026, 07:40 PM UTC
We are investigating an issue affecting OTLP ingestion in the prod-us-east-3 region. Customers may experience intermittent failures when writing telemetry to the OTLP endpoint. Based on current information, logs are confirmed to be affected, and metrics and traces may also be impacted. Our investigation indicates intermittent failures began around 08:00 UTC, with the primary period of impact occurring between 15:30 UTC and 17:45 UTC. We are continuing to investigate the scope and root cause of this issue and will provide additional updates as more information becomes available.
- resolved Jul 28, 2026, 07:43 PM UTC
The issue affecting OTLP ingestion in the prod-us-east-3 region has been resolved. Between 13:30 UTC and 17:45 UTC, some customers experienced intermittent failures when writing telemetry to the OTLP endpoint. Logs were confirmed to be affected, and metrics and traces may also have experienced intermittent ingestion failures. Our investigation has concluded, and the affected services have recovered.
Critical July 24, 2026 Write Outage 1h 18m AWS US East (VA) - prod-us-east-3
Started Jul 24, 2026, 05:02 PM UTC· resolved Jul 24, 2026, 06:20 PM UTC
- investigating Jul 24, 2026, 05:02 PM UTC
We are currently investigating a Write outage ongoing, starting at 16:30 UTC for the impacted region.
- monitoring Jul 24, 2026, 05:18 PM UTC
Healthy as of 17:00 UTC. We are continuing to monitor.
- resolved Jul 24, 2026, 06:20 PM UTC
This incident has been resolved.
Major July 24, 2026 Errors creating new Slack integration for Grafana IRM 4h 45m Incident Management and Response (IRM)
Started Jul 24, 2026, 09:17 AM UTC· resolved Jul 24, 2026, 02:03 PM UTC
- investigating Jul 24, 2026, 09:17 AM UTC
We are investigating a possible error creating Slack integrations. Will update the status soon.
- identified Jul 24, 2026, 09:49 AM UTC
We have identified the root cause of the problem, and we are working on a fix
- identified Jul 24, 2026, 10:12 AM UTC
We have verified the fix, and we are starting to roll it out
- resolved Jul 24, 2026, 02:03 PM UTC
This incident has been resolved.
Notice July 23, 2026 Intermittent write outage from 21:20-21:26, 21:54-21:56, and 22:22-22:23 6h 31m
Started Jul 23, 2026, 11:00 PM UTC· resolved Jul 24, 2026, 05:32 AM UTC
- monitoring Jul 23, 2026, 11:00 PM UTC
status is health in prod-us-east-3.loki-prod-042 and we're continuing to monitor
- resolved Jul 24, 2026, 05:32 AM UTC
Writes are looking stable in the last 6h
Major July 23, 2026 Metrics Write Path Errors 2h 33m AWS US East - prod-us-east-0: Ingestion
Started Jul 23, 2026, 05:14 PM UTC· resolved Jul 23, 2026, 07:47 PM UTC
- investigating Jul 23, 2026, 05:14 PM UTC
We are currently investigating an elevated rate of error and latency in the impacted write path.
- monitoring Jul 23, 2026, 05:37 PM UTC
Error rates have dropped, and we are monitoring this issue for any recurrence.
- resolved Jul 23, 2026, 07:47 PM UTC
This incident has been resolved.
Major July 23, 2026 Stacks using SCIM user provisioning are currently unable to log into Grafana 5h 24m AWS Australia - prod-au-southeast-2, AWS Brazil - prod-sa-east-1, AWS Canada - prod-ca-east-0, AWS Germany - prod-eu-west-2 and 27 more
Started Jul 23, 2026, 10:13 AM UTC· resolved Jul 23, 2026, 03:37 PM UTC
- investigating Jul 23, 2026, 10:13 AM UTC
We are currently facing an issue where Stacks using SCIM user provisioning are currently unable to log into Grafana. We are currently investigating this issue and working on a fix.
- identified Jul 23, 2026, 11:26 AM UTC
The issue has been identified and we are currently working on the mitigations now. The scope is much more limited than initially thought only specific SCIM configurations are impacted.
- identified Jul 23, 2026, 01:23 PM UTC
We have applied a fix and are currently awaiting feedback from affected customers.
- resolved Jul 23, 2026, 03:37 PM UTC
This incident has been resolved.
Minor July 22, 2026 K6 - Cloud output test-runs are failing to fetch the script logs 57m Cloud Output Test Runs
Started Jul 22, 2026, 10:27 AM UTC· resolved Jul 22, 2026, 11:25 AM UTC
- investigating Jul 22, 2026, 10:27 AM UTC
We are currently investigating an issue cloud output test-runs which is resulting in failure to fetch the script logs. We are working on identifying the cause and working on a fix.
- monitoring Jul 22, 2026, 10:52 AM UTC
We have identified the cause and applied a fix which has has resulted in significant recovery. We will continue to monitor this before resolving.
- resolved Jul 22, 2026, 11:25 AM UTC
This incident has been resolved.