Grafana incident

Prometheus writes in prod-eu-west-3 are degraded

Resolved after 29 days 5 hours, affecting Azure Netherlands - prod-eu-west-3: Ingestion, on March 25, 2026.

monitoring Apr 20, 2026, 03:08 PM UTC

We are continuing to monitor for any further issues.

Started
Mar 25, 2026, 02:11 PM UTC
Resolved
Apr 23, 2026, 08:07 PM UTC
Duration
29d 5h
Detected by Pingoru
Mar 25, 2026, 02:11 PM UTC

Affected components

Azure Netherlands - prod-eu-west-3: Ingestion

Update timeline

  1. investigating Mar 25, 2026, 02:11 PM UTC

    The metric writes issue reported in https://status.grafana.com/incidents/gfshj17lxj5z is still ongoing. Our Engineering team is actively investigating this and we will provide further updates as our investigation progresses.

  2. investigating Mar 25, 2026, 09:35 PM UTC

    We are continuing to investigate this issue.

  3. monitoring Mar 26, 2026, 12:04 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. monitoring Mar 26, 2026, 05:45 PM UTC

    We are continuing to monitor the previously impacted environments.

  5. monitoring Mar 27, 2026, 09:05 PM UTC

    We are continuing to monitor this through the weekend.

  6. monitoring Apr 02, 2026, 09:38 PM UTC

    We are continuing to monitor for any further issues.

  7. monitoring Apr 08, 2026, 08:32 PM UTC

    We are still seeing intermittent issues and continue to seek a resolution

  8. monitoring Apr 14, 2026, 08:11 PM UTC

    We have deployed mitigation and seen improvement in write failures over the past week. We are still seeing intermittent spikes in latency and continue to monitor.

  9. monitoring Apr 20, 2026, 03:08 PM UTC

    We are continuing to monitor for any further issues.

  10. resolved Apr 23, 2026, 08:07 PM UTC

    This incident has been resolved. Thank you for your patience.

Your product runs on more than Grafana

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →