Treefort incident

Authentication intermittently unavailable

Minor

Resolved after 1 hour, on March 27, 2026.

monitoring Mar 27, 2026, 11:25 PM UTC

We've performed a database failover to a new primary node and are seeing services recover. We're continuing to monitor.

Started
Mar 27, 2026, 10:04 PM UTC
Resolved
Mar 27, 2026, 11:04 PM UTC
Duration
1h
Detected by Pingoru
Mar 27, 2026, 10:04 PM UTC

Update timeline

  1. investigating Mar 27, 2026, 09:09 PM UTC

    We're seeing a spike in failed requests to our API and are actively investigating. Some thumbnails and content may not load, and user authentication may be slow or encounter errors.

  2. monitoring Mar 27, 2026, 09:19 PM UTC

    An upstream provider is experiencing capacity issues resulting in intermittent network failures. Services are recovering at this time and error rates have dropped significantly. We are continuing to monitor apps to ensure full recovery.

  3. identified Mar 27, 2026, 11:08 PM UTC

    Our upstream provider is having hardware issues with the machines our authentication service runs on. We are working to provision additional machines to restore full service. Some actions that require authentication may intermittently fail in the meantime.

  4. monitoring Mar 27, 2026, 11:25 PM UTC

    We've performed a database failover to a new primary node and are seeing services recover. We're continuing to monitor.

  5. resolved Mar 28, 2026, 04:48 AM UTC

Your product runs on more than Treefort

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →