Scalyr incident

ingestions and queries are affected on app.scalyr.com, app.eu.scalyr.com, app.dataset.com, and app.eu.dataset.com

Resolved after 5 hours 42 minutes, on January 16, 2025.

monitoring Jan 16, 2025, 10:01 PM UTC

Recovery is currently in progress. The estimated time for full ingestion restoration is approximately 60 to 90 minutes from now.

Started
Jan 16, 2025, 06:39 PM UTC
Resolved
Jan 17, 2025, 12:22 AM UTC
Duration
5h 42m
Detected by Pingoru
Jan 16, 2025, 06:39 PM UTC

Update timeline

  1. investigating Jan 16, 2025, 06:39 PM UTC

    We are currently investigating this issue.

  2. investigating Jan 16, 2025, 07:10 PM UTC

    We are in the process of rolling back the change and restarting the affected servers. Some accounts may observe a gradual recovery of their logs.

  3. identified Jan 16, 2025, 07:48 PM UTC

    A misconfiguration deployment caused server resource allocation issues. A rollback and server restart have been initiated to address the problem.

  4. monitoring Jan 16, 2025, 10:01 PM UTC

    Recovery is currently in progress. The estimated time for full ingestion restoration is approximately 60 to 90 minutes from now.

  5. resolved Jan 17, 2025, 12:22 AM UTC

    The backlog data caused by the incident has been fully recovered in all regions

Your product runs on more than Scalyr

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →