Resolved after 40 minutes, affecting App and App and 1 more, on February 6, 2025.
Incident: At 2025-02-06 15:37 UTC, a configuration change caused some of our servers to crash, leading to complete downtime for 20 minutes. We rolled back the configuration, but this event triggered a cascade failure in network routing components; our systems recovered over the next 30 minutes, with full recovery by 2025-02-06 16:30. Impact: Users were unable to use Asana during this event in all regions, with downtime including App, API, and Mobile. No customer data was lost. Moving forward: We have added validation to prevent a repeat of this issue, and are updating our networking components to avoid this form of cascade failure. We will be moving to staged configuration rollouts to identify this class of issue without causing downtime for customer traffic. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.
Affected components
Update timeline
- investigating Feb 06, 2025, 03:52 PM UTC
We are seeing elevated errors for App and API, and are investigating.
- identified Feb 06, 2025, 03:57 PM UTC
We have identified a cause and are implemented a fix.
- monitoring Feb 06, 2025, 04:21 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 06, 2025, 04:29 PM UTC
This incident has been resolved.
- postmortem Feb 07, 2025, 10:47 PM UTC
Incident: At 2025-02-06 15:37 UTC, a configuration change caused some of our servers to crash, leading to complete downtime for 20 minutes. We rolled back the configuration, but this event triggered a cascade failure in network routing components; our systems recovered over the next 30 minutes, with full recovery by 2025-02-06 16:30. Impact: Users were unable to use Asana during this event in all regions, with downtime including App, API, and Mobile. No customer data was lost. Moving forward: We have added validation to prevent a repeat of this issue, and are updating our networking components to avoid this form of cascade failure. We will be moving to staged configuration rollouts to identify this class of issue without causing downtime for customer traffic. Our metric considers a weighted average of uptime experienced by users at each data center. The number of minutes of downtime shown reflects this weighted average.
Your product runs on more than Asana
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services