Resolved after 33 minutes, affecting EU-CENTRAL-1, on March 12, 2025.
On 09:43 UTC, QStash experienced degraded service when a high number of requests to a specific domain were throttled, resulting in timeouts during an unexpected phase of the TCP connection establishment. These requests and resulting retries triggered excessive consumption of network resources and negatively impacted all users. We have added more resources to QStash as a quick remediation and as for the resolution, we have improved the timeout mechanism to detect and fail-faster in such cases.
Affected components
Update timeline
- investigating Mar 12, 2025, 09:43 AM UTC
We are currently investigating this issue.
- monitoring Mar 12, 2025, 10:08 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Mar 12, 2025, 10:16 AM UTC
This incident has been resolved.
- postmortem Mar 12, 2025, 12:26 PM UTC
On 09:43 UTC, QStash experienced degraded service when a high number of requests to a specific domain were throttled, resulting in timeouts during an unexpected phase of the TCP connection establishment. These requests and resulting retries triggered excessive consumption of network resources and negatively impacted all users. We have added more resources to QStash as a quick remediation and as for the resolution, we have improved the timeout mechanism to detect and fail-faster in such cases.
Your product runs on more than Upstash
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services