Resolved after 51 minutes, affecting US Central, on October 30, 2025.
At approximately 4:10 PM ET, ClearlyIP's monitoring systems detected registration issues within US-Central trunking. Within a few minutes, it was determined that one of the two primary trunking servers was maintaining its active calls \(2.us-central\) and that its counterpart \(1.us-central\) had stalled and was not accepting traffic. It was quickly determined that the issue stemmed from a database cluster deadlock within the US-Central footprint. Upon analysis, the deadlock's cause was identified to be related to unsuccessful credential change activity. A series of actions were taken during the next 30 minutes which restored 2.us-central to service. Following that, maintenance was performed to resolve version differences on the database node not currently in production before restoring it \(and 1.us-central\) to normal service shortly after 6PM ET.
Affected components
Update timeline
- investigating Oct 30, 2025, 08:21 PM UTC
We are currently investigating this issue.
- monitoring Oct 30, 2025, 08:51 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Oct 30, 2025, 09:13 PM UTC
This incident has been resolved.
- postmortem Nov 03, 2025, 08:39 PM UTC
At approximately 4:10 PM ET, ClearlyIP's monitoring systems detected registration issues within US-Central trunking. Within a few minutes, it was determined that one of the two primary trunking servers was maintaining its active calls \(2.us-central\) and that its counterpart \(1.us-central\) had stalled and was not accepting traffic. It was quickly determined that the issue stemmed from a database cluster deadlock within the US-Central footprint. Upon analysis, the deadlock's cause was identified to be related to unsuccessful credential change activity. A series of actions were taken during the next 30 minutes which restored 2.us-central to service. Following that, maintenance was performed to resolve version differences on the database node not currently in production before restoring it \(and 1.us-central\) to normal service shortly after 6PM ET.
Your product runs on more than ClearlyIP
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services