Resolved after 40 minutes, affecting Clearly Cloud USA - w, on October 29, 2025.
At approximately 2:30 PM ET on October 29, alerts began to be received by ClearlyIP's Network Operations team indicating problems accessing the database cluster used by Clearly Cloud for most US-based customers. This was interfering with call routing for a portion of customers and prevented access to Call Panel and other system components. The team immediately began investigating and observed the problem related to a protective mechanism which had detected large amounts of database request traffic and proactively restarted services to prevent data loss or corruption. The source of the requests was quickly identified, stopped, and determined to be legitimate. The team then checked each server in the cluster before determining it was safe to restore normal service by approximately 2:45 PM ET. A postmortem analysis on the source of the database traffic identified it to be related to delivering an unusual backlog of SMS and email messages from the platform. Temporary measures were taken to prevent a recurrence of the same issue while long-term improvements are scheduled.
Affected components
Update timeline
- investigating Oct 29, 2025, 06:37 PM UTC
Clearly Cloud USA is currently down and the team is investigating.
- identified Oct 29, 2025, 06:56 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Oct 29, 2025, 07:02 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Oct 29, 2025, 07:17 PM UTC
This incident has been resolved.
- postmortem Nov 03, 2025, 04:09 PM UTC
At approximately 2:30 PM ET on October 29, alerts began to be received by ClearlyIP's Network Operations team indicating problems accessing the database cluster used by Clearly Cloud for most US-based customers. This was interfering with call routing for a portion of customers and prevented access to Call Panel and other system components. The team immediately began investigating and observed the problem related to a protective mechanism which had detected large amounts of database request traffic and proactively restarted services to prevent data loss or corruption. The source of the requests was quickly identified, stopped, and determined to be legitimate. The team then checked each server in the cluster before determining it was safe to restore normal service by approximately 2:45 PM ET. A postmortem analysis on the source of the database traffic identified it to be related to delivering an unusual backlog of SMS and email messages from the platform. Temporary measures were taken to prevent a recurrence of the same issue while long-term improvements are scheduled.
Your product runs on more than ClearlyIP
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services