JustCall incident
Partial Dashboard, Calling, SMS, and MMS Service Disruption
Resolved after 23 hours 30 minutes, affecting SMS Service and JustCall Dashboard and 1 more, on July 8, 2025.
**Date:** 9th July 2025 **Duration:** 14:51 UTC - 15:25 UTC **Affected Services:** * Customer-facing Dashboard * SMS Delivery * MMS Delivery * Inbound Calling * Outbound Calling * Associated internal services **Impact** A subset of customers experienced degraded performance or complete unavailability in: * Accessing the dashboard * Users experienced delays in sending and receiving SMS and MMS messages. * Intermittent degradation in both inbound and outbound call performance. * Delayed or failed service actions tied to within a designated cloud service region. **Root Cause** A malfunction in one of our cache systems triggered a reboot, leading to the invalidation of all cache entries in that system: * Massive cache miss rate * Significant surge in requests directly hitting primary databases and downstream services * Snowball/piggyback effect where database latency grew, increasing queue sizes and causing cascading slowdowns across services * Overwhelmed compute resources and degraded service health in that region. **Resolution** * Engineers manually restored cache to a known good state from cache backups. * Temporarily scaled up database resources and async queues.
Affected components
Update timeline
- investigating Jul 09, 2025, 02:42 PM UTC
Some customers are experiencing issues with loading the JustCall dashboard, and failures in sending SMS, MMS, and making or receiving calls via the web interface. Investigation is ongoing.
- monitoring Jul 09, 2025, 02:44 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jul 09, 2025, 02:45 PM UTC
This incident has been resolved.
- postmortem Jul 10, 2025, 03:50 PM UTC
**Date:** 9th July 2025 **Duration:** 14:51 UTC - 15:25 UTC **Affected Services:** * Customer-facing Dashboard * SMS Delivery * MMS Delivery * Inbound Calling * Outbound Calling * Associated internal services **Impact** A subset of customers experienced degraded performance or complete unavailability in: * Accessing the dashboard * Users experienced delays in sending and receiving SMS and MMS messages. * Intermittent degradation in both inbound and outbound call performance. * Delayed or failed service actions tied to within a designated cloud service region. **Root Cause** A malfunction in one of our cache systems triggered a reboot, leading to the invalidation of all cache entries in that system: * Massive cache miss rate * Significant surge in requests directly hitting primary databases and downstream services * Snowball/piggyback effect where database latency grew, increasing queue sizes and causing cascading slowdowns across services * Overwhelmed compute resources and degraded service health in that region. **Resolution** * Engineers manually restored cache to a known good state from cache backups. * Temporarily scaled up database resources and async queues.
Your product runs on more than JustCall
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services