Resolved after 2 hours 13 minutes, affecting Websites and API and 1 more, on August 5, 2025.
We experienced a service disruption that affected our API and portal services. The incident was caused by database performance issues during routine maintenance, resulting in degraded service for our users.Timeline * **9:16 AM** - Routine database maintenance initiated * **9:25 AM** - Performance degradation detected; investigation began * **9:29 AM** - API services temporarily suspended to allow systems to recover * **9:50 AM** - Partial service restoration began * **10:15 AM** - Rate limiting implemented to ensure stable recovery * **10:23 AM** - Full service restored; all systems operational Impact During this incident: * API responses experienced significant delays * Some API endpoints were temporarily unavailable * Portal functionality was degraded or unavailable * Charging system was not affected — OCPP and chargepoint manager were operating normally. Root Cause The incident occurred during routine database maintenance when a schema update on a frequently accessed data table caused unexpected performance impacts. This led to cascading effects throughout our database infrastructure.Preventive Measures We are implementing the following improvements to prevent similar incidents: 1. **Enhanced Monitoring** - Implementing more granular performance monitoring to better predict the impact of maintenance operations 2. **Improved Maintenance Procedures** - Establishing stricter review processes for database changes, particularly for high-traffic components 3. **Application Resilience** - Improving our application's ability to handle traffic surges during recovery scenarios
Affected components
Update timeline
- investigating Aug 05, 2025, 07:36 AM UTC
We are currently investigating the issue, the pin seems to be hidden.
- investigating Aug 05, 2025, 07:45 AM UTC
We are continuing to investigate this issue.
- investigating Aug 05, 2025, 08:16 AM UTC
We are continuing to investigate this issue.
- identified Aug 05, 2025, 08:29 AM UTC
The issue has been identified and a fix is being implemented.
- monitoring Aug 05, 2025, 08:37 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Aug 05, 2025, 09:50 AM UTC
This incident has been resolved
- postmortem Aug 05, 2025, 09:50 AM UTC
We experienced a service disruption that affected our API and portal services. The incident was caused by database performance issues during routine maintenance, resulting in degraded service for our users.Timeline * **9:16 AM** - Routine database maintenance initiated * **9:25 AM** - Performance degradation detected; investigation began * **9:29 AM** - API services temporarily suspended to allow systems to recover * **9:50 AM** - Partial service restoration began * **10:15 AM** - Rate limiting implemented to ensure stable recovery * **10:23 AM** - Full service restored; all systems operational Impact During this incident: * API responses experienced significant delays * Some API endpoints were temporarily unavailable * Portal functionality was degraded or unavailable * Charging system was not affected — OCPP and chargepoint manager were operating normally. Root Cause The incident occurred during routine database maintenance when a schema update on a frequently accessed data table caused unexpected performance impacts. This led to cascading effects throughout our database infrastructure.Preventive Measures We are implementing the following improvements to prevent similar incidents: 1. **Enhanced Monitoring** - Implementing more granular performance monitoring to better predict the impact of maintenance operations 2. **Improved Maintenance Procedures** - Establishing stricter review processes for database changes, particularly for high-traffic components 3. **Application Resilience** - Improving our application's ability to handle traffic surges during recovery scenarios
Your product runs on more than Zaptec
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services