Overall Degraded Performance

Update timeline

1. investigating Sep 16, 2026, 11:15 AM UTC

   Impact Intermittent delays and timeouts across all product surfaces Root Cause: Traffic spikes from background operations overloading ClickHouse Incident Summary Spotflow experienced periods of degraded performance and intermittent timeouts today across the entire platform. The root cause was identified as multiple sharp spikes in background operations traffic, which overloaded our primary ClickHouse instance and saturated database query resources. Key Impacts & Mitigations Product Surfaces: Front-end dashboards and analytics queries experienced elevated response latency and intermittent execution timeouts. Background Queues: Non-essential asynchronous background operations were temporarily throttled to relieve pressure on the ClickHouse cluster. Database Recovery: Query execution times are returned to baseline levels as background traffic normalizes.

2. resolved Sep 16, 2026, 04:23 PM UTC

   Incident has been resolved. Next Steps Implement stricter rate-limiting and queue pacing for background jobs during peak hours. Audit ClickHouse resource allocation and concurrency limits to safely isolate background processing from user-facing workload traffic.

3. resolved Sep 16, 2026, 04:23 PM UTC

   Incident has been resolved. ### Next Steps * Implement stricter rate-limiting and queue pacing for background jobs during peak hours. * Audit ClickHouse resource allocation and concurrency limits to safely isolate background processing from user-facing workload traffic.