Pinpoint incident

Investigating Service Disruption

Resolved after 20 minutes, affecting Pinpoint, on March 19, 2026.

postmortem Mar 19, 2026, 01:37 PM UTC

**Date:** 19 March 2026 **Duration:** Approximately 35 minutes \(11:46 to 12:21 UTC\). ### What Happened Pinpoint experienced a service outage beginning at approximately 11:46 UTC on 19 March. Users were unable to access the platform and received 503 errors. Full service was restored by approximately 12:21 UTC. No data was lost or corrupted. ### Timeline \(UTC\) * **11:46** - Automated monitoring alerts triggered for service unavailability. * **11:47** - Engineering team began investigating. * **12:01** - Status page updated to reflect the incident. * **12:10** - Root cause identified as a database firewall misconfiguration introduced during an infrastructure change. * **12:20** - Fix applied; service restored. * **12:21** - Incident marked as resolved. ### Why It Happened During an infrastructure configuration change, a database firewall rule was inadvertently applied with an overly restrictive scope. This prevented our application servers from connecting to the primary database, causing all requests to fail. ### What We're Doing About It * **Immediate fix:** The firewall configuration has been corrected and verified. * **Hardening:** We are making production database firewall rules explicitly immutable and tightening API token permissions to prevent unexpected firewall modifications. We sincerely apologise for the disruption and appreciate your patience.

Started
Mar 19, 2026, 12:01 PM UTC
Resolved
Mar 19, 2026, 12:21 PM UTC
Duration
20m
Detected by Pingoru
Mar 19, 2026, 12:01 PM UTC

Affected components

Pinpoint

Update timeline

  1. investigating Mar 19, 2026, 12:01 PM UTC

    We’re aware of a current issue causing 503 errors and are investigating urgently.

  2. resolved Mar 19, 2026, 12:21 PM UTC

    This has been resolved. A full post mortem will follow soon.

  3. postmortem Mar 19, 2026, 01:37 PM UTC

    **Date:** 19 March 2026 **Duration:** Approximately 35 minutes \(11:46 to 12:21 UTC\). ### What Happened Pinpoint experienced a service outage beginning at approximately 11:46 UTC on 19 March. Users were unable to access the platform and received 503 errors. Full service was restored by approximately 12:21 UTC. No data was lost or corrupted. ### Timeline \(UTC\) * **11:46** - Automated monitoring alerts triggered for service unavailability. * **11:47** - Engineering team began investigating. * **12:01** - Status page updated to reflect the incident. * **12:10** - Root cause identified as a database firewall misconfiguration introduced during an infrastructure change. * **12:20** - Fix applied; service restored. * **12:21** - Incident marked as resolved. ### Why It Happened During an infrastructure configuration change, a database firewall rule was inadvertently applied with an overly restrictive scope. This prevented our application servers from connecting to the primary database, causing all requests to fail. ### What We're Doing About It * **Immediate fix:** The firewall configuration has been corrected and verified. * **Hardening:** We are making production database firewall rules explicitly immutable and tightening API token permissions to prevent unexpected firewall modifications. We sincerely apologise for the disruption and appreciate your patience.

Your product runs on more than Pinpoint

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →