Mixpanel incident

Degraded Service for Requests to decide.mixpanel.com

Resolved after 7 hours 33 minutes, on October 21, 2026.

postmortem Oct 24, 2026, 01:58 AM UTC

# **Mixpanel RCA: Requests to** [**decide.mixpanel.com**](http://decide.mixpanel.com) **Failed, October 15–21, 2026** ### **Summary** From Oct 15, 2026 to Oct 21, 2026, requests sent to `decide.mixpanel.com` and `api-decide.mixpanel.com` failed. This affected customers who sent event tracking, profile updates, or feature flag requests to this hostname, usually through a proxy. Events sent to this domain during this window were not received by Mixpanel and this data cannot be restored by Mixpanel. **Traffic sent to** `api.mixpanel.com`**,** `api-js.mixpanel.com,` `api-eu.mixpanel.com`**, and** `api-in.mixpanel.com` **were not affected.** Service on this hostname was unintentionally impacted while we were removing the `/decide` endpoint infrastructure. The hostnames were treated as part of the endpoint due to an incorrect understanding of system dependencies. To prevent this class of issue, we are adding traffic monitoring for the `decide.mixpanel.com` and `api-decide.mixpanel.com` hostnames. We will also require confirmed zero traffic at the hostname level before any hostname is removed. **We have since restored service on** `decide.mixpanel.com` . ### **What happened** Older Mixpanel SDKs called the `/decide` endpoint to fetch configuration for [**Legacy Automatically Tracked Mobile Events**](https://docs.mixpanel.com/docs/tracking-methods/sdks/ios#legacy-automatically-tracked-events). We [announced the endpoint’s retirement](https://docs.mixpanel.com/changelogs#2026-01-26), and for some time `/decide` has returned a `410` response. Separately, `decide.mixpanel.com` and `api-decide.mixpanel.com` acted as aliases for our standard API hostnames. For example, `decide.mixpanel.com/track` ingested events the same way `api-js.mixpanel.com/track` does. Some customers had set up proxies or SDK configurations to send their traffic to these hostnames. On October 15 at 7:40 PM CDT, during the final cleanup of the `/decide` endpoint, we removed the DNS records for `decide.mixpanel.com` and `api-decide.mixpanel.com` and removed both hostnames from our load balancer. Before making the change, we confirmed that traffic to the `/decide` endpoint had dropped to zero. However, our monitoring measured traffic by endpoint rather than by hostname. It therefore did not show that other requests, such as `/track` and `/flags`, were still reaching Mixpanel through these hostnames. From then on, all requests to this hostname failed, including endpoints such as `/track`, `/engage`, `/groups`, and `/flags`. * **Events and Profiles:** Events, Profiles, and any data intended for [Ingestion API endpoints](https://docs.mixpanel.com/reference/ingestion-api) to these hostnames during this window never reached Mixpanel, so we do not have a copy to restore. If you retained this data elsewhere, such as in proxy logs or your data warehouse, you can re-send it to Ingestion API’s [/import](https://docs.mixpanel.com/reference/import-events), [/engage,](https://docs.mixpanel.com/reference/profile-set) and [/groups](https://docs.mixpanel.com/reference/group-set-property). * **Feature flags:** Feature flag requests sent to these hostnames failed. SDKs served the last stored variant if one existed, or otherwise the fallback value configured in your code. ### Timeline \(Central Time, October 2026\) * **October 15, 7:40 PM CDT:** The hostname removal is applied. Requests to `decide.mixpanel.com` and `api-decide.mixpanel.com` begin failing. * **October 16, 5:22 AM CDT:** We receive the first customer report. It is initially understood to be related to the planned `/decide` retirement. * **October 21, 11:42 AM CDT:** Following an increase in customer reports about tracking failing for proxies, we identify the hostname removal as the cause. * **October 21, 12:26 PM CDT:** We restore the hostnames. `/decide` remains retired. * **October 21, 1:51 PM CDT:** We confirm traffic to the restored hostnames has recovered. * **October 21, 8:34 PM CDT:** The incident is resolved. ### **Root cause** 1. During the `/decide` retirement, `decide.mixpanel.com` and `api-decide.mixpanel.com` were unintentionally removed. 2. These hostnames had been configured to accept all API traffic, not only `/decide` requests, and this behavior was undocumented. 3. Our monitoring tracked traffic to the `/decide` endpoint but not to these hostnames, so pre-change checks did not show that other traffic was still active. 4. Because the hostnames and the retired endpoint share a name, early customer reports were initially interpreted as expected effects of the announced `/decide` retirement. This delayed escalation. ### **What we’re changing** * **Monitoring and alerting:** We are adding traffic and error-rate monitoring and alerting for `decide.mixpanel.com` and `api-decide.mixpanel.com`, so failures are detected automatically rather than through customer reports. * **Hostname audit:** We are auditing all public API hostnames and routing aliases for similar undocumented entry points, and we will review new hostnames and aliases before they go live. We are also tracking traffic that arrives on any unexpected hostname. * **Safer removals:** Before any future change to a public hostname, we will confirm zero traffic at the hostname level, and we will identify and contact any projects still using it. * **Customer communication:** We will give customers advance notice of material infrastructure changes that affect public hostnames, including publishing them on a timeline in our changelog. **Already deployed:** * Restored `decide.mixpanel.com` and `api-decide.mixpanel.com` * `/decide` remains retired #### **Common questions** * **Was my data affected?** Only if your SDK or proxy sent requests to `decide.mixpanel.com` or `api-decide.mixpanel.com` between October 15 and October 21. Data sent there during that window were not received and cannot be restored by Mixpanel. If you retained a copy of that data, you can re-send it to Mixpanel using our Ingestion API endpoints such as `/import` for events. * **Do I need to change anything?** No, the domains work again. We recommend using `api.mixpanel.com` \(or `api-eu.mixpanel.com` / `api-in.mixpanel.com` for EU and India data residency\) as your API domain. * **Is** `/decide` coming back? No. It remains retired as announced, and it is separate from the `decide.mixpanel.com` hostname. We’re sorry for the disruption. If you have questions about your project, please contact your account team or Mixpanel Support.

Started
Oct 21, 2026, 06:00 PM UTC
Resolved
Oct 22, 2026, 01:34 AM UTC
Duration
7h 33m
Detected by Pingoru
Oct 21, 2026, 06:00 PM UTC

Update timeline

  1. monitoring Oct 21, 2026, 10:28 PM UTC

    We've received reports that requests to decide.mixpanel.com are failing. Any traffic sent through decide.mixpanel.com may be affected, including feature flags and event tracking. To restore service now, point requests to api.mixpanel.com (US), api-eu.mixpanel.com (EU), or api-in.mixpanel.com (IN), depending on your project's Data Residency. We've rolled out a mitigation and are monitoring the results. We're seeing traffic restored. We will post progress updates on our status page. If you have any questions, please contact support.

  2. resolved Oct 22, 2026, 01:34 AM UTC

    This incident has been resolved.

  3. postmortem Oct 24, 2026, 01:58 AM UTC

    # **Mixpanel RCA: Requests to** [**decide.mixpanel.com**](http://decide.mixpanel.com) **Failed, October 15–21, 2026** ### **Summary** From Oct 15, 2026 to Oct 21, 2026, requests sent to `decide.mixpanel.com` and `api-decide.mixpanel.com` failed. This affected customers who sent event tracking, profile updates, or feature flag requests to this hostname, usually through a proxy. Events sent to this domain during this window were not received by Mixpanel and this data cannot be restored by Mixpanel. **Traffic sent to** `api.mixpanel.com`**,** `api-js.mixpanel.com,` `api-eu.mixpanel.com`**, and** `api-in.mixpanel.com` **were not affected.** Service on this hostname was unintentionally impacted while we were removing the `/decide` endpoint infrastructure. The hostnames were treated as part of the endpoint due to an incorrect understanding of system dependencies. To prevent this class of issue, we are adding traffic monitoring for the `decide.mixpanel.com` and `api-decide.mixpanel.com` hostnames. We will also require confirmed zero traffic at the hostname level before any hostname is removed. **We have since restored service on** `decide.mixpanel.com` . ### **What happened** Older Mixpanel SDKs called the `/decide` endpoint to fetch configuration for [**Legacy Automatically Tracked Mobile Events**](https://docs.mixpanel.com/docs/tracking-methods/sdks/ios#legacy-automatically-tracked-events). We [announced the endpoint’s retirement](https://docs.mixpanel.com/changelogs#2026-01-26), and for some time `/decide` has returned a `410` response. Separately, `decide.mixpanel.com` and `api-decide.mixpanel.com` acted as aliases for our standard API hostnames. For example, `decide.mixpanel.com/track` ingested events the same way `api-js.mixpanel.com/track` does. Some customers had set up proxies or SDK configurations to send their traffic to these hostnames. On October 15 at 7:40 PM CDT, during the final cleanup of the `/decide` endpoint, we removed the DNS records for `decide.mixpanel.com` and `api-decide.mixpanel.com` and removed both hostnames from our load balancer. Before making the change, we confirmed that traffic to the `/decide` endpoint had dropped to zero. However, our monitoring measured traffic by endpoint rather than by hostname. It therefore did not show that other requests, such as `/track` and `/flags`, were still reaching Mixpanel through these hostnames. From then on, all requests to this hostname failed, including endpoints such as `/track`, `/engage`, `/groups`, and `/flags`. * **Events and Profiles:** Events, Profiles, and any data intended for [Ingestion API endpoints](https://docs.mixpanel.com/reference/ingestion-api) to these hostnames during this window never reached Mixpanel, so we do not have a copy to restore. If you retained this data elsewhere, such as in proxy logs or your data warehouse, you can re-send it to Ingestion API’s [/import](https://docs.mixpanel.com/reference/import-events), [/engage,](https://docs.mixpanel.com/reference/profile-set) and [/groups](https://docs.mixpanel.com/reference/group-set-property). * **Feature flags:** Feature flag requests sent to these hostnames failed. SDKs served the last stored variant if one existed, or otherwise the fallback value configured in your code. ### Timeline \(Central Time, October 2026\) * **October 15, 7:40 PM CDT:** The hostname removal is applied. Requests to `decide.mixpanel.com` and `api-decide.mixpanel.com` begin failing. * **October 16, 5:22 AM CDT:** We receive the first customer report. It is initially understood to be related to the planned `/decide` retirement. * **October 21, 11:42 AM CDT:** Following an increase in customer reports about tracking failing for proxies, we identify the hostname removal as the cause. * **October 21, 12:26 PM CDT:** We restore the hostnames. `/decide` remains retired. * **October 21, 1:51 PM CDT:** We confirm traffic to the restored hostnames has recovered. * **October 21, 8:34 PM CDT:** The incident is resolved. ### **Root cause** 1. During the `/decide` retirement, `decide.mixpanel.com` and `api-decide.mixpanel.com` were unintentionally removed. 2. These hostnames had been configured to accept all API traffic, not only `/decide` requests, and this behavior was undocumented. 3. Our monitoring tracked traffic to the `/decide` endpoint but not to these hostnames, so pre-change checks did not show that other traffic was still active. 4. Because the hostnames and the retired endpoint share a name, early customer reports were initially interpreted as expected effects of the announced `/decide` retirement. This delayed escalation. ### **What we’re changing** * **Monitoring and alerting:** We are adding traffic and error-rate monitoring and alerting for `decide.mixpanel.com` and `api-decide.mixpanel.com`, so failures are detected automatically rather than through customer reports. * **Hostname audit:** We are auditing all public API hostnames and routing aliases for similar undocumented entry points, and we will review new hostnames and aliases before they go live. We are also tracking traffic that arrives on any unexpected hostname. * **Safer removals:** Before any future change to a public hostname, we will confirm zero traffic at the hostname level, and we will identify and contact any projects still using it. * **Customer communication:** We will give customers advance notice of material infrastructure changes that affect public hostnames, including publishing them on a timeline in our changelog. **Already deployed:** * Restored `decide.mixpanel.com` and `api-decide.mixpanel.com` * `/decide` remains retired #### **Common questions** * **Was my data affected?** Only if your SDK or proxy sent requests to `decide.mixpanel.com` or `api-decide.mixpanel.com` between October 15 and October 21. Data sent there during that window were not received and cannot be restored by Mixpanel. If you retained a copy of that data, you can re-send it to Mixpanel using our Ingestion API endpoints such as `/import` for events. * **Do I need to change anything?** No, the domains work again. We recommend using `api.mixpanel.com` \(or `api-eu.mixpanel.com` / `api-in.mixpanel.com` for EU and India data residency\) as your API domain. * **Is** `/decide` coming back? No. It remains retired as announced, and it is separate from the `decide.mixpanel.com` hostname. We’re sorry for the disruption. If you have questions about your project, please contact your account team or Mixpanel Support.

Your product runs on more than Mixpanel

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →