Is Synthflow AI down?

Status: Resolved Summary From 17:15 UTC on 27 October to 08:42 UTC on 28 October, some customers in our US and EU regions could not open the Settings page, and some could not log in with SSO. Both problems are now fixed and have not come back. Impact Affected users saw the Settings page fail to load, and some SSO logins failed. Calls and voice agents kept running as normal. Data impact A small number of workspaces may have lost some workspace-level settings, such as the workspace name. No customer data was lost. We will contact each affected workspace directly to help restore these settings. Root cause An issue with one of our third-party providers caused some workspace settings, including the workspace’s region, to be cleared. Without these settings, the Settings page and SSO login stopped working for affected workspaces. Our first fix restored the settings, but they were cleared again until we found and fixed the underlying cause. Timeline (UTC) • 27 Oct, 17:15: We started investigating reports of Settings and SSO login failures. • 27 Oct, 18:33: We restored settings for the affected workspaces. • 27 Oct, 21:11: Some settings were being cleared again. We found the cause and began restoring all affected workspaces. • 28 Oct, 08:42: We finished restoring every affected workspace and added safeguards. Our checks found no affected workspaces after this. What we’re doing to prevent this • We’ve added safeguards that stop workspace settings from being cleared if a third-party provider fails. • To migrate off this provider to prevent this issue from happening again in the future We’re sorry for the disruption. If you still see problems with Settings or SSO login, please contact support at [email protected], or reach out to your account representative.

Status: Resolved Summary From 17:15 UTC on 27 October to 08:42 UTC on 28 October, some customers in our US and EU regions could not open the Settings page, and some could not log in with SSO. Both problems are now fixed and have not come back. Impact Affected users saw the Settings page fail to load, and some SSO logins failed. Calls and voice agents kept running as normal. Data impact A small number of workspaces may have lost some workspace-level settings, such as the workspace name. No customer data was lost. We will contact each affected workspace directly to help restore these settings. Root cause An issue with one of our third-party providers caused some workspace settings, including the workspace’s region, to be cleared. Without these settings, the Settings page and SSO login stopped working for affected workspaces. Our first fix restored the settings, but they were cleared again until we found and fixed the underlying cause. Timeline (UTC) • 27 Oct, 17:15: We started investigating reports of Settings and SSO login failures. • 27 Oct, 18:33: We restored settings for the affected workspaces. • 27 Oct, 21:11: Some settings were being cleared again. We found the cause and began restoring all affected workspaces. • 28 Oct, 08:42: We finished restoring every affected workspace and added safeguards. Our checks found no affected workspaces after this. What we’re doing to prevent this • We’ve added safeguards that stop workspace settings from being cleared if a third-party provider fails. • To migrate off this provider to prevent this issue from happening again in the future We’re sorry for the disruption. If you still see problems with Settings or SSO login, please contact support at [email protected], or reach out to your account representative.

Status: Identified We are aware of an issue affecting access to the Synthflow dashboard. Some users may also see degraded API performance, such as slower response times or intermittent errors. Our team is working to restore full service as quickly as possible. Live calling is operating normally. Agents continue to place and receive calls without interruption. We will post another update within 30 minutes, or sooner as new information becomes available. We apologise for the inconvenience. Affected components Synthflow API (Degraded performance) Synthflow Dashboard (Full outage)

Status: Resolved Access to the Synthflow dashboard and normal API performance have been restored. The issue was caused by a downstream vendor and has now been fixed. Live calling remained operational throughout the incident. We apologize for the disruption. Affected components Synthflow API (Operational) Synthflow Dashboard (Operational)

Status: Resolved Access to the Synthflow dashboard and normal API performance have been restored. The issue was caused by a downstream vendor and has now been fixed. Live calling remained operational throughout the incident. We apologize for the disruption. Affected components Synthflow API - Legacy infrastructure (Operational) Synthflow Dashboard - Legacy infrastructure (Operational)

Status: Resolved Summary Between 12:00 UTC 23th October and 16:00 26th October UTC, some requests to update a subaccount through our public API failed with a 400 error if they included the operation_type or products fields. The cause was an internal infrastructure change that moved some accounts to a new system for subaccount settings before that system supported these two fields. We reverted the change for all affected accounts, and updates are working normally again. Impact • Affected endpoint: PUT /v2/subaccounts/{subaccount_id} • Symptom: requests that included operation_type or products returned 400 "Not supported for this workspace." No changes from those requests were saved, including other fields sent in the same request. • Not affected: creating, listing, retrieving and deleting subaccounts; updates that did not include these two fields; live agents and calls. Root cause A recent update to how we process subaccount changes did not support the operation_type and products fields, so requests that included them were rejected. Our pre-release checks did not catch the change in API behaviour. Resolution We reverted the change for all affected accounts. Subaccount updates that include operation_type and products are now accepted and saved as expected. Prevention We are strengthening our checks so that changes cannot alter documented API behaviour without being caught before release. We are also reviewing the other subaccount fields to confirm they are not affected. What you need to do If you received a 400 error on a subaccount update during this window, please retry the request. The failed request saved nothing, so a retry is safe. We apologise for the disruption. If you have questions, please contact [email protected].

Status: Resolved Summary Between 12:00 UTC 23th October and 16:00 26th October UTC, some requests to update a subaccount through our public API failed with a 400 error if they included the operation_type or products fields. The cause was an internal infrastructure change that moved some accounts to a new system for subaccount settings before that system supported these two fields. We reverted the change for all affected accounts, and updates are working normally again. Impact • Affected endpoint: PUT /v2/subaccounts/{subaccount_id} • Symptom: requests that included operation_type or products returned 400 "Not supported for this workspace." No changes from those requests were saved, including other fields sent in the same request. • Not affected: creating, listing, retrieving and deleting subaccounts; updates that did not include these two fields; live agents and calls. Root cause A recent update to how we process subaccount changes did not support the operation_type and products fields, so requests that included them were rejected. Our pre-release checks did not catch the change in API behaviour. Resolution We reverted the change for all affected accounts. Subaccount updates that include operation_type and products are now accepted and saved as expected. Prevention We are strengthening our checks so that changes cannot alter documented API behaviour without being caught before release. We are also reviewing the other subaccount fields to confirm they are not affected. What you need to do If you received a 400 error on a subaccount update during this window, please retry the request. The failed request saved nothing, so a retry is safe. We apologise for the disruption. If you have questions, please contact [email protected].

Status: Investigating We are aware of an issue preventing end users from logging in to white-label workspaces, those configured with a customer’s own domain and branding. This issue is limited to custom-domain workspaces. All other services are operating normally. Affected components Synthflow Dashboard (Degraded performance)

Status: Resolved This issue is now resolved. RCA: Impact: Users were unable to log in to white-label workspaces using custom domains. Standard workspaces were unaffected. Root cause: A change introduced during an internal migration caused an expected issue affecting only custom domain workspaces. Resolution: We have applied the fix, and introduced enhanced QA processes specific for custom domain workspaces We apologise for the disruption. Affected components Synthflow Dashboard (Operational)

Status: Resolved This issue is now resolved. RCA: Impact: Users were unable to log in to white-label workspaces using custom domains. Standard workspaces were unaffected. Root cause: A change introduced during an internal migration caused an expected issue affecting only custom domain workspaces. Resolution: We have applied the fix, and introduced enhanced QA processes specific for custom domain workspaces We apologise for the disruption. Affected components Synthflow Dashboard - Legacy infrastructure (Operational)

Status: Resolved What happened A change to our internal telephony configuration was rolled out to production and introduced an unexpected fault in call routing. What was affected • EU region: calls failed from 10:32–10:44 UTC (12:32–12:44 CEST) — 12 minutes. • US region: calls failed from 10:33–10:42 UTC (12:33–12:42 CEST) — 9 minutes. • All regions: warm transfers failed intermittently from 07:00–10:42 UTC (09:00–12:42 CEST) — 3 hours 42 minutes. All services were fully restored by 10:44 UTC. No customer data was affected. What we're doing next We have rolled back the change and added a pre-deployment check to catch this class of routing fault before it reaches production. A full root cause analysis will be published here shortly. We're sorry for the disruption.

Status: Resolved What happened A change to our internal telephony configuration was rolled out to production and introduced an unexpected fault in call routing. What was affected • EU region: calls failed from 10:32–10:44 UTC (12:32–12:44 CEST) — 12 minutes. • US region: calls failed from 10:33–10:42 UTC (12:33–12:42 CEST) — 9 minutes. • All regions: warm transfers failed intermittently from 07:00–10:42 UTC (09:00–12:42 CEST) — 3 hours 42 minutes. All services were fully restored by 10:44 UTC. No customer data was affected. What we're doing next We have rolled back the change and added a pre-deployment check to catch this class of routing fault before it reaches production. A full root cause analysis will be published here shortly. We're sorry for the disruption.