Central 1 incident
Resolved – INC229690 – Application login using 2SS (2-Step-Security) unavailable
Resolved on October 14, 2026, affecting Payment Services.
**Postmortem: INC229690 – 2SS Applications Unavailable \(UM, Wires, AFT, EFM\)** On October 14, 2026, users experienced authentication and access failures across applications that rely on 2-Step Security \(2SS\), including User Management \(UM\), Wires, AFT, and EFM. Multiple clients reported being unable to access affected applications, including sign-in failures and HTTP 500 errors. The issue was resolved following a restart of the 2SS service, restoring access to the affected applications. **Point of Failure:** The outage was caused by an unintended application dependency on libraries contained within a backup folder located in the active 2SS webapps directory. Earlier in October, vulnerability remediation updates had been deployed to the 2SS environment, with a backup copy of the application retained in this directory as a rollback option. The backup was believed to be inactive and not part of the production application path. Following the deployment, a hard-token provisioning issue was investigated under a separate incident, INC229174. During troubleshooting, an older library within the backup folder was identified as potentially interfering with the newly deployed code. Because the folder was understood to be used solely for rollback purposes, its contents were removed to eliminate the possibility that the older libraries were contributing to the provisioning issue. However, the running application was actively referencing libraries contained within this location. Removal of those files therefore caused the 2SS service to fail, resulting in authentication and access failures across dependent applications. The vulnerability remediation deployment itself did not directly cause the outage. Rather, the incident exposed an unintended dependency within the application environment: files retained as a backup were being referenced by the running production application. The risk associated with removing the backup contents was therefore not known prior to the troubleshooting activity. During the outage, customers were unable to access applications dependent on 2SS, including User Management, Wires, AFT, and EFM. Reported symptoms included general sign-in failures, inability to access User Management and Wires, and HTTP 500 errors when accessing EFM. The 2SS service was restarted at 2:10 p.m. PT and service was restored at 2:11 p.m. PT. Affected customers subsequently confirmed successful access to their applications. **Corrective Actions:** Follow-up actions are focused on addressing the underlying application dependency and strengthening deployment and troubleshooting practices within the 2SS environment. * **Separate backup artifacts from active application paths:** Review the storage of application backups and rollback artifacts to ensure they are isolated from active production application directories and cannot inadvertently be referenced at runtime. * **Validate application dependencies:** Review the 2SS application configuration and dependencies to identify and eliminate unintended references to libraries or components outside the intended production application structure. * **Strengthen deployment and rollback procedures:** Update deployment and rollback practices to clearly distinguish active application components from backup copies and include validation following changes to application files. * **Improve hard-token provisioning validation:** Review opportunities to improve validation of hard-token provisioning following application changes, reducing reliance on post-deployment external validation. We sincerely apologize for the disruption caused by this incident and are taking these steps to strengthen our deployment and application management practices and reduce the likelihood of recurrence. **Enquiries:** Jason Seale | Director, Incident & Service Management | [[email protected]](mailto:[email protected]) | 778.558.5627
Affected components
Update timeline
- resolved Oct 14, 2026, 11:39 PM UTC
Central 1 has continued to monitor User Management (UM), EFM, AFT and Wires following the earlier sign-in issue. Services have remained stable and no further issues have been observed. This incident is now considered resolved. If you continue to experience sign-in issues, please contact Central 1 Support. Central 1 - [email protected] - 1.888.889.7878, press 1
- postmortem Oct 23, 2026, 10:02 PM UTC
**Postmortem: INC229690 – 2SS Applications Unavailable \(UM, Wires, AFT, EFM\)** On October 14, 2026, users experienced authentication and access failures across applications that rely on 2-Step Security \(2SS\), including User Management \(UM\), Wires, AFT, and EFM. Multiple clients reported being unable to access affected applications, including sign-in failures and HTTP 500 errors. The issue was resolved following a restart of the 2SS service, restoring access to the affected applications. **Point of Failure:** The outage was caused by an unintended application dependency on libraries contained within a backup folder located in the active 2SS webapps directory. Earlier in October, vulnerability remediation updates had been deployed to the 2SS environment, with a backup copy of the application retained in this directory as a rollback option. The backup was believed to be inactive and not part of the production application path. Following the deployment, a hard-token provisioning issue was investigated under a separate incident, INC229174. During troubleshooting, an older library within the backup folder was identified as potentially interfering with the newly deployed code. Because the folder was understood to be used solely for rollback purposes, its contents were removed to eliminate the possibility that the older libraries were contributing to the provisioning issue. However, the running application was actively referencing libraries contained within this location. Removal of those files therefore caused the 2SS service to fail, resulting in authentication and access failures across dependent applications. The vulnerability remediation deployment itself did not directly cause the outage. Rather, the incident exposed an unintended dependency within the application environment: files retained as a backup were being referenced by the running production application. The risk associated with removing the backup contents was therefore not known prior to the troubleshooting activity. During the outage, customers were unable to access applications dependent on 2SS, including User Management, Wires, AFT, and EFM. Reported symptoms included general sign-in failures, inability to access User Management and Wires, and HTTP 500 errors when accessing EFM. The 2SS service was restarted at 2:10 p.m. PT and service was restored at 2:11 p.m. PT. Affected customers subsequently confirmed successful access to their applications. **Corrective Actions:** Follow-up actions are focused on addressing the underlying application dependency and strengthening deployment and troubleshooting practices within the 2SS environment. * **Separate backup artifacts from active application paths:** Review the storage of application backups and rollback artifacts to ensure they are isolated from active production application directories and cannot inadvertently be referenced at runtime. * **Validate application dependencies:** Review the 2SS application configuration and dependencies to identify and eliminate unintended references to libraries or components outside the intended production application structure. * **Strengthen deployment and rollback procedures:** Update deployment and rollback practices to clearly distinguish active application components from backup copies and include validation following changes to application files. * **Improve hard-token provisioning validation:** Review opportunities to improve validation of hard-token provisioning following application changes, reducing reliance on post-deployment external validation. We sincerely apologize for the disruption caused by this incident and are taking these steps to strengthen our deployment and application management practices and reduce the likelihood of recurrence. **Enquiries:** Jason Seale | Director, Incident & Service Management | [[email protected]](mailto:[email protected]) | 778.558.5627
Your product runs on more than Central 1
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.