Voyado Outage History
Voyado is up right nowPingoru has recorded 106 incidents for Voyado: 86 in the last 12 months and 33 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 8 | 280 of 720 (39%) |
| May 2026 | 10 | 603 of 744 (81%) |
| June 2026 | 11 | 591 of 720 (82%) |
| July 2026 | 11 | 684 of 744 (92%) |
| August 2026 | 11 | 601 of 744 (81%) |
| September 2026 | 12 | 628 of 720 (87%) |
| October 2026 | 10 | 490 of 593 (83%) |
Incidents, newest first
Minor June 4, 2025 Engage: performance issues affecting tracking pipeline (abandoned cart/products of interests) and reports 23h 48m Automations, Other
Started Jun 04, 2025, 02:19 PM UTC· resolved Jun 05, 2025, 02:07 PM UTC
- investigating Jun 04, 2025, 02:19 PM UTC
We are currently investigating the issue which seems to be related to scaling and could lead to delays in abandoned cart/browse automations to be triggered. Also affecting loading of dashboard/reports in the UI.
- investigating Jun 04, 2025, 03:55 PM UTC
The issue is still under active investigation. We appreciate your patience while we work to resolve it.
- investigating Jun 04, 2025, 06:10 PM UTC
The issue remains under investigation. We will provide updates as they become available.
- investigating Jun 04, 2025, 08:16 PM UTC
We are actively investigating the issue but have not yet identified the root cause.
- investigating Jun 05, 2025, 05:16 AM UTC
The root cause is still unknown. We’re continuing our investigation and will share updates as soon as we have more information. Apologies for the inconvenience.
- investigating Jun 05, 2025, 07:31 AM UTC
We are actively working on the issue. We do not yet have a complete understanding of the root cause. Some abandoned cart requests are currently being processed successfully. We appreciate your patience and will continue to provide updates.
- investigating Jun 05, 2025, 08:08 AM UTC
We are continuing to explore alternative solutions, though the root cause has not yet been fully identified. We are also engaging with our cloud provider to assist in the ongoing investigation. Further updates will be provided as new information becomes available. Our monitoring shows that Abandoned Browse is functioning as expected, whereas Products of Interest is currently impacted.
- identified Jun 05, 2025, 11:22 AM UTC
We have identified a solution and we are now working to deploy this change to production as soon as possible.
- monitoring Jun 05, 2025, 12:28 PM UTC
The fix has now been implemented and the system is operating normally. We are continuing monitoring the situation closely.
- monitoring Jun 05, 2025, 12:35 PM UTC
We are continuing to monitor for any further issues.
- resolved Jun 05, 2025, 02:07 PM UTC
After deploying the fix and monitoring the system, we can confirm that the issue is resolved.
- postmortem Jun 10, 2025, 11:43 AM UTC
## Summary Between June 4 at 14:39 and June 5 at 14.10, one of our backend services experienced performance issues leading to delays and partial unavailability. The issue has been resolved, and systems are now operating normally. ## Customer Impact During the incident, features such as ROD, Abandoned Cart, Register Status, Automation Overview and Product of interest were affected. Customers may have experienced slow response times or temporary unavailability when using these functionalities. No data loss occurred, and all services were restored by June 5 at 14.10. ## Root Cause and Mitigation The issue stemmed from an unexpected delay in the cleanup of background processes connected to secure storage systems, following a minor system update. While the connections themselves were successful, the delayed termination caused performance degradation. We mitigated the issue by safely bypassing the affected cleanup process and restored service stability. A support case was also raised with Microsoft Azure to investigate further. ## Next Steps * Improve automated testing to detect performance issues before deployment. * Review and update the logic used for secure storage connections in production environments. * Monitor Azure infrastructure updates for changes that may affect our services. * Continue to enhance logging and observability for faster diagnostics.