Phrase incident
Degraded Performance of Translation Memory Component in Phrase TMS (EU) between August 19, 2016 2:11 PM CEST and August 19, 2016 4:10 PM CEST
Phrase experienced a major incident on August 19, 2026 affecting Translation memory, lasting 1h 52m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 19, 2026, 12:16 PM UTC
We are currently investigating an issue affecting Translation Memory functionality in Phrase TMS (EU). Users may experience problems with creating Translation Memories and other related operations during this time. Our team is actively working to identify the cause and restore full functionality as quickly as possible.
- identified Aug 19, 2026, 12:34 PM UTC
The issue has been identified, and a fix is being implemented.
- monitoring Aug 19, 2026, 01:19 PM UTC
A fix has been implemented and we are monitoring the situation.
- resolved Aug 19, 2026, 02:09 PM UTC
The incident has been resolved.
- postmortem Oct 06, 2026, 08:14 AM UTC
## **Introduction** We would like to share more details about the events that occurred with Phrase between 2:11 PM CEST and 4:10 PM CEST on August 19, 2026, which led to degraded Translation Memory functionality in Phrase TMS \(EU\) and what Phrase engineers are doing to prevent these issues from recurring. ## **Timeline** * **August 19, 2026 at 2:11 PM CEST:** Phrase identified an issue affecting Translation Memory functionality in Phrase TMS \(EU\). Users experienced errors when creating translation memories and performing related operations, including confirming segments, elevating error rates across dependant services and use-cases. * **August 19, 2026 at 2:32 PM CEST:** Root cause was identified and engineers began applying a fix to the affected environment by rebalancing the cluster and restarting affected nodes to speed up recovery. * **August 19, 2026 at 3:10 PM CEST:** Translation memory creation and related operations were confirmed to be functioning normally again. * **August 19, 2026 at 4:10 PM CEST:** After a sustained monitoring period showed no recurrence, the incident was marked resolved. ## **Root Cause** Investigation revealed that the incident originated in the database cluster that stores Translation Memory data. A single Translation Memory had, over time, grown very large and required a disproportionate share of read activity to be served by the node that also coordinated the health and metadata management of the wider cluster. Because the cluster was undergoing zero-downtime maintenance spanning days, there was a higher amount of coordination actions and an elevated load on coordination nodes in the cluster. Those two combined with peak load caused multiple coordination nodes performance to slowly deteriorate, leading to elevated error rate and degraded performance. ## **Actions to Prevent Recurrence** To prevent similar incidents in the future and mitigate any negative effects, we: * Rebalanced the cluster and its backing infrastructure to create more headroom for Translation Memory searches and updates. * Paused the ongoing maintenance on impacted Translation Memory cluster. * Improved capacity of coordination node and will be improving cluster’s topology to prevent cluster impact during sudden requests peaks.