TRMNL incident

API non-responsiveness

Resolved after 1 hour 1 minute, affecting Display API, on January 28, 2026.

resolved Jan 28, 2026, 07:21 PM UTC

An infrastructure change to our underlying cache system was made around 9 AM Eastern time, which caused one of our cache entries to start growing without bounds. Eventually this entry got so big it ate up all the CPU time just serializing this value and writing it back out to the cache. The cache for this value was disabled and performance has returned to normal.

Started
Jan 28, 2026, 06:19 PM UTC
Resolved
Jan 28, 2026, 07:21 PM UTC
Duration
1h 1m
Detected by Pingoru
Jan 28, 2026, 06:19 PM UTC

Affected components

Display API

Update timeline

  1. investigating Jan 28, 2026, 06:19 PM UTC

    Today at 12:46pm ET we deployed a domain switchover and our services are catching up.

  2. identified Jan 28, 2026, 06:48 PM UTC

    CPU usage on our API server began spiking at 9a ET for approximately 4 hours and our DNS changes may have exacerbated the latency. We've disabled some logging and are seeing a return to normal levels.

  3. resolved Jan 28, 2026, 07:21 PM UTC

    An infrastructure change to our underlying cache system was made around 9 AM Eastern time, which caused one of our cache entries to start growing without bounds. Eventually this entry got so big it ate up all the CPU time just serializing this value and writing it back out to the cache. The cache for this value was disabled and performance has returned to normal.

Your product runs on more than TRMNL

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →