Degraded performance: login failures and slow loading of emails and account data

Update timeline

1. investigating Jul 31, 2026, 08:30 AM UTC

   We are investigating reports of users being unable to log in, seeing missing team members, or experiencing slow loading of emails. Our team is working to identify the cause.

2. identified Jul 31, 2026, 08:30 AM UTC

   We have identified the root cause: an internal caching service (Redis) ran out of memory and began rejecting writes. As a result, permission checks bypassed the cache and put excessive load on the account service, causing slow responses, login failures, and missing account data. We have increased the cache capacity and are preparing a permanent fix.

3. monitoring Jul 31, 2026, 08:31 AM UTC

   Cache capacity has been increased and the system is recovering. Response times are returning to normal as caches warm up. A permanent fix is being deployed.

4. resolved Jul 31, 2026, 08:49 AM UTC

   The permanent fix has been deployed to all production clusters and all systems are operating normally. During the incident (July 30, 22:48 UTC – July 31, ~04:30 UTC, with residual slowness until ~08:49 UTC), some users experienced login failures, missing team member data, and slow loading of emails. No data was lost. We have added memory-usage alerting to detect this class of issue before it affects users. We apologize for the inconvenience.