Is Treefort down?

Our upstream hosting provider is restarting host machines across the fleet over the next several hours. This may result in brief periods of downtime for web apps as traffic is routed to other machines.

Our upstream hosting provider is continuing to apply a security patch for a critical vulnerability to their host machines, which requires restarting the machine. We are continuing to see brief outages among apps and the admin dashboard as infrastructure is restarted, but downtime affecting users has remained very brief thanks to cross-region redundancy. We expect full stability to return within a few hours as our hosting provider finishes applying updates.

Our hosting provider reports that all machines have been restarted, and we expect no further disruptions at this time.

We're seeing some failures in eBook processing jobs. We're currently investigating the problem.

We've identified the problem and put a mitigation in place while we continue to work on a permanent fix. All eBooks are now processing normally.

Some pages on the admin dashboard are intermittently not loading. Refreshing the page sometimes works to load the affected page. We're investigating the problem.

We've identified a faulty API machine instance which was causing some network requests to hang indefinitely. We've replaced the instance and are continuing to monitor performance.