Treefort Outage History
Treefort is up right nowPingoru has recorded 15 incidents for Treefort: 15 in the last 12 months and 6 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 4 | 21 of 720 (3%) |
| May 2026 | 1 | 1 of 744 (0%) |
| June 2026 | 2 | 4 of 720 (1%) |
| July 2026 | 0 | 0 of 744 (0%) |
| August 2026 | 1 | 13 of 744 (2%) |
| September 2026 | 2 | 2 of 720 (0%) |
| October 2026 | 3 | 11 of 545 (2%) |
Incidents, newest first
Minor October 16, 2026 Web apps briefly unavailable 8h 37m
Started Oct 16, 2026, 07:19 PM UTC· resolved Oct 17, 2026, 03:56 AM UTC
- identified Oct 16, 2026, 07:19 PM UTC
Our upstream hosting provider is restarting host machines across the fleet over the next several hours. This may result in brief periods of downtime for web apps as traffic is routed to other machines.
- identified Oct 17, 2026, 12:42 AM UTC
Our upstream hosting provider is continuing to apply a security patch for a critical vulnerability to their host machines, which requires restarting the machine. We are continuing to see brief outages among apps and the admin dashboard as infrastructure is restarted, but downtime affecting users has remained very brief thanks to cross-region redundancy. We expect full stability to return within a few hours as our hosting provider finishes applying updates.
- resolved Oct 17, 2026, 03:56 AM UTC
Our hosting provider reports that all machines have been restarted, and we expect no further disruptions at this time.
Minor October 15, 2026 eBook processing errors 1h 46m
Started Oct 15, 2026, 03:48 PM UTC· resolved Oct 15, 2026, 05:34 PM UTC
- investigating Oct 15, 2026, 03:48 PM UTC
We're seeing some failures in eBook processing jobs. We're currently investigating the problem.
- resolved Oct 15, 2026, 05:34 PM UTC
We've identified the problem and put a mitigation in place while we continue to work on a permanent fix. All eBooks are now processing normally.
Minor October 13, 2026 Partial auth outage 7m
Started Oct 13, 2026, 10:37 PM UTC· resolved Oct 13, 2026, 10:44 PM UTC
- monitoring Oct 13, 2026, 10:37 PM UTC
- monitoring Oct 13, 2026, 10:42 PM UTC
- resolved Oct 13, 2026, 10:44 PM UTC
Minor September 28, 2026 Admin dashboard pages intermittently not loading 1h 10m
Started Sep 28, 2026, 05:15 PM UTC· resolved Sep 28, 2026, 06:26 PM UTC
- investigating Sep 28, 2026, 05:15 PM UTC
Some pages on the admin dashboard are intermittently not loading. Refreshing the page sometimes works to load the affected page. We're investigating the problem.
- monitoring Sep 28, 2026, 05:42 PM UTC
We've identified a faulty API machine instance which was causing some network requests to hang indefinitely. We've replaced the instance and are continuing to monitor performance.
- resolved Sep 28, 2026, 06:26 PM UTC
Minor September 1, 2026 Authentication system errors 1h 9m
Started Sep 01, 2026, 12:02 PM UTC· resolved Sep 01, 2026, 01:12 PM UTC
- investigating Sep 01, 2026, 12:02 PM UTC
Our authentication system is intermittently returning errors when users try to sign in. We're investigating the problem.
- monitoring Sep 01, 2026, 12:28 PM UTC
One of the authentication app servers encountered a fault. We've replaced the virtual machine with a new one and are seeing errors subside.
- resolved Sep 01, 2026, 01:12 PM UTC
Minor August 25, 2026 File upload outage 13h
Started Aug 25, 2026, 03:00 AM UTC· resolved Aug 25, 2026, 04:00 PM UTC
- resolved Aug 25, 2026, 03:00 AM UTC
File uploads were broken in the admin dashboard.
Minor June 17, 2026 Some tags/collections not loading 30m
Started Jun 17, 2026, 09:39 PM UTC· resolved Jun 17, 2026, 10:10 PM UTC
- investigating Jun 17, 2026, 09:39 PM UTC
The Tags tab of the admin dashboard is currently not display tags, and some collections are not being populated in apps. We're actively investigating the issue.
- identified Jun 17, 2026, 09:57 PM UTC
We're currently testing and deploying a fix for this issue.
- resolved Jun 17, 2026, 10:10 PM UTC
The fix is live and full app and admin dashboard functionality is restored.
Minor June 8, 2026 Treefort Analytics down 3h 32m
Started Jun 08, 2026, 03:00 PM UTC· resolved Jun 08, 2026, 06:32 PM UTC
- investigating Jun 08, 2026, 03:00 PM UTC
We're seeing heightened error rates in our API and some requests are intermittently failing. Dashboard analytics are affected and some content requests are loading slowly or not at all. We're currently investigating.
- investigating Jun 08, 2026, 03:47 PM UTC
We've identified an issue with our analytics system. We are isolating the issue so that other aspects of the platform can fully recover while we work on a fix for analytics.
- identified Jun 08, 2026, 04:16 PM UTC
We are working on a fix for the analytics system. In the meantime analytics are still unavailable via the admin dashboard or API. We have finished processing webhooks that we backed up due to this issue and general access to the admin dashboard has been restored.
- resolved Jun 08, 2026, 06:32 PM UTC
All systems have recovered. We will be sending out a post-mortem to customers shortly.
Minor May 28, 2026 Network connectivity issues 1h
Started May 28, 2026, 09:33 PM UTC· resolved May 28, 2026, 10:33 PM UTC
- investigating May 28, 2026, 09:35 PM UTC
We are investigating network connectivity issues with an upstream provider. The Treefort admin dashboard is currently unavailable for some users.
- investigating May 28, 2026, 09:45 PM UTC
The connectivity issues are affecting apps as well. Users on the west coast may experience slow load times. The upstream provider is investigating the issue
- monitoring May 28, 2026, 10:14 PM UTC
A fix is being implemented. In the meantime we are continuing to see sporadically slow load times.
- resolved May 28, 2026, 10:40 PM UTC
The fix is complete and network connectivity has been fully restored.
Minor April 20, 2026 Bad deployment 47m
Started Apr 20, 2026, 04:55 PM UTC· resolved Apr 20, 2026, 05:42 PM UTC
- identified Apr 20, 2026, 04:55 PM UTC
We've identified a bad deployment that is causing articles to fail to load in apps and video series to fail to load in the admin dashboard.
- resolved Apr 20, 2026, 05:42 PM UTC
Both issues have been resolved. App downtime for articles was ~33 minutes. Admin dashboard downtime for video series was ~43 minutes.
Major April 19, 2026 Major Outage detected on status page 18h 37m Overall
Started Apr 19, 2026, 08:47 PM UTC· resolved Apr 20, 2026, 03:24 PM UTC
No vendor updates were posted for this incident.
Minor April 17, 2026 Partial outage 45m
Started Apr 17, 2026, 01:48 PM UTC· resolved Apr 17, 2026, 02:34 PM UTC
- monitoring Apr 17, 2026, 01:48 PM UTC
We are monitoring lingering issues with several apps related to the previous outage.
- resolved May 12, 2026, 11:19 PM UTC
Minor April 17, 2026 Outage 18m
Started Apr 17, 2026, 12:51 PM UTC· resolved Apr 17, 2026, 01:10 PM UTC
- investigating Apr 17, 2026, 12:57 PM UTC
We are investigating a major platform outage.
- identified Apr 17, 2026, 01:08 PM UTC
Our upstream hosting provider is addressing an issue related to handling secure network traffic. We are seeing most services start to recover.
- resolved Apr 17, 2026, 01:15 PM UTC
All services have recovered.
Minor March 27, 2026 Authentication intermittently unavailable 1h
Started Mar 27, 2026, 10:04 PM UTC· resolved Mar 27, 2026, 11:04 PM UTC
- investigating Mar 27, 2026, 09:09 PM UTC
We're seeing a spike in failed requests to our API and are actively investigating. Some thumbnails and content may not load, and user authentication may be slow or encounter errors.
- monitoring Mar 27, 2026, 09:19 PM UTC
An upstream provider is experiencing capacity issues resulting in intermittent network failures. Services are recovering at this time and error rates have dropped significantly. We are continuing to monitor apps to ensure full recovery.
- identified Mar 27, 2026, 11:08 PM UTC
Our upstream provider is having hardware issues with the machines our authentication service runs on. We are working to provision additional machines to restore full service. Some actions that require authentication may intermittently fail in the meantime.
- monitoring Mar 27, 2026, 11:25 PM UTC
We've performed a database failover to a new primary node and are seeing services recover. We're continuing to monitor.
- resolved Mar 28, 2026, 04:48 AM UTC
Minor March 27, 2026 Admin dashboard and user authentication problems 49m
Started Mar 27, 2026, 05:02 PM UTC· resolved Mar 27, 2026, 05:52 PM UTC
- investigating Mar 27, 2026, 05:02 PM UTC
We're investigating problems with our authentication service and admin dashboard.
- identified Mar 27, 2026, 05:10 PM UTC
We've identified unusually high load on one of our authentication database servers. We've restarted the database cluster to restore baseline service but response times may continue to be slow as the database catches up.
- monitoring Mar 27, 2026, 05:23 PM UTC
- resolved Mar 27, 2026, 05:53 PM UTC