Depot Outage History
Depot is up right nowPingoru has recorded 77 incidents for Depot: 77 in the last 12 months and 9 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 12 | 12 of 720 (2%) |
| May 2026 | 10 | 6 of 744 (1%) |
| June 2026 | 6 | 5 of 720 (1%) |
| July 2026 | 5 | 6 of 744 (1%) |
| August 2026 | 2 | 1 of 744 (0%) |
| September 2026 | 3 | 2 of 720 (0%) |
| October 2026 | 3 | 5 of 574 (1%) |
Incidents, newest first
Minor October 8, 2026 Issue starting CI jobs 1h 54m us-east-1, Depot CI, us-east-1
Started Oct 08, 2026, 07:13 AM UTC· resolved Oct 08, 2026, 09:08 AM UTC
- investigating Oct 08, 2026, 07:13 AM UTC
We're investigating an issue preventing some CI jobs from starting
- identified Oct 08, 2026, 07:57 AM UTC
We have identified the issue and are deploying a fix.
- monitoring Oct 08, 2026, 08:53 AM UTC
The underlying issue preventing jobs from starting has been fixed, we are monitoring performance as deployment proceeds.
- resolved Oct 08, 2026, 09:08 AM UTC
This incident has been resolved.
Minor October 6, 2026 Slow disk I/O for US container builds 1h 40m us-east-1, us-east-1
Started Oct 06, 2026, 08:14 PM UTC· resolved Oct 06, 2026, 09:54 PM UTC
- monitoring Oct 06, 2026, 08:14 PM UTC
Additional capacity is online and the cluster rebalance is underway. We expect build performance to have returned to normal, but will keep monitoring until the cluster has completed rebalancing.
- resolved Oct 06, 2026, 09:54 PM UTC
This incident has been resolved.
Minor October 5, 2026 Slow disk I/O for EU container builds 1h 48m eu-central-1
Started Oct 05, 2026, 11:34 AM UTC· resolved Oct 05, 2026, 01:23 PM UTC
- identified Oct 05, 2026, 11:34 AM UTC
We have identified a slowdown in our EU storage cluster for container builds and have deployed additional capacity. Please update your project region to the US if you need to unblock immediately, otherwise the additional capacity will be online momentarily.
- monitoring Oct 05, 2026, 11:58 AM UTC
Additional capacity is online and the cluster rebalance is underway.
- monitoring Oct 05, 2026, 12:49 PM UTC
We expect build performance to have returned to normal, but will keep monitoring until the cluster has completed rebalancing.
- resolved Oct 05, 2026, 01:23 PM UTC
This incident has been resolved.
Minor September 28, 2026 Elevated queue times for Sandboxes and Depot CI Jobs 19m Depot CI
Started Sep 28, 2026, 06:52 PM UTC· resolved Sep 28, 2026, 07:11 PM UTC
- investigating Sep 28, 2026, 06:52 PM UTC
We're currently investigating this incident.
- identified Sep 28, 2026, 06:56 PM UTC
We've identified a slow query and are deploying a mitigation.
- monitoring Sep 28, 2026, 07:02 PM UTC
Query performance is back to baseline after rolling out a fix. We'll continue to monitor.
- resolved Sep 28, 2026, 07:11 PM UTC
This incident has been resolved.
Minor September 20, 2026 Some CI jobs failing to start 1h 27m us-east-1, Depot CI, us-east-1
Started Sep 20, 2026, 06:09 PM UTC· resolved Sep 20, 2026, 07:37 PM UTC
- investigating Sep 20, 2026, 06:09 PM UTC
We are investigating reports of some CI jobs failing to start.
- resolved Sep 20, 2026, 06:14 PM UTC
This incident has been resolved.
- investigating Sep 20, 2026, 06:29 PM UTC
We are seeing some lingering failures and are investigating.
- monitoring Sep 20, 2026, 07:25 PM UTC
Service appears to be restored, we are continuing to monitor.
- resolved Sep 20, 2026, 07:37 PM UTC
This incident has been resolved.
Minor September 4, 2026 Sandboxes are having problems dispatching 6m Depot CI
Started Sep 04, 2026, 01:01 PM UTC· resolved Sep 04, 2026, 01:07 PM UTC
- investigating Sep 04, 2026, 01:01 PM UTC
Some organizations are having their sandboxes delayed or unable to dispatch.
- resolved Sep 04, 2026, 01:07 PM UTC
This incident has been resolved.
Minor August 28, 2026 We're seeing an increased amount of errors loading pages in the Dashboard. 22m Dashboard
Started Aug 28, 2026, 08:59 PM UTC· resolved Aug 28, 2026, 09:22 PM UTC
- investigating Aug 28, 2026, 08:59 PM UTC
We're investigating an increased volume of errors loading pages in the Dashboard.
- resolved Aug 28, 2026, 09:22 PM UTC
This incident has been resolved.
Minor August 28, 2026 Builds are experiencing a slowdown. 30m us-east-1, us-east-1
Started Aug 28, 2026, 08:00 PM UTC· resolved Aug 28, 2026, 08:31 PM UTC
- investigating Aug 28, 2026, 08:00 PM UTC
We're seeing a slow down in builds in use-east-1 and are investigating.
- monitoring Aug 28, 2026, 08:24 PM UTC
We are seeing improvements and are monitoring.
- resolved Aug 28, 2026, 08:31 PM UTC
This incident has been resolved.
Minor July 28, 2026 Increased queue time for GHA jobs 45m us-east-1, us-east-1
Started Jul 28, 2026, 01:51 AM UTC· resolved Jul 28, 2026, 02:36 AM UTC
- investigating Jul 28, 2026, 01:51 AM UTC
Increased queue time for GHA jobs
- resolved Jul 28, 2026, 02:36 AM UTC
Queue times have returned to normal
Minor July 16, 2026 Dashboard is down 9m Dashboard
Started Jul 16, 2026, 10:00 AM UTC· resolved Jul 16, 2026, 10:09 AM UTC
- investigating Jul 16, 2026, 10:00 AM UTC
Due to an upstreams provider's CDN outage, Depot dashboard is down. We are working with them on a mitigation
- investigating Jul 16, 2026, 10:05 AM UTC
Dashboard is recovering
- monitoring Jul 16, 2026, 10:08 AM UTC
Dashboard has recovered, we are monitoring
- resolved Jul 16, 2026, 10:09 AM UTC
This incident has been resolved.
Minor July 9, 2026 Elevated queue times for Github Actions jobs 1h 42m us-east-1, us-east-1
Started Jul 09, 2026, 05:44 PM UTC· resolved Jul 09, 2026, 07:27 PM UTC
- identified Jul 09, 2026, 05:44 PM UTC
We're currently seeing elevated queue times for GitHub Actions jobs due to upstream capacity constraints. We'll continue to monitor the situation.
- monitoring Jul 09, 2026, 07:09 PM UTC
Capacity appears to be less constrained upstream, and jobs are currently starting at a normal rate. We will continue to monitor the situation.
- resolved Jul 09, 2026, 07:27 PM UTC
This incident has been resolved.
Minor July 8, 2026 Delays acquiring builder machines 51m us-east-1, us-east-1
Started Jul 08, 2026, 09:40 PM UTC· resolved Jul 08, 2026, 10:31 PM UTC
- identified Jul 08, 2026, 09:40 PM UTC
There are delays in acquiring machines to run docker builds due to upstream capacity constraints
- monitoring Jul 08, 2026, 10:13 PM UTC
Upstream capacity constraints have improved, and we're able to acquire builder machines once again. We will continue to monitor the situation.
- resolved Jul 08, 2026, 10:31 PM UTC
This incident has been resolved.
Minor July 7, 2026 Unable to start container builds 2h us-east-1, us-east-1, eu-central-1
Started Jul 07, 2026, 05:20 PM UTC· resolved Jul 07, 2026, 07:20 PM UTC
- investigating Jul 07, 2026, 05:20 PM UTC
We're currently investigating why we're unable to start container builds.
- monitoring Jul 07, 2026, 05:49 PM UTC
We were intermittently unable to acquire builder machines due to upstream AWS capacity constraints. We are no longer seeing the same capacity constraints, and builds are processing normally. We will continue to monitor the situation.
- resolved Jul 07, 2026, 06:05 PM UTC
This incident has been resolved.
- identified Jul 07, 2026, 06:14 PM UTC
Upstream capacity appears to be constrained once again. We are actively working on mitigations.
- monitoring Jul 07, 2026, 07:09 PM UTC
Mitigations are deployed, and capacity is recovering. We are monitoring
- resolved Jul 07, 2026, 07:20 PM UTC
This incident has been resolved.
Minor June 29, 2026 Increased queue time for GitHub Actions jobs 27m us-east-1, us-east-1
Started Jun 29, 2026, 04:12 PM UTC· resolved Jun 29, 2026, 04:40 PM UTC
- investigating Jun 29, 2026, 04:12 PM UTC
We are investigating delays acquiring EC2 instances for GitHub Actions jobs.
- monitoring Jun 29, 2026, 04:22 PM UTC
Capacity has recovered, we are monitoring.
- resolved Jun 29, 2026, 04:40 PM UTC
This incident has been resolved.
Minor June 25, 2026 Unable to start some GHA Jobs 29m us-east-1, us-east-1
Started Jun 25, 2026, 08:09 PM UTC· resolved Jun 25, 2026, 08:39 PM UTC
- investigating Jun 25, 2026, 08:09 PM UTC
Investigating increased levels of GHA jobs not starting.
- investigating Jun 25, 2026, 08:27 PM UTC
Jobs are starting but may still be experiencing elevated queue times.
- resolved Jun 25, 2026, 08:39 PM UTC
This incident has been resolved.
Minor June 23, 2026 GHA jobs are having increased queue times 1h 3m us-east-1, us-east-1
Started Jun 23, 2026, 06:50 PM UTC· resolved Jun 23, 2026, 07:53 PM UTC
- investigating Jun 23, 2026, 06:50 PM UTC
Investigating increased queue times for Depot GHA ubuntu jobs.
- resolved Jun 23, 2026, 07:53 PM UTC
This incident has been resolved.
Minor June 17, 2026 Increased queue times for GitHub Actions jobs 9m us-east-1, us-east-1
Started Jun 17, 2026, 02:13 PM UTC· resolved Jun 17, 2026, 02:23 PM UTC
- investigating Jun 17, 2026, 02:13 PM UTC
We are investigating increased queue times for GitHub Actions jobs
- resolved Jun 17, 2026, 02:23 PM UTC
This incident has been resolved, all jobs are processing normally.
Minor June 9, 2026 GitHub Actions runner disconnections 42m GitHub.com
Started Jun 09, 2026, 06:31 PM UTC· resolved Jun 09, 2026, 07:13 PM UTC
- investigating Jun 09, 2026, 06:31 PM UTC
We are receiving reports of GitHub Actions runners disconnecting from GitHub's control plane. We are investigating.
- monitoring Jun 09, 2026, 06:46 PM UTC
We have deployed a mitigation for the disconnect and are monitoring
- resolved Jun 09, 2026, 07:13 PM UTC
This incident is resolved
Minor May 29, 2026 Depot CI jobs not syncing checks to GitHub 19m Depot CI
Started May 29, 2026, 12:07 AM UTC· resolved May 29, 2026, 12:26 AM UTC
- identified May 29, 2026, 12:07 AM UTC
Depot CI jobs not syncing checks to GitHub.
- monitoring May 29, 2026, 12:21 AM UTC
We've applied a fix and are monitoring.
- resolved May 29, 2026, 12:26 AM UTC
This incident has been resolved.
Minor May 28, 2026 GitHub jobs using Depot managed are unable to start 1h 23m us-east-1, us-east-1
Started May 28, 2026, 11:04 PM UTC· resolved May 29, 2026, 12:28 AM UTC
- investigating May 28, 2026, 11:04 PM UTC
GitHub jobs using Depot managed are unable to start.
- monitoring May 29, 2026, 12:21 AM UTC
We've applied a fix and are monitoring
- resolved May 29, 2026, 12:28 AM UTC
This incident has been resolved.
Minor May 28, 2026 Depot CI has higher queue times 15m Depot CI
Started May 28, 2026, 10:32 PM UTC· resolved May 28, 2026, 10:48 PM UTC
- investigating May 28, 2026, 10:32 PM UTC
We are seeing high queue times and are scaling up.
- resolved May 28, 2026, 10:48 PM UTC
This incident has been resolved.
Minor May 26, 2026 Delays starting GitHub jobs 24m us-east-1
Started May 26, 2026, 02:23 PM UTC· resolved May 26, 2026, 02:47 PM UTC
- identified May 26, 2026, 02:23 PM UTC
We are deploying a fix for an issue causing delays starting GitHub Actions runners in us-east-1
- identified May 26, 2026, 02:28 PM UTC
Fix is deployed, we are processing the backlog of jobs now
- resolved May 26, 2026, 02:47 PM UTC
This incident has been resolved.
Minor May 18, 2026 slow container builds in us-east-1 for some customers 19m us-east-1
Started May 18, 2026, 09:04 PM UTC· resolved May 18, 2026, 09:23 PM UTC
- identified May 18, 2026, 09:04 PM UTC
We are seeing some customers with very slow I/O at the moment and are starting to mitigate.
- monitoring May 18, 2026, 09:10 PM UTC
We are seeing recovery now and new builds are having normal disk i/o.
- resolved May 18, 2026, 09:23 PM UTC
This incident has been resolved.
Minor May 17, 2026 Some Depot CI jobs failing to start 1h 10m Depot CI
Started May 17, 2026, 08:11 AM UTC· resolved May 17, 2026, 09:21 AM UTC
- investigating May 17, 2026, 08:11 AM UTC
We are investigating an issue preventing some Depot CI jobs from starting.
- monitoring May 17, 2026, 08:21 AM UTC
We have deployed a fix and jobs are processing. We are continuing to monitor.
- identified May 17, 2026, 08:28 AM UTC
We are still seeing some intermittent failures and are investigating.
- monitoring May 17, 2026, 09:02 AM UTC
Jobs are processing again, we are continuing to monitor.
- resolved May 17, 2026, 09:21 AM UTC
This incident has been resolved.
Minor May 14, 2026 Unable to start some Depot CI jobs 36m Depot CI
Started May 14, 2026, 09:05 PM UTC· resolved May 14, 2026, 09:41 PM UTC
- investigating May 14, 2026, 09:05 PM UTC
Investigating the failure of some Depot CI jobs being able to start.
- monitoring May 14, 2026, 09:36 PM UTC
We identified the issue as upstream rate limits. We are monitoring to make sure the issue has cleared.
- resolved May 14, 2026, 09:41 PM UTC
This incident has been resolved.