Replicate Outage History
Replicate is up right nowPingoru has recorded 37 incidents for Replicate: 32 in the last 12 months and 7 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 2 | 1 of 720 (0%) |
| May 2026 | 4 | 0 of 744 (0%) |
| June 2026 | 2 | 0 of 720 (0%) |
| July 2026 | 5 | 0 of 744 (0%) |
| August 2026 | 4 | 0 of 744 (0%) |
| September 2026 | 4 | 39 of 720 (5%) |
| October 2026 | 0 | 0 of 744 (0%) |
| November 2026 | 0 | 0 of 19 (0%) |
Incidents, newest first
Minor September 28, 2026 Replicate Flux Hotswap Models Stuck 17m Replicate
Started Sep 28, 2026, 08:11 PM UTC· resolved Sep 28, 2026, 08:28 PM UTC
- investigating Sep 28, 2026, 08:11 PM UTC
We are investigating Flux hotswap models that haven't made progress since last Thursday, 2026-09-24.
- resolved Sep 28, 2026, 08:28 PM UTC
This incident has been resolved.
Minor September 24, 2026 Intermittent 500 errors for backend services on Replicate 1h 54m Replicate
Started Sep 24, 2026, 08:40 PM UTC· resolved Sep 24, 2026, 10:34 PM UTC
- investigating Sep 24, 2026, 08:40 PM UTC
We have a backend component that entered an unexpected reboot, causing issues for services downstream.
- resolved Sep 24, 2026, 10:34 PM UTC
The impacted backend has been redeployed, and all related services should now be back up and running. There may be some increased latency as they continue to come online.
Minor September 17, 2026 Replicate Pruna issue 20h 42m Replicate
Started Sep 17, 2026, 01:09 AM UTC· resolved Sep 17, 2026, 09:51 PM UTC
- identified Sep 17, 2026, 01:09 AM UTC
Some Pruna models may not be able to scale up and are therefore either experiencing long queue times or entirely unavailable. We are working with Pruna to resolve the issue.
- resolved Sep 17, 2026, 09:51 PM UTC
This incident has been resolved.
Minor September 14, 2026 Some Replicate models are failing to start 15h 40m Replicate
Started Sep 14, 2026, 10:25 PM UTC· resolved Sep 15, 2026, 02:06 PM UTC
- investigating Sep 14, 2026, 10:25 PM UTC
We are investigating an issue which is preventing Replicate models from failing to start new replicas.
- identified Sep 15, 2026, 12:03 AM UTC
Some third party models as well as https://replicate.com/qwen/qwen-edit-multiangle are affected and are unable to scale out. Please use alternate models while we work with providers to publish fixed versions.
- resolved Sep 15, 2026, 02:06 PM UTC
This incident has been resolved.
Minor August 27, 2026 A100 hardware partial outage – Replicate
Started Aug 27, 2026, 12:25 AM UTC· resolved Aug 27, 2026, 12:25 AM UTC
- investigating Aug 26, 2026, 09:17 PM UTC
We're seeing most A100 hardware automatically cordoned due to possible hardware failure.
- monitoring Aug 26, 2026, 11:07 PM UTC
We are seeing pods start up again now and they are working through the backlog.
- resolved Aug 27, 2026, 12:25 AM UTC
We've been below capacity for A100 hardware for the past 30 minutes and all queues have been drained. Thanks for your patience 💖
Minor August 10, 2026 Delayed scaling due to node failure – Replicate
Started Aug 10, 2026, 09:45 PM UTC· resolved Aug 10, 2026, 09:45 PM UTC
- monitoring Aug 10, 2026, 09:05 PM UTC
Scaling decisions were delayed by nearly 1 hour after the controller responsible for emitting queue metrics failed to schedule on a soft-failed node. We have since cordoned and drained the node, and the controller is emitting queue metrics again.
- resolved Aug 10, 2026, 09:45 PM UTC
All affected models have been scaling correctly for more than 30 minutes at this point, and we see no residual prediction queues. Thank you for your patience!
Minor August 5, 2026 Significant degradation – Replicate
Started Aug 05, 2026, 11:33 AM UTC· resolved Aug 05, 2026, 11:33 AM UTC
- investigating Aug 05, 2026, 09:30 AM UTC
We are aware of a significant degradation with the service. We have identified the root cause and are taking steps to mitigate. Please hold tight for further information.
- monitoring Aug 05, 2026, 11:25 AM UTC
We have made significant progress in recovering the services, and most traffic should be flowing normal. We are monitoring to make sure all traffic is recovered before closing the incident
- resolved Aug 05, 2026, 11:33 AM UTC
We have been monitoring for a while and services seem to be back. Thank you for your patience
Minor August 2, 2026 API degraded for A100s – Replicate
Started Aug 02, 2026, 04:18 PM UTC· resolved Aug 02, 2026, 04:18 PM UTC
- identified Aug 02, 2026, 04:09 PM UTC
The API for creating predictions for A100s is currently degraded as a piece of backing infrastructure failed. We are working on getting it back online
- resolved Aug 02, 2026, 04:18 PM UTC
We have restarted the affected component and service should be back to normal. Thank you for your patience
Minor July 31, 2026 Degraded scale-out due to failed setups pulling from huggingface – Replicate
Started Jul 31, 2026, 11:39 PM UTC· resolved Jul 31, 2026, 11:39 PM UTC
- identified Jul 31, 2026, 09:25 PM UTC
Models that pull from huggingface during setup have not been succeeding, which is resulting in scale-out delays and queue backups.
- identified Jul 31, 2026, 09:26 PM UTC
We have confirmed that turning off our caching layer for huggingface downloads allows models to successfully set up, so we are making that change for all models while we continue to troubleshoot.
- monitoring Jul 31, 2026, 10:27 PM UTC
All affected hardware types have recovered and we are continuing to monitor and troubleshoot. Thanks for your patience!
- resolved Jul 31, 2026, 11:39 PM UTC
We're looking stable now, and with caching back on. Thanks again for your patience!
Minor July 23, 2026 Hitting GPU Capacity for H100s creating large queue times for some models – Replicate
Started Jul 23, 2026, 02:19 PM UTC· resolved Jul 23, 2026, 02:19 PM UTC
- identified Jul 22, 2026, 11:51 AM UTC
We are over provisioned on H100s currently which is causing long queue times for some models running on H100s
- monitoring Jul 22, 2026, 05:11 PM UTC
GPU usage is back below capacity. We're continuing to monitor.
- monitoring Jul 23, 2026, 03:05 AM UTC
This issue is now resolved
- resolved Jul 23, 2026, 02:19 PM UTC
This issue is resolved
Minor July 16, 2026 HuggingFace download issues – Replicate
Started Jul 16, 2026, 04:04 PM UTC· resolved Jul 16, 2026, 04:04 PM UTC
- investigating Jul 16, 2026, 08:31 AM UTC
We are aware of 504s being returned by models that reach out to HuggingFace during setup. We believe this is likely related to the disruption visible at https://status.huggingface.co/. We are monitoring and will update when we can identify the HuggingFace CDN is back up/
- monitoring Jul 16, 2026, 09:10 AM UTC
HuggingFace is reporting recovery, which we are also seeing on our end. However, we still see minor degradation, which we believe is due to thundering herd once the main issue was resolved. We are still keeping an eye out, but believe we are recovering
- monitoring Jul 16, 2026, 09:28 AM UTC
We have seen most HuggingFace calls start succeeding, except for a small subset that are being downloaded through cas-bridge.xethub.hf.co "cas-bridge.xethub.hf.co": http://cas-bridge.xethub.hf.co. Most models are now succeeding to spin up
- monitoring Jul 16, 2026, 09:46 AM UTC
Still seeing signs of improvement, most models are able to spin up with a few still waiting their turn from HuggingFace. Models should spin up after enough tries if they are failing for HuggingFace related reasons.
- resolved Jul 16, 2026, 04:04 PM UTC
We have not seen elevated rates of HuggingFace/model setup errors for a couple hours now, so we believe this incident is cleared
Minor July 14, 2026 H100 GPU shortage resulting in high queue times – Replicate
Started Jul 14, 2026, 05:27 PM UTC· resolved Jul 14, 2026, 05:27 PM UTC
- investigating Jul 13, 2026, 04:26 PM UTC
Long queue times, especially on BFL models
- monitoring Jul 13, 2026, 05:48 PM UTC
We pushed a fix and are monitoring
- resolved Jul 14, 2026, 05:27 PM UTC
H100 capacity has returned to normal levels
Minor July 10, 2026 High contention on H100 hardware – Replicate
Started Jul 10, 2026, 11:04 PM UTC· resolved Jul 10, 2026, 11:04 PM UTC
- monitoring Jul 10, 2026, 02:50 PM UTC
We are seeing high contention on H100 hardware which is resulting in delays on predictions and scale-out.
- resolved Jul 10, 2026, 11:04 PM UTC
We are back under max capacity for H100 hardware. Thank you for your patience!
Minor June 30, 2026 Limited H100 capacity – Replicate
Started Jun 30, 2026, 07:29 PM UTC· resolved Jun 30, 2026, 07:29 PM UTC
- identified Jun 30, 2026, 01:55 PM UTC
We recently received a sharp increase in demand for H100 capacity which is resulting in delayed scale-out and queue backup.
- resolved Jun 30, 2026, 07:29 PM UTC
The H100 capacity is back in good health. Thanks for your patience!
Minor June 3, 2026 We're seeing long setup times and high contention for models on some L40S and H200 clusters. – Replicate
Started Jun 03, 2026, 07:06 PM UTC· resolved Jun 03, 2026, 07:06 PM UTC
- investigating Jun 03, 2026, 05:59 PM UTC
We're seeing long setup times and high contention for models on some L40S and H200 clusters.
- investigating Jun 03, 2026, 07:06 PM UTC
System is back to operating normally
- resolved Jun 03, 2026, 07:06 PM UTC
System is back to operating normally
Minor May 28, 2026 Degraded performance on flux-2-klein-4b – Replicate
Started May 28, 2026, 02:17 PM UTC· resolved May 28, 2026, 02:17 PM UTC
- investigating May 28, 2026, 12:30 PM UTC
Long queue times for black-forest-labs/flux-2-klein-4b resulting in canceled predictions
- resolved May 28, 2026, 02:17 PM UTC
This issue has been resolved and queue times are back to normal
Minor May 21, 2026 Prediction and Training status updates delayed – Replicate
Started May 21, 2026, 11:28 PM UTC· resolved May 21, 2026, 11:28 PM UTC
- identified May 21, 2026, 09:38 PM UTC
Our message queues for prediction and training status updates are hitting capacity limits which are causing connection failures for queue consumers. We are in the process of bringing additional capacity online.
- resolved May 21, 2026, 11:28 PM UTC
Message flows are healthy.
Minor May 21, 2026 Constrained H100 capacity – Replicate
Started May 21, 2026, 10:05 PM UTC· resolved May 21, 2026, 10:05 PM UTC
- identified May 21, 2026, 03:09 PM UTC
We are seeing heightened demand for H100 hardware which is causing severe queue delays.
- resolved May 21, 2026, 10:05 PM UTC
H100 hardware contention has resolved. Thank you for your patience!
Minor May 12, 2026 Constrained capacity for H100 hardware – Replicate
Started May 12, 2026, 07:43 PM UTC· resolved May 12, 2026, 07:43 PM UTC
- identified May 12, 2026, 03:27 PM UTC
Demand for constrained H100 hardware is causing scaling delays. This impacts queue size and inference speed for any models running on H100s.
- resolved May 12, 2026, 07:43 PM UTC
There is no more contention for H100 hardware. Thank you for your patience!
Minor April 19, 2026 Degraded A100 hardware 26m Replicate
Started Apr 19, 2026, 02:36 PM UTC· resolved Apr 19, 2026, 03:02 PM UTC
- monitoring Apr 19, 2026, 02:36 PM UTC
All predictions and trainings targeting A100 hardware are experiencing degraded performance while control plane nodes restart.
- resolved Apr 19, 2026, 03:02 PM UTC
All A100 capacity is back. Thanks for your patience!
Minor April 9, 2026 A100 capacity unavailable during storage maintenance 47m Replicate
Started Apr 09, 2026, 04:43 PM UTC· resolved Apr 09, 2026, 05:30 PM UTC
- investigating Apr 09, 2026, 04:43 PM UTC
The persistent storage for all A100 hardware is under maintenance and is expected to be degraded until completion.
- resolved Apr 09, 2026, 05:30 PM UTC
The maintenance is complete and all systems are reporting healthy. Thank you for your patience!
Minor March 23, 2026 Downstream errors for Black Forest Labs models 9h 37m Replicate
Started Mar 23, 2026, 03:57 PM UTC· resolved Mar 24, 2026, 01:34 AM UTC
- identified Mar 23, 2026, 03:57 PM UTC
Some Black Forest Labs models are failing due to downstream errors from BFL. We are monitoring the situation and working on work arounds. BFL Status page: https://status.bfl.ml/
- resolved Mar 24, 2026, 01:34 AM UTC
Black Forest Labs has resolved the issue.
Minor March 10, 2026 Degraded performance on Flux Schnell 5h 21m Replicate
Started Mar 10, 2026, 12:56 PM UTC· resolved Mar 10, 2026, 06:17 PM UTC
- investigating Mar 10, 2026, 12:56 PM UTC
We are investigating an outage that is only affecting Flux Schnell.
- monitoring Mar 10, 2026, 01:11 PM UTC
A GPU provider has an outage. Traffic is being rerouted and we are processing new Flux Schnell requests.
- resolved Mar 10, 2026, 06:17 PM UTC
Flux Schnell requests are being served normally.
Minor February 20, 2026 Model Predictions Stuck at "Starting" 59m Replicate
Started Feb 20, 2026, 12:13 PM UTC· resolved Feb 20, 2026, 01:12 PM UTC
- investigating Feb 20, 2026, 12:13 PM UTC
We are currently investigating why a large number of models are not currently processing requests, with predictions stalled with a "starting" status.
- monitoring Feb 20, 2026, 12:59 PM UTC
We have identified the root cause and have made an update. We are continuing to monitor as we start to see things improve.
- resolved Feb 20, 2026, 01:12 PM UTC
Models are once again operational
Notice January 26, 2026 Increased setup failures for T4 models 7h 43m Replicate
Started Jan 26, 2026, 07:33 AM UTC· resolved Jan 26, 2026, 03:16 PM UTC
No vendor updates were posted for this incident.