CloudCannon Outage History
CloudCannon is up right nowPingoru has recorded 50 incidents for CloudCannon: 16 in the last 12 months and 0 in the last 90 days.
Incidents, newest first
Notice July 22, 2026 Assets failing to load in the Visual Editor 28m App (app.cloudcannon.com)
Started Jul 22, 2026, 03:29 AM UTC· resolved Jul 22, 2026, 03:57 AM UTC
- investigating Jul 22, 2026, 03:29 AM UTC
We are currently investigating this issue.
- resolved Jul 22, 2026, 03:57 AM UTC
This incident has been resolved.
Notice May 21, 2026 Syncing Issues 2h 57m
Started May 21, 2026, 07:55 PM UTC· resolved May 21, 2026, 10:53 PM UTC
- identified May 21, 2026, 07:55 PM UTC
Some customers are experiencing issues syncing with GitHub. We are working on a fix
- monitoring May 21, 2026, 08:05 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring May 21, 2026, 08:27 PM UTC
We are continuing to monitor for any further issues.
- resolved May 21, 2026, 10:53 PM UTC
This incident has been resolved.
Minor May 4, 2026 Intermittent build errors 14h 31m App (app.cloudcannon.com)
Started May 04, 2026, 02:07 PM UTC· resolved May 05, 2026, 04:39 AM UTC
- investigating May 04, 2026, 02:07 PM UTC
We are currently investigating this issue.
- identified May 04, 2026, 02:38 PM UTC
The issue has been identified and a fix is being implemented.
- identified May 04, 2026, 03:12 PM UTC
Certain sites are stuck trying to sync at the start of a build. This is not as widespread as initially expected. We will continue our investigation during business hours. Thank you for your patience.
- identified May 04, 2026, 10:11 PM UTC
We have identified the issue and we are working on a fix. Sites will be rebuilt when resolved.
- identified May 05, 2026, 12:30 AM UTC
The issue is patched and we are working through the recovery process.
- identified May 05, 2026, 04:07 AM UTC
Recovery is complete. We are working to rebuild sites with a broken build.
- resolved May 05, 2026, 04:39 AM UTC
All files have been restored and sites are rebuilt. Thanks for your patience surrounding this issue. If you experience any further issues, please contact support.
- postmortem May 05, 2026, 09:04 PM UTC
## Postmortem: S3 Storage Incident \(May 3–5, 2026\) ### Root Cause Between May 3–5, 2026, a bug in CloudCannon's site-deletion logic caused a background job to delete a large portion of files from our production S3 bucket. When a site is deleted, a background job removes its associated files from S3 using the site's storage prefix as the deletion target. The issue was caused when a site with a blank storage prefix — a value that should not be permitted but lacked validation — was deleted. The deletion function had no guard against a blank prefix, so it resolved to match every object in the bucket. The job's automatic retry behavior meant that when it timed out mid-deletion, it resumed on each retry, progressively deleting further through the bucket alphabetically, and the loss of source files caused builds to fail. All affected files have been fully recovered using S3 versioning. ### Impact * Sites whose storage prefix fell alphabetically before the job's timeout point lost all source files in S3. * All builds for affected sites failed. * Syncing diffs were also deleted and were unavailable during this window. These have been recovered. * Deployed sites, served from a separate bucket, were not affected. * Hosting was relatively unaffected due to caching layers. * No customer data was permanently lost — S3 versioning was enabled, so all files were recoverable. ### Resolution Files were recovered by removing the S3 delete markers created by the bulk deletion, restoring all objects in place. No database changes or manual re-syncing were required. The following fixes have been shipped: * The S3 deletion function now returns immediately if given a blank prefix. * The site deletion job now validates the storage prefix before running. * The site model now enforces a non-blank storage prefix at the data layer.
Notice April 29, 2026 Elevated Error Rate on Hosting –
Started Apr 29, 2026, 11:09 AM UTC· resolved Apr 29, 2026, 11:09 AM UTC
- resolved Apr 29, 2026, 11:09 AM UTC
This incident has been resolved.
Minor March 9, 2026 Slow requests 10m Hosting (sites.cloudcannon.com)
Started Mar 09, 2026, 04:36 PM UTC· resolved Mar 09, 2026, 04:47 PM UTC
- identified Mar 09, 2026, 04:36 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Mar 09, 2026, 04:42 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Mar 09, 2026, 04:47 PM UTC
This incident has been resolved.
Minor March 4, 2026 Slow hosting requests 29m Hosting (sites.cloudcannon.com)
Started Mar 04, 2026, 09:10 AM UTC· resolved Mar 04, 2026, 09:40 AM UTC
- monitoring Mar 04, 2026, 09:10 AM UTC
Requests were reporting higher than normal response times. We have released a fix and numbers are returning to normal. We are continuing to monitor the results.
- resolved Mar 04, 2026, 09:40 AM UTC
This incident has been resolved.
Minor February 24, 2026 Slow builds 16m App (app.cloudcannon.com)
Started Feb 24, 2026, 10:18 PM UTC· resolved Feb 24, 2026, 10:35 PM UTC
- identified Feb 24, 2026, 10:18 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Feb 24, 2026, 10:29 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 24, 2026, 10:35 PM UTC
This incident has been resolved.
Notice February 24, 2026 Error when resetting password for site-level user account authentication 16h 44m
Started Feb 24, 2026, 04:09 AM UTC· resolved Feb 24, 2026, 08:54 PM UTC
- investigating Feb 24, 2026, 04:09 AM UTC
Some users may not be able to reset the password used to access a Site with User Account authentication. We are currently investigating this issue.
- resolved Feb 24, 2026, 08:54 PM UTC
This incident has been resolved.
Minor February 20, 2026 Elevated 503 Error Rate on Hosting 2h 26m Hosting (sites.cloudcannon.com)
Started Feb 20, 2026, 02:08 PM UTC· resolved Feb 20, 2026, 04:35 PM UTC
- monitoring Feb 20, 2026, 02:08 PM UTC
Some 503 errors on hosted sites. We are monitoring the problem and working to release further improvements.
- monitoring Feb 20, 2026, 02:10 PM UTC
We are continuing to monitor for any further issues.
- monitoring Feb 20, 2026, 04:00 PM UTC
Further issues have been identified and fixes have been released.
- resolved Feb 20, 2026, 04:35 PM UTC
The incident has been resolved. We will produce a post mortem early next week.
Minor February 19, 2026 Elevated 503 Error Rate on Hosting 9h 50m Hosting (sites.cloudcannon.com)
Started Feb 19, 2026, 10:44 PM UTC· resolved Feb 20, 2026, 08:34 AM UTC
- monitoring Feb 19, 2026, 10:44 PM UTC
Some 503 errors on hosted sites. We are monitoring the problem and working to release further improvements.
- monitoring Feb 20, 2026, 02:59 AM UTC
We have released more improvements for this issue today, and we are continuing to monitor the results.
- monitoring Feb 20, 2026, 05:13 AM UTC
Servers haven't reported a 503 error in the last 6 hours. The improvements appear to be working well. We will continue to monitor the results.
- resolved Feb 20, 2026, 08:34 AM UTC
This incident has been resolved.
Minor February 12, 2026 Elevated 503 Error Rate on Hosting –
Started Feb 12, 2026, 07:00 AM UTC· resolved Feb 12, 2026, 07:00 AM UTC
- resolved Feb 12, 2026, 07:00 AM UTC
Increased 503 errors on the hosting. We have identified an issue and we are deploying a fix.
- postmortem Feb 13, 2026, 02:41 AM UTC
## Incident: Intermittent 503 errors on hosted sites **Duration:** Recurring over approximately one week, with individual episodes lasting 30-90 minutes before self-recovering. **Impact:** During episodes, some requests to sites hosted on CloudCannon returned error responses. Static assets and HTML pages were all affected. Sites recovered without manual intervention, but the disruption was repeated. ### Summary Over the past week, a significant increase in bot and crawler traffic drove higher-than-usual request volumes to our file serving infrastructure. This elevated load exposed a weakness in how our application managed database connections, leading to periodic service degradation. ### What happened Our file serving layer uses a pool of database connections to resolve site configuration for each incoming request. Under the increased traffic load, a combination of network conditions between our application servers and the database caused some of these connections to silently break mid-use. Under normal traffic levels, the small number of affected connections would have had minimal impact. However, the sustained increase in request volume meant the connection pool was under constant pressure, and broken connections accumulated faster than the system could recover from them. Once enough dead connections built up, the pool became saturated, and new requests could not be served. The service self-recovered each time as the underlying connections were eventually recycled, but the recovery was slower than acceptable. ### Root cause The database connection pool lacked client-side safeguards for detecting and recovering from broken connections. Specifically: * There were no client-side timeouts on database queries, so a query sent over a broken connection would wait indefinitely for a response that would never arrive. * TCP keepalive was not enabled, so broken connections were not detected until the operating system's own timeout expired, which could take over an hour. Under the increased traffic volume, these gaps were exposed more frequently and with greater impact. ### Resolution We have deployed a series of changes to harden the connection pool and ensure rapid recovery: * **Client-side query timeouts** ensure that broken connections are detected and evicted from the pool within seconds, rather than blocking indefinitely. * **TCP keepalive** is now enabled on all database connections, allowing the operating system to actively detect broken connections. * **Connection acquisition timeouts** ensure that requests fail fast with a clear error rather than queuing indefinitely when the pool is under pressure. * **Improved error observability** now captures which specific operations are affected, enabling faster diagnosis. * **Graceful shutdown handling** prevents request drops during deployments. We are also working on broader architectural improvements to reduce per-request database dependency through better caching and conditional responses, which will provide additional resilience under high traffic conditions. ### Current status We have deployed fixes that we believe address the root cause and are actively monitoring. We will continue to update this page as we confirm the effectiveness of these changes over the coming days.
Minor February 11, 2026 Elevated 503 Error Rate on Hosting 11m Hosting (sites.cloudcannon.com)
Started Feb 11, 2026, 02:40 PM UTC· resolved Feb 11, 2026, 02:52 PM UTC
- identified Feb 11, 2026, 02:40 PM UTC
The issue has been identified and a fix is being implemented.
- identified Feb 11, 2026, 02:41 PM UTC
We are continuing to work on a fix for this issue.
- monitoring Feb 11, 2026, 02:45 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 11, 2026, 02:52 PM UTC
The incident is resolved and a permanent solution will be released later today.
Minor February 10, 2026 Elevated 503 Error Rate on Hosting 27m
Started Feb 10, 2026, 03:56 PM UTC· resolved Feb 10, 2026, 04:24 PM UTC
- investigating Feb 10, 2026, 03:56 PM UTC
We are currently investigating this issue.
- resolved Feb 10, 2026, 04:24 PM UTC
This incident has been resolved.
Minor February 5, 2026 Elevated 503 Error Rate on Hosting –
Started Feb 05, 2026, 02:00 AM UTC· resolved Feb 05, 2026, 02:00 AM UTC
- resolved Feb 05, 2026, 02:00 AM UTC
A server in our hosting infrastructure became unresponsive, resulting in an elevated rate of 503 (Service Unavailable) errors for a subset of requests. Our monitoring systems detected the issue and the affected server was terminated. A healthy replacement was brought online, restoring normal service.
Minor February 3, 2026 Delay in builds 26m App (app.cloudcannon.com)
Started Feb 03, 2026, 08:43 PM UTC· resolved Feb 03, 2026, 09:09 PM UTC
- identified Feb 03, 2026, 08:43 PM UTC
The issue has been identified and a fix is being implemented.
- resolved Feb 03, 2026, 09:09 PM UTC
This incident has been resolved.
Minor November 5, 2025 Syncing backlog 29m App (app.cloudcannon.com)
Started Nov 05, 2025, 10:03 PM UTC· resolved Nov 05, 2025, 10:32 PM UTC
- monitoring Nov 05, 2025, 10:03 PM UTC
We have identified a backlog of syncs which is causing saves to become backed up. We are monitoring the situation.
- resolved Nov 05, 2025, 10:32 PM UTC
This incident has been resolved.
Minor October 29, 2025 Syncing paused 6m App (app.cloudcannon.com)
Started Oct 29, 2025, 07:16 AM UTC· resolved Oct 29, 2025, 07:23 AM UTC
- identified Oct 29, 2025, 07:16 AM UTC
The issue has been identified and a fix is being implemented.
- monitoring Oct 29, 2025, 07:20 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Oct 29, 2025, 07:23 AM UTC
This incident has been resolved.
Minor October 17, 2025 Syncing downtime 12m App (app.cloudcannon.com)
Started Oct 17, 2025, 05:04 PM UTC· resolved Oct 17, 2025, 05:17 PM UTC
- investigating Oct 17, 2025, 05:04 PM UTC
We are currently investigating an issue preventing saves and syncs to start.
- resolved Oct 17, 2025, 05:17 PM UTC
Syncing has returned to normal. We are investigating the root cause for the delays.
Minor October 17, 2025 Syncing paused 8m App (app.cloudcannon.com)
Started Oct 17, 2025, 03:09 AM UTC· resolved Oct 17, 2025, 03:18 AM UTC
- identified Oct 17, 2025, 03:09 AM UTC
We are working on a fix for syncing downtime
- resolved Oct 17, 2025, 03:18 AM UTC
This incident has been resolved.
Minor September 23, 2025 Syncing downtime 36m App (app.cloudcannon.com)
Started Sep 23, 2025, 03:38 PM UTC· resolved Sep 23, 2025, 04:15 PM UTC
- identified Sep 23, 2025, 03:38 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Sep 23, 2025, 03:54 PM UTC
A fix has been implemented and we are monitoring to ensure the back log of events are processed.
- resolved Sep 23, 2025, 04:15 PM UTC
This incident has been resolved.
Major August 14, 2025 Syncing downtime 3h 6m App (app.cloudcannon.com)
Started Aug 14, 2025, 11:06 PM UTC· resolved Aug 15, 2025, 02:13 AM UTC
- identified Aug 14, 2025, 11:06 PM UTC
We have identified an issue preventing syncs and saves from starting. We are restarting the failed services.
- monitoring Aug 14, 2025, 11:20 PM UTC
Syncing is back to operating as usual. Working through backlog of syncs.
- resolved Aug 15, 2025, 02:13 AM UTC
This incident has been resolved.
Major June 27, 2025 Issues syncing 57m App (app.cloudcannon.com)
Started Jun 27, 2025, 02:04 PM UTC· resolved Jun 27, 2025, 03:02 PM UTC
- identified Jun 27, 2025, 02:04 PM UTC
We have identified an issue which is preventing sites from saving and syncing. We are working on a fix.
- monitoring Jun 27, 2025, 02:11 PM UTC
A fix has been applied and syncs are now running as expected. We are monitoring the back log to ensure all sites sync back up.
- monitoring Jun 27, 2025, 02:43 PM UTC
The back log is starting to clear well now. We are continuing to monitor the results.
- resolved Jun 27, 2025, 03:02 PM UTC
The back log is cleared and syncing has returned to normal.
Major May 6, 2025 Increased hosting error rate 20m Hosting (sites.cloudcannon.com)
Started May 06, 2025, 04:27 PM UTC· resolved May 06, 2025, 04:47 PM UTC
- investigating May 06, 2025, 04:27 PM UTC
We are currently investigating an issue which is causing uncached sites to take longer to load.
- monitoring May 06, 2025, 04:41 PM UTC
Problematic servers have been restarted and hosting is responding correctly. We are continuing to monitor for changes.
- resolved May 06, 2025, 04:47 PM UTC
The incident has been resolved.
Minor April 15, 2025 Invalid 404 responses –
Started Apr 15, 2025, 01:30 PM UTC· resolved Apr 15, 2025, 01:30 PM UTC
- resolved Apr 15, 2025, 01:30 PM UTC
A erroneous deployment resulted in freshly built sites returning more 404 responses than normal. The release was rolled back and caches were cleared to return service to normal. A fix has since been implemented for the cause.
Minor March 26, 2025 Screenshots are failing to load assets 4h 50m App (app.cloudcannon.com)
Started Mar 26, 2025, 04:14 AM UTC· resolved Mar 26, 2025, 09:05 AM UTC
- investigating Mar 26, 2025, 04:14 AM UTC
We are investigating an issue where screenshots are not loading all assets on a website. This results in broken looking screenshots on some sites.
- resolved Mar 26, 2025, 09:05 AM UTC
We have released a fix and screenshots have returned to normal. Site with broken screenshots will still be broken until a change occurs updating the effected pages. Changing a shared dependency across the sites will re-trigger screenshots for impacted sites (e.g. a layout or shared css file). Sites impacted are limited to sites which rely on case insensitivity of URLs in CloudCannon hosting.