Netdata Outage History
Netdata is up right nowPingoru has recorded 51 incidents for Netdata: 8 in the last 12 months and 1 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 1 | 0 of 720 (0%) |
| May 2026 | 3 | 5 of 744 (1%) |
| June 2026 | 0 | 0 of 720 (0%) |
| July 2026 | 0 | 0 of 744 (0%) |
| August 2026 | 0 | 0 of 744 (0%) |
| September 2026 | 0 | 0 of 720 (0%) |
| October 2026 | 1 | 234 of 744 (31%) |
| November 2026 | 0 | 19 of 19 (100%) |
Incidents, newest first
Notice May 29, 2026 Agent connectivity issue on legacy systems –
Started May 29, 2026, 06:00 AM UTC· resolved May 29, 2026, 06:00 AM UTC
- resolved May 29, 2026, 06:00 AM UTC
Following a scheduled TLS certificate renewal, a subset of Netdata agents failed to reconnect to Netdata Cloud. The issue stems from outdated CA (Certificate Authority) trust stores on certain legacy Linux distributions, like: • CentOS 7 / RHEL 7 • Amazon Linux 2 • Debian 9 (Jessie) • Ubuntu 16.04 LTS • Oracle Linux 7 A quick way to test for this: curl -v https://app.netdata.cloud If the output shows an error indicating the certificate has expired, the system needs an updated CA bundle.
Minor May 8, 2026 False-Positive Node Reachability Alerts 4h 47m Cloud Alerts, Agent - Cloud Connection (ACLK)
Started May 08, 2026, 01:13 AM UTC· resolved May 08, 2026, 06:00 AM UTC
- monitoring May 08, 2026, 01:13 AM UTC
We are currently tracking an issue with our upstream cloud provider that is affecting a subset of our Kubernetes nodes. As a result, some customers may receive false-positive alerts regarding node reachability. Please be assured that your underlying services and workloads remain operational, and these alerts can be safely ignored at this time. We are closely monitoring the situation with our provider.
- resolved May 08, 2026, 06:00 AM UTC
This incident has been resolved.
Minor May 8, 2026 False-Positive Node Reachability Alerts 20m Cloud Alerts, Agent - Cloud Connection (ACLK)
Started May 08, 2026, 12:11 AM UTC· resolved May 08, 2026, 12:32 AM UTC
- monitoring May 08, 2026, 12:11 AM UTC
We are currently tracking an issue with our upstream cloud provider that is affecting a subset of our Kubernetes nodes. As a result, some customers may receive false-positive alerts regarding node reachability. Please be assured that your underlying services and workloads remain operational, and these alerts can be safely ignored at this time. We are closely monitoring the situation with our provider.
- monitoring May 08, 2026, 12:12 AM UTC
We are continuing to monitor for any further issues.
- resolved May 08, 2026, 12:32 AM UTC
This incident has been resolved.
Notice April 15, 2026 Kubernetes node failure. –
Started Apr 15, 2026, 06:38 AM UTC· resolved Apr 15, 2026, 06:38 AM UTC
- resolved Apr 15, 2026, 06:38 AM UTC
At ~02:20 UTC, a Kubernetes node running part of our ingress controller failed and was auto-replaced. Some clients experienced brief agent disconnections and false-positive node-unavailability alerts during the failover. Service has fully recovered after few minutes. We apologize for the inconvenience.
Notice March 8, 2026 Unexpected Loadbalancer restart –
Started Mar 08, 2026, 03:30 PM UTC· resolved Mar 08, 2026, 03:30 PM UTC
- resolved Mar 08, 2026, 03:30 PM UTC
Yesterday, a brief infrastructure incident occurred due to certain processes on our load balancer failing (exactly at 16:34 UTC). This caused more than half of our globally connected nodes to reconnect, which in turn triggered false positive alerts notifying clients of node downtime. We apologize for the confusion and are taking steps to ensure this does not happen again.
Major January 22, 2026 404 errors with Netdata RHEL 8.x repositories with latest repository configuration packages. 2h 32m Agent Repositories
Started Jan 22, 2026, 12:47 PM UTC· resolved Jan 22, 2026, 03:20 PM UTC
- identified Jan 22, 2026, 12:47 PM UTC
We have identified an issue with the latest version of the repository configuration packages for our RHEL 8.x repositories that causes systems which update to that version of the repository configuration package to become unable to access our repositories. This affects all systems that use our repository configuration packages to access our RHEL 8.x repositories, including but not limited to RHEL 8.x, Alma Linux 8.x, Rocky Linux 8.x, and Oracle Linux 8.x. Affected systems will report a failure to download repository metadata for Netdata repositories. A fix is currently being tested and is expected to be deployed soon.
- identified Jan 22, 2026, 12:48 PM UTC
We are continuing to work on a fix for this issue.
- monitoring Jan 22, 2026, 02:06 PM UTC
The primary fix has been deployed and appears to be working correctly. We are currently working on a followup fix to allow affected systems to update properly without requiring user intervention. Until that additional fix is deployed, affected systems can be manually updated to fully resolve the issue locally by fetching the latest version of the repository configuration package manually and installing it on the system: - For most RHEL 8 equivalent systems using our stable repositories: https://repository.netdata.cloud/repos/repoconfig/el/8/x86_64/netdata-repo-5-4.noarch.rpm - For most RHEL 8 equivalent systems using our nightly repositories: https://repository.netdata.cloud/repos/repoconfig/el/8/x86_64/netdata-repo-edge-5-4.noarch.rpm - For Oracle Linux 8 systems using our stable repositories: https://repository.netdata.cloud/repos/repoconfig/ol/8/x86_64/netdata-repo-5-4.noarch.rpm - For Oracle Linux 8 systems using our nightly repositories: https://repository.netdata.cloud/repos/repoconfig/ol/8/x86_64/netdata-repo-edge-5-4.noarch.rpm
- resolved Jan 22, 2026, 03:20 PM UTC
In addition to the fix for the repository configuration packages, a fix has been deployed to the repositories themselves that will enable affected systems to update successfully to the latest version of the repository configuration packages, and as such no further issues are expected.
Critical November 18, 2025 Services Unavailable 7h 11m Cloud Web UI, Cloud Charts, Agent - Cloud Connection (ACLK), Agent Repositories
Started Nov 18, 2025, 12:11 PM UTC· resolved Nov 18, 2025, 07:22 PM UTC
- investigating Nov 18, 2025, 12:11 PM UTC
As Cloudflare is experiencing network issues, Netdata Cloud, as well as our package repositories are currently unavailable.
- monitoring Nov 18, 2025, 12:23 PM UTC
Connectivity is recovering, we'll keep this open until we see traffic at previous levels. See https://www.cloudflarestatus.com/incidents/8gmgl950y3h7 for details on the Cloudflare outage itself.
- resolved Nov 18, 2025, 07:22 PM UTC
This incident has been resolved.
Major September 29, 2025 Agent v2.7.0 on Windows doesn't start go.d plugin 1d 6h Agent (all platforms)
Started Sep 29, 2025, 11:36 AM UTC· resolved Sep 30, 2025, 06:29 PM UTC
- identified Sep 29, 2025, 11:36 AM UTC
We've identified an issue with the latest Agent release on Windows that causes the go.d plugin to not be started. The impact is that for Windows nodes, none of the metrics from the go.d plugin are being collected and corresponding alerts don't fire. We will be releasing a patch release to address this as soon as possible today. As a workaround the plugin binary in Program Files can be renamed to add the missing .exe extension.
- monitoring Sep 30, 2025, 05:04 AM UTC
We have released Netdata Agent v2.7.1 and recommend all Windows users to upgrade. Release notes: https://github.com/netdata/netdata/releases/tag/v2.7.1
- resolved Sep 30, 2025, 06:29 PM UTC
This incident has been resolved.
Major August 21, 2025 Package availability issues with native DEB/RPM packages. 22m Agent Repositories
Started Aug 21, 2025, 07:35 PM UTC· resolved Aug 21, 2025, 07:58 PM UTC
- investigating Aug 21, 2025, 07:35 PM UTC
We are currently investigating issues with availability of our nightly DEB and RPM packages which are preventing updates and installs from working correctly.
- identified Aug 21, 2025, 07:39 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Aug 21, 2025, 07:42 PM UTC
A fix has been implemented and we expect the repositories to be functioning normally within the next 30 minutes.
- resolved Aug 21, 2025, 07:58 PM UTC
All repositories are functioning normally again.
Notice August 14, 2025 Issues with package availability in official DEB/RPM package repositories 9m Agent Repositories
Started Aug 14, 2025, 03:03 PM UTC· resolved Aug 14, 2025, 03:12 PM UTC
- monitoring Aug 14, 2025, 03:03 PM UTC
Due to issues in our package upload process, the v2.6.2 stable release and the nightly builds for 2025-08-13 and 2025-08-14 were not fully uploaded to our official DEB/RPM repositories. We have already identified the root cause and implemented a fix, and are currently monitoring the processing of the package upload backlog to ensure that all of the packages are properly available.
- resolved Aug 14, 2025, 03:12 PM UTC
We have confirmed that the issue that caused the incomplete package uploads has been fully fixed, and all packages that should have been uploaded since the issue first occurred have been fully uploaded and are available in the repositories.
Minor August 11, 2025 Increased data request latencies 9d Cloud Charts, Agent - Cloud Connection (ACLK)
Started Aug 11, 2025, 12:27 PM UTC· resolved Aug 20, 2025, 01:09 PM UTC
- investigating Aug 11, 2025, 12:27 PM UTC
We are investigating an issue with increased data request latencies that can result in slow or completely empty charts for some customers. The affected Netdata Agent versions are v2.6.0 and later, for all stable and nightly builds.
- investigating Aug 11, 2025, 02:24 PM UTC
We will be adding additional debugging code in both the Agent (next nightly) and Cloud Backend to track what is causing these spurious latencies.
- identified Aug 12, 2025, 09:28 AM UTC
We have identified the cause of this issue and have been creating a fix which will appear in the upcoming nightly build. When we are confident that there are no negative effects on that build, we will do a new patch release on the stable channel the next day. What happened is that we were not correctly clearing MQTT messages that had been acknowledged by the broker, causing newer messages to not be correctly recognized as acknowledged in certain conditions.
- monitoring Aug 13, 2025, 03:30 PM UTC
We've implement a change for this issue which appeared in our latest nightly, and resolved the problem. We are now issuing a new stable release v2.6.2 to include that fix, and will resolve this issue when all packages are available.
- resolved Aug 20, 2025, 01:09 PM UTC
This incident has been resolved. We recommend that Agents are upgraded to v2.6.2 on the stable channel, or the latest nightly build at the earliest convenience.
Notice July 25, 2025 Windows Agent Monitoring Issues 2d 22h Agent (all platforms)
Started Jul 25, 2025, 12:27 PM UTC· resolved Jul 28, 2025, 10:44 AM UTC
- monitoring Jul 25, 2025, 12:27 PM UTC
We identified monitoring issues affecting customers using Netdata Agent version 2.6.0 on Windows servers. These issues were caused by upstream changes in Windows libraries that impacted agent functionality. Resolution Fixed Version Available: Netdata Agent v2.6.1 Release Date: July 25, 2025 (16:00 UTC) Preview Available: Nightly release v2.6.0-2 Required Action Immediate upgrade recommended - All users running Netdata Agent v2.6.0 on Windows servers should upgrade to v2.6.1 as soon as possible to restore full monitoring capabilities. We appreciate your patience as we continue working on additional improvements related to the upstream Windows library changes.
- resolved Jul 28, 2025, 10:44 AM UTC
This incident has been resolved.
Minor June 4, 2025 MQTT broker failure 49m Agent - Cloud Connection (ACLK)
Started Jun 04, 2025, 12:59 PM UTC· resolved Jun 04, 2025, 01:49 PM UTC
- identified Jun 04, 2025, 12:59 PM UTC
We had a major problem with our MQTT broker. It is currently up and running and agents are reconnecting to the cloud.
- monitoring Jun 04, 2025, 01:42 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jun 04, 2025, 01:49 PM UTC
This incident has been resolved.
Major April 1, 2025 Nightly static builds overwrite node and metric data upon install 1d Agent (all platforms)
Started Apr 01, 2025, 09:00 AM UTC· resolved Apr 02, 2025, 09:25 AM UTC
- investigating Apr 01, 2025, 09:00 AM UTC
We have found that a recent change in the nightly static builds of Netdata Agent causes meta data on the Agent to be overwritten. Specifically the sqlite3 database that keeps meta data on which timeseries stored in dbengine correspond with which metrics, and the information on the Agent's "machine GUID" are overwritten with the same copy in the build package. Not affected are: - All stable releases - Native packages (.deb and .rpm) Affected are all nightly static builds with the following version numbers: - 2.3.0-50-nightly - 2.3.0-60-nightly - 2.3.0-72-nightly - 2.3.0-78-nightly - 2.3.0-87-nightly The initial impact is that all affected Agent installs, even though they still have the timeseries data stored on disk, have lost all meta data associated with it, so these timeseries become inaccessable. This is unrecoverable. Additionally, the main form of identification is overwritten, too. We are assessing what the impact is for users of Netdata Cloud, and will update this incident with more information when the investigation is completed. The bug itself has been fixed and merged. We will issue a new nightly build shortly.
- resolved Apr 02, 2025, 09:25 AM UTC
Affected Agents can cause the creation of multiple duplicate nodes in Netdata Cloud. All but the last one will appear as offline, and the last one will be as if it was created from scratch with no data. Unfortunately, the previously stored metrics for the affected nodes can not be recovered. The duplicate offline nodes can be safely deleted from Space Settings -> Nodes. Note that you may have to add the newest copy of these nodes to the appropriate rooms. The fixed nightly static build is v2.3.0-102.
Minor February 25, 2025 Lastest Agent nightly build (v2.2.0-245) broken at first start 5h 44m Agent (all platforms)
Started Feb 25, 2025, 09:41 AM UTC· resolved Feb 25, 2025, 03:25 PM UTC
- investigating Feb 25, 2025, 09:41 AM UTC
We are investigating an issue with today's nightly (v2.2.0-245), causing alerting to not work ("health") and external plugins, including go.d, to not connect properly. This may be resolved by restarting the Agent. Stable versions of the Agent are not affected.
- identified Feb 25, 2025, 11:04 AM UTC
We have identified the issue, committed a fix, and initiated new nightly builds for all platforms. This will take several hours. In the mean time, please restart Netdata to work around the issue.
- monitoring Feb 25, 2025, 01:14 PM UTC
The builds are completed, so we are watching out for any remaining related issues.
- monitoring Feb 25, 2025, 01:29 PM UTC
We are continuing to monitor for any further issues.
- resolved Feb 25, 2025, 03:25 PM UTC
This incident has been resolved.
Notice November 15, 2024 Alarm Processing Delays –
Started Nov 15, 2024, 07:00 AM UTC· resolved Nov 15, 2024, 07:00 AM UTC
- resolved Nov 15, 2024, 07:00 AM UTC
Our alarm processing infrastructure was running behind which is causing inaccuracies alarms for some nodes. No data has been lost and the systems should be already up to date.
Minor November 8, 2024 Alerting is working slower 5h 29m Cloud Alerts
Started Nov 08, 2024, 08:07 AM UTC· resolved Nov 08, 2024, 01:36 PM UTC
- investigating Nov 08, 2024, 08:07 AM UTC
Due to the release of Netdata Agent 2.0 we have quite a big backlog for alarms. We are investigating this issue.
- identified Nov 08, 2024, 09:49 AM UTC
The issue has been identified and a fix is being implemented.
- monitoring Nov 08, 2024, 11:01 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Nov 08, 2024, 01:36 PM UTC
This incident has been resolved.