BCGov incident
CHG0080163 - CITZ MCS GOLDDR - Apply updates and perform maintenance
Resolved after 52 days 12 hours, on August 31, 2026.
We will update some software, firmware, and cluster settings and perform a rolling restart of all nodes in the cluster. We will apply the latest firmware on the physical servers. We will upgrade Trident to v26.06.0 and enable concurrency. This has shown great speed improvements to PVC mount/unmount operations in the LABs. Switch the container runtime from runc to crun , as it is the default for OpenShift now. When? Aug 31st to Sept 4th Trident will be upgraded at 6am on the first day of the change. At 9am on the first day of the change we will begin restarting the master nodes first, then the infra nodes, and start on the worker nodes last. Node reboots will continue until 5pm each day, and resume at 9am the next day. Silver will pause over the weekend. Will there be an impact on the Platform apps? As nodes are rebooted pods will be rescheduled to another node. We expect the past issues with Trident taking a long time to start up pods will be solved, but will continue to use slow manual node drains and monitoring of Trident health. Do I need to do anything? Monitor your apps health.
Update timeline
- investigating Aug 31, 2026, 12:21 PM UTC
This work was successfully completed.
- scheduled Aug 31, 2026, 01:00 PM UTC
We will update some software, firmware, and cluster settings and perform a rolling restart of all nodes in the cluster. We will apply the latest firmware on the physical servers. We will upgrade Trident to v26.06.0 and enable concurrency. This has shown great speed improvements to PVC mount/unmount operations in the LABs. Switch the container runtime from runc to crun , as it is the default for OpenShift now. When? Aug 31st to Sept 4th Trident will be upgraded at 6am on the first day of the change. At 9am on the first day of the change we will begin restarting the master nodes first, then the infra nodes, and start on the worker nodes last. Node reboots will continue until 5pm each day, and resume at 9am the next day. Silver will pause over the weekend. Will there be an impact on the Platform apps? As nodes are rebooted pods will be rescheduled to another node. We expect the past issues with Trident taking a long time to start up pods will be solved, but will continue to use slow manual node drains and monitoring of Trident health. Do I need to do anything? Monitor your apps health.
Your product runs on more than BCGov
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services