GitLab incident

Delay in CI job processing

Minor Resolved View vendor source →

GitLab experienced a minor incident on September 9, 2026 affecting CI/CD and GitLab agent server for Kubernetes (Google Compute Engine), lasting 1h 39m. The incident has been resolved; the full update timeline is below.

Started
Sep 09, 2026, 04:32 PM UTC
Resolved
Sep 09, 2026, 06:12 PM UTC
Duration
1h 39m
Detected by Pingoru
Sep 09, 2026, 04:32 PM UTC

Affected components

CI/CDGitLab agent server for Kubernetes (Google Compute Engine)

Update timeline

  1. investigating Sep 09, 2026, 04:32 PM UTC

    We're observing an increase in CI job queues. Users may see jobs stuck in a pending state for longer than usual. Jobs may not fire their associated webhooks or integrations. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22897 Back to current status Status History Subscribe to receive status updates by email Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates by webhook Each status update will POST a JSON payload to this URL Email address for managing webhook Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates in Microsoft Teams Enter your Microsoft Teams webhook. View Instructions Email address for managing subscription Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates via RSS RSS Feed Close Follow our status updates on Twitter/X @gitlabstatus Close Subscribe to receive status updates in Slack Slack channel ID Find the channel ID: Select the channel in your Slack workspace. The channel ID is displayed in the browser URL. Example: https://app.slack.com/client/T04SJBK1C/ C03SKGJ1P Email address Close Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates via iCalendar iCalendar Feed Close

  2. investigating Sep 09, 2026, 04:32 PM UTC

    We're observing an increase in CI job queues. Users may see jobs stuck in a pending state for longer than usual. Jobs may not fire their associated webhooks or integrations. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22897

  3. investigating Sep 09, 2026, 04:58 PM UTC

    We're still observing elevated CI job queues and are working to address the problem. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22897

  4. investigating Sep 09, 2026, 04:58 PM UTC

    We're still observing elevated CI job queues and are working to address the problem. Details in: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/22897

  5. monitoring Sep 09, 2026, 05:35 PM UTC

    CI job execution delays have improved significantly, and the backlog of pending jobs is shrinking. We're continuing to monitor the situation.

  6. monitoring Sep 09, 2026, 05:35 PM UTC

    CI job execution delays have improved significantly, and the backlog of pending jobs is shrinking. We're continuing to monitor the situation.

  7. resolved Sep 09, 2026, 06:12 PM UTC

    We are no longer observing delays in CI job execution and the backlog of pending jobs has been worked through. We'll now be marking this as resolved.