GitLab incident

Security Reports not loading for some users

Resolved after 18 hours 10 minutes, affecting CI/CD and GitLab agent server for Kubernetes (GitLab agent server for Kubernetes — Google Compute Engine), on September 29, 2026.

resolved Sep 30, 2026, 12:26 PM UTC

We observed that the backlog of jobs have been processed and new reports should be coming in on the normal schedule. With this update the incident is resolved. More info: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

Started
Sep 29, 2026, 06:15 PM UTC
Resolved
Sep 30, 2026, 12:26 PM UTC
Duration
18h 10m
Detected by Pingoru
Sep 29, 2026, 06:15 PM UTC

Affected components

CI/CDGitLab agent server for Kubernetes (GitLab agent server for Kubernetes — Google Compute Engine)

Update timeline

  1. investigating Sep 29, 2026, 06:15 PM UTC

    Security and vulnerability reports are not loading or being ingested for some users. We are investigating now. For more info: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024 Back to current status Status History Subscribe to receive status updates by email Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates by webhook Each status update will POST a JSON payload to this URL Email address for managing webhook Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates in Microsoft Teams Enter your Microsoft Teams webhook. View Instructions Email address for managing subscription Close Subscribe Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates via RSS RSS Feed Close Follow our status updates on Twitter/X @gitlabstatus Close Subscribe to receive status updates in Slack Slack channel ID Find the channel ID: Select the channel in your Slack workspace. The channel ID is displayed in the browser URL. Example: https://app.slack.com/client/T04SJBK1C/ C03SKGJ1P Email address Close Manage Subscription Manage Existing Subscription Create New Subscription Subscribe to receive status updates via iCalendar iCalendar Feed Close

  2. investigating Sep 29, 2026, 06:15 PM UTC

    Security and vulnerability reports are not loading or being ingested for some users. We are investigating now. For more info: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  3. identified Sep 29, 2026, 06:36 PM UTC

    We've identified the cause and are working on mitigation. More info: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  4. identified Sep 29, 2026, 06:36 PM UTC

    We've identified the cause and are working on mitigation. More info: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  5. monitoring Sep 29, 2026, 07:06 PM UTC

    The mitigation we implemented is reducing the backlog of events. We are now monitoring the situation, see also: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  6. monitoring Sep 29, 2026, 07:06 PM UTC

    The mitigation we implemented is reducing the backlog of events. We are now monitoring the situation, see also: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  7. monitoring Sep 29, 2026, 08:01 PM UTC

    We continue to see improvements across our graphs, see also: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  8. monitoring Sep 29, 2026, 08:01 PM UTC

    We continue to see improvements across our graphs, see also: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  9. monitoring Sep 29, 2026, 09:12 PM UTC

    We continue to see improvements across our graphs and reduced backlog, more info: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  10. monitoring Sep 29, 2026, 09:12 PM UTC

    We continue to see improvements across our graphs and reduced backlog, more info: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  11. monitoring Sep 29, 2026, 10:29 PM UTC

    Monitoring continues. A backlog of ingestion events is currently being processed, which we expect will take some time. We will keep posting updates as we have them. See: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  12. monitoring Sep 29, 2026, 10:29 PM UTC

    Monitoring continues. A backlog of ingestion events is currently being processed, which we expect will take some time. We will keep posting updates as we have them. See: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  13. monitoring Sep 30, 2026, 12:02 AM UTC

    Monitoring continues. A backlog of ingestion events is currently being processed, which we expect will take some time. See: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  14. monitoring Sep 30, 2026, 12:02 AM UTC

    Monitoring continues. A backlog of ingestion events is currently being processed, which we expect will take some time. See: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  15. monitoring Sep 30, 2026, 07:19 AM UTC

    No material updates to report. We are continuing to monitor the report-ingestion backlog and working on mitigation. More info: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  16. monitoring Sep 30, 2026, 07:19 AM UTC

    No material updates to report. We are continuing to monitor the report-ingestion backlog and working on mitigation. More info: https://gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  17. monitoring Sep 30, 2026, 09:17 AM UTC

    No material updates to report. We are continuing to monitor the report-ingestion backlog and working on mitigation. More info: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  18. monitoring Sep 30, 2026, 09:17 AM UTC

    No material updates to report. We are continuing to monitor the report-ingestion backlog and working on mitigation. More info: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

  19. resolved Sep 30, 2026, 12:26 PM UTC

    We observed that the backlog of jobs have been processed and new reports should be coming in on the normal schedule. With this update the incident is resolved. More info: gitlab.com/gitlab-com/gl-infra/production/-/work_items/23024

Your product runs on more than GitLab

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →