Dataclay incident

QUE API — some 504 gateway timeouts

Resolved after 1 day 22 hours, affecting API, on May 14, 2025.

resolved May 16, 2025, 03:42 PM UTC

After investigation, we resolved an issue causing slowdowns and some timeouts in job logging queries. We also added rules to mitigate a series of malformed requests that were not successfully connecting to QUE endpoints, but were also slowing or causing some timeouts for QUE user clients. This appears to have resolved the issues we were seeing. If you continue to experience any connection timeouts, please contact Dataclay support.

Started
May 14, 2025, 05:07 PM UTC
Resolved
May 16, 2025, 03:42 PM UTC
Duration
1d 22h
Detected by Pingoru
May 14, 2025, 05:07 PM UTC

Affected components

API

Update timeline

  1. investigating May 14, 2025, 05:07 PM UTC

    We are investigating some QUE API gateway timeouts on the /jobs endpoint. We will update as we discover more information.

  2. monitoring May 15, 2025, 02:47 PM UTC

    After investigating the timeout issue with the QUE API endpoints, we have identified two possible causes. We have issued a fix and are monitoring activity.

  3. resolved May 16, 2025, 03:42 PM UTC

    After investigation, we resolved an issue causing slowdowns and some timeouts in job logging queries. We also added rules to mitigate a series of malformed requests that were not successfully connecting to QUE endpoints, but were also slowing or causing some timeouts for QUE user clients. This appears to have resolved the issues we were seeing. If you continue to experience any connection timeouts, please contact Dataclay support.

Your product runs on more than Dataclay

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →