Crossref incident

Metadata indexing failure and instability on the REST API (Starting Friday 12th 20:00 UTC and continuing) affecting several other tools

Resolved after 17 days 6 hours, affecting Public REST API and Polite REST API and 1 more, on September 15, 2025.

resolved Oct 02, 2025, 03:31 PM UTC

We have updated the service that synchronises new and updated metadata records with the REST API; all three pools of the REST API are performing well as a result (as are the services that run on it). Thus, we are resolving this incident. -IF

Started
Sep 15, 2025, 09:13 AM UTC
Resolved
Oct 02, 2025, 03:31 PM UTC
Duration
17d 6h
Detected by Pingoru
Sep 15, 2025, 09:13 AM UTC

Affected components

Public REST APIPolite REST APIMetadata searchParticipation reportsCrossmark dialog server

Update timeline

  1. investigating Sep 15, 2025, 09:13 AM UTC

    We have found that some records failed to be indexed by the REST API on Friday 12th September, and continuing. Our technical team is investigating.

  2. investigating Sep 15, 2025, 04:33 PM UTC

    We are continuing to investigate this issue.

  3. investigating Sep 16, 2025, 02:14 PM UTC

    Our technical team is re-pushing all of the records that have failed to be indexed since Friday, 12 September. Those records should be indexed in the REST API in the next 24 hours. -IF

  4. monitoring Sep 17, 2025, 12:48 PM UTC

    A fix has been implemented and we are monitoring the results. Do to the heavy workload of indexing a large volume of records yesterday, we have observed longer response times in the Public and Polite pools of the REST API. We believe those response times should improve throughout the day. -IF

  5. monitoring Sep 19, 2025, 10:15 AM UTC

    The public and polite pools are still showing as unstable with long response times and timeouts occurring. We are still looking into this as a priority and will post with more information as we know it.

  6. monitoring Sep 22, 2025, 07:08 PM UTC

    We are continuing to monitor for any further issues.

  7. identified Sep 23, 2025, 01:26 PM UTC

    The public and polite pools are still experiencing some instability, which is affecting downstream tools such as Participation Reports, Search, and Crossmark. We're working on a fix incl. infrastructure upgrades that should help long-term. -GH

  8. identified Sep 26, 2025, 01:59 PM UTC

    We've identified and fixed one issue which was affecting citation counts. We're still working on two additional tasks: upgrading Elasticsearch in the REST API and updating the service that synchronises new and updated metadata records with the REST API. -GH

  9. resolved Oct 02, 2025, 03:31 PM UTC

    We have updated the service that synchronises new and updated metadata records with the REST API; all three pools of the REST API are performing well as a result (as are the services that run on it). Thus, we are resolving this incident. -IF

Your product runs on more than Crossref

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →