Latest blog posts
Martyn Rittman
2024 November 15

Research Integrity Roundtable 2024

For the third year in a row, Crossref hosted a roundtable on research integrity prior to the Frankfurt book fair. This year the event looked at Crossmark, our tool to display retractions and other post-publication updates to readers. Since the start of 2024, we have been carrying out a consultation on Crossmark, gathering feedback and input from a range of members. The roundtable discussion was a chance to check and refine some of the conclusions we’ve come to, and gather more suggestions on the way forward.

...Find out more
Dominika Tkaczyk
2024 November 06

How good is your matching?

In our previous blog post in this series, we explained why no metadata matching strategy can return perfect results. Thankfully, however, this does not mean that it’s impossible to know anything about the quality of matching. Indeed, we can (and should!) measure how close (or far) we are from achieving perfection with our matching. Read on to learn how this can be done! How about we start with a quiz? Imagine a database of scholarly metadata that needs to be enriched with identifiers, such as ORCIDs or ROR IDs.

...Find out more
Kornelia Korzec
2024 October 28

Update on the Resourcing Crossref for Future Sustainability research

We’re in year two of the Resourcing Crossref for Future Sustainability (RCFS) research. This report provides an update on progress to date, specifically on research we’ve conducted to better understand the impact of our fees and possible changes. Crossref is in a good financial position with our current fees, which haven’t increased in 20 years. This project is seeking to future-proof our fees by: Making fees more equitable Simplifying our complex fee schedule Rebalancing revenue sources In order to review all aspects of our fees, we’ve planned five projects to look into specific aspects of our current fees that may need to change to achieve the goals above.

...Find out more
Lucy Ofiesh
2024 September 24

Meet the candidates and vote in our 2024 Board elections

On behalf of the Nominating Committee, I’m pleased to share the slate of candidates for the 2024 board election. Each year we do an open call for board interest. This year, the Nominating Committee received 53 submissions from members worldwide to fill four open board seats. We maintain a balanced board of 8 large member seats and 8 small member seats. Size is determined based on the organization’s membership tier (small members fall in the $0-$1,650 tiers and large members in the $3,900 - $50,000 tiers).

...Find out more
  • Menu
search

Documentation

Metadata Plus snapshots

Metadata Plus snapshots provide access to our 160,104,382 metadata records in a single file, providing an easy way to retrieve an up-to-date copy of our records. The files are made available via a /snapshots route in the REST API which offers a compressed .tar file (tar.gz) containing the full extract of the metadata corpus in either JSON or XML formats.

How to access snapshots

New snapshots are created each month, available by the 5th day, providing all records up to and including the previous month.

If you’re looking for the most up-to-date snapshot (all records up to and including the previous month), you can use the following URLs which will always alias to the current month:

  • JSON output: https://api.crossref.org/snapshots/monthly/latest/all.json.tar.gz
  • XML output: https://api.crossref.org/snapshots/monthly/latest/all.xml.tar.gz

If you want to test to see if a particular snapshot is available, you can do a HTTPS HEAD request using the following URL patterns:

  • JSON output: https://api.crossref.org/snapshots/monthly/{YYYY/MM}/all.json.tar.gz
  • XML output: https://api.crossref.org/snapshots/monthly/{YYYY/MM}/all.xml.tar.gz

Please note that XML snapshots are available in UNIXSD format only.

As snapshots are available to Metadata Plus users only, you will need to identify yourself in the request by using a “Crossref-Plus-API-Token” HTTPS header with your access token. The example below shows how this should be formatted, with XXX replaced by your token:

Crossref-Plus-API-Token: Bearer XXX

The files will be very large (>42GB) so may take a while to download depending on the speed of your internet connection.

Please contact us if you’re unable to access snapshots.

Keeping your data current

For applications where you want to keep a copy of our metadata records current, use OAI-PMH Plus (as described above) or the REST API to query for new records at your preferred interval.

Snapshots FAQs

Are snapshots for ‘all time’ available?

Snapshots are available for current and previous quarters. With each new snapshot, we may remove files older than the current and previous quarters. For example, on 1 April the files from the previous October, November, and December may be removed.

I’m seeing a 404 error when I request the URL

If you’re looking for the current month, this may be because the archive hasn’t yet been created for that month. Snapshots are usually available by the 5th of each month.

If you’re looking for a month that’s more than 6 months old, it may be that the snapshot has been deleted. If the archive you looking isn’t particularly new or old and you’re still seeing a 404 error, please contact us.

I’m seeing a 401 error when I request the URL

Snapshots are only available to Metadata Plus users. This 401 message means that the system doesn’t recognise you as a Metadata Plus user. If you’re already a Metadata Plus user, make sure you’re using your correct token in the header of your query. If you’re still having problems, please contact us.

I need a full snapshot mid-month

Snapshot archives are provided at the start of each month. The archive contains all the registered content received by Crossref up until that time. (Really? Yeah, all of it.) If you need a snapshot mid-month, you should download and ingest the latest archive and then harvest and ingest the registered content that has changed since then.

To get the registered content that has changed since an archive was created, use OAI-PMH Plus or the REST API. For example, if the archive was created on January 31, 2018 then the OAI-PMH Plus harvest’s initial URL is

https://oai.crossref.org/oai?verb=ListRecords&set=J&from=2018-01-31&metadataPrefix=cr_unixsd

This will harvest journal data. If you are interested in book data then use the “B” set.

https://oai.crossref.org/oai?verb=ListRecords&set=B&from=2018-01-31&metadataPrefix=cr_unixsd

If you are interested in series data then use the “S” set.

https://oai.crossref.org/oai?verb=ListRecords&set=S&from=2018-01-31&metadataPrefix=cr_unixsd

It is important to use the created date and not the completed date. It takes time to build the archive, so changes will occur during the build. Using the created date ensures those changes are harvested too.

Page owner: Patrick Polischuk   |   Last updated 2020-April-08