2017 on Crossref

Metadata and integrity: the unlikely bedfellows of scholarly research

Damian Pattinson — Thu, 14 Dec 2017 00:00:00 +0000

I was invited recently to present parliamentary evidence to the House of Commons Science and Technology Select Committee on the subject of Research Integrity. For those not familiar with the arcane workings of the British Parliamentary system, a Select Committee is essentially the place where governments, and government bodies, are held to account. So it was refreshing to be invited to a hearing that wasn’t about Brexit.

The interest of the British Parliament in the integrity of scientific research confirms just how far science’s ongoing “reproducibility crisis” has reached. The fact that a large proportion of the published literature cannot be reproduced is clearly problematic, and this call to action from MPs is very welcome. And why would the government not be interested? At stake is the process of how new knowledge is created, and how reliable that purported knowledge is.

The other issue driving this overview of research practices are the cases of deliberate fraud and wrongdoing that have recently created headlines (e.g., the STAP papers concerning the reprogramming of stem cells). While these cases are clearly dramatic outliers, they nevertheless serve to diminish public confidence in scholarly research and the findings that come out of this enterprise.

As with most inquiries, the question quickly boiled down to: who is to blame? As Bill Grant MP asked me directly, “Where does the responsibility lie?”

My answer was lifted from an article by Ginny Barbour and colleagues in F1000Research this November (https://doi.org/10.12688/f1000research.13060.1): publishers are responsible for the integrity of the published literature, while institutions and employers are ultimately responsible for the conduct of their staff. Misconduct entails intent, usually to deceive the reader into believing a conclusion that the researcher wishes them to believe. But journal editors can never know, and are not in a position to investigate, whether a researcher has deliberately falsified their data.

However, there are things that publishers can do to ensure high standards of integrity. Much of this involves making a study’s authors publish as much information about what they have done as possible - the more the reader can see of how data were generated, the more that reader can trust the findings communicated in the published article.

Article metadata directly supports this function. It provides structure and transparency to information pertaining to ethics and integrity. And because metadata is independent of the main article, it can be readable even if the article itself is locked behind a paywall.

Crossref already provides metadata that can demonstrate the integrity of published articles. The metadata collected on 91+ million scholarly works across publishers and disciplines is open and freely accessible to all. Bibliographic information, for example, allows readers to see who the authors of the article are, where they are from, and what else they have published. Similarly, funding data allows readers to identify potential conflicts of interest, for example if the funder has commercial or political affiliations. Even if the reader cannot see the conflict of interest statement (or if the journal has not provided one), they can use the funding statement to surface potential conflicts.

And if they wanted, publishers could provide additional metadata to add still more transparency to the research process. Ethical approval by institutional review boards, for example, could be captured, and any protocol numbers traced back to the original ethics committee approval. At present the process of ethical approval varies from country to country, and from institution to institution. Encouraging authors and journals to deposit information on the approval process would both demonstrate the high ethical standards the author is working to, and also improve the standards themselves, since institutions would have to encode their approval processes in a way that is understandable to others. This could pave the way to significantly higher international ethical standards, all through a simple addition to the indexed metadata underlying the scholarly literature.

One key recommendation that I and many others made to the Committee was, in short, “show your work”. As a researcher, that means showing your data. As a publisher, that means showing what checks you have done. In both cases, metadata can help.

A major issue that publishers and researchers can – and should – address is the provision of actual scientific data. Most papers, today, present only the end results of the authors’ (often quite extensive) analyses. The case for sharing data is an obvious one - many recent cases of misconduct could have been identified earlier, or even avoided altogether, if editors and readers had had access to underlying datasets.

With images, a requirement to submit raw images alongside the edited figures would dramatically reduce the cases of manipulation that are rife in the literature (studies suggest up to 20% of papers have some kind of inappropriate figure manipulation, with around 1 in 40 papers showing manipulation beyond that which can be expected to be a result of error). Similarly, providing the numbers that a paper’s analyses are based upon would allow readers to fully assess if datasets are distributed as would be expected through random sampling, and, if they choose, to determine if the data are sufficient to support the statistical inferences made in the paper. The Crossref schema – by providing unique identifiers to data citations - makes this link between data and paper possible. (See the recent blog post on the Research Nexus for more information.)

For publishers, showing your work also means being transparent to your readers about the editorial checks that a manuscript has undergone. Crossref has a tool that enables this editorial transparency: it’s called Crossmark. Crossmark allows readers to see the most up-to-date information about an article, even on downloaded PDFs. In most cases it is used to show whether the version of an article is most recent one, or whether any corrigenda or retractions have been subsequently added. But it can also be used to provide whatever information a publisher wishes to share about the paper. Some journals have experimented with using Crossmark to ‘thread’ publications together, for example, by linking all the outputs generated from a single clinical trial registration number (blog post here). But publishers could go further and display metadata pertaining to the editorial checks they have performed on a paper. So Crossmark could tell readers that the paper has been checked for plagiarism, or figure manipulation, or reporting standards such as CONSORT or ARRIVE guidelines. Here at Research Square we have been addressing this with a series of Badges that researchers can apply to their papers to demonstrate what checks have been performed.

Together, these implementations would provide value to the reader, who can see exactly what has been checked, and to the publisher, who can show how rigorous their editorial processes are. It would also serve to highlight the integrity of the authors who have passed all of these checks.

Research integrity is not something that can be easily measured but, unlike wit or charm, it is something that people generally know that they have.* This means that they just need to be transparent in their output to demonstrate this to the world. Metadata provides a simple way of doing this, so researchers and publishers should make sure they provide it as openly as they can.

*with apologies to Laurie Lee for the mangled quote

Dr. Livingstone, I presume…a two month expedition deep into the heart of research publishing

Amanda Bartell — Wed, 13 Dec 2017 00:00:00 +0000

Hello there. I’m Amanda Bartell, and I joined the Crossref team in mid-October as the new Head of Member Experience. My new Member Experience team will be responsible for metadata users as well as members, onboarding new accounts, supporting existing ones, and making sure that everyone can make the most of Crossref services - an an easy and efficient way. I have spent the last couple of months exploring the world of academic publishing and what our members need - and it’s been fascinating!

Expedition members

The new Member Experience team is made up of some people who are new to Crossref and Scholarly Publishing and some whose names you’ll probably recognize!

Anna Tolwinska (Member Experience Manager) will support existing members in understanding the quality of metadata they deposit with us, and how they can best make use of our other products and services.
Paul Davis (Product Support Specialist) and Shayn Smulyan (Product Support Associate) will continue to provide excellent technical support to all creators and consumers of our metadata.
Gurjit Bhullar (Membership Coordinator) will help new applicants who want to join Crossref understand the member obligations and have a smooth induction journey.

We’ll be expanding the team in 2018 to support you further - watch this space!

What a diverse ecosystem

My background is educational publishing, so this has been my first foray into the world of scholarly publishing. In my first few months I’ve been lucky enough to attend three very different events with Crossref - Frankfurt Book Fair, our annual meeting (LIVE17) in Singapore, and an OpenCon event in Oxford. Each one has given me the chance to talk to our members and other constituents, and I’ve been really struck by what a diverse bunch you are: from small volunteer-led society journals through universities to commercial behemoths; from Albania to Zambia (and 125 countries in between); covering everything from Ancient History to X-Ray Spectrometry.

Expedition equipment to suit the climate

This diversity gives my team a huge responsibility. We need to make sure that the support we provide to you can meet the needs of everyone - whether you’re a multinational publisher with a large team of xml specialists, or a small team of enthusiastic academics. Everyone should be able to clearly understand and take advantage of what Crossref offers both to you as an organisation and to the wider scholarly community.

With this in mind, we’re going to be making a few changes to the support materials we provide over the next 12 months—rewriting them so they’re clearer for everyone, re-structuring our support center so there’s a separate route through depending on your level of technical expertise and closer links with our main website, plus providing support in different languages and different formats.

Sticking together in a harsh environment

As someone who has previously worked in commercial publishing, something else that has struck me about working in a member organisation is the difference between members and traditional “customers”. It’s been fantastic to see how involved many of you are in Crossref. From taking part in our various committees and working groups, to helping to organize LIVE Local events, to attending webinars and training, it’s obvious that you feel a real sense of ownership over Crossref and our shared mission.

We’re hoping to make use of that great sense of community in 2018 by improving our member center, giving you more access to see the level of metadata you’re sharing with the community (and that others are sharing) and providing more options for you to communicate with, and support each other. We’re also going to be improving the education we offer for new members, to make sure that everyone is aware of the joint mission we all have to improve research communications. Most long time members know it’s so much more than just having a DOI, and we need to make sure that our new members are aware of this too and share our vision.

Leaving no-one behind

We have a lot of plans for the Member Experience team in 2018, but it’s key that everything we do meets the needs of all our members. If you have any suggestions for how we can improve your member experience, [do let me know](mailto:feedback@crossref.org?subject=Member Experience suggestion).

Global Persistent Identifiers for grants, awards, and facilities

Ginny Hendricks — Wed, 13 Dec 2017 00:00:00 +0000

Crossref’s Open Funder Registry (neé FundRef) now includes over 15 thousand entries. Crossref has over 2 million metadata records that include funding information - 1.7 million of which include an Open Funder Identifier. The uptake of funder identifiers is already making it easier and more efficient for the scholarly community to directly link funding to research outputs, but lately we’ve been hearing from a number of people that the time is ripe for a global grant identifier as well.

To that end, Crossref convened its funder advisory group along with representatives from our collaborator organisations, ORCID and DataCite, to explore the creation of a global grant identifier system.

We thought you might like to know about what we’ve been discussing…

The First Rule of Grant Identifiers

The first rule of grant identifiers is that they probably should not be called “grant identifiers”. Research is supported in a variety of ways—through grants, endowments, secondments, loans, use of facilities/equipment and even crowd-funding. In any of these cases, it is important to be able to link researchers and research outputs to details about the sources of support. This is true for prosaic reasons—to understand ROI, to map the competitive landscape, to ensure that mandates are fulfilled, to avoid double payment. But it is also true for epistemic reasons; understanding how research was funded can help contextualise that research, and help expose potential conflicts of interest or specific agendas.

The Open Funder Registry which provides a coarse mapping between research outputs and funders, but it is becoming clear that we need more fine-grained mapping directly to information about the kind of support that was provided.

Awkwardly, none of us had any great ideas about alternative nomenclature, so we’ve made the eminently practical decision to continue to use the term “grant identifier” whilst being aware that our aim is to define a system that applies more broadly to any form of funding or support of research. So +1 for practicality.

Why do we need an open, global, grant identifier?

With the steady increase in research outputs, and the growing number of active researchers from both academia and industry, research stakeholders find they need to be able to automate workflows in order to scale their systems efficiently. Funders want to be able to track the outputs that arise from research they have funded. As a result, institutions find themselves having to regularly analyse and summarise the research their faculty produces. Faculty, in turn, face increasing accounting bureaucracy in order to meet all the reporting requirements that are cascading through the system. And finally, publishers are seeking to make the manuscript submission and evaluation process more efficient as well as to increase the discoverability and contextual richness of their publications.

Most funders already have local, internal grant identifiers. But there are over 15K funders currently listed in the aforementioned Open Funder Registry. The problem is that each funder has its own identifier scheme and (sometimes) API. It is very difficult for third parties to integrate with so many different systems. Open, global, persistent and machine-actionable identifiers are key to scaling these activities.

We already have a sophisticated open, global, interoperable infrastructure of persistent identifier systems for some key elements of scholarly communications. We have persistent identifiers for researchers and contributors (ORCID iDs), for data and software (DataCite DOIs), for journal articles, preprints, conference proceedings, peer reviews, monographs and standards (Crossref DOIs), and for Funders (Open Funder Registry IDs).

And there are similar systems under active development for research organisations, conferences, projects and resources reported in the biomedical literature (e.g. antibodies, model organisms). At a minimum, open, persistent identifiers address the inherent difficulty in disambiguating entities based on textual strings (structured or otherwise). This precision, in turn, allows automated cross-walking of linked identifiers through APIs and metadata which enable advanced applications.

For example, the use of identifiers can simplify user interfaces and save users time. Almost everybody in scholarly communications spends a frustrating portion of their lives copying information from one system to another. This process is not just tedious, it is also error-prone. But we are increasingly seeing systems make use of identifiers to eliminate the need for a lot of this manual copying. For example, researchers using an ORCID iD when they submit a manuscript can start to expect that their relevant ORCID biographical data will simply be imported into the manuscript tracking system so that it doesn’t have to be manually copied over. And if said researcher has their manuscript accepted, they can also expect that their ORCID record will automatically be updated with the publication information and that their institution and/or their funder can be automatically notified of the impending publication so that relevant repositories and CRIS systems can be populated automatically.

Additionally, there is a growing list of services that have been built on top of these standard identifiers. Profile systems (e.g. VIVO, Impact Story, Kudos) can automatically retrieve the latest information from a researcher’s ORCID record. Bibliographic management tools (EasyBib, Zotero, Papers) allow researchers to cite content with the latest metadata. And similarity checking services can harvest and index the latest scholarly literature for inclusion in the tools they have developed for detecting plagiarism and fraud. Funder identifiers are already playing an important role in this metadata workflow. As of November 2017, there are 1.7 million Crossref publication DOIs that are explicitly linked to an Open Funder Registry ID. These linkages serve as a foundation for initiatives like SHARE, CHORUS, and the Jisc Publications Router. But there are another 1+ million records that have funding information without an associated ID and, of course, 90+ million records that have no funding information at all.

So If we have global funder identifiers and they are already working, why do we need global grant identifiers as well? Don’t we just need to increase uptake of funder identifiers? How will grant identifiers help?

First, global grant identifiers could greatly reduce the UX complexity of gathering funder information. This, in turn, would boost the collection of funding information from researchers and ensure that the information that they provide to publishers, institutions and other funders is accurate and complete.

Second, the introduction of global grant identifiers would further increase the utility of links between research outputs and funding information. A grant identifier provides more granular information about the funding. Instead of just linking to information about the funder, a grant identifier would allow linking research outputs to particular research programs along with the information relating to those programs, such as grant durations, award amounts, etc. It would also allow analysis of relationships between multiple co-funding bodies.

To DOI or not to DOI?

Clearly, we think DOIs are pretty good things. But we also aren’t zealots. Sometimes DOIs are appropriate and sometimes they are not. For example, we were instrumental in defining the structure of the ORCID identifier and, in that case, we decided that DOIs were not appropriate.

But in the case of a global grant identifier system, we think there are a number of reasons adopting DOIs would be useful:

It is easy to “overlay” the global DOI system onto existing local identifier systems. An organisation does not need to abandon their internal identifier scheme in order to use DOIs. They can instead incorporate their local scheme into the DOI structure via the simple mechanism of prepending their existing identifiers with an assigned DOI prefix and registering relevant metadata with a DOI registration agency like Crossref or DataCite.
DOI links are “persist-able”. That is they can resolve to different online locations even if domain names change and/or the DNS system itself is replaced. This characteristic is important for a grant identifier because funding agencies - particularly government funding agencies - tend to undergo frequent reorganisations (e.g. splitting, merging, restructuring) and renaming. An indirectly resolvable identifier like a DOI (or ARK, Handle, etc.) is critical to ensure the long-term integrity of identifiers in these situations.
There are 15K+ funders currently listed in the Open funder Registry. Each has their own grant identifier scheme and different levels of technical support for them (APIs, etc.). This makes it very difficult for 3rd parties to build tools that work “generically” with grant identifiers. But once a local identifier scheme had been “globalised” by making it a DOI, third parties can build tools without having to worry about the differences between individual funder systems.
Crossref and DataCite DOIs are deeply embedded in the tools and workflows of scholarly communications. Manuscript tracking systems, bibliographic management systems, metrics systems, CRIS systems, profile systems, etc. often have built-in mechanisms for consuming and making use of DOIs and their associated metadata.
Crossref and DataCite DOIs are cross-disciplinary. They are used in the humanities, social sciences, sciences and in a host of communities that frequently interact with the scholarly literature for example- NGOs, IGOs, patent systems, and standards bodies.
Crossref and DataCite provide a variety of APIs (e.g. REST, OAI-PMH) and services (e.g. search, Crossmark, Similarity Check, Scholix) built around DOIs.
DOI’s have a useful characteristic, which is that the “prefix” of a DOI can be used to determine who originally created the record with which the DOI is associated. In the case of grant identifiers, this means that the prefix of a DOI-based grant identifier could be used to automatically determine the correct funder responsible for the initial grant. This means that the UIs for entering funder/grant information could be both simplified and made more robust—which would likely increase the number of parties that collect and propagate id-based funder information.

But the use of DOIs as the basis for grant identifiers also introduces some potential barriers to adopting a standard funding identifier. For example:

Funders would need to be able to join a suitable DOI registration agency (e.g. Crossref, DataCite). Some funders (e.g. government agencies) may be restricted in their ability to “join” external organisations.
Funders would need to be able to create new DOIs and register associated metadata with their chosen registration agency in a timely manner. Some funders may be unable to generate metadata or may not have the technical capacity to automatically register metadata.
Funders would need to be able to provide an openly available (e.g. not behind access control) online resource to which the DOI would resolve. For example, a landing page describing the grant or a digital copy of the grant itself. Again, some funders may face technical barriers to providing an online resource to resolve to. In other cases there may be privacy or security reasons for not providing an open resource to which a DOI can resolve.

Still, the advisory group consensus has been that these barriers are generally surmountable. Most of the questions they had revolved around understanding what a DOI-based workflow would look like from the funder’s perspective, and so we outlined the steps a funder would need to take in order to adopt DOI-based global identifiers.

The DOI-based grant identifiers workflow

A funder registering metadata and creating DOIs for grants would need to support the following workflow:

When a grant is submitted, the funder would assign their own internal identifier for tracking, etc. For example 00-00-05-67-89.
If the grant is accepted, the funder would:

generate a global public identifier for the grant based on the DOI. For example, assuming their prefix was 10.4440, then the global public identifier might become https://doi.org/10.4440/00-00-05-67-89.
create a “landing page” on their website (or wherever they make their grants available online) to which the global public identifier will resolve. The landing page would display a TBD set of metadata describing the grant, as well as a link to the grant itself.
register the generated DOI and a TBD set of metadata with their registration agency (RA) (e.g. Crossref or DataCite). This metadata would include the URL of the landing page defined above.

Once metadata and DOIs are registered with an RA, the funder would have a series of ongoing obligations:

Update locations: If the location of the landing page changes (for example, because of a site restructuring, merger of split of the funding organisation, etc.), the funder would need to update their metadata records to point the DOI to the new location.
Update metadata: If metadata becomes out-of-date (e.g. the status of a grant changes, additional grant-related metadata is added, etc.), the funder would update the relevant records.
Promote the use of the the DOI as the preferred global, public identifier for the grant. That is - the one that people should use when referring to or citing the grant (the funder can continue to use the original local identifier for their internal systems, etc.).

Again, the advisory group thought that this workflow seemed tractable and agreed that the best way to ensure that would be to proceed to creating a working pilot of a global grant identifier system based on the DOI.

Next steps

Crossref is starting a grant identifier pilot. We will create two sub-groups of the funder advisory group.

Group for “Governance, membership, and fees”

This group will look at governance and financial issues raised by the introduction of grant identifiers. For example, it will look at whether Crossref’s membership model works as is or might need to be adjusted in order to accommodate a new constituency. We know, for example, that some funders find it hard to become “members” of organisations. We might need to create other participation categories in order to accommodate these restrictions. Similarly the group will look design a pricing model of DOIs for grants in order to make sure that they cover the costs of modifying and sustaining the system for them, as well as to ensure that the pricing incentivises funders to participate. This sub-group will work closely with Crossref’s membership and fees committee.

Group for “Technical and metadata”

This group will look at any technical changes that need to be made to registration process in order to accommodate the new participants. If there are, they are likely to center around specific metadata requirements for grants. As such, the group will likely spend most of its time agreeing to a practical metadata schema for capturing relevant information about the myriad of ways in which organisations support research. This group will also liaise with other relevant technical working groups, such as those who are looking at organisational identifiers and conference identifiers.

The two sub-groups will first meet in January and, after a few meetings, will report back the advisory group with recommendations. Using these recommendations, we will develop an implementation plan which will include testing the infrastructure, testing metadata deposits, fee modelling, etc, with a small group of participants.

If you are a funder, and you would like to have somebody from your origanization participate in one of these working groups, please contact Ginny Hendricks. Note that joining the above groups does not commit you to anything other than engaging in the discussion. We want to make sure we create a system that works for a range of funders, not just those who can start testing something right away.

And our survey says...

Christine Cormack Wood — Mon, 11 Dec 2017 00:00:00 +0000

Earlier this year we sent out a short survey inviting members to rate our performance. We asked what you think we do well, what we don’t do so well, and one thing we could do to improve our rating.

We were delighted to receive 313 responses and relieved that 93% of those were positive (phew!). It was very useful to hear your thoughts and to get such a variety of comments covering Product, Outreach, Marketing and Member Experience. There were a few recurring themes, three of which we’d like to address here:

1. Providing information in different languages

Not surprisingly, given the growing diversity of our member base, some respondents asked us to share information in languages other than English. We have been aware of this growing need for some time and have been working on a few developments in this area:

In January 2018 we will be launching a series of seven service videos in six different languages—French, Spanish, Brazilian Portuguese, Chinese, Korean, and Japanese.
January also sees the launch of a new initiative called the Ambassador Program. Ambassadors will work closely with Crossref to help spread the word about our services, and support our global members in their own languages.
During 2017 we hosted two webinars in Brazilian Portuguese and one in Turkish, and aim to increase this in 2018.

2. Member-to-member discussion forum

Some respondents asked for a facility to enable members to reach out to each other, giving direct opportunity for discussions and/or sharing experiences online (and in their own languages). We have been working for a few months now to provide a member-to-member discussion area, which is planned for 2018. Following a soft launch covering a few areas/topics, we’ll broaden the scope to include technical support, too.

3. Registering metadata more easily using the web deposit form

Many respondents requested a more user-friendly process for registering metadata through our webform. Our Product and DevOps teams have been working on this for some time and have created a new interface called the Metadata Manager, which is currently in Beta but scheduled to launch in Q1 of 2018.

Finally, we’d like to thank you for participating in our survey. Your valuable feedback and suggestions help us understand your experience, improve our service, shape the course of particular projects and even direct our future strategy.

As this survey was anonymous, we are unable to respond to anyone on an individual basis, however, if you’d like to have your particular comments addressed, we would love to hear from you directly.

Working with universities at Crossref LIVE Yogyakarta

Rachael Lammey — Mon, 11 Dec 2017 00:00:00 +0000

Following on from our LIVE Annual Meeting in Singapore, my colleague, Susan Collins, and I held a local LIVE event in Yogyakarta thanks to support from Universitas Ahmad Dahlan (UAD), Universitas Muhammadiyah Sidoarjo and one of Crossref’s new Sponsoring Affiliates, Relawan Jurnal Indonesia.

Over the past two years, we’ve seen accelerated growth in our membership in Asia Pacific (making up a quarter of all new members in the last two years). A lot of those new members have come from Indonesia, so it was great to have the opportunity to meet up, answer questions and to share knowledge between all our different organisations.

We welcomed speakers such as Dr. Muhammad Dimyati, from the Directorate General of Strengthening for Research and Development, Ministry of Research, Technology and Higher Education. Dr. Dimyati talked about the importance of Indonesian research and presented statistics on its growth, but also its coverage in different databases like Scopus and DOAJ.

Dr. Lukman from LIPI, the Indonesian Institute of Sciences also joined us to explain the importance of identifiers within the research ecosystem. As any identifier buff will know, we’re keen to talk more about how organisations are using Crossref metadata and identifiers, and the importance of providing good, complete metadata (Metadata2020) so this, plus a remote presentation from Nobuko Miyari from ORCID helped provide great context for the day.

Metadata and identifiers are of course just one part of the process, and Mr. Tole Sutikno from UAD gave an overview of good practice publishing by looking at some of the wider issues that journal editors (and researchers) need to know.

We had time in the afternoon to talk to our audience about Crossref - our different services, OJS integrations, funding data and our APIs, and thanks to our moderators we were able to take lots of questions from members who had specific questions about Crossmark, Cited-by and depositing references.

A few weeks later, and I’m still absorbing all of the things that happened on our (too) quick trip to Yogyakarta.

Thanks again to our members and hosts for attending the event and sharing their questions, ideas and plans with us, and we plan to come back to continue to build on these in future.

The PIDapalooza lineup is out; come rock out with us at the open festival of persistent identifiers

Ginny Hendricks — Fri, 01 Dec 2017 00:00:00 +0000

PIDs’R’Us and if they’re you, too, please join us for the second PIDapalooza, in Girona, Spain on January 23-24, for a two-day celebration of persistent identifiers.

Together, we will achieve the incredible - make a meeting about persistent identifiers and networked research fun! Brought to you by California Digital Library, Crossref, DataCite, and ORCID, this year’s sessions are organized around eight themes:

PID myths
Achieving persistence
PIDs for emerging uses
Legacy PIDs
Bridging worlds
PIDagogy
PID stories
Kinds of persistence

The program is now final and there really is something for everyone (well, every PID geek)

Hmm, Do Researchers Need to Care about PID Systems? Excellent question.
We’ll hear Stories from the PID Roadies: Scholix.
Nevermind the The Bollockschain and other PID Hallucinations.
An intriguing session on #ResInfoCitizenshipIs?.
There will be a plenary by Johanna McEntyre on As a biologist I want to reuse and remix data so that I can do my research.
And we’ll enjoy another plenary from Melissa Haendel (title to be confirmed).

With half the places already booked, now’s the time to register and plan your trip. We hope to see fellow festival-goers there for some PIDtastic party time (and actually some epic serious conversations).

Contact me via the steering committee at PIDapalooza@datacite.org with any questions, music requests, or backstage passes.

Full lineup

View the Crossref LIVE17 agenda.

Using the Crossref REST API. Part 7 (with CHORUS)

Christine Cormack Wood — Mon, 27 Nov 2017 00:00:00 +0000

Continuing our blog series highlighting the uses of Crossref metadata, we talked to Sara Girard and Howard Ratner at CHORUS about the work they’re doing, and how they’re using our REST API as part of their workflow.

Introducing CHORUS

CHORUS (www.chorusaccess.org) is an innovative non-profit organisation that supports funders, publishers, authors and institutions to deliver public access to articles reporting on funded research. Our vision is to create a future where the output flowing from funded research is easily and permanently discoverable, accessible and verifiable by anyone in the world.

CHORUS currently monitors over 400,000 articles for more than 20 US federal and two international funding agencies, and has partnerships with Department of Defense, Department of Energy, National Science Foundation, National Institute of Standards and Technology, Office of the Director National of Intelligence: Intelligence Advanced Research Projects Activity, Smithsonian Institution, US Department of Agriculture, US Geological Survey, Japan Science and Technology Agency, and the Australian Research Council. CHORUS is supported by over 50 publisher and affiliate members who represent the majority of funded published research.

What problem is your service trying to solve?

CHORUS is the first service of CHOR Inc., founded in 2013 in response to the directive of the US Office of Science and Technology Policy (OSTP) for all US federal research agencies to develop and implement plans to widen public access to publications and data associated with federally funded research.

CHORUS aims to minimize public access compliance burdens and ensure the long-term preservation and accessibility of articles reporting on funded research. We provide the necessary metadata infrastructure and governance to enable a smooth, low-friction interface between funders, authors, institutions and publishers in a distributed network environment. CHORUS’ services track public accessibility of articles regardless of whether they are published Gold OA or made open by the publisher.

Can you tell us how you are using the Crossref REST API at CHORUS?

The Crossref REST API is a key source for the metadata database that powers the CHORUS Dashboard, Search and Reporting services for Funders, Institutions and Publishers.

What metadata values do you pull from the API?

We pull the basic bibliographic information such as publisher, journal title, article title, authors and publication date. Perhaps even more important to our area of focus are the funder, grant and license information.

How often do you extract/query data?

CHORUS uses the Crossref REST API every day.

Can you describe your workflow using Crossref metadata?

Every night we query the Crossref API to send us metadata for all article or conference proceeding records for our member publishers that have funder metadata matching the funders monitored by CHORUS.

CHORUS monitors these DOIs for public accessibility on publisher websites; inclusion in agency search tools; deposit in a growing list of funder repositories (e.g.,US DOE PAGES,NSF PAR, and USGS Publications Warehouse and NIH PubMed Central); and for associated ORCID researcher records. CHORUS also uses the reuse license metadata to identify when an article is expected to be made publicly accessible.

Finally, we check for ingestion in CLOCKSS and/or Portico to ensure long-term preservation and accessibility of research findings reported in journal and proceedings articles. Our preservation partners keep the full text in their dark archives, only making it available when the content may no longer be made publicly accessible by the publisher.

The collected and enhanced metadata is presented in our dashboard, search and reporting services all including links back to the publisher sites via the Crossref DOI.

What are the future plans for CHORUS?

Following the success of our Funder and Publisher Dashboards, CHORUS is expanding the services we provide to international funders, non-governmental funders, and institutions. Our first funder partnership outside of the United States is with the Japan Science and Technology Agency (JST). CHORUS announced its new Institution Dashboard service this Autumn after successfully concluding pilots with the University of Florida and University of Denver. CHORUS will also be adding links to relevant datasets and other metadata utilizing forthcoming identifiers and metadata standards.

What else would you like to see the REST API offer

It would be great to see more identification of funders from Crossref members. While we have seen great leaps since 2013, we all have a long way to go. We are also eager to see Crossref incorporate the organisation Identifiers that they have begun with ORCID, DataCite and others.

Thanks, CHORUS! If you would like to contribute a case study on the uses of Crossref Metadata APIs please contact the Community team.

The research nexus - better research through better metadata

Jennifer Lin — Tue, 14 Nov 2017 00:00:00 +0000

Researchers are adopting new tools that create consistency and shareability in their experimental methods. Increasingly, these are viewed as key components in driving reproducibility and replicability. They provide transparency in reporting key methodological and analytical information. They are also used for sharing the artifacts which make up a processing trail for the results: data, material, analytical code, and related software on which the conclusions of the paper rely. Where expert feedback was also shared, such reviews further enrich this record. We capture these ideas and build on the notion of the “article nexus” blogpost with a new variation: “the research nexus.”

Some of Crossref’s publishing community are encouraging the scholarly communication practices surrounding these tools in a variety of ways: incorporating them into the publishing workflow, integrations between the tools and publishing systems, as well as linking and exposing the artifacts in the publications for readers to access. A special set of publishers have gone all the way and included these links into their Crossref metadata record. They insert them directly into the metadata deposit when they register the content (technical documentation). Doing so, these connections reach further than the publisher platform and propagate to systems across the research ecosystem including places like indexers, research information management systems, sharing platforms (oh, the list goes on!). We highlight a small set of examples to illustrate how these outstanding publishing practices are supporting good research.

1. Linking to an entire collection of methods

Crossref member, Protocols.io, is supporting transparency and methods reproducibility with their open access repository of science methods. Leitão-Goncalves R, Carvalho-Santos Z, Francisco AP, et al. investigated the concerted action of the commensal bacteria Acetobacter pomorum and Lactobacilli in Drosophila melanogaster, demonstrating how the interaction of specific nutrients within the microbiome can shape behavioral decisions and life history traits. Findings were published in PLOS Biology earlier this year: https://doi.org/10.1371/journal.pbio.2000862. Authors deposited detailed methods and protocols used in the project (Drosophila rearing, media preparations, and microbial manipulations) as a collection in Protocols.io: https://doi.org/10.17504/protocols.io.hdtb26n. So Protocols.io registered their content with us, linking the protocol to the paper. This creates the crosswalk between both so that users can get from one to the other through the metadata. The full metadata record can be found here.

2. Linking to video protocol

If a picture is worth a thousand words, the truism might apply to moving pictures many times over. Fasel B, Spörri J, Schütz P, et al. proposed a set of calibration movements optimized for alpine skiing and validated the 3D joint angles of the knee, hip, and trunk during alpine skiing in a PLOS ONE paper: https://doi.org/10.1371/journal.pone.0181446. These movements consisted of squats, trunk rotations, hip ad/abductions, and upright standing. The specific team responsible for designing them (Fasel B, Spörri J, Kröll J, and Aminian K) described the set of calibration movements performed but found videos to be a far more effective way to communicate the technical movements used in their study. They made the visuals available too: https://doi.org/10.17504/protocols.io.itrcem6. So Protocols.io deposited the link between video protocol and paper to the Crossref metadata record (full metadata record).

3. Linking to software and peer reviews

The Journal of Open Source Software (JOSS) is an academic journal about high quality research software across broadly diverse disciplines. Sara Mahar works on the effectiveness of organisations funded by the US Department of Housing and Urban Development to combat homelessness. She collaborated with computational physicist Matthew Bellis to create a python tool for researchers to visualize and analyze data from the Homeless Management Information System:https://doi.org/10.21105/joss.00384. The software was archived in Zenodo: https://doi.org/10.5281/zenodo.13750 and the peer review artifacts were also published. JOSS deposited all these links in the metadata record (found here).

4. Linking to preprint, data, code, source code, peer reviews

Gigascience, published by Oxford University Press, is experimenting with a number of new tools in their mission to promote reproducibility of analyses and data dissemination, organisation, understanding, and use. In a recent paper Luo R, Schatz M, and Salzberg S shared the results of the firstly publicly available implementation of variant calling using a 16-genotype probabilistic model for germline variant detection: https://doi.org/10.1093/gigascience/gix045. Prior to formal peer review, the group posted the preprint in bioRxiv: https://doi.org/10.1101/111393. When the paper was published, the authors made the supporting data available, including snapshots of the test and result data, in a public repository: http://dx.doi.org/10.5524/100316. OUP included this data citation in their Crossref metadata record via the routes recommended in our previous blog post about depositing data citations. The researchers made the code available in Github, and the algorithm is ready for researchers to run on Code Ocean, a cloud-based computational reproducibility platform that allows researchers to wrap and encapsulate the data, code, and computation environment linked to an article: https://doi.org/10.24433/CO.0a812d9b-0ff3-4eb7-825f-76d3cd049a43. For further transparency, expert reviews of the manuscript from the peer review history were published in Publons: http://dx.doi.org/10.5524/review.100737 and http://dx.doi.org/10.5524/review.100738. (As of last month, publishers can register peer reviews at Crossref). The full metadata record contains links to the entire set of materials listed above.

5. Linking to preprint, Code, Docker hub, video, reviews

Narechania A, Baker R, DeSalle R, et al. used bird flocking behavior to design an algorithm, Clusterflock, for optimizing distance-based clusters in orthologous gene families that share an evolutionary history. Their paper was published in Gigascience last year: https://doi.org/10.1186/s13742-016-0152-3. Supporting data, code snapshots and video were published in GigaDB: http://dx.doi.org/10.5524/100247. Code was maintained in GitHub. And authors also created a Docker application for Clusterflock, a lightweight, stand-alone, executable package of the software which includes everything needed to run it: code, runtime, system tools, system libraries, settings (Docker Hub link here). They created a video demo of the algorithm. Publons reviews were published http://dx.doi.org/10.5524/review.100507 and http://dx.doi.org/10.5524/review.100508. Gigascience shared all these assets in their publication, including the link to the original bioRxiv preprint: https://www.biorxiv.org/content/early/2016/03/25/045773). The full metadata record containing these links can be found here.

The Research Nexus: better research through better metadata

These five are just a few exemplary cases showing how publishers are declaring the relationships between their publications and other associated artifacts to support reproducibility and discoverability of their content. We welcome you to check out our overview of relationships between DOIs and other materials for more information. Members who are enriching your publishing pipeline in similar ways, please register these links to make your reach go further. We also welcome everyone to retrieve these relations in our REST API (technical documentation).

A transparent record of life after publication

Madeleine Watson — Wed, 01 Nov 2017 00:00:00 +0000

Crossref Event Data and the importance of understanding what lies beneath the data.

Some things in life are better left a mystery. There is an argument for opaqueness when the act of full disclosure only limits your level of enjoyment: in my case, I need a complete lack of transparency to enjoy both chicken nuggets and David Lynch films. And that works for me. But metrics are not nuggets. Because in order to consume them, you really need to know how they’re made. Knowing the provenance of data, along with the context with which it was derived, provides everyone with the best chance of creating indicators which are fit for purpose. This is just one of the reasons why we built the Event Data infrastructure with transparency in mind.

The transparency problem

For the scholarly community, alternative metrics to citation count (‘altmetrics’) are becoming increasingly popular as they can offer rich and expedited insight into today’s diverse and dynamic research environment. Research artifacts undergo an extended life online as they’re linked, shared, saved and discussed in forums both within and beyond the traditional academic ecosystem. Data on these interactions are initially fragmented and buried within platforms like social media, blogs and news sites. Downstream, there are several value-add services that collate and present that data as a single, aggregated count. We see individual data points like ‘paper X was tweeted 22 times’, and ‘paper X is referenced 16 times on Wikipedia’ being combined, homogenised, weighted and expressed as a single figure, a calculated number serving as a proxy for value. But altmetrics alone don’t tell the whole story, and how they are calculated is not without idiosyncrasy or politics. As we each have our own unique voice and perspective, we need to ensure we understand the lenses through which these metrics are made in order to consume them effectively.

The 2015 Metric Tide report highlighted transparency as one of the five dimensions of responsible metrics. Having access to the context used to create a metric — the provenance of the original data as well as full transparency around its extraction, processing and aggregation — helps consumers to use the data meaningfully and allows for comparison across third-party vendors. But transparency is difficult to achieve when, as the report notes, the systems and infrastructure for collecting and curating altmetrics-style data are fragmented and have limited interoperability.

In the academic community, underlying centralised systems include ORCIDs to identify people and DOIs to identify items. But we’re missing a transparent, centralised infrastructure for describing and recording the relationships between objects and resources¹. These relationships, or links, occur outside publisher platforms and can provide valuable information about the interconnectivity and dissemination of research. Dedicated infrastructure for collecting these relationships would provide a data source for those interested in altmetrics to build upon.

Figure 1.1 Example of some relationships between articles and activity on the web

At Crossref, we call these relationships Events. An Event is the record of a claim made about the existence of a relationship between a registered content item (i.e. a DOI) and a specific activity on the web. Events include:

a DataCite dataset DOI contains a link to a Crossref article DOI
an article was referenced in Wikipedia
an article was mentioned on Twitter
an article has a Hypothes.is annotation
a blog contains a link to an article

In collaboration with DataCite, we are collecting Events for the DOIs registered with our organisations and are making that data available for others in the community to use. This is the Event Data infrastructure, with which we’re plugging the gap in open scholarly relationships infrastructure. https://doi.org/10.64000/kattr-5k219

The Event Data infrastructure

Crossref and DataCite have for many years provided a centralised location for bibliographic metadata and links, and a facility to help our members register Persistent Identifiers (DOIs) for their content. With nearly 100 million DOIs registered with Crossref, we know where research lives. Which got us thinking — could we use these links to find out more about the journey research undertakes after publication? Could we express these interactions as links without any aggregation or counts so it could be maximally reused? And if so, could we then provide this data in an open, centralised, structured format? The answer was yes, subject to some challenges:

Querying for individual DOIs wasn’t scalable for our full corpus of 100 million items, so we had to find something else.
Not everyone uses the DOI link (not a surprise!). Most people will link directly to the publisher’s site. This means we need to look for links using both the DOI and article landing page URLs.
When we find people referring to registered content using its landing page, we find the DOI for that content item so that the link can be referenced in our data set in a stable, link-rot-proof way.
We don’t always know the article landing page URL for every DOI upfront because like many relationships, the one between DOIs and URLs is complicated.

We began by asking the wrong questions and as a result we got the wrong type of data back: instead of returning a record of individual actions, we were returning aggregated counts. Aside from not meeting our use case, aggregation requires the curation of an ever-churning dataset in order to keep totals updated, which is not scalable for the number of DOIs in our corpus.

We soon learnt to ask the right questions. One pivotal change in approach was that instead of counts, we asked instead ‘what activity is happening on Twitter for this article?’. Our data went from ‘DOI X was mentioned 20 times on Twitter as of this date’ to ‘tweet X mentions DOI X on this date’. The data are now represented as a subject-verb-object triple:

Figure 1.2 Triple table.

Ultimately this has allowed us to represent actions like Wikipedia page edits as individual atomic actions (i.e an Event) rather than as a dataset that changes over time. Being open about the provenance of altmetrics with Event Data Crossref Event Data (the Crossref-specific service powered by the shared Event Data infrastructure) has evolved beyond a link store to become a continual stream of Events; each Event tells a new part of the story. Rather than constantly updating an Event whenever a new action takes place, we add a new one instead:

Figure 1.3 A Wikipedia Event.

Events answer a whole range of questions, such as:

What links to what?
How was the link made?
Which Agent collected the Event?
Which data source?
When was the link observed?
When do we think the link actually happened?
What algorithms were used to collect it?
Where’s the evidence?

We’re collecting data from a diverse range of platforms including Twitter, Wikipedia, blogs and news sites, Reddit, StackExchange, Wordpress.com and Hypothes.is. This means that when we observe a link in these platforms to what we think is a DOI, we create an Event and a corresponding Evidence Record to represent our observation. We also have Events to represent the links between research items registered with Crossref and DataCite - for example, when a Crossref DOI cites a DataCite DOI and vice versa.

The provenance of the data is fully transparent and is made available to everyone via an open API. We call this the evidence trail. The record of each link (‘Events’) as well as the corresponding evidence can then be used to feed into tools for impact measurement, discoverability, collaboration and network analysis.

Therefore, one application of Event Data is as an underlying, transparent data source for altmetrics calculations. For example, you might want to know the total number of times your paper has been mentioned on Twitter to date. If I told you that the number was 22, what does that actually mean? Do you know whether I counted both tweets and retweets? Do you consider both of these actions as equal? Is the sentiment of the tweet important to you? Was it a human or a bot that initiated a tweet? Are you interested in tweets containing links to multiple representations of your paper or do you only want to track mentions of your version of record (the final published copy)? With Event Data as your underlying data source, you can answer these questions.

Not only transparent in data, transparent by design

The National Information Standards Organisation (NISO), a US organisation responsible for technical standards for publishing, bibliographic and library applications, has developed a set of recommendations for transparency in their Alternative Assessment Metrics Project report, as well as a Code of Conduct for both altmetric practitioners and aggregators that aims to help improve the quality of altmetrics data. The working groups recognised that without transparency and conforming to a recognised standard, altmetric indicators “are difficult to assess, and thus may be seen as less reliable for purposes of measuring influence or evaluation”¹.

Crossref Event Data is one of the example altmetric data providers listed in the NISO recommendations. My colleague Joe Wass participated in the development and specification of the NISO “Altmetrics Recommended Practices on Data Metrics, Alternative Outputs, and Persistent Identifiers” at the same time as we were working with DataCite on Event Data, so they have mutually informed one another.

Figure 1.4 Martin Fenner (DataCite) and Joe Wass (Crossref) drawing plans for the Event Data infrastructure.

The outcome of our involvement in the NISO recommendations is that Crossref Event Data is a service that is transparent by design. We have opened up our entire extraction and processing workflow so that we can clearly demonstrate the context and environment that was used to generate an Event. This evidence is a core component of our transparency-first principle.

Building services on Event Data

There are some really exciting ways that people are already using Event Data, and we’re still only in beta. Our aim has always been to create an open, portable, transparent data set that can be used by our diverse community including researchers, application developers, publishers, funders and third-party service providers. We have already seen data from our service used in recent research studies, impact reports and even a front-end tool. Launched recently as a prototype, ImpactStory’s Paperbuzz.org uses Event Data as one of its data sources for tracking the online buzz around scholarly articles. Jason Priem, cofounder of ImpactStory, notes:

“Because Crossref Event Data is completely open data, we believe it’s a game-changer for altmetrics. Our latest project, Paperbuzz.org, is just the first of a whole constellation of upcoming tools that will add value on top of Crossref’s open data.”

We are working towards launching Crossref Event Data as a production service. In the meantime though, please do take a look at our comprehensive User Guide. Hopefully you’ll be inspired to go make something cool using the data! Events are being collected constantly; take a look below as they stream in from our data sources or visit our live stream demo site to watch in real time.

Figure 1.5 Screen capture of Crossref Event Data live stream demo.

As the service matures, we’ll continue to add new platforms to track and I also encourage anyone with article link data to get in touch to discuss how we can share it with the community via Event Data.

For researchers in particular, I’m really keen to hear your thoughts on our data model and about the things we could additionally provide you with from an infrastructure perspective that would best support your research needs.

And if you’re a publisher, take a look at our Event Data best practice guidelines — there’s some really important information in there about how you can help give us the best chance possible of collecting Events for your registered content.

And finally, if you’re a consumer of altmetrics data, I encourage you to ask questions. Ask your altmetrics vendors about how they gather their data and what context they apply to the aggregation of the metrics they supply. Ask yourself what behaviours you are interested in tracking and equally those you are not. Think about the endgame; about the type of impact you’re truly trying to measure and the story you want to tell. Because it’s these questions that will help you choose indicators that are the best fit for your own unique narrative.

This content is cross-posted on eLife Labs.

References

¹ Bilder, Geoffrey; Lin, Jennifer; Neylon, Cameron (2015): What exactly is infrastructure? Seeing the leopard’s spots. Retrieved: Oct 16, 2017; https://doi.org/10.6084/m9.figshare.1520432.v1

² NISO, Outputs of the NISO Alternative Assessment Metrics Project. Retrieved: 6th October 2017; https://www.niso.org/publications/rp-25-2016-altmetrics , p.2.

Meet the members, Part 1 (with Oxfam)

Christine Cormack Wood — Mon, 30 Oct 2017 00:00:00 +0000

Introducing our new blog series Meet the members; where we talk to some of our members and find out a little bit more about them, ask them to share how they use our services, and discuss what their plans for the future are. To start the series we talk to Liam Finnis of Oxfam.

Can you tell us a little bit about Oxfam?

Around the globe, Oxfam works to find practical, innovative ways for people to lift themselves out of poverty and thrive. We save lives and help rebuild livelihoods when crisis strikes. And we campaign so that the voices of the poor influence the local and global decisions that affect them.

Oxfam’s Policy & Practice platform is the gateway to Oxfam’s knowledge, experience, and thinking. Policy & Practice aims to influence, enable and learn from others by sharing and collaborating online with professionals and practitioners.

What’s your role within Oxfam?

My name is Liam Finnis and I am the Website Manager for Oxfam GB’s Policy & Practice site and the Oxfam Digital Repository. In addition to maintenance and development of our platforms, my role focuses on raising the visibility of our programme work including approach and methodology, while also ensuring the availability and accessibility of our publications and resources.

What’s your participation level?

We joined Crossref in 2016, but only really began fully implementing DOIs this year. We have registered 139 content items as of October, with the majority assigned in 2017. While this only constitutes a small number of our total publications (roughly 6%), we’ve focused on current and future publications rather than retroactive application (with a handful of exceptions).

Tell us a bit about what you publish and for whom

We produce roughly 220 publications each year, with a library of 4,450 spanning 40 years. Roughly half of this would be considered grey literature and includes: research reports; evaluations; briefing papers; technical briefings; case studies; guidelines and toolkits. We also publish the Gender & Development Journal with Routledge/Taylor & Francis.

While our organisational focus is on inequalities and the eradication of poverty, this isn’t something we can achieve by looking solely at economic models. Our publications span a range of subject areas including: climate change; food and livelihoods; economics; gender; conflicts and disasters; land rights; and Water, Sanitation and Hygiene (WASH).

Our audience ranges from humanitarian and development practitioners to policy makers to researchers and academics. We publish the research that underpins our campaigns advocacy work; the evaluations of our emergency response efforts; reports outlining the methodologies we’ve applied; briefings on policy and recommendations; and, toolkits and guidelines for research, programme quality and responsible data management.

What do you think makes your publications unique?

Oxfam is one of the only NGOs that is actively sharing an extensive body of knowledge and experience. With 75 years of experience working on a global scale, our publications help to share learning and encourage best practice. Further to that, they showcase the changes (gradual or sudden) that we’ve seen in how development and humanitarian aid is defined and approached through the decades.

The Oxfam Gender Training Manual, published in 1991, remains one of our most frequently accessed resources; still widely regarded as a relevant, unique and valuable resource within the sector. Another of our key publications, Wealth: Having it all and wanting more, was published in 2015, outlining the methodology and data sources for Oxfam’s frequently cited fact ‘85 billionaires have the same wealth as the bottom half of the world’s population’.

The diversity in subject and format of our publications isn’t necessarily unique, but I’m reasonably confident that there is something in our publications that will relate to everyone.

What trends are you seeing in your part of the scholarly publishing community?

This is difficult to quantify as, while we have been lightly engaged with the scholarly publishing community in the past, we’ve been significantly more active in the past year. In addition to more actively applying DOIs, early in 2017 we were included in EBSCO Discovery and in March we made efforts to improve the visibility of our Digital Repository. Previously, the key route was through the Policy & Practice website, which brought together publications with blogs and pages focused on programmes, projects, approaches and methodology.

Since making these two changes we’ve seen a significant increase in access of our resources directly from the repository. This has come in addition to the general usage through Policy & Practice. We are also working with Research4Life, INASP and TEEAL to improve visibility and accessibility of our publications more widely.

How would you describe the value of being a Crossref member?

In the past two years, we’ve been looking into how we can ensure that our publications are visible and accessible to a wider audience. Becoming a member of Crossref and registering content with Crossref is a big part of that. It helps to give us a place in the discussions and events as well as enabling us to better understand and meeting scholarly publishing standards and implement best practice.

What are Oxfam’s plans for the future?

In terms of our work with Crossref and an active role in the scholarly publishing community, we’re still fairly new to it and we’re starting to see some of the benefits of our efforts. In the future, we’re looking to get a better idea of the opportunities available and build on our recent work.

Personally, I’m really interested in exploring Crossref Event Data in greater detail and seeing how it can help us map the impact of our work more effectively.

Thanks, Liam!

Peer reviews are open for registering at Crossref

Jennifer Lin — Tue, 24 Oct 2017 00:00:00 +0000

About 13-20 billion researcher-hours were spent in 2015 doing peer reviews. What valuable work! Let’s get more mileage out of these labors and make these expert discussions citable, persistent, and linked up to the scholarly record. As we previously shared during Peer Review week, Crossref is lauintroducing support for a new record type to support the registration of peer reviews. We’re one step closer to changing that. Today, we are excited to announce that we’re open for deposits.

If you missed the first episode, here’s a recap:

Publishers have been registering reviews with us for a while (ex: Example 1, Example 2, and Example 3). But these have been shoehorned into other content: article, dataset, or component. So we are extending Crossref’s infrastructure to properly treat this special scholarly artifact. This includes a range of outputs made publicly available from the peer review history (referee reports, decision letters, author responses, community comments) across any and all review rounds. We welcome scholarly discussions of journal articles before or after publication (e.g. “post-publication reviews”).

We collect metadata that characterizes the peer review asset (for example: recommendation, type, license, contributor info, competing interests). We also collect metadata, which offers a view into the review process (e.g. pre/post-publication, revision round, review date).

This special set will support the discovery and investigation of peer reviews as it is linked up to the article discussed. It will also enable the following:

Enable tracking of the evolution of scholarly claims through the lineage of expert discussion
Support enrichment of scholarly discussion
Enable reviewer accountability
Credit reviewers and editors for their scholarly contribution
Support publisher transparency
Connect reviews to the full history of the published results
Provide data for analysis and research on peer review

Please come check out our documentation for more information.

As publishers are implementing this, we are finishing up the delivery of this metadata for machine and human access, across all the Crossref interfaces (REST API, OAI-PMH, Crossref Metadata Search) to enable discoverability across the research ecosystem. We are also working to make it possible for members to get Cited-by data for the peer reviews they register.

If you are interested in registering your peer review content with us, please get in touch.

What happened at last month's LIVE local in London

Vanessa Fairhurst — Sun, 22 Oct 2017 00:00:00 +0000

So much has happened since we held LIVE16 (our annual meeting) in London last year that we wanted to check-in with our UK community and share the year’s developments around our tools, teams and services ahead of LIVE17 next month in Singapore.

And so, on 26th September we held a half-day ‘LIVE local’, covering a wide range of strategic topics, well-attended by a diverse representation of our UK community of publishers, funders, researchers, and tool-makers.

What we discussed on the day:

Ed Pentz, Crossref’s Executive Director, kicked the day off with ‘What’s new at Crossref’
Geoffrey Bilder, Strategic Director, talked us through ‘Crossref’s Strategic Initiatives’
Ginny Hendricks, Director of Member and Community Outreach introduced ‘Metadata 2020’
Rachael Lammey, Head of International Outreach discussed the ‘Global reach of Crossref metadata’
Jure Triglav from Coko Foundation presented some interesting ‘Metadata Use Case Studies’
Jennifer Lin, Director of Product Management, spoke about Crossref’s ‘New Product Developments’
Ed Pentz concluded the day leading a discussion on ‘Crossref’s Future Direction’

This event was one in a series of smaller, regional events which aim to better cater to our global membership and provide a tailored program of activities. You can read more about this series of events on our LIVE locals page, and if you are interested in hosting an event near you or have suggestions for one in your region then please contact me to get involved.

Celebrating ORCID at five

Ed Pentz — Mon, 16 Oct 2017 00:00:00 +0000

Happy birthday, ORCID! It’s their fifth birthday today and it’s gratifying to me—as a founding board member and former Chair of the board—to see how successful it has become. ORCID has a great staff, over 700 members from 41 countries and is quickly approaching 4 million ORCID iDs. Crossref—it’s board, staff, and members—has been an ORCID supporter from the start. One example of this support is that we seconded Geoffrey Bilder to be ORCID’s interim CTO for about eight months.

Actually, Crossref has been involved with ORCID even before the start.

ORCID’s birthday recognizes when the registry went live in 2012 but the origins of what became ORCID stretch back to a meeting that Crossref organized back in February 2007 on “Author IDs”. After this meeting there were many follow on discussions but it was clear that as an association of scholarly publishers Crossref didn’t have suitable governance for an researcher identifier registry which needed support from a broader group of stakeholders.

Subsequent discussions between Nature and Thomson Reuters (represented by Howard Ratner Dave Kochalko) led—after many more meetings—to ORCID being set up as a new organisation. ORCID was incorporated in September 2010 and the first meeting of the board of directors of ORCID was on October 8th, 2010.

A lot of people and organisations have contributed to getting ORCID to where it is today and it’s been great to be a part of it and continue to contribute to their future.

Reflecting on the creation of ORCID: it has shown the power of collaboration in improving scholarly research, and in making life easier and better for researchers.

Today they celebrate in a number of fun ways and, in particular, mark the occasion with the release of a new set of educational resources.

From everyone in the Crossref community, here’s to ORCID’s continuing success!

Changes to the 2018 membership agreement for better metadata distribution

Ginny Hendricks — Mon, 09 Oct 2017 00:00:00 +0000

We are making a change to section 9b of the standard Crossref membership agreement which will come into effect on January 1, 2018. This will not change how members register content, nor will it affect membership fees in any way. The new 2018 agreement is on our website, and the exact wording changes are highlighted below. The new membership agreement will automatically replace the previous version from January 1, 2018 and members will not need to sign a new agreement.

What’s changing?

At its July meeting the Crossref board unanimously approved recommendations from the Membership and Fees Committee to update Crossref’s metadata delivery offerings. One of the recommendations was to remove the option for case-by-case opt outs of metadata delivery through the OAI-PMH channel used for Enhanced Crossref Metadata Services.

This opt-out was only used by a small number of our members (around 40 of nearly 9,000), who have been contacted directly. This means that for the vast majority of members there is no change in how Crossref makes their metadata available but we wanted to make everyone aware of the change to the membership agreement.

So, as is currently the case, all metadata registered with Crossref is available via all the Metadata APIs under an appropriate agreement with the user or terms and conditions for the service. The one exception to this is how references are distributed - we will contact members next week about the options for references.

Why are we making this change?

Our metadata services have become very popular with users of all kinds throughout scholarly communications–including search and discovery platforms, libraries, other publishers, reference managers, sharing services, and analytics providers. More and better metadata means more and better discoverability of publisher content.
The change also brings this service into line with our mission to improve scholarly communications through quality metadata and related infrastructure services, removing the need for bilateral agreements between publishers and third parties.
Many members complained when we contacted them about opt-outs whenever a new OAI-PMH user came on board. It is better for our members and for our staff if there is a common standard across the board.

Changes to 2018 membership agreement

9) Sharing of Metadata by PILA

a) Local Hosting. [no change]

b) Other Metadata Services. Subject to compliance by the entity receiving the Metadata and Digital Identifiers with the terms and conditions ~~set forth in a separate agreement between~~ established by PILA for the particular service through which access is provided, and ~~the entity receiving the Metadata and Digital Identifiers~~, PILA may ~~license~~ authorize third parties to receive and use ~~bulk deliveries of~~ Metadata and Digital Identifiers from ~~the~~ PILA ~~System from members who have chosen to participate in Metadata Services,~~ which PILA shall provide directly to such third parties. At least thirty (30) days prior to making such Metadata delivery PILA will notify each PILA Member whose Metadata and Digital Identifiers are intended to be included in such delivery of the anticipated delivery date, the identity of the third party and the purpose for which the delivery is being made. Metadata and Digital Identifiers belonging to any PILA Member who notifies PILA in writing prior to the specified delivery date of its desire to be excluded from such delivery will be excluded or removed from such delivery.

Please contact our membership specialist if you have any feedback or questions.

Using the Crossref REST API. Part 6 (with NLS)

Christine Cormack Wood — Fri, 06 Oct 2017 00:00:00 +0000

Continuing our blog series highlighting the uses of Crossref metadata, we talked to Ulf Kronman, Bibliometric Analyst at the National Library of Sweden about the work they’re doing, and how they’re using our REST API as part of their workflow.

Introducing the National Library of Sweden (NLS)

The NLS is a state agency, has a staff of about 320, and its main offices in Stockholm. Its primary duty is to preserve the Swedish cultural heritage by collecting everything printed in Sweden, and has been doing so since 1661. Nowadays the library also collects Swedish TV and radio programs, movies, videos, music, and computer games.

The National Library coordinates services and programs for all publicly funded libraries in Sweden and runs the national library catalogue system Libris and the national database for Swedish scholarly output, SwePub. The library also runs the Bibsam consortium, negotiating national subscription licenses and open access publishing agreements with publishers.

Images left to right: External and internal view of the National Library of Sweden, and Ulf Kronman, Bibliometric Analyst at NLS.

What problem is your service trying to solve?

The metadata in the national scholarly publication database SwePub is harvested from the Swedish universities’ local publication systems, where data often is entered manually by librarians and researchers. This means that the metadata can contain a lot of omissions, synonyms, spelling variants and errors. Using Crossref, we can enhance and correct the metadata delivered to us, if we just have a correct DOI.

Can you tell us how you are using Crossref metadata at the National Library of Sweden?

The Crossref metadata is presently used in two projects; Open APC Sweden and in our local analysis database for publication statistics used in negotiations with publishers.

Open APC Sweden is a pilot project to gather data on open access publication costs (APC’s – Article Processing Charges) from Swedish universities. The project is modelled from the German Bielefeld University Open APC initiative, which is a part of the INTACT project. After APC data has been delivered to the APC system, scripts are run against the Crossref API to fetch information about publishers and journals. A description of Open APC Sweden can be found here.

When building our local analysis database for publisher statistics, we download data from the SwePub database, use the Crossref DOIs for API lookup against Crossref to add correct ISSN and publisher data to the records and then match the records against a list of publisher serials. In this way, we can get information about how much Swedish researchers have been publishing with a certain publisher and use this data when negotiating conditions for open access publishing with the publisher in question.

What metadata values do you pull from the API?

In Open APC Sweden, a Python script supplied by staff at the Bielefeld University is used to pull metadata about publisher and journal names and ISSN’s from the Crossref API. The result is entered into an enriched version of the APC data files delivered by the universities and then statistics can be calculated on the result using an R script. The result can be seen here.

In the local analysis database, a modified copy of the Bielefeld Python script is used to add the same metadata to the records before matching them against publisher serial ISSNs.

Have you built your own interface to extract this data?

In Open APC Sweden, the Python script is developed and maintained at the Bielefeld University and an exact copy is being run in the Swedish project.

In the local analysis system, the Python script is somewhat modified to suit the special demands of this system.

But sometimes it is very convenient just to use the main DOI lookup to do a manual check-up of problematic records.

How often do you extract/query data?

In Open APC Sweden, usually about two-three times a month, when new datasets are delivered from the universities. In the local analysis database, usually lookups are being done on a daily basis as development of the database continues.

What do you do with the metadata once it’s pulled from the API?

In Open APC Sweden, the metadata is going into the APC data files for processing of statistics. In the local analysis database, the metadata is used to match against publisher journal ISSN’s.

What plans do you have for the future?

For the Open APC Sweden I would like to build a database system to make the system more scalable than just working with flat data files.

With both the SwePub system and the local analysis system, we are now using the new service oaDOI and their API to look up metadata about the open access status of the publications to enrich our local systems.

What else would you like to see the REST API offer?

In the process of normalising the publishers’ names, the names returned are sometimes at a “too high” or on a too generic level to be used to generate good statistics. For instance, Springer Nature are sometimes returned as Springer Nature, sometimes as Springer Science + Business Media and sometimes as Nature Publishing Group. A similar thing is valid for Taylor & Francis, where the mother company Informa UK Limited is returned instead of the publishing subsidiary of the company. One thing to wish for here is that we could agree on some kind of normalisation of the publishers’ names and that Crossref could return this as a supplement to the present metadata.

Thanks Ulf! If you would like to contribute a case study on the uses of Crossref Metadata APIs please contact the Community team.

Publishers, help us capture Events for your content

Madeleine Watson — Mon, 02 Oct 2017 00:00:00 +0000

The day I received my learner driver permit, I remember being handed three things: a plastic thermosealed reminder that age sixteen was not a good look on me; a yellow L-plate sign as flimsy as my driving ability; and a weighty ‘how to drive’ guide listing all the things that I absolutely must not, under any circumstances, even-if-it-seems-like-a-really-swell-idea-at-the-time, never, ever do.

The margin space dedicated to finger-wagging left little room for championing any driving-do’s. And as each page delivered a fresh new warning, my enthusiasm for hitting the road sunk to levels usually reserved for activities like trigonometry and visits to my orthodontist.

Many years (and an excellent driving record) later, I’m reminded of this again now when thinking about our own Event Data User Guide. Because it contains a chapter with some really important don’ts for our members. Really good, we’d-love-you-to-consider-not-doing-these-things type of advice. But despite our intent to encourage, I feel the ghost of finger-waggers past. So in the spirit of championing enthusiasm over ennui, I thought I’d attempt to contextualise our Event Data Best Practices Guide for Publishers and show you why there’s a lot of good reasons for publishers to be enthusiastic about these rules.

So if you’re a publisher, I encourage you to read on to learn more about how you can help us have the best chance possible of capturing Events for your content.

What’s in it for you? Well, collecting this data helps to give everyone (Crossref, yourself, and others) a better picture of how your content is being used, including for altmetrics.

1. Please let us in

Please do open the door when we come knocking, we promise not to stay long. You can do this by allowing the User Agent CrossrefEventDataBot to visit your site, and whitelisting it if necessary. The bot is how we visit URLs to confirm if they are for an item of content registered with us. The reason why we’re visiting your site could include:

someone tweeted an article landing page
someone discussed it on Reddit
it was linked to from a blog post

The Bot has only one job: to work out the DOI. No information beyond this is stored. Whenever we become aware of a link that we think points to a DOI or an Article Landing Page, we follow it so we can collect the required metadata. Everything in Crossref Event Data is linked via its DOI, so it’s important that we can collect this information.

The bot will identify itself using the standard method. It sets two headers:

Referer: https://eventdata.crossref.org
User-Agent: CrossrefEventDataBot (eventdata@crossref.org)

Once we confirm that a link points to registered content, we then log an Event for the DOI. You should expect our bot to visit no more than once or twice per second, although if there is a period of activity around your articles, you may see higher rates. The bot also takes a sample of DOIs and visits them to work out which domain names belong to our members, so it can maintain a list. This can happen every few weeks. You may see a small number of requests from the bot, but limited to one per second.

If we can’t enter your site to look for metadata though, then we won’t be able to collect Events for your DOIs. So by allowing our bot, you will be helping us to collect Event Data for your registered content.

If you’re worried about traffic on your site, consider sending us your mapping of article landing pages to DOIs. Because Resource URLs aren’t the same as article landing pages, we need more information than the DOI Resource URLs that you already send us.

If you’re running a blog or website (and you’re not a member of Crossref), you may also see our bot visiting, to look for links that comprise Events. Please allow us to visit, so we can record in our Event Data service the fact that your website links to registered content.

2. We ❤️ robots.txt

Robots.txt files are important and we ensure our Event Data Bot respects yours. If we are instructed not to visit a site, we won’t. So if you want us to visit your site in order to check the metadata of your article landing page, please ensure you provide an exception for our Bot, or make sure that you’re not blocking it. Check the restrictions in your file to see if we’re allowed to visit. This is just another way you can help us work for you.

3. Include the DC Identifier

Including good metadata is general best practice for scholarly publishing. When we visit a publisher’s site, we look for metadata embedded in the HTML document (such as DC.Identifier tags that, amongst other things, enable Crossmark to work).

By ensuring you include a Dublin Core identifier meta tag in each of your articles pages, our system can match your landing pages back to DOIs.

Here’s an example:

4. Let us in, even if we don’t bring cookies

We’re like that friend who turns up for dinner without bringing a bottle of wine. And we hope that you’ll be ok with that. Some Publisher sites don’t allow browsers to visit unless cookies are enabled and they block visitors that don’t accept them. If your site does this, we will be unable to collect Events for your DOIs. Allowing your site to be accessed without cookies will help give us the best chance of successfully reading your metadata.

5. We may not speak your language

Sometimes we come across a publisher’s site that won’t render unless JavaScript is enabled. This means that the site won’t show any content to browsers that don’t execute JavaScript. The Event Data Bot does not execute JavaScript when looking for a DOI. This means that if your site requires JavaScript, then we will be unable to collect DOIs for your Events. Consider allowing your site to be accessed without JavaScript. And if this is not possible, then if you ensure you include the tag in the HTML header, then we’ll do our best to collect Events for your registered content.

If you want to pass this on to your friendly system administrator, the best practice is documented in full here: https://www.eventdata.crossref.org/guide/best-practice/publishers-best-practice/. And sorry about all the don’ts you’ll find on that page…. don’t let them curb your enthusiasm for taking Event Data out for a spin!

BestBlogsRead

Christine Cormack Wood — Tue, 26 Sep 2017 00:00:00 +0000

We know that research communication happens everywhere, and we want your help in finding it!

From October 9th we will be collecting links sent in by you through a social campaign across Twitter and Facebook called #BestBlogsRead.

Simply send us links to the blogs YOU like to read It’s easy to participate, all you have to do is watch out for the daily tweets and facebook posts and then send us links to the blogs (and news sites) you read.

From gardening to gaming, recipes to rock climbing, tennis to taxidermy - whatever blogs you read, we want to hear about them!

Because research happens everywhere! And you’ll be surprised where it is mentioned - for example:

We found a Wiley article mentioned in a blog about the eclipse
An American Chemical Society article in a blog about food allergies
A blog about Neanderthals on the Atlantic links to and article from the American Association for the Advancement of Science

So, watch out for the campaign on Twitter and Facebook, and tell us about your #BestBlogsRead.

Crossref at the Frankfurt Book Fair

Christine Cormack Wood — Tue, 26 Sep 2017 00:00:00 +0000

We’ll be at booth M82 in the Hotspot area of Hall 4.2 and would love to meet with you. Let us know if you’re interested in chatting with one of us - about anything at all.

Kirsty Meddings, Product Manager: Here to help with Crossref services such as Crossmark and funding data, and happy to talk about your metadata and how you can deposit more.

Paul Davis, Support Specialist: Any issues with metadata deposit, or anything technical, I’m your man.

Susan Collins, Publisher Outreach Manager: If you’re a member and have questions about how things are going, or try out additional services, I can help.

Jennifer Kemp, Affiliate Outreach Manager: Come to me if you want to get Metadata from Crossref, or discuss our imminent new service for social mentions and data links: Event Data (in Beta).

Ginny Hendricks, Member & Community Outreach Director: I’d love to talk to publishers and platforms about the new Metadata 2020 initiative.

Amanda Bartell, Head of Member Experience: This will be my first day at Crossref! If there is something you’d like the Membership team to do or change, please let me know.

Chrissie Cormack-Wood, Head of Marketing Communications: I’ll be acting as “host” so ask me anything about our booth and activities at the Fair. Ideas for joint campaigns or co-promotion are welcome too.

If some of these topics are on your agenda, or if you’re not sure who to contact, please let me know and I’ll set up a 30-minute meeting at our booth, M82 in Hall 4.2.

And, if you don’t get a chance to visit us at our stand, make sure you don’t miss Ginny’s Metadata 20/20 talk at 2.30pm on Wednesday 11th, at the Hot Spot stage in the corner of Hall 4.2, area N99.

We hope you have a great Book Fair!

Event Data as Underlying Altmetrics Infrastructure at the 4:AM Altmetrics Conference

Joe Wass — Mon, 25 Sep 2017 00:00:00 +0000

I’m here in Toronto and looking forward to a busy week. Maddy Watson and I are in town for the 4:AM Altmetrics Conference, as well as the altmetrics17 workshop and Hack-day. I’ll be speaking at each, and for those of you who aren’t able to make it, I’ve combined both presentations into a handy blog post, which follows on from my last one.

But first, nothing beats a good demo. Take a look at our live stream. This shows the Events passing through Crossref Event Data, live, as they happen. You may need to wait a few seconds before you see anything.

Crossref and scholarly links

You may know about Crossref. If you don’t, we are a non-profit organisation that works with Publishers (getting on for nine thousand) to register scholarly publications, issue Persistent Identifiers (DOIs) and maintain the infrastructure required to keep them working. If you don’t know what a DOI is, it’s a link that looks like this:

https://doi.org/10.5555/12345678

When you click on that, you’ll be taken to the landing page for that article. If the landing page moves, the DOI can be updated so you’re taken to the right place. This is why Crossref was created in the first place: to register Persistent Identifiers to combat link rot and to allow Publishers to work together and cite each other’s content. A DOI is a single, canonical identifier that can be used to refer to scholarly content.

Not only that, we combine that with metadata and links. Links to authors via ORCIDs, references and citations via DOIs, funding bodies and grant numbers, clinical trials… the list goes on. All of this data is provided by our members and most of it is made available via our free API.

Because we are the central place that publishers register their content, and we’ve got approaching 100 million items of Registered Content, we thought that we could also curate and collect altmetrics type data for our corpus of publications. After all, a reference from a Tweet to an article is a link, just like a citation between two articles is a link.

An Experiment

So, a few years back we thought we would try and track altmetrics for DOIs. This was done as a Crossref Labs experiment. We grabbed a copy of PLOS ALM (since renamed Lagotto), loaded a sample of DOIs into it and watched as it struggled to keep up.

It was a good experiment, as it showed that we weren’t asking exactly the right questions. There were a few things that didn’t quite fit. Firstly, it required every DOI to be loaded into it up-front, and, in some cases, for the article landing page for every DOI to be known. This doesn’t scale to tens of millions. Secondly, it had to scan over every DOI on a regular schedule and make an API query for each one. That doesn’t scale either. Thirdly, the kind of data it was requesting was usually in the form of a count. It asked the question:

“How many tweets are there for this article as of today?”

This fulfilled the original use case for PLOS ALM at PLOS. But when running it at Crossref, on behalf of every publisher out there, the results raised more questions than they answered. Which was good, because it was a Labs Experiment.

Asking the right question

The whole journey to Crossref Event Data has been a process of working out how to ask the right question. There are a number of ways in which “How many tweets are there for this article as of today?” isn’t the right question. It doesn’t answer:

Tweeted by who? What about bots?
Tweeted how? Original Tweets? Retweets?
What was tweeted? The DOI? The article landing page? Was there extra text?
When did the tweet occur?

We took one step closer toward the right question. Instead of asking “how many tweets for this article are there as of today” we asked:

“What activity is happening on Twitter concerning this article?”

If we record each activity we can include information that answers all of the above questions. So instead of collecting data like this:

Registered Content	Source	Count	Date
10.5555/12345678	twitter	20	2017-01-01
10.5555/87654321	twitter	5	2017-01-15
10.5555/12345678	twitter	23	2017-02-01

We’re collecting data like this:

Subject	Relation	Object	Source	Date
twitter.com/tweet/1234	references	10.5555/12345678	twitter	2017-01-01
twitter.com/tweet/5678	references	10.5555/987654321	twitter	2017-01-11
twitter.com/tweet/9123	references	10.5555/12345678	twitter	2017-02-06

Now we’re collecting individual links between tweets and DOIs, we’re closer to all the other kinds of links that we store. It’s like the “traditional” links that we already curate except:

It’s not provided by publishers, we have to go and collect it ourselves.
It comes from a very diverse range of places, e.g. Twitter, Wikipedia, Blogs, Reddit, random web pages
The places that the Events do come from don’t play by the normal rules. Web pages work differently to articles.

Non-traditional Publishing is Untraditional

This last point caused us to scratch our heads for a bit. We used to collect links within the ’traditional’ scholarly literature. Generally, journal articles:

get published once
have a publisher looking after them, who can produce structured metadata
are subject to a formal process of retractions or updates

Now we’re collecting links between things that aren’t seen as ’traditional’ scholarship and don’t play by the rules.

The first thing we found is that blog authors don’t reference the literature using DOIs. Instead they use article landing pages. This meant that we had to put in the work to collect links to article landing pages and turn them back into DOIs so that they can be referenced in a stable, link-rot-proof way.

When we looked at Wikipedia we noticed that, as pages are edited, references are added and removed all the time. If our data set reflected this, it would have to evolve over time, with items popping into existence and then vanishing again. This isn’t good.

Our position in the scholarly community is to provide data and infrastructure that others can use to create services, enrich and build things. Curating an ever changing data set, where things can disappear, is not a great idea and is hard to work with.

We realised that a plain old link store (also known as an assertion store, triple store, etc.) wasn’t the right approach as it didn’t capture the nuance in the data with sufficient transparency. At least, it didn’t tell the whole picture.

We settled on a new architecture, and Crossref Event Data as we now know it was born. Instead of a dataset that changes over time, we have a continual stream of Events, where each Event tells a new part of the story. An Event is true at the time it is published, but if we find new information we don’t edit Events, we add new ones.

An Event is the way that we tell you that we observed a link. It includes the link, in “subject - relation type - object” format, but so much more. We realised that one question won’t do, so Events now answer the following questions:

What links to what?
How was the link made? Was it with a article’s DOI or straight to an Article landing page?
Which Agent collected it?
Which data source were they looking at?
When was the link observed?
When do we think the link actually happened?
What algorithms were used to collect it?
How do you know?

I’ll come back to the “how do you know” a bit later.

What is an altmetrics Event?

So, an Event is a package that contains a link plus lots of extra information required to interpret and make sense of it. But how do we choose what comprises an Event?

An Event is created every time we notice an interaction between something we can observe out on the web and a piece of registered content. This simple description gives rise to some interesting quirks.

It means that every time we see a tweet that mentions an article, for example, we create an Event. If a tweet mentions two articles, there are two events. That means that “the number of Twitter events” is not the same as “the number of tweets”.

It means that every time we see a link to a piece of registered content in a webpage, we create an Event. The Event Data system currently tries to visit each webpage once, but we reserve the right to visit a webpage more than once. This means that the number of Events for a particular webpage doesn’t mean there are that many references.

We might go back and check a webpage in future to see if it still has the same links. If it does, we might generate a new set of Events to indicate that.

Because of the evolving nature of Wikipedia, we attempt to visit every page revision and document the links we find. This means that if an article has a very active edit history, and therefore a large number of edits, we will see repeated Events to the literature, once for every version of the page that makes references. So the number of Events in Wikipedia doesn’t mean the number of references.

An Event is created every time we notice an interaction. Each source (Reddit, Wikipedia, Twitter, blogs, the web at large) has different quirks, and you need to understand the underlying source in order to understand the Events.

We put the choice into your hands.

If you want to create a metric based on counting things, you have a lot of decisions to make. Do you care about bots? Do you care about citation rings? Do you care about retweets? Do you care about whether people use DOIs or article landing pages? Do you care what text people included in their tweet? The answer to each of these questions means that you’ll have to look at each data point and decide to put a weighting or score on it.

If you wanted to measure how blogged about a particular article was, you would have to look at the blogs to work out if they all had unique content. For example, Google’s Blogger platform can publish the same blog post under multiple domain names.

A blog full of link spam is still a blog. You may be doing a study into reputable blogs, so you may want to whitelist the set of domain names to exclude less reputable blogs. Or you may be doing a study into blog spam, so lower quality blogs is precisely what you’re interested in,

If you wanted to measure how discussed an article was on Reddit, you might want to go to the conversation and see if people were actually talking about it, or whether it was an empty discussion. You might want to look at the author of the post to see if they were a regular poster, whether they were a bot or an active member of the community.

If you wanted to measure how referenced an article was in Wikipedia, you might want to look at the history of each reference to see if it was deleted immediately. Or if it existed for 50% of the time, and to give a weighting.

We don’t do any scoring, we just record everything we observe. We know that everyone will have different needs, be producing different outcomes and use different methodologies. So it’s important that we tell you everything we know.

So that’s an Event. It’s not just a link, it’s the observation of a link, coupled with extra information to help you understand it.

How do you know?

But what if the Event isn’t enough? To come back to the earlier question, “how do you know?”

Events don’t exist in isolation. Data must be collected and processed. Each Agent in Crossref Event Data monitors a particular data source and feeds data into the system, which goes and retrieves webpages so it can make observations. Things can go wrong.

Any one of these things might prevent an Event from being collected:

We might not know about a particular DOI prefix immediately after it’s registered.
We might not know about a particular landing page domain for a new member immediately.
Article landing pages might not have the right metadata, so we can’t match them to DOIs.
Article landing pages might block the Crossref bot, so we can’t match DOIs.
Article landing pages might require cookies, or convoluted JavaScript, so the bot can’t get the content.
Blogs and webpages might require cookies or JavaScript to execute.
Blogs might block the Event Data bot.
A particular API might have been unavailable for a period of time.
We didn’t know about a particular blog newsfeed at the time.

This is a fact of life, and we can only operate on a best-effort basis. If we don’t have an Event, it doesn’t mean it didn’t happen.

This doesn’t mean that we just give up. Our system generates copious logs. It details every API call it made, the response it got, every scan it made, every URL it looked at. This amounts to about a gigabyte of data per day. If you want to find out why there was no Wikipedia data at a given point in time, you can go back to the log data and see what happened. If you want to see why there was no Event for an article by publisher X, you can look at the logs and see, for example, that Publisher X prevented the bot from visiting.

Every Event that does exist has a link to an Evidence Record, which corresponds with the logs. The Evidence Record tells you:

which version of the Agent was running
which Artifacts and versions it was working from
which API requests were made
which inputs looked like possible links
which matched or failed
which Events were generated

Artifacts are versioned files that contain information that Agents use. For example, there’s a list of domain names, a list of DOI prefixes, a list of blog feed urls, and so on. By indicating which version of these Artifacts were used, we can explain why we visited a certain domain and not another.

All the code is open source. The Evidence Record says which version of each Agent was running so you can see precisely which algorithms were used to generate the data.

Between the Events, Evidence Records, Evidence Logs, Artifacts and Open Source software, we can pinpoint precisely how the system behaved and why. If you have any questions about how a given Event was (or wasn’t) generated, every byte of explanation is freely available.

This forms our “Transparency first” idea. We start the whole process with an open Artifact Registry. Open source software then produces open Evidence Records. The Evidence Record is then consulted and turned into Events. All the while, copious logs are being generated. We’ve designed the system to be transparent, and for each step to be open to inspection.

We’re currently in Beta. We have over thirty million Events in our API, and they’re just waiting for you to use them!

Head over to the User Guide and get stuck in!

If you are in Toronto, come and say hi to Maddy or me.

Organisation Identifier Working Group Update

Ed Pentz — Mon, 18 Sep 2017 00:00:00 +0000

About 1 year ago, Crossref, DataCite and ORCID [announced a joint initiative] (https://orcid.org/blog/2016/10/31/organisation-identifier-project-way-forward) to launch and sustain an open, independent, non-profit organisation identifier registry to facilitate the disambiguation of researcher affiliations. Today we publish governance recommendations and product principles and requirements for the creation of an open, independent organisation identifier registry and invite community feedback.

The organisation Identifier (OrgID) Working Group was established as a joint effort by Crossref, DataCite and ORCID in January 2017. The members of the group bring a broad range of experience and perspectives, including expertise in research data discovery, data management, persistent identifiers, economics research, funding, archiving, non-profit membership organisations, academia, publishing, and metadata development.

The Working Group was charged with refining the structure, principles, and technology specifications for an open, independent, non-profit organisation identifier registry to facilitate the disambiguation of researcher affiliations.

The group has been working in three interdependent areas: Governance, Registry Product Definition, and Business Model & Funding, and today releases for public comment its findings and recommendations for governance and product requirements.

We invite your feedback!

Please send comments by October 15th, 2017.

PIDapalooza is back and wants your PID stories

Ginny Hendricks — Thu, 14 Sep 2017 00:00:00 +0000

Now in its second year, this “open festival of persistent identifiers” brings together people from all walks of life who have something to say about PIDs. If you work with them, develop with them, measure or manage them, let us know your PID adventures, pitfalls, and plans by submitting a talk by September 18. It’ll be in Girona, Spain, January 23-24, 2018.

One of the great strengths of last year’s PIDapalooza was the number of people who spoke and all the conversations that were kindled. So if you’re thinking of going, we encourage you to propose a talk, so we can hear what you’re working on and you can get some feedback.

At the inaugural PIDapalooza event Crossref took to the stage twice, with Ed Pentz covering Org IDs and Joe Wass talking about Event Data.

Here we have Joe’s memories of the event and Ed’s update on the Org ID status.

Joe Wass reflects:

At Crossref, the subject of Persistent Identifiers is something we care deeply about, and linking between DOIs, ORCID iDs and other identifiers is the reason we get up in the morning. But a whole conference dedicated to them? If I’m honest, the first time I heard about PIDapalooza I thought the subject was rather niche.

How wrong I was. It turns out there are people from all walks of life who care about “things” using persistent identifiers to link, describe and reference them. There was a great balance between presenters and attendees, and the programme meant that lots of people had a chance to speak. We heard about identifiers for research vessels, pieces of scientific equipment, individual bottles of milk, plus the usual subjects like scholarly publishing, datasets, organisations and funders, and how to cite them.

Between sessions we chatted over a wide range of subjects, noted similarities between subject areas, offered advice and exchanged ideas. Who knew this stuff was all related?

Ed Pentz on plans for the new organisation IDs

An important presentation at the 2016 PIDapalooza meeting was on organisation identifiers. A week before the conference Crossref, DataCite and ORCID released three documents for public comment outlining a proposed way forward. The goal is launch and sustain an open, independent, non-profit organisation identifier registry to facilitate the disambiguation of researcher affiliations. At the packed PIDapalooza session Crossref, DataCite and ORCID gave an update on their work over the previous year and their proposals going forward.

There was a lively discussion and debate about the issues. Following the meeting the three organisations set up the OI Project Working Group with a broad group of stakeholders. The group has been meeting over the last year and will release two documents next week - a set of Governance Recommendations and Product Principles and Recommendations for community feedback. So watch this space.

The PIDapalooza conference really helped galvanize the work in this area by bringing together a broad range of people interested in persistent identifiers. If you have an idea about PIDs, please come and tell us about it.

Check out the decks from last year's talks, the PIDapalooza website with all the info, and sumbit a proposal for your talk before September 18.

Making peer reviews citable, discoverable, and creditable

Ginny Hendricks — Mon, 11 Sep 2017 00:00:00 +0000

A number of our members have asked if they can register their peer reviews with us. They believe that discussions around scholarly works should have DOIs and be citable to provide further context and provenance for researchers reading the article. To that end, we can announce some pertinent news as we enter Peer Review Week 2017 : Crossref infrastructure is soon to be extended to manage DOIs for peer reviews. Launching next month will be support for this new resource/record type, with schema specifically dedicated to the reviews and discussions of scholarly content.

Not disimilar to other registered resources (datasets, working papers, preprints, translations, etc.) publication peer reviews are important scholarly contributions in their own right and form a part of the scholarly record. In addition to the members who have been registering them, many more are looking to better handle these contributions and give recognition to this process which is so critical to maintaining scientific quality.

Here are a few examples of existing Crossref DOIs for peer reviews: https://doi.org/10.1016/j.engfracmech.2015.01.019 and https://doi.org/10.5194/wes-1-177-2016 and https://doi.org/10.14322/PUBLONS.R518142.

We are extending our infrastructure to support all members who make these scholarly discussions available to readers. To accommodate a wide range of publisher practices, this will include a range of outputs made publicly available from the peer review history, across any and all review rounds, including referee reports, decision letters, and author responses. Members will be able to include not only scholarly discussions of journal articles before but also after publication (e.g. “post-publication reviews”).

Central to this new feature of the Crossref Content Registration service is the special set of metadata dedicated to supporting the discovery and investigation of peer reviews as it is linked up to the article discussed. The peer review schema will provide a characterization of the peer review asset (for example: recommendation, type, license, contributor info, competing interests) as well as offer a view into the review process (e.g. pre/post-publication, revision round, review date).

Our custom support for peer reviews will ensure that:

Readers can see provenance and get context of a work
Links to this content persist over time
The metadata is useful
They are connected to the full history of the published results
Contributors are given credit for their work (we will ask for ORCID iDs)
The citation record is clear and up-to-date.

As with all the content registered with Crossref, we will make peer review metadata available for machine and human access, across multiple interfaces (e.g. REST API, OAI-PMH, Crossref Metadata Search) to enable discoverability across the research ecosystem. This metadata may also support enrichment of scholarly discussion, reviewer accountability, publishing transparency, analysis or research on peer reviews, and so on.

To reflect the nature of this special content, we will bundle the fees for peer review content fees into the cost of registering the article for members who publish the journal article and its peer reviews. No matter how many reviews are associated with a paper, there will be a fixed fee for the full set.

Peer review infrastructure will arrive at Crossref in one month, and we are excited to engage our members who want to assign DOIs to peer reviews or migrate previously registered review content to the new schema. A special thanks to the members so far who have given feedback and advice to develop the schema: BMC, The BMJ, Copernicus, eLife, PeerJ, and Publons.

Please contact our membership specialist if you’d like to know more.

More metadata for machines-citations, relations, and preprints arrive in the REST API

Kirsty Meddings — Mon, 11 Sep 2017 00:00:00 +0000

Over the past few months we have been adding to the metadata and functionality of our REST API, Crossref’s public machine interface for the metadata of all 90 million+ registered content items. Much of the work focused on a review and upgrade of the API’s code and architecture in order to better support its rapidly growing usage. But we have also extended the types of metadata that the API can deliver.

One of the biggest changes is that references are now available if the publisher has made them public (a simple email instruction to us). Currently 45% of all publications with deposited references are now accessible. For example:

This article studying fluid ejection from animals has 55 references and they are all in the metadata here. You can also see that the article has an is-referenced-by count of 6.
This article exploring whether people bitten by their cat are more likely to develop depression has 142 references and is referenced by 12.

We recently announced that we would be accepting preprints, and the metadata for 15,000 preprints registered to date is now in the API, labelled as posted-content. Over 4,000 have been subsequently published in a journal, and the Crossref metadata now links these preprints to their respective articles (and vice versa). For example this article in Biorxiv has since been published in a journal, and this relationship is recorded in its metadata as is-preprint-of.

Also new to the API:

Cited-by counts - the number of times each work has been referenced by other content registered with us. Look for is-referenced-by-count within a record.
This article from 1953 about a fairly notable discovery has been cited 4832 times, but the two most cited articles both have over 100,000 citations and thousands have been cited more than Watson and Crick.
Abstracts for over 1 million works.
Similarity Check URLs–the ones that Turnitin crawl to add content to the database–are now showing so that participating publishers can check that they are including them in their metadata deposits.
Subject categories have been added for an additional 7000 journal titles, taking the total number of classified titles to ~45,000.

Are you already using our Metadata APIs for your system or project? We’re always keen to hear new use cases and happy to answer any questions.

You may need to install a JSON viewer extension in your browser to render API queries in a human-friendly way.

Using the Crossref REST API. Part 5 (with OpenCitations)

Christine Cormack Wood — Sun, 10 Sep 2017 00:00:00 +0000

As part of our blog post series on the Crossref REST API, we talked to Silvio Peroni and David Shotton of OpenCitations (OC) about the work they’re doing, and how they’re using the Crossref REST API as part of their workflow.

Introducing OpenCitations

OpenCitations employs Semantic Web technologies to create an open repository of the citation data that publishers have made available. This repository, called the OpenCitations Corpus (OCC), contains RDF-based scholarly citation data that are made freely available so that others may use and build upon them. All the resources published by OC – namely the data within the OCC, the ontologies describing the data, and the software developed to build the OCC – are available to the public with open licenses.

What problem is your service trying to solve?

OC was started to address the lack of RDF-based open citation data. To our knowledge, when the project formally started with Jisc funding in 2010 the prototype OCC was the first RDF-based dataset of open citation data.

We collect accurate scholarly citation data derived from bibliographic references harvested from the scholarly literature, so as to make them available under a Creative Commons public domain dedication (CC0) by means of Semantic Web technologies, thus making them findable, accessible, interoperable, and re-usable, as well as structured, separable, and open.

OCC citation data are described using standard and/or well-known vocabularies, including the SPAR Ontologies , PROV-O, the Data Catalog Vocabulary, and VoID. The use of such vocabulary is described in the OCC metadata document, and is implemented by means of the OpenCitations Ontology (OCO).

The OCC resources are made available and accessible in different ways, so as to facilitate their reuse in different contexts: as monthly dumps, via the SPARQL endpoint, and by accessing them directly by means of the HTTP URIs of the stored resources (via content negotiation; example)

Can you tell us how you are using the Crossref Metadata API at OpenCitations?

At present, basic citation information is retrieved from PubMed Central, and the Crossref API is then used to retrieve additional metadata describing the citing and cited articles, and to disambiguate bibliographic resources and agents by means of the identifiers retrieved (e.g., DOI, ISSN, ISBN, URL, and Crossref member URL). In future, we will retrieve full citation data direct from Crossref.

What metadata values do you pull from the API?

We pull the titles, subtitles, identifiers (e.g. DOI, ISSN, ISBN, URL, and Crossref member URL), author list, publisher, container resources (issue, volume, journal, book, etc.), publication year and pages.

Have you built your own interface to extract this data?

The SPAR Citation Indexer, a.k.a. SPACIN, is a script and a series of Python classes that allow one to process particular JSON files containing the bibliographic reference lists of papers, produced from the PubMed Central API by another script included in the OpenCitations GitHub repository.

SPACIN processes such JSON files and retrieves additional metadata information about all the citing and cited articles by querying the Crossref API, among others. Once SPACIN has retrieved all these metadata, RDF resources are created (or reused, if they have been already added in the past) and stored in the file system in JSON-LD format. In addition, they are also uploaded to the OCC triplestore (via the SPARQL UPDATE protocol).

How often do you extract/query data?

The entire OpenCitations ingestion workflow is running continuously, processing about half a million citations per month.

What do you do with the metadata once it’s pulled from the API?

All the metadata relevant to bibliographic entities are stored by using the OCC metadata model. The ontological terms of such metadata model are collected within an ontology called the OpenCitations Ontology (OCO), which includes several terms from the SPAR Ontologies and other vocabularies. In particular, the following six bibliographic entity types occur in the datasets created by SPACIN:

bibliographic resources (br), class fabio:Expression – resources that either cite or are cited by other bibliographic resources (e.g. journal articles), or that contain such citing/cited resources (e.g. journals);
resource embodiments (re), class fabio:Manifestation – details of the physical or digital forms in which the bibliographic resources are made available by their publishers;
bibliographic entries (be), class biro:BibliographicReference – literal textual bibliographic entries occurring in the reference lists of bibliographic resources;
responsible agents (ra), class foaf:Agent – names of agents having certain roles with respect to the bibliographic resources (i.e. names of authors, editors, publishers, etc.);
agent roles (ar), class pro:RoleInTime – roles held by agents with respect to the bibliographic resources (e.g. author, editor, publisher);
identifiers (id), class datacite:Identifier – external identifiers (e.g. DOI, ORCID, PubMedID) associated to bibliographic resources and agents.

Do you have plans to enhance your metadata input?

We already handle additional information, such as ORCIDs, that are extracted by means of the ORCID API applied to the citing and cited articles included in the OCC. In addition, we are developing scripts in order to use all the new citation data Crossref now makes available as consequence of the Initiative for Open Citations (I4OC).

What are the future plans for OpenCitations?

With funding received from the Alfred P. Sloan Foundation, we will shortly extend the current infrastructure and the rate of data ingest. Our immediate goal is to increment the daily ingestion of citation data from about half a million citations per month to about half a million citations per day. In addition, we plan to analyse the OCC so as to understand the quality of its current data, and to develop new user interfaces, including graph visualizations of citation networks, that will expand the means whereby users can interact with the OpenCitations data.

What else would you like to see our REST API offer?

Categorising articles/journals/any bibliographic resources according to their main discipline (Computer Science, Biology, etc.) and, eventually, by means of subject terms and/or keywords. Additionally, provision of authors’ institutional affiliations and funder information would be extremely valuable.

Thank you Silvio and David!

If you are keen to share what you’re doing with the our Metadata APIs, contact feedback@crossref.org and share your story.

LIVE17 in Singapore is taking shape!

Ginny Hendricks — Tue, 29 Aug 2017 00:00:00 +0000

Our annual meeting on 14th and 15th November, LIVE17 is shaping up nicely with an exciting line-up of respected speakers talking around the theme of “Metadata + Infrastructure + Relations = Context”, with each half day covering some element of the main theme.

Day one, AM: Metadata enables connections
Day one, PM: How research and infrastructure is changing
Day two, AM: Social challenges in the scholarly community
Day two, PM: Who is using your metadata and what are they doing with it?

This years updated format means both days will be packed with a mixture of plenary and breakout sessions and interactive activities. A cocktail reception with entertainment will be held in the Grand Marquee on the first evening.

A comprehensive agenda of the two-day event will be available shortly, but in the meantime we’ve provided a few talk teasers from six of our plenary speakers to whet your appetite:

Speaker	Title and organisation	Talk title
Theodora Bloom	Executive Editor, The BMJ	Preparing to handle dynamic scholarly content: Are we ready?
Casey Green	Assistant Professor, Perelman School of Medicine, University of Pennsylvania	Research and literature parasites in a culture of sharing.
Leonid Teytelman	Co-founder and CEO, Protocols.io	A call to reduce random collisions with information; we can automatically connect scientists to the knowledge that they need.
Nicholas Bailey	Data Science Team, Royal Society of Chemistry	What does data science tell us about social challenges in scholarly publishing?
Miguel Escobar Varela	Assistant Professor of Theatre Studies, National University of Singapore	Digital Humanities in Singapore: some thoughts for the future.
Kuansan Wang	Managing Director, Microsoft Research Outreach	Democratize access to scholarly knowledge with AI.

Theodora Bloom - Preparing to handle dynamic scholarly content: Are we ready?

Historically, journals might expect a few ‘Letters to the Editor" to discuss ‘matters arising’ after an article was published. But scholarly communications are becoming much more dynamic, with versions posted as ‘preprints’ before publication, corrections after publication, and potentially multiple versions of the same study appearing at different times. How should we handle this changing landscape for the benefits of researchers and consumers of the literature?

About Theodora Bloom

Theodora Bloom has been executive editor of The BMJ since June 2014. She has a PhD in developmental cell biology from the University of Cambridge and worked as a postdoctoral fellow at Harvard Medical School. She moved into publishing as an editor on the biology team at Nature, and in 1992 joined the fledgling journal Current Biology. After a number of years helping to develop Current Biology and its siblings Structure and Chemistry & Biology, Theo joined the beginnings of the open access movement. As the founding editor of Genome Biology she was closely involved in the birth of the commercial open access publisher BioMed Central. She joined the non-profit open access publisher Public Library of Science (PLOS) in 2008, first as chief editor of PLOS Biology and later as biology editorial director. She took the lead for PLOS on issues around data access and availability and launched PLOS’s data sharing policy. At The BMJ she is responsible for operations, delivering the journal online and in print.

Casey has been a strong champion of preprints and will discuss his efforts in this area including resources that he has shared to help advance the spread of preprints not only amongst researchers but publishers. These include letters to respond to journals that invite reviews but have unclear preprint policies. His lab members have also analyzed the licensing of preprints and the coverage of literature provided by the pirate repository, Sci-Hub. His talk will touch on each of these areas, and also a discussion of the Research Parasite and Symbiont Awards, which aim to advance recognition for data sharing and reuse.

About Casey Greene

Casey is an assistant professor in the Department of Systems Pharmacology and Translational Therapeutics in the Perelman School of Medicine at the University of Pennsylvania and the director of the Childhood Cancer Data Lab for Alex’s Lemonade Stand Foundation. His lab develops deep learning methods that integrate distinct large-scale datasets to extract the rich and intrinsic information embedded in such integrated data. Before starting the Integrative Genomics Lab in 2012, Casey earned his PhD for his study of gene-gene interactions in the field of computational genetics from Dartmouth College in 2009 and moved to the Lewis-Sigler Institute for Integrative Genomics at Princeton University where he worked as a postdoctoral fellow from 2009-2012. The overarching theme of his work has been the development and evaluation of methods that acknowledge the emergent complexity of biological systems.

Leonid Teytelman - Call to reduce random collisions with information; we can automatically connect scientists to the knowledge that they need.

Every scientist knows that virtually all papers, including their own, contain mistakes. A key motivation for creating protocols.io was to make it possible to share corrections and optimizations of published research protocols and to have this information automatically reach the scientists using these methods. While pushing relevant knowledge to the users is built into all aspects of protocols.io, we can do a lot more. If publishers, Crossref, and reference management platforms collaborate, we can move beyond the search towards a point where important information automatically reaches the appropriate researchers.

About Leonid (Lenny) Teytelman

Lenny is the Co-founder and CEO of protocols.io, an open access platform to share and discover research protocols. It enables scientists to make, exchange, improve and discuss protocols and it is poised to dramatically accelerate and to increase reproducibility of scientific research. Lenny did his graduate studies at UC Berkeley and finished his postdoctoral research at MIT. Lenny has a strong passion for sharing science and improving research efficiency through technology.

How can we facilitate the fair advancement and dissemination of knowledge? The risks and shortcomings within scholarly publishing are always under scrutiny, but some problems don’t seem to be going away. What should we do about obvious gender inequality within some disciplines, or the weight given to Impact Factor as a measure of quality? The Royal Society of Chemistry has a royal charter to publish scientific content in a way that serves the public interest, and as such its Data Science team devotes part of its time to analysing the social challenges facing scholarly publishing. In this talk, Nicholas Bailey will share some examples.

About Nicholas Bailey

Nicholas Bailey is a web analytics expert, a swimmer, a father, and a data geek. After spending several years in the Marketing team at the Royal Society of Chemistry, ultimately managing the database marketing team, he moved out of Marketing and into the Data Science team in order to work more closely with agile teams of developers and strengthen his data analysis and coding skills. Nicholas has a lot to say about measuring digital products, machine learning, and the potential of data science to contribute to positive social outcomes.

Miguel Escobar Varela - Digital Humanities in Singapore: some thoughts for the future.

Singapore-based researchers from a variety of disciplines are currently using digital tools to study the humanities, in areas as diverse as history and dance studies. This talk will present an overview of current projects and suggest a path for the growth of this field in Singapore. It argues that the future of DH requires better inter-institutional infrastructure for long-term data storage, clearer protocols for interoperability and more freely available and reusable datasets. This is easier said than done, but looking at the examples of other countries can provide some sources for inspiration.

About Miguel Escobar Varela

Miguel Escobar Varela is an assistant professor in the University Scholars Programme (USP) at the National University of Singapore. At the USP, Dr. Varela teaches in the domain of Humanities and Social Sciences. He is a theatre researcher and software programmer. His interests are in teaching theatre through interactive websites and applying computational methods to study performances in Singapore and Indonesia.

Kuansan Wang - Democratize access to scholarly knowledge with AI.

With the advent of big data and cloud computing, artificial intelligence has made tremendous strides in recent years. Not only has machine surpassed humans in playing the chess game Go and Jeopardy game shows, reports of superhuman performance in other highly cognitive tasks, ranging from image classification to speech recognition, also abound. Have we reached a stage where the advancements in AI can help tackle a problem in scientific pursuits, namely, the access and the dissemination of scholarly knowledge? This talk describes Microsoft Academic, a project inside Microsoft Research that uses the state-of-the-art AI in natural language understanding and knowledge acquisition to harvest knowledge from scholarly communications and make it available on the web. The talk will describe the technical challenges that have been overcome, the world-wide research collaborations that have since been enabled, and discuss the potentials of making knowledge more readily available to the mass.

About Kuansan Wang

Kuansan Wang is the Managing Director at Microsoft Research Outreach (MSR), where he started in March 1998 as a Researcher in the speech technology group working. In 2004, he moved to the speech product group and became a software architect where he helped create and ship the product Microsoft Speech Server, which is still powering the corporate call center for Microsoft. Since September 2007, he has been back at MSR, joining the newly founded Internet Service Research Center with a mission to revolutionize online services and make Web more intelligent. In March 2016, he took on an additional role as a Managing Director of MSR Outreach, an organisation with the mission to serve the research community.

Scenario planning for our future

Ed Pentz — Mon, 28 Aug 2017 00:00:00 +0000

Crossref is governed by a board of directors that meets in person three times a year in March, July and November. At the July meeting the board typically spends a significant amount of time on strategic planning in addition to its usual activities such as financial oversight, approving investment in new services based on staff and committee recommendations, reviewing and approving policies and fees for new and existing services and generally making sure Crossref is healthy and well run.

This year we worked with a facilitator to look farther into the future than normal using a technique called scenario planning to map out “strategic agendas” for the next five years. Scenario-based strategic planning doesn’t try to predict the future but allows us to be flexible in planning by looking at a range of different possible eventualities. This is particularly useful for Crossref because scholarly research and communications is changing rapidly and we operate in a very complex environment.

To prepare for the meeting our facilitator, Susan Stickely, prepared 12 “critical uncertainties” - impactful issues that could go either way and that will affect how Crossref works, its mission and even whether it needs to exist. To develop the critical uncertainties Susan interviewed Crossref staff, board members, general members and scholarly communications community influencers and we held a preparatory group exercise at the March board meeting. The critical uncertainties are:

Scholarly Communication Landscape: Increasing diversity? Or publishing disintermediated?
Machine Learning / Artificial Intelligence: Supporting? Or obsoleting the researcher and publishers?
Policy and Regulation: Limiting? Or visionary?
Financing of Scholarly Communication: Shrinking Pool? Or Expanding Pool?
Rise of Pre-print, New Content Sources: New, non-traditional? Or De-formalizing?
Tracking and Privacy: Increased Privacy? Or Loss of Privacy?
Cybersecurity: Secure? Or Vulnerable, Insecure?
Publisher Sustainability: Slow Progress? Or Fast Progress?
Impact of Open: Open or Closed? Or Slow to Change?
Source of Prestige and Recognition: New Source? Or Publisher, Institution?
Quality and Accuracy of Content: High? Or Low?
Geopolitical Stability and Stance: Stable, Unified? Or Unstable, Fragmented

In addition, from the interviews Susan was able to summarize Crossref’s distinctive competencies as:

Having a reputation as a trusted, neutral one-stop source of metadata and services
Managing scholarly infrastructure with technical knowledge and innovation
Convening and facilitating scholarly communications community collaboration

To be successful Crossref will need to continue to invest in, apply, and evolve these distinctive competencies and strategic dilemmas and challenges.

Over a day and half of discussions and breakout sessions the board and staff drew up a number of scenarios and created a draft strategic agenda for Crossref. Over the next couple of months we’ll be working on refining the strategic agenda and will be presenting the results to members in the next couple of months.

One theme that emerged is for Crossref to engage more with funders and build on the work with done with them in creating the Crossref Funder Registry. We have started a new Funder Advisory Group and, among other things, are working with them on a prototype for a new registry of grant identifiers.

In the regular board session the board approved three recommendations from the Membership and Fees Committee:

To approve the recommendations with respect to volume discounts for current deposits of posted content (i.e. preprints).
To create a new “peer review report” record type with a specific metadata schema and a bundled fee of $1.25 to be charged for a content item and all the reports associated with it.
To update the metadata delivery offering to have a single agreement that covers all metadata APIs/delivery routes, to adopt a single (updated) fee structure, and to remove case-by-case opt-outs for metadata.

Item number 3 involves a number of big changes - for example the removal of the case-by-case opt outs requires a change to the main Membership Agreement - so we will be sending out more information to members and Affiliates in September and October about the changes and our implementation plans.

You can see the full history of the motions from every Board meeting on our website.

Another major issue that the board discussed is the upcoming election for the board of directors. In order to broaden participation and be inclusive there was a new process this year. The Nominating Committee put out a call for expressions of interest for candidates to be on the slate for the election. We had a great response and there were 25 expressions of interest reviewed by the Nominating Committee who came up with a slate of nine excellent candidates for the six seats up for election. This is the first time that there are more candidates than seats on the slate so it’s particularly important for members to vote this year. See the recent blog post about the election process and the slate for more details.

The next board meeting is in November in conjunction with Crossref LIVE17 in Singapore.

Coming to a venue near you

Vanessa Fairhurst — Thu, 24 Aug 2017 00:00:00 +0000

First of all – hello! I’m Vanessa. I’m fairly new to Crossref, having just joined our outreach team a few weeks ago. I previously worked in International Development, enabling individuals and institutions in Africa, Asia and Latin America to access cutting edge scholarly research and knowledge, supporting national development and transforming lives.

A firm belief in the importance of connecting research and information around the world led me to Crossref where my role of International Community Outreach Manager connects me with a range of different people working across diverse disciplines and sectors. I’ll be supporting the coordination of our local LIVE events and helping to set up an ambassador program (more information on this coming soon) to deepen regional connections around the globe. You can read more about myself and my colleagues at Crossref on our People page.

As Crossref membership continues to grow globally, it becomes increasingly important for us to look at new ways to engage with our international membership base.

You may have heard about our LIVE local events, or even attended one in person before. These are free-to-attend, one day, regional events (local to you), providing a tailored program of activities which include information on the key concepts of Crossref, the services we offer and our future plans.

In the past year we have held LIVE local events in Brazil, Beijing, Boston and most recently Seoul. We also have a London LIVE event coming up soon. Next year we are aiming to be even more ambitious, hoping to expand our activities to a number of different countries around the world.

Images left to right, Crossref LIVE participants in Seoul, Crossref LIVE speakers in Brazil, and literature we use at our LIVE events

||||

When running our LIVE local events, we collaborate with local organisations to ensure they are appropriate, accessible, and applicable to the country context. Members support us by lending their local expertise with regards to venue selection, suggestions for speakers, tailored content, translation of materials and participant enrolment. We collaborate on logistics, content, Crossref speakers and the promotion of the event to our members and the wider community.

When running our LIVE local events, we collaborate with local organisations to ensure they are appropriate, accessible, and applicable to the country context. Members support us by lending their local expertise with regards to venue selection, suggestions for speakers, tailored content, translation of materials and participant enrollment. We collaborate on logistics, content, Crossref speakers and the promotion of the event to our members and the wider community.

We will release more information of upcoming regional events in due course, but we are working on the following countries as priorities for 2018-19:

Asia-Pacific: Malaysia, Indonesia, Japan, Taiwan, Australia
Central Asia: India
Latin America: Mexico, Colombia, Chile, Brazil
Middle East: UAE (Dubai or Abu Dhabi)
Africa: South Africa, Kenya
Eastern Europe: Turkey, Greece, Bulgaria, Romania, Serbia, Poland
Western Europe: Germany, Spain, UK
North America: Canada, USA

If you are interested in hosting a LIVE local event or have any suggestions for one in your region, then we would love to hear from you. View more information on our LIVE locals page or contact us to hear more or get involved.

2017 election slate

Lisa Hart Martin — Thu, 17 Aug 2017 00:00:00 +0000

Slate of 2017 board candidates announced, and it’s going to be exciting

Crossref is always evolving and the board knows it must evolve with us so we can continue to provide the right kind of services and support for you, as members of the research community.

This year two things happened for the first time: we used our updated bylaws see article VII, section 2 agreed by the board last year, to allow more candidates than available seats; and secondly, to issue an open call for expressions of interest. Many members of the current board felt it was vital to move to this more transparent process.

With Crossref developing new services for new types of members at a rapid pace, it’s an exciting time to be on the board of directors. With 25 expressions of interest it seems we’re not the only ones who think so!

From these 25 applications, the Nominating Committee has proposed the following nine candidates to fill the six seats open for election to our board of directors:

American Institute of Physics (AIP), Jason Wilde, USA
F1000 Research, Liz Allen, UK
Institute of Electronic and Electrical Engineers (IEEE), Gerry Grenier, USA
The Institution of Engineering and Technology (IET), Vincent Cassidy, UK
Massachusetts Institute of Technology Press (MIT Press), Amy Brand, USA
OpenEdition, Marin Dacos, France
SciELO, Abel Packer, Brazil
SPIE, Eric Pepper, USA
Vilnius Gediminas Technical University Press (VGTU Press), Eleonora Dagiene, Lithuania

Read the candidates’ organisational and personal statements

Candidates were chosen based on the following criteria:

That board representation should be reflective of membership
A balance of types and sizes of organisations
That all committee choices and recommendations were unanimous

You can be part of this important process, by voting in the election

If your organisation is a member of Crossref on September 15 2017, you are eligible to vote when voting opens on September 28 (affiliates, however, are not eligible to vote).

How can you vote?

On September 28, your organisation’s designated voting contact will receive an email with a link to the formal Notice of Meeting and Proxy Form with concise instructions on how to vote. An additional email will be sent with a username and password along with a link to our online voting platform. It is important to make sure your voting contact is up-to-date.

Want to add your voice?

We are accepting independent nominations until 7 November 2017. organisations interested in standing as an independent candidate should contact me by this date with the endorsements of ten other Crossref members.

The election itself will be held at LIVE17 Singapore, our annual meeting, on 14 November 2017. We hope you’ll be there to hear the results.

You do want to see how it's made — seeing what goes into altmetrics

Joe Wass — Mon, 14 Aug 2017 00:00:00 +0000

There’s a saying about oil, something along the lines of “you really don’t want to see how it’s made”. And whilst I’m reluctant to draw too many parallels between the petrochemical industry and scholarly publishing, there are some interesting comparisons to be drawn.

Oil starts its life deep underground as an amorphous sticky substance. Prospectors must identify oil fields, drill, extract the oil and refine it. It finds its way into things as diverse as aspirin, paint and hammocks. And as I lie in my hammock watching paint dry, I’m curious to know how crude oil made its way into the aspirin that I’ve taken for the headache brought on by the paint fumes. Whilst it would be better if I did know how these things were made, not knowing doesn’t impair the efficacy of my aspirin.

Altmetrics start life deep inside a number of systems. Data buried in countless blogs, social media and web platforms must be identified, extracted and refined before it can be used in products like impact assessments, prompts to engagement, and even tenure decisions. But there the similarity ends. Like the benzene in my aspirin, the data that goes into my favourite metric has come a long way from its origins. But that doesn’t mean that I shouldn’t know how it was made. In fact, knowing what went into it can help me reason about it, explain it and even improve it.

Heavy industry or backyard refinery?

When you head out to fill your car, you buy fuel from a company that probably did the whole job itself. It found the crude oil, extracted it, refined it, transported it and pumped it into your car. Of course there are exceptions, but a lot of fuel is made by vertically integrated companies who do the whole job. And whilst there are research scientists who brew up special batches for one-off pieces of research, if you wanted to make a batch of fuel for yourself you’d have to set up your own back-yard fractional distillation column.

Because the collection of a huge amount of data must be boiled down into altmetrics, organisations who want to produce these metrics have a big job to do. They must find data sources, retrieve the data, process it and produce the end product. The foundation of altmetrics is the measurement of impact, and whilst the intermediary data is very interesting, the ultimate goal of a metric is the end product. If you wanted to make a new metric you’d have two choices: set up an oil refinery (i.e. build a whole new system, complete with processing pipeline) or a back-yard still (a one-off research item). Either option involves going out and querying different systems, processing the data and producing an output.

Being able to demonstrate the provenance of a given measurement is important because no measurement is perfect. It’s impossible to query every single extant source out there. And even if you could, it would be impossible to prove that you had. And even then, the process of refinement isn’t always faultless. Every measurement out there has a story behind it, and being able to tell that story is important when using the measurement for something important. Data sources and algorithms change over time, and comparing a year-old measurement to one made today might be difficult without knowing what underlying observations went into it. A solution to this is complete transparency about the source data, how it was processed, and how it relates to the output.

Underlying data

This is where Crossref comes in. It turns out that the underlying data that goes into altmetrics is just our kind of thing. As the DOI Registration Agency for scholarly literature, it’s our job to work with publishers to keep track of everything that’s published, assign DOIs and be the central collection and storage point for metadata and links. Examples of links stored in Crossref are between articles and funders, clinical trial numbers, preprints, datasets etc. With the Event Data project, we are now collecting links between places on the web and our registered content when they’re made via DOIs or article landing pages.

This data has wider use than just than altmetrics. For example, an author might want to know over what time period a link to their article was included in Wikipedia, and which edit to the article was responsible for removing it and why. Or, in these days of “fake news”, someone may want to know everywhere on Twitter that a particular study is referenced so they can engage in conversation.

Whilst the field of altmetrics was the starting point for this project, our goal isn’t to provide any kind of metric. Instead, we provide a stream of Events that occurred concerning a given piece of registered content with a DOI. If you want to build a metric out of it, you’re welcome to. There are a million different things you could build out of the data, and each will have a different methodology. By providing this underlying data set, we hope we’ve found the right level of abstraction to enable people to build a wide range of things.

Every different end-product will use different data and use different algorithms. By providing an open dataset at the right level of granularity, we allow the producers of these end-products to say exactly which input data they were working with. By making the data open, we allow anyone else to duplicate the data if they wish.

Sticky mess

To finish, let me return to the sticky mess of the distillation column. We identify sources (websites, APIs and RSS feeds). We visit each one, and collect data. We process that data into Events. And we provide Events via an API. At each stage of processing, we make the data open:

The Artifact Registry lists all of the sources, RSS feeds and domains we query.
The Evidence Registry lists which sites we visited, what input we got, what version of each Artifact was used, and which Events were produced.
The Evidence Log describes exactly what every part of the system did, including if it ran into problems along the way.
The Events link back to the Evidence so you can trace exactly what activity led up to the Event.
All the code is open source and the version is linked in the Evidence Record, so you can see precisely which algorithms were used to generate a given Event.
Anyone using the Data can link back to Events, which in turn link back to their Evidence.

The end-product, Events, can be used to answer altmetrics-y questions like “who tweeted my article?”. But the layers below that can be put to a range of other uses. For example:

“Why does publisher X have a lower Twitter count?”. The Evidence Logs might show that they tend to block bots from their site, preventing data from being collected.
“Why did their Twitter count rise?”. The Evidence Logs might show that they stopped blocking bots.
“What does Crossref think the DOI is for landing page X?”. A search of the Evidence Logs might show that the Event Data system visited the page on a given date and decided that it corresponded to DOI Y.
“Which domains hold DOI landing pages?”. The “Domains” Artifact will show the domains that Event Data looked at, and the Evidence Logs will show which versions were used over time.

By producing not only Events, but being completely transparent about the refinement process, we hope that people can build things beyond traditional altmetrics, and also make use of the intermediary products as well. And by using open licenses, we allow reuse of the data.

See you in Toronto!

There’s so much more to say but I’ve run out of ink. To find out more, come to 4:AM Altmetrics Conference! I’ll be speaking at the conference in Session 10 on the 28th. I’ll also be at the Altmetrics Workshop on the 26th. Stacy Konkiel and I are hosting the Hackathon on the 29th, where you can get your hands on the data. See you there!

This blog post was originally posted on the 4:AM Altmetrics Conference Blog.

Using the Crossref REST API. Part 4 (with CLA)

Rachael Lammey — Tue, 25 Jul 2017 00:00:00 +0000

As a follow-up to our blog posts on the Crossref REST API we talked to the Copyright Licensing Agency (CLA) about the work they’re doing, and how they’re using the Crossref REST API as part of their workflow.

Alex Cole, Senior Business Analyst at the Copyright Licensing Agency introduces the DCS

The Digital Content Store (DCS) is an innovative rights, technology and content platform for UK Higher Education Institutions (HEIs), which was developed collaboratively with HEIs, publishers and technology partners. The platform is included in the CLA annual licence fee and is an optional tool for licensees.

At its core, the system is a searchable repository of digital copies that have been created under the licence by HEIs (the CLA Digital Content Store), it also functions as a workflow management tool. When extracts are digitised by HEIs under the CLA Licence, they are uploaded directly to the DCS. Once an extract is uploaded and assigned to a course, students are able to access the extract via a secure link. Every year HEIs are obliged to report all of these digitised items to CLA as part of the terms of their copyright blanket licence. Prior to the DCS, HEIs were having to submit this data manually, a process that could take days, if not weeks. The system removes the need for annual census reporting to CLA, reducing the data collection burden on the HE sector and creating administrative efficiencies through streamlining the digital course pack creation process.

Can you talk about how you’re using the Crossref REST API within CLA Digital Content Store (DCS)?

When a DCS user adds a new extract to a course they need to include relevant metadata. This metadata is necessary, as it ultimately helps CLA in correctly identifying the copyright owner of the extract so that we can make sure they receive fair payment in our royalties distributions. The Crossref REST API supplies the DCS user with article and journal metadata so that they can provide the correct information about the content they are uploading. Using the API saves the user the time they would have otherwise spent searching for this data, streamlining their workflow and making the process more efficient.

Searching for and adding content in the DCS

What are your future development plans?

We’re continuing to develop the DCS in order to improve user experience for our customers. We’re currently looking into opening up access for our users by allowing academics to submit requests to the DCS via a web-form and our own DCS Course Content URL API. We are also looking into incorporating the Crossref REST API into some of our back office workflows to improve efficiency and simplify our workflow. The metadata that we can retrieve from Crossref can help us match customer usage to our rights database.

What else would you like to see in Crossref metadata?

Going forward we’d like to see:

More books included in the database.
Indicating if an ISSN is associated with the print or digital edition of a journal.

Thanks Alex!

Event Data enters Beta

Jennifer Kemp — Wed, 05 Jul 2017 00:00:00 +0000

We’ve been talking about it at events, blogging about it on our site, living it, breathing it, and even sometimes dreaming about it, and now we are delighted to announce that Crossref Event Data has entered Beta.

A collaborative initiative by Crossref and DataCite, Event Data offers transparency around the way interactions with scholarly research occur online, allowing you to discover where it’s bookmarked, linked, liked, shared, referenced, commented on etc., across the web, and beyond publisher platforms.

The name Event Data reflects the nature of the service, as it collects and stores digital actions that occur on the web, from the quick and simple, such as bookmarking and referencing, through to deeper interconnectivity such as exposing the links between research artifacts. Each individual action is timestamped and recorded in our system as an Event, and made available to the community via an API.

Event Data will be available for absolutely anyone to use; publishers, third party vendors, editors, bibliometricans, researchers, authors, funders etc., and with tens of thousands of events occurring every day, there’s a wealth of insight to be gained for those interested in analyzing and interpreting the data.

It’s important to note that Event Data does not provide metrics. What is does provide is the raw data to help you facilitate your own analysis, giving you the freedom to integrate the data into your own systems.

We are currently working very closely with a few organisations with specific use cases who are helping us to test and refine Beta before we launch our production service later this year. If you decide to take a look at Beta yourself, all the data you collect from Event Data is licensed for public sharing and reuse according to our Terms of Use.

Until Event Data is in production mode, we do not recommend building any commercial or customer-based tools off the data. If you are not in the Beta test group but are interested in participating, please contact me below. For more information about Event Data, please see our user guide.

Please contact me, Jennifer Kemp—Outreach Manager for Event Data—with any questions.

Crossref and colleagues in South Korea

Ginny Hendricks — Fri, 30 Jun 2017 00:00:00 +0000

Connecting Crossref, ORCID, DataCite, and our communities

Q: What do you get if you combine our three organisations for a week to catch up with our Korean community - publishers, librarians, universities, researchers, and service providers? A: Two events, plenty of meetings, great conversations and feedback, fabulous Korean hospitality, and a little jet-lag.

Over the past few years, Crossref has seen huge growth in our members in Korea. We have nine Sponsoring Affiliates (who look after nearly 1,000 members between them), two Sponsoring Members and nearly 80 Library members. With the International DOI Foundation (IDF) strategy meeting taking place in Daejon, it seemed sensible to combine that with our own events and meetings with key organisations. This also fitted nicely with some plans that ORCID and DataCite had, so we combined forces.

We (that’s me, Rachael Lammey, Ed Pentz, and Geoffrey Bilder) hosted a Crossref LIVE local event on Monday 12th June for around 80 members and affiliates. We were joined by Alice Meadows and Nobuko Maiyairi (ORCID), Martin Fenner (DataCite), and Professor Sun-Tae Hong (Seoul National University) as co-presenters. We looked at the global reach of Korean research, and how registering content with Crossref and participating in services like Reference Linking helps create valuable connections between research outputs. With so many established members in Korea, we were able to go beyond the basics and emphasize the importance of metadata input, metadata delivery, and preview our upcoming Event Data service. We also talked data-sharing and the value of integrating ORCID iDs into publisher and institution workflows.

_Growth in research outputs in Asia Pacific 2009-2017. Source: Web of Science databases SCI-E, SSCI and AHCI only, downloaded 19/4/2017. Data provided by Wiley (thank you!)_

Later in the week we took a multi-pronged approach to highlight the many shared principles of our organisations and discuss the specific initiatives we’re collaborating on. We held the Joint Global Infrastructure Conference covering the global nature of what we do and the connections/interoperability between ORCID, DataCite and Crossref. This interoperability and our governance structures lend themselves to cooperation on other initiatives such as Metadata 2020 and The OI Project, which we were able to share.

Check out all #jgic_seoul tweets.

Guest speakers volunteered to talk about how they work with our organisations - we were joined by Choon Shil Lee from the Korean Association of Medical Journal Editors (KAMJE) to demonstrate their ORCID integrations, and Hideaki Takeda from the Japan Link Centre (JaLC) who discussed the infrastructure and services they use to register and disseminate content globally. User stories like this are great - they highlight how people work with our services, give others ideas, and also flag up where we can do more.

Part of doing more involved providing clarification on Crossref’s position alongside other DOI Registration Agencies. With a new Registration Agency in Korea, we needed to communicate the global nature of what we do to help our members achieve their discoverability goals, as not all DOIs are made equal. Through working with ORCID and DataCite colleagues we were able to place great importance both on our work worldwide, and on the benefits to Korean societies in collaborating outside national boundaries.

Combining talks from our three organisations was a great opportunity to emphasize the importance of shared global infrastructure. Geoffrey Bilder’s plug socket analogy is apt - services that work cross-border, cross-language, and cross-subject areas streamline processes for all of our different communities and enable research to travel beyond national boundaries and help it be found, linked, cited and assessed.

Want to find out more? Slides from both meetings are available here and here, and watch out for further collaborative events.

Crossref receives SOC accreditation for data integrity and security

Lisa Hart Martin — Wed, 21 Jun 2017 00:00:00 +0000

We are delighted to announce that Crossref has been awarded the Service organisation Control (SOC) 2® accreditation after an independent assessment of our controls and procedures by the American Institute of CPA’s (AICPA).

The SOC 2® accreditation is awarded to service organisations that have passed standard trust services criteria relating to the security, availability, and processing integrity of systems used to process users’ data and the confidentiality and privacy of the information processed by these systems.

The AICPA’s assessment also reviewed our vendor management programs, internal corporate governance and risk management processes, and regulatory oversight.

Find out more about the SOC accreditation structure

Now put your hands up! (for a Similarity Check update)

Madeleine Watson — Tue, 06 Jun 2017 00:00:00 +0000

Today, I’m thinking back to 2008. A time when khaki and gladiator sandals dominated my wardrobe. The year when Obama was elected, and Madonna and Guy Ritchie parted ways. When we were given both the iPhone 3G and the Kindle, and when the effects of the global financial crisis lead us to come to terms with the notion of a ‘staycation’. In 2008 we met both Wall-E and Benjamin Button, were enthralled by the Beijing Olympics, and became addicted to Breaking Bad. And lest we forget, 2008 was also the year in which Beyoncé brought us Single Ladies; in all its sassy hand-waving, monochrome glory. For Crossref though, 2008 holds another important milestone as it was the year we launched our Similarity Check initiative. Today, the artist formerly known as CrossCheck provides our members with cost-effective access to Turnitin’s powerful text comparison tool, iThenticate.

Fast forward nearly a decade, and it’s wonderful to see just how Similarity Check membership has grown in the nine years since launch; from 16 original members in 2008 to over 1,300 today.

Figure 1.1 The number of publishers participating in the Similarity Check service each year between 2008 – 2017 (to April)

Usage of iThenticate is also consistent with this growth in membership, and throughout 2016 our members checked over four million manuscripts for similarity using the tool. As Similarity Check members contribute their full-text content into Turnitin’s database, this increase in membership also has a dramatic impact on the volume of content indexed by Turnitin. Today, members can compare their manuscripts against Turnitin’s database of over 60 million full-text works provided by Similarity Check members. With over 88 million works currently registered with Crossref, this means that 68% of all content deposited with us is now available for comparison in iThenticate.

Over the years we have worked very closely with Turnitin to help champion new iThenticate feature developments that best support our member’s use of the tool as a core function of their editorial workflow. Many of our members too have also worked together with Turnitin to provide feedback on user experience and design.

Below, Turnitin’s Product Manager for iThenticate, Sun Oh, shares an insight into their research process and how Similarity Check member’s feedback has been critical in developing new and improved functionality in iThenticate.

Read on to learn more from Sun…

Sun Oh is a Senior Product Manager at Turnitin. She is currently the Product Manager for iThenticate and backend systems including the Content Intake System and similarity reports.

Last year we surveyed our Crossref customers to find out what Similarity Check improvements they would like to see and noticed a recurring request for the ability to compare two or more personally sourced documents.

We were intrigued and decided to run with it. We contacted the respondents who had asked for this, and started conversations to find out more. This helped us gather invaluable data, which in turn helped us to build the feature based on real use cases and with a clear view of what was wanted.

The design prototypes were reviewed for usability and effectiveness each step of the way by the respondents and once we had the feature up and running, those who requested it in our initial survey were among the first to trial it.

We’re thrilled to announce that we’ve now launched the new Doc-to-Doc comparison feature, available through iThenticate’s native interface. Simply select the Doc-to-Doc comparison upload method from the document submission panel.

If you are a Crossref member using Similarity Check, you have exclusive early access to this new feature, which allows you to use iThenticate’s powerful similarity check functionality and apply it to your own, private documents.

How does Doc-to-Doc Comparison work?

Doc-to-Doc comparison allows users to upload one primary document and compare it against up to five other documents.

Figure 1.2 The document upload screen for Doc-to-Doc comparison

When the upload is complete, a similarity score is generated for the primary document based on the amount of similar content found in the comparison documents. A full comparison report is also available.

The comparison report will open in the document viewer, and will display the primary document along with a list of the comparison documents and with their similarity percentage. If one of the comparison documents doesn’t include text that matches the primary document, iThenticate will still display it anyway, with a 0% score, allowing users to rule it out of their inspection. The similarity report will be stored securely in the user’s folder until they delete it.

Figure 1.3 Similarity report for Doc-to-Doc comparison

As these documents will not be stored in a shared database, they won’t affect the similarity score of any future submissions. Primary and comparison documents remain completely private and will not be indexed into the shared iThenticate content database.

To get a better idea of how Doc-to-Doc comparison works, check out the iThenticate feature guide on the Turnitin website.

Start using Doc-to-Doc Comparison now!

If you’re a Crossref member using Similarity Check, you can log in to your iThenticate account now and select the Doc-to-Doc comparison link on the homepage.

What else is new in iThenticate in this new release?

New Look

In addition to Doc-to-Doc comparison, we decided to refresh the look and feel of iThenticate; the same tools our users know and trust, now with a modern interface. Users will also notice that iThenticate now has more readable font and friendlier styling throughout.

Report Mode Memory

To make life easier, iThenticate now remembers whether users were in the All Sources or Match Overview mode when they last used the Document Viewer. iThenticate will then open documents in this mode automatically hereafter.

Improved Submission Process

We’re also enhancing our submission process by making the upload requirements more inclusive. We’ve increased the possible file size limit from 40MB to 100MB when uploading to either the database or to Doc-to-Doc comparison, and PowerPoint (.ppt) and Excel (.xlsm) file formats are now accepted.

Developments completed in 2016

If Similarity Check members haven’t had a chance to check out the improvements we introduced in iThenticate throughout 2016, here’s a quick recap. You can always find our updates on the What’s New page of the iThenticate website.

Download User List

The ability for administrators to download a list of all the users in their account has been added. This list will allow administrators to easily send emails to users.

Similarity Score Calculation Update

We updated how the similarity score is calculated when bibliographic material is excluded from a similarity report. Now, when bibliography exclusion is enabled, the word count of the bibliography is not included when calculating the overall percentage. This update to the similarity report calculation helps to provide users with a more accurate similarity score.

Improved Security

We are fully committed to keeping user’s data safe and secure at all times. To that end, we’ve added additional security logging, put in measures to enforce stronger passwords, and enabled Captcha after failed login attempts.

Faster Report Generation

We’ve increased the number of resources dedicated to the generation of similarity reports for our iThenticate service. As a result, users should see faster turnaround times for similarity reports.

Support for Eight Additional Languages

The iThenticate user interface is now available in eight additional languages: German, Dutch, Latin American Spanish, Brazilian Portuguese, Italian, French, and both Simplified & Traditional Chinese. When adding new users to an account, administrators can specify the language of the new user, which will then send a welcome email in the selected language. Individual users can also set their preferred language by selecting a language from the Language dropdown in the Settings menu.

Content Intake System

We’ve developed a new Content Intake System which enables our publication content database to scale so that our users can compare against a constantly growing database of the most recently published content. This allows us to index Similarity Check members’ data in a much more reliable and efficient way than legacy intake methods. And recently, we’ve made the collecting and processing of content from Crossref members using Similarity Check even faster by parallelising our processors. This means that we have more processors running simultaneously to process data.

By removing the need for crawling, we will also minimize our impact on traffic to a Similarity Check member’s public-facing website. The Content Intake System is able to directly collect full text URLs from members DOI metadata. This results in a huge reduction in the time it takes from when a publisher first deposits a new DOI with Crossref, to when the content is indexed by us into our full-text publication database. To date, we’ve been able to index the content associated with 60 million Crossref DOIs, and have indexed more than 165 million published works in total which submissions are compared against in iThenticate.

Walker (web crawler)

We’ve developed a new web crawler. Referred to as “Walker”, the crawler makes it possible to provide quicker and more reliable similarity matches to content available on the web. Not to be confused with the Content Intake System mentioned above, Walker’s purpose is to crawl the public web and is not used for indexing full-text content from Similarity Check members.

Using Walker, we’re adding an average of nearly 10 million new web pages to our content database per day, ensuring we have the freshest internet content available to find matches against.

We’d love to get your feedback!

As we design and develop new features, we want to make sure we’re fully understanding Similarity Check member’s needs and would love the opportunity to engage with users for further research. If you’d like to sign up to participate in user research for upcoming feature developments, please take a few minutes to fill out our Feedback Program Form. We look forward to connecting with you!

Contact Turnitin (EDIT 30/04/24: Support for iThenticate, contact details updated )

Please go to our Get help with Similarity Check page

~~For iThenticate technical and billing support, please email tiisupport@turnitin.com~~

~~For questions about content indexing, please contact Gareth at gmalcolm@turnitin.com~~

~~For iThenticate product development questions, please contact Sun at soh@turnitin.com~~

* Sun Oh, Product Manager for iThenticate*

**Thanks to Sun and the whole team at Turnitin for sharing this update.**

For more information about Similarity Check, visit our service page.

Want to join Crossref Similarity Check? Please contact our membership specialist.

Data citations and the eLife story so far

Melissa Harrison — Thu, 18 May 2017 00:00:00 +0000

When we set up the eLife journal in 2012, we knew datasets were an important component of research content and decided to give them prominence in a section entitled ‘Major datasets’ (see images below). Within this section, major previously published and generated datasets are listed. We also strongly encourage data citations in the reference list.

Major Datasets for “Structural basis of protein translocation by the Vps4-Vta1 AAA ATPase” by N. Monroe, H. Han, P. Shen, et. al.

Almost five years on and I feel we have still not cracked it! We have signed up to the Force11 data citation principles, which were published three years back; we have been actively involved in working groups of Force11 and others, for example the Data Citation Roadmap for Scientific Publishers and the JATS XML data citation recommendation of JATS4R. I am also currently working with other publishers to come up with recommended JATS XML tagging for data availability statements, which is easier said than done considering the nuances of dataset uses and also how different publishers approach this.

Added to this, there is still significant push-back from authors about putting all dataset citations in the reference list (for example, authors are concerned about self-citing by citing a dataset created as part of the research article; “dataset citations” that are in effect a link to a search results page on a database; and the necessitation of hundreds of reference entries if an author has used a large base for the research).

While eLife is very active in this space, and aims to arrange and mark up the datasets and citations produced by our authors in line with recommendations, the recommendations still have some gaps and the complete picture is not yet clear.

In late 2014, we brought in-house the process of depositing Crossref metadata (previously our online host did this for us). It gave us control of our processes and, at the time, we sent all the information we could to Crossref and have ensured our references are open and available in the Crossref public API. The code for this conversion process is all open-source and available for reuse. It can be found on GitHub. Since then, besides small improvements to the code and troubleshooting problems, we’ve not updated the code. I have been keeping a list of Crossref features and new deposit metadata we can add to our deposits, and now is the time for us to start working on this again.

One of the items we’ll be addressing is data citations.

The Crossref reference schema does not cater well for non-book or -journal content, and if an item does not have a DOI, the “reference” is not very useful because of the few tags available in the Crossref schema.

However, Crossref have introduced the relationship type to their schema, so data references can be well linked and mineable. As I see Crossref as a potential broker between publishers and data repositories in the future, using the relationship-type deposit for all datasets will assist this and also allow these data points to more easily be seen within the article Nexus framework (see the recent blog post, How do you deposit data citations?).

At eLife, we already distinguish between Dataset generated as part of research results (relationship type in the Crossref schema: “isSupplementedBy”) and Dataset produced by a different set of researchers or previously published (relationship type: “references”). Therefore, it will not be hard for us to convert all the information about data referencing that is within the dataset section into a relationship-type deposit in the conversion to Crossref XML.

We have also recently gone through an exercise of defining a set of rules for all our references and, of the 12 allowed types, one is data. The rules for Schematron (a rule-based validation language for making assertions about the presence or absence of patterns in XML trees; see also this useful article about Schematron on the JATS4R learning centre) have been written for the eLife ‘business’ rules. Subject to final testing, these will be integrated into our workflow (the Schematron is open source and available for reuse on GitHub, and we will also build an API for people to use the Schematron direct). This will allow us to easily identify all data references and convert them into relationship types in the XML delivered to Crossref. This way, they will not be lost in the references section of our deposits, but properly identified.

However, we do appreciate this will become harder for us as authors become more familiar with datasets as references, because we will not be able to identify the difference between generated and analysed datasets so easily.

The code developed and used to complete these conversions will, again, be on Github and open source, and we actively encourage the reuse of this.

While the industry is still working on the best way to deal with data and ensuring it is given the prominence it requires, we feel this is the best approach we can take. Nothing is forever and we can still change what we do in the future. The beauty of open-source code also means that if there is an alternative approach now or in the future, the code we wrote at eLife can be developed by someone else in the future and we can all benefit.

If you have any questions, please do not hesitate to contact us.

Want to be on our Board?

Lisa Hart Martin — Fri, 28 Apr 2017 00:00:00 +0000

Do you want to affect change for the scholarly community?

Our Nominating Committee is inviting expressions of interest to serve on the Board as it begins its consideration of a slate for the November 2017 election.

Key responsibilities of the Board are setting the strategic direction for the organisation, providing financial oversight, and approving new policies and services. Some of the decisions the board has made in recent years include:

Establishing The OI Project to create a persistent organisation Identifier;
Inclusion of preprints in the Crossref metadata; and
The approval to develop Event Data which will track online activity from multiple sources.

Any member can express interest in serving on the Board

We are seeking people who know about scholarly communications and would like to be part of our future. If you have a vision for the international Crossref community, we are interested in hearing from you. Crossref members that are eligible to vote, and would like to be considered, can express their interest together with statements of interest from you and from your organisation. The form should be completed and sent to us before 01 June 2017.

The role of the Nominating Committee

The Nominating Committee meets to discuss change, process, criteria, and potential candidates, ensuring a fair representation of membership. The Committee is made up of three board members not up for election, and two non-board members.

Current Nominating Committee members:

John Shaw, Sage (Chair)
Mark Patterson, eLife
Paul Peters, Hindawi
Chris Fell, Cambridge University Press
Rebecca Lawrence, F1000 Research

About the election and our Board

We have a principle of one member, one vote; our board comprises a cross-section of members and it doesn’t matter how big or small you are, every member gets a single vote. Board terms are three years, and one third of the Board is eligible for election every year. There are six seats up for election in 2017. The board meets in a variety of international locations in March, July, and November each year. View a list of the current Crossref Board members and a history of the decisions they’ve made (motions). The election opens online in late September 2017 and voting is done by proxy online or in person at the annual business meeting during Crossref LIVE in November 2017. Election materials and instructions for voting will be available to all Crossref members online in late September 2017. The board needs to be truly representative of Crossref’s global and diverse membership of organisations who publish.

Please express interest using the form, or email me with any questions.

The OI Project gets underway planning an open organisation identifier registry

Ed Pentz — Tue, 28 Mar 2017 00:00:00 +0000

At the end of October 2016, Crossref, DataCite, and ORCID reported on collaboration in the area of organisation identifiers. We issued three papers for community comment and after input we subsequently announced the formation of The OI Project, along with a call for expressions of interest from people interested in serving on the working group.

We had a great response and are happy to report that the Working Group has now been established, and is already underway with work to develop a plan for an open, independent, not-for-profit, sustainable, organisation identifier registry.

There is information about the OI Project Working Group on the ORCID website including a list of the 17 working group members. They represent a broad range of scholarly communications stakeholders. Our scope of work includes three separate but interdependent areas:

Governance;
Registry Product Definition; and
Business Model & Funding.

The initial goal of the Working Group is to create a thorough and robust implementation plan by the end of 2017.

Please take a look at the website for more information and we’ll provide updates as things progress throughout the course of the year.

Please contact us with any questions.

Revised Crossref DOI display guidelines are now active

Ed Pentz — Wed, 15 Mar 2017 00:00:00 +0000

We have updated our DOI display guidelines as of March 2017, this month! I described the what and the why in my previous blog post New Crossref DOI display guidelines are on the way and in an email I wrote to all our members in September 2016. I’m pleased to say that the updated Crossref DOI display guidelines are available via this fantastic new website and are now active. Here is the URL of the full set of guidelines in case you want to bookmark it (https://doi.org/10.13003/5jchdy) and a shareable image to spread the word on social media.

This blog is a quick reminder that all Crossref members should now be displaying DOIs in the recommended new format from this month, on any new content you publish online. Please note these guidelines are for Crossref DOIs only, we have nearly 90 million registered but there are others, and not all DOIs are made equal.

The main changes are to display the DOI as a full, linked URL using HTTPS:

https://doi.org/10.xxxx/xxxxx

For background on the HTTPS issue please read Geoffrey Bilder’s blog post, Linking DOIs using HTTPS.

What will happen if you don’t update your Crossref DOI display?

We tell members that they should be working towards making the change even if they can’t do it until later - we recognize that it is not always an easy change to make.

However, if members don’t make the change, nothing immediate will happen (Crossref won’t fine you!) although as more members make the change your display will look odd and out of place compared with other members’ content.

If you have any questions please do not hesitate to contact us.

How do you deposit data citations?

Jennifer Lin — Thu, 02 Mar 2017 00:00:00 +0000

Please visit Crossref’s official Data & Software Citations Deposit Guide for deposit details.

Very carefully, one at a time? However you wish.

Last year, we introduced linking publication metadata to associated data and software when registering publisher content with Crossref Linking Publications to Data and Software. This blog post follows the “whats” and “whys” with the all-important “how(s)” for depositing data and software citations. We have made the process simple and fairly straightforward: publishers deposit data & software links by adding them directly into the standard metadata deposit via relation type and/or references. This is part of the **existing Content Registration ** process and requires no new workflows.

Relationships

Data & software citations are a valuable part of the “research article nexus”, comprised of the publication linked to a variety of associated research objects, including data and software, supporting information, protocols, videos, published peer reviews, a preprint, conference papers, etc. For all of these resources, we use relation types in the metadata deposit to “anchor” the article in the article nexus and link to it.

For data & software, we ask for:

identifier of the dataset/software
identifier type: “DOI”, “Accession”, “PURL”, “ARK”, “URI”, “Other” *
relationship type: “isSupplementedBy” or “references”
description of dataset or software.
*Additional identifier types beyond those used for data or software are also accepted, including ARXIV, ECLI, Handle, ISSN, ISBN, PMID, PMCID, and UUID.

Crossref maintains an expansive set of relationship types to support the various resources linked in the research article nexus. For data and software, we recommend “isSupplementedBy” and “references” as relationship types in the metadata. Use the former if it was generated de novo as part of the research results. For those generated by another project and then reused, we recommend applying “references” in the relationship type. These were selected in consultation with DataCite and data working groups. They will provide the level of specificity requested by the community.

To illustrate how to represent the link within the metadata deposit, we offer two examples from two popular dataset identifiers, one for each of the relationship types.

Dataset	Snippet of deposit XML containing link
Dataset with DOI: Data from: Extreme genetic structure in a social bird species despite high dispersal capacity. Database: Dryad Digital Repository DOI: https://doi.org/10.5061/dryad.684v0	`<program xmlns="http://www.crossref.org/relations.xsd">` `<related_item>` `<description>Data from: Extreme genetic structure in a social bird species despite high dispersal capacity</description>` `<inter_work_relation relationship-type="isSupplementedBy" identifier-type="doi">10.5061/dryad.684v0</inter_work_relation>` `</related_item>` `</program>`
Dataset with accession number: NKX2-5 mutations causative for congenital heart disease retain functionality and are directed to hundreds of targets Database: Gene Expression Omnibus (GEO) Accession number: GSE44902 URL: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE44902	`<program xmlns="http://www.crossref.org/relations.xsd">` `<related_item>` `<description>NKX2-5 mutations causative for congenital heart disease retain and are directed to hundreds of targets</description>` `<inter_work_relation relationship-type="references" identifier-type="Accession">GSE44902</inter_work_relation>` `</related_item>` `</program>`

In the examples above, the Dryad dataset was generated as part of the research published in an article. Hence, it contains the “isSupplementedBy” relationship type. The GEO dataset was reused by and referenced in a scholarly article published separate from the project that generated this dataset. Hence, it contains the “references” relationship type.

Both Crossref and DataCite employ this method of linking. Data repositories who register their content with DataCite follow the same process and apply the same metadata tags. This means that we achieve direct data interoperability with links in the reverse direction (data and software repositories to journal articles).

References

Another mechanism for depositing data and software citations is to insert it into the manuscript’s references. Publishers then deposit it as part of the article’s references. To do so, publishers follow the general process for depositing references. (Visit Crossref’s Support page for step-by-step instructions.)

Publishers can deposit the full data or software citation as a unstructured reference.
<citation key="ref=3">
<unstructured_citation>Morinha F, Dávila JA, Estela B, Cabral JA, Frías Ó, González JL, Travassos P, Carvalho D, Milá B, Blanco G (2017) Data from: Extreme genetic structure in a social bird species despite high dispersal capacity. Dryad Digital Repository. http://dx.doi.org/10.5061/dryad.684v0</unstructured_citation\>
</citation>
</citation_list>

Or they can employ any number of reference tags currently accepted by Crossref. Most do not readily suit datasets and software as the suite was originally established to match article and book references. This leaves out substantial metadata needed to identify and describe the dataset, however, if the resource does not have a DOI.
<citation key="ref2">
<doi>10.5061/dryad.684v0</doi>
<cYear>2017</cYear>
<author>Morinha F, Dávila JA, Estela B, Cabral JA, Frías Ó, González JL, Travassos P, Carvalho D, Milá B, Blanco G</author>
</citation>
We are exploring the JATS4R recommendations while we consider expanding the current collection. We welcome additional suggestions from the community.

Precise, accessible links

Crossref’s infrastructure is setup to facilitate the flow of information about scholarly works across the research network. We maintain a fair degree of flexibility both in the structure and completeness of metadata deposited. The aim, though, is to make the links rich in metadata, accurate in associating literature to corresponding resource, and available to both human and machine consumers as per Principle #5 and #7 in the Joint Declaration of Data Citation Principles.

As with the other associated resources in the article nexus, we recommend depositing data/software links in the publication metadata via relationships. Publishers are free to do this on top of or independent of references. Relationship metadata offer a high degree of precision. References are a hodgepodge of various resources cited by the publication, including articles, books, media, blogs, reference materials, etc. and data citations are hard to isolate. Furthermore, the unstructured, “spaghetti string” text is difficult for systems to parse and extract specific information.

With relationship metadata, data and software resources are expressly designated. We obtain a more accurate link that specifies identifier type and explicitly identifies data generated as part of the research shared in the paper or as reuse of existing data). The richer metadata contained here enables consumers to conduct powerful queries based on different attributes (identifier type, description, relationship), taking data discovery and mining to the next level.

Furthermore, relationships are important for achieving full accessibility of data and software citations. Access to references is based on publisher permission so not all data citations can be shared (excluding DataCite DOIs). In contrast, all links deposited via relationships are publicly available.

Publishers play an important role in supporting research validation and reproducibility. Data & software citation is a basic part of of this practice, and instrumental in enabling the reuse and verification of these research outputs, tracking their impact, and creating a scholarly structure that recognizes and rewards those involved in producing them. For the full scoop of how to deposit (i.e., technical details and more), we encourage you to reference the Crossref Data & Software Citations Deposit Guide and contact us (support@crossref.org) with questions or feedback.

Taking the "con" out of conferences

Geoffrey Bilder — Wed, 15 Feb 2017 00:00:00 +0000

TL;DR

Crossref and DataCite are forming a working group to explore conference identifiers and project identifiers. If you are interested in joining this working group and in doing some actual work for it, please contact us at community@crossref.org and include the text conference identifiers WG in the subject heading.

All the times I could have gone to Walt Disney World…

Back around 2010 I added a filter to my email settings that automatically flagged and binned any email that contained the word “Orlando.” Back then this was a remarkably effective way of detecting and ignoring spam from the numerous fake technology conferences that all seemed to advertise the city of Orlando, Florida as the location for their non-events. I suspected they all chose Orlando as it would provide the punter that little bit of extra motivation to pay and register for the conference as they simultaneously plotted how they could tag-on some holiday time at Walt Disney World. I finally had to remove the filter last year when I realised that the scammers had moved on to advertising more realistically gritty cities in their calls for submissions and that meanwhile I had managed to miss all the mail informing me of the ALA’s summer 2016 meeting held in, you guessed it… Orlando.

Clearly we need better mechanisms to flag dubious conferences.

Late last year Crossref’s Strategic initiatives group was approached by “CounterMock,” a group of Crossref members (including major proceedings publishers like Springer Nature, Elsevier, IEEE, ACM, IET, etc) who were actively exploring the establishment of an identifier system and registry for scholarly conferences.

The long term goal of the group is to make it easier for publishers, researchers and other stakeholders to identify fraudulent and/or low-quality conferences. There has recently been a proliferation of conferences that seem to have been developed specifically to dupe international and early-career researchers into paying substantial conference and publication fees. Sometimes these conferences are intentionally named after long-standing and well-respected conferences. At worst these conferences are entirely fake - no meetings are held and no publications are issued. At best they produce subpar publications of questionable academic integrity. Members of the group are concerned that these “mock conferences” (Hence “COUNTERMOCK”) will:

Waste researcher time.
Waste publisher time.
Undermine academic trust in conferences and conference proceedings as a trustworthy means of scholarly communication.

The group understands that the “evaluation of a conference quality” and the “unambiguous identification of conferences” are separate concerns (as they are with publications, contributors, etc). But they also realise that it will be hard to address the quality issue without an infrastructure for unambiguously identifying conferences and providing meaningful provenance metadata about those conferences. Moreover, having unique identifiers for conference series would enable a number of other applications. Examples include conference-level metrics, better and more structured info about forthcoming conferences on a certain topic, and more visibility of conferences in research evaluation.

Springer Nature has built a POC prototype of a conference identifier system and shown it to a number of other parties. The feedback has been that there is interest in the project, but that the consensus is that it should be managed a run by a neutral industry group. They have approached us to form a working group and explore how this project can be advanced.

This is all good. Crossref itself doesn’t make value judgements on the quality of content registered with us. Crossref DOIs are not quality marks. But we do believe that unambiguous identification of research artifacts is a perquisite to building effective trust and reputation tools.

It is possible that the issue of conference identifiers can be folded into the work we are doing with DataCite and ORCID on organisation identifiers. For example, some have argued that organisation identifiers should include identifiers for projects or other less formal and more ephemeral corporate entities that are often included in affiliation and/or bibliographic data. It is possible to make the similar arguments in the case of conferences.

On the other hand we have also been interested in the issue of “project identifiers.” Martin Fenner and Tom Demeranville have made a strong argument that ‘projects’ can be thought of as containers for collections of project outputs, project members and project funders. Again, it seems plausible that one could make the same case for conferences.

At the very least it is important to coordinate any work that is done on conference, project and organisation identifiers. This why we have decided to form a joint Crossref/DataCite working group to specifically explore conference and project identifiers and determine how they relate both to each other and to our already ongoing work with ORCID on organisation identifiers.

Additionally, it is likely that the working group will discuss and explore how conference/project identifiers might be used for increasing the transparency of peer review at conferences, better attribution for programme chairs and program committee members, and how they might be incorporated into other services like Crossref Metadata Search, DataCite search, CrossMark, etc.

If you are interested in doing some work on this- then please indicate your interest in joining a working group by sending email to community@crossref.org and include the text conference identifiers WG in the subject heading.

We will update this blog as the group convenes and makes progress.

Linking DOIs using HTTPs: the background to our new guidelines

Geoffrey Bilder — Tue, 17 Jan 2017 00:00:00 +0000

Recently we announced that we were making some new recommendations in our DOI display guidelines. One of them was to use the secure HTTPS protocol to link Crossref DOIs, instead of the insecure HTTP.

Some people asked whether the move to HTTPS might affect their ability to measure referrals (i.e. where the people who visit your site come from).

TL;DR: Yes

Yes. If you do not move your DOI links to HTTPS, Crossref, its members and the members of other DOI registration agencies (e.g. DataCite, JLC, CNKI) will find it increasingly difficult to accurately measure referrals. You should link DOIs using HTTPS.

In fact, if you do not support HTTPS on your site now, it is likely that your ability to measure referrals is already impaired. If you do not already have a plan to move your site to HTTPS, you should develop one.

If you have already transitioned your site to HTTPS, you should follow the new guidelines and link DOIs via HTTPS as soon as possible. As it stands, you are not sending any referrer information when DOIs are clicked on and followed from your site. You should also make sure that the URLs you have registered with Crossref are HTTPS URLs, otherwise you will not get referrer information on your site when they are followed.

Read on if you want some grody details. We’ll try to keep it as non-technical as possible.

Two protocols, one web

To start with your web browser supports two closely related protocols, HTTP and HTTPS.

The first, HTTP, is the protocol that the web started out with. It is an unencrypted protocol and it is also easy to intercept and modify. It is also very easy and inexpensive to implement.

The second protocol, HTTPS, is a secure version of the first protocol. It is very difficult to intercept and modify. It has historically been more complex and expensive to implement.

Here you might say - “Great, but HTTPS has been around for a long time. We’ve used it for sensitive transactions like authentication and credit card transactions. Why do we want to use DOI links with HTTPS?” Why are you suggesting that we should even consider moving our entire site to HTTPS?

The pressure to move to HTTPS

The insecure HTTP protocol has become a major vector for a lot of security issues on the web. It allows user web pages to be intercepted and modified between the server and the browser. This flaw is being abused for everything from spying, to inserting unwanted advertisements into web pages, to distributing viruses, ransomware and botnets.

As such, there has been a steady drumbeat of industry encouragement to move to the more secure HTTPS protocol for all website functions.

We are not going to argue all the points here. Instead we will mention the major constituencies that are advocating for a move to HTTPS and provide you with some pointers. We apologise that these are all so US-centric, but a lot of the web’s global direction does seem to be presaged by US adoption trends.

Google

It is probably easiest to start with Google, since its practices tend to focus the attention of those managing websites.

Back in 2014 Google announced that they would slowly move toward including the use of HTTPS as a ranking signal. In 2015 they upped the ante by announcing that they would start indexing HTTPS versions of pages by default. It looks like in early 2017 they will really start to take the gloves off as they modify their Chrome browser to flag sites that do not use HTTPS as being insecure.

Every top website, evah

It looks like Google's plan is working too. Their 2016 transparency report shows that most top websites have already transitioned to HTTPS and that this translates to approximately 25% of all web traffic worldwide taking place using HTTPS. Indeed, over 50% of all web pages viewed by desktop users are delivered via HTTPS.

Government agencies

The USA’s Whitehouse issued [a directive instructing all Federal websites to adopt HTTPS]. As of December 2016 64% of federal websites have made the transition.

Libraries

Much of the pressure to move to HTTPS is coming from the library community who have a historical tradition of protecting patron privacy and resisting efforts to censor content. The third principle of the American Library Association's code of ethics reads:

We protect each library user’s right to privacy and confidentiality with respect to information sought or received and resources consulted, borrowed, acquired or transmitted.

Recently there has been a major push by the Electronic Frontier Foundation to get libraries to adopt a number of security and privacy practices, including the use of HTTPS by all library systems as well as those used by library vendors.

What are Crossref members doing about HTTPS?

How big an issue is this? How many of our members have moved to HTTPS? How many plan to? Well, we looked at the URLs that are registered with Crossref and we tested them with both protocols. Eventually we will write a blog post detailing our findings - but the highlights are:

Slightly fewer than half of the member domains tested only support HTTP.
Slightly fewer than half of the member domains tested support both HTTP and HTTPS.
About 370 of the member domains tested only support HTTPS.

The transition to HTTPS and the issue of DOI referrals

The HTTP referrer is a piece of information passed on by a browser that indicates the site from which the user navigated.

So, for example, if a user visiting site A clicks on a link which takes them to site B, site B will then record in its logs that a user visited them from site A. Obviously, this is important information for understanding where your web site traffic comes from.

The default rules for referrals are¹:

If you link between two sites with the same level of security, all referral information is retained.
When you follow a link from an insecure (HTTP) web site to a secure (HTTPS) site, referral data is passed on to the secure web site.
If you follow a link from a secure (HTTPS) web site to an insecure (HTTP) site, referral data is not passed on to the insecure web site.

So let's see what the situation would look like with normal links. If we had two sites, `A` & `B`, the following table maps the possible combinations of protocols that can be used to link from `A` to `B`. So, for example, row #2 reads:

A user browses site A using HTTP and clicks on a HTTPS link to publisher B who hosts their site using HTTPS.

The last column indicates if the referrer information is passed along by the browser. In the case of row #2, the answer is “yes”. The user has navigated from a less secure site to a more secure site.

User views site A using	Site A links to site B using	Browser reports referrer to site B
HTTP	HTTP	Yes
HTTP	HTTPS	Yes
HTTPS	HTTP	No
HTTPS	HTTPS	Yes

But this gets a little more complicated with DOIs. In this case publisher `A` links to publisher `B` through the DOI system. This means there are two parts to the link. The first `(A->doi.org)` results in a redirect (A->B). Again we use the last columns to indicate when referrer information is passed along to site B. Again, let’s look at row #2. It reads:

A user browses the site of member A using HTTP and clicks on a HTTP DOI link. The DOI system redirects the browser to member B using an HTTPS link registered with Crossref by member B. The middle column and the last column records whether Crossref and the publisher were able to see referrer information. The answer in both cases is “yes”. In the first case (A->DOI) because the link was from a less secure site (HTTP on A) to a more secure site (HTTPS at DOI). The second case because the link is between two sites at the same security level (HTTP).

	User views site A using	Site A links DOI using	Browser reports referrer to Crossref²	Crossref redirects to site B using³	Browser reports referrer to site B
1	HTTP	HTTP	Yes	HTTP	Yes
2	HTTP	HTTP	Yes	HTTPS	Yes
3	HTTP	HTTPS	Yes	HTTP	Yes
4	HTTP	HTTPS	Yes	HTTPS	Yes
5	HTTPS	HTTP	No	HTTP	No
6	HTTPS	HTTP	No	HTTPS	No
7	HTTPS	HTTPS	Yes	HTTP	No
8	HTTPS	HTTPS	Yes	HTTPS	Yes

So what does this mean?

Our old display guidelines recommended linking DOIs using HTTP. Rows #1, #2, #5, #6 represent the status quo.

About half of our members support HTTPS. A few support it exclusively and it seems, given the industry pressures mentioned above, those who support both protocols are likely doing so as a transition stage to HTTPS-only sites.

This means that the scenarios represented in row #5 & #6 are already happening. The referral information for any user viewing one of our member sites using HTTPS is being lost when they click on DOIs that use the HTTP protocol. Crossref doesn’t get the referral data and neither does the member whose DOI has been clicked on.

Of course this applies to non-member sites that link to DOIs as well. Wikipedia is the largest referrer of DOIs from outside the industry. In 2015 The Wikimedia Foundation made a highly publicised transition to HTTPS on all of their sites. This means that any of our members who are running HTTP sites have already lost the ability to see any referral information from Wikipedia on their own sites. However, Crossref worked closely with Wikimedia to ensure that, at the very least, Crossref was still able to record Wikimedia referral data on behalf of our members.

A solution

It is largely this work with Wikimedia that has helped us to understand just how important it is for Crossref to get ahead of the curve in helping our community to transition to HTTPS.

As long as our members are running a combination of HTTP and HTTPS sites, there is no way for our community to avoid some disruption in the flow of referral data. And we certainly would never entertain the notion of asking our members to keep using HTTP.The best we can do is recommend a practice that will help smooth the transition to HTTPS. That is what we are doing.Our new recommendation is to move to linking DOIs using HTTPS. This is represented in rows #3, #4, #7 and #8 in the table above.

This is a particularly important step for our members who have already moved to hosting their sites on HTTPS. As long as they are using HTTP DOIs on their site, they will be sending no referral traffic to Crossref, other Crossref members or other users of the DOI infrastructure. This is captured in scenarios #5 and #6.

If our linking guidelines are followed during the industry’s transition to HTTPS, then scenario #5 and #6 will eventually be replaced with scenario #7. It is still not perfect, but at least it means that, during the transition, publishers who are still running HTTP sites will be able to get some DOI referral data via Crossref. And of course, once our members have widely transitioned to HTTPS, everything will go back to normal and they will be able to see referral data on their own sites as well (i.e.they will have moved from the state represented in row #1 to state represented in row #8.)

In summary, please change your sites to use HTTPS to link DOIs. They should look like this:

https://doi.org/10.7554/eLife.20320

FAQ

Q: If I have moved my site to HTTPS, do I need to redeposit my URLs to that they use the HTTPS protocol instead?

A: Yes. If you want to be able to still collect referrer information on your site (scenario #8) as opposed to via Crossref (scenario #7).

Q: But can’t I avoid redepositing my URLs and get referrer data again if I simply redirect HTTP URLs to HTTPS on my own site?

A: No. The browser will strip referrer information if there is any HTTP step in the redirects. Even if the redirect is done on your own site.

Q: Can I avoid having to redeposit all my URLs? Can’t Crossref just update the protocol on our existing DOIs for us?

A: Contact support@crossref.org. We’ll see what we can do.

Q: What about all the old PDFs that are are there? They link to DOIs using HTTP.

A: That is true. But links followed from PDFs don’t send referrer information anyway.

Q: And what about my new PDFs? Should I start linking DOIs from them using HTTPS.

A: Probably. But not because of the DOI referrer problem. Simply because HTTPS is a more secure, private, and future-proof protocol.

Q: Don’t some countries block HTTPS?

A: Typically countries block specific sites and/or services. We do not know of any countries that have a blanket block on the HTTPS protocol.

Q: I use a link resolver that uses OpenURL + a cookie pusher to redirect my users to local resources. What do I need to do?

A: You need to change your cookie pusher script to enable the Secure attribute for cookies for HTTPS-linked DOIs.

Q: Can I use protocol-relative URLs (e.g. //doi.org/10.7554/eLife.20320)?

A: Protocol-relative URLs can be used in HTML HREFs to help ease the transition from HTTP to HTTPS, but use the full protocol in the text of the DOI link itself. So, for example, the following is fine:

https://doi.org/10.7554/eLife.20320

Q: I hear that HTTP and HTTPS versions of URI identifiers are considered to be different identifiers. Doesn’t this mean that by moving to HTTPS we are essentially doubling the number of DOI-based identifiers out there?

A: Yes. It isn’t a problem that is only being faced by DOIs. Basically all HTTP-URI based identifiers face the same issue. We will put in place appropriate same-as assertions in our metadata and HTTP headers to allow people to understand that the HTTP and HTTPS representations of the DOI point to the same thing.

On a personal note (@gbilder speaking- don’t blame @CrossrefOrg) - it breaks my brain that the official line is that the protocol difference means they are different identifiers. As a practical matter (a concept the W3C seems to be increasingly alienated from), it would be insane for anybody to follow this policy to the letter. You can probably be pretty safe swapping the protocols on DOIs and being sure you will get the same thing.

Q: I see that the Crossref site isn’t running on HTTPS. Are you just a bunch of hypocrites?

A: ~~Yes. The site will be moving to HTTPS-only very soon. Then we won’t be.~~ We do now.

References

These rules can be tweaked using meta referrer tags (https://www.w3.org/TR/referrer-policy/), but not in any way that both avoids the fundamental problems outlined here and that preserves the security/privacy characteristics that are the very reason to implement HTTPS in the first place.
To be pedantic- it actually passes referrer information to the DOI proxy (https://doi.org/), which in turn is reported to Crossref.
To continue with the pedantry- the DOI proxy does the redirect based on the URL member B has deposited with Crossref.