Blog

Integrity of the Scholarly Record (ISR): what do research institutions think?

Madhura Amdekar – 2024 May 09

In Research IntegrityTrustworthinessStrategy

Earlier this year, we reported on the roundtable discussion event that we had organised in Frankfurt on the heels of the Frankfurt Book Fair 2023. This event was the second in the series of roundtable events that we are holding with our community to hear from you how we can all work together to preserve the integrity of the scholarly record - you can read more about insights from these events and about ISR in this series of blogs.

Research institutions are one of the most important stakeholders in the endeavour of research integrity, and any conversation around ISR is incomplete without the views of this key community. This fact was acknowledged at the second ISR roundtable event, and one of the main takeaways from the discussions was to make more focused efforts to hear the viewpoints of researchers and academics.

As the first step in this direction, we organised an online discussion on the integrity of the scholarly record, to which we invited: researchers and academics, research integrity experts based at academic institutions, Crossref members, as well as other organisations working on this topic such as COPE and Digital Science. The primary objective of this event was to hear from this community their perspectives on preserving and leveraging the integrity of the scholarly record and to identify opportunities for collaboration in this area. To ensure common ground, we also wanted to share information about Crossref metadata, the Research Nexus vision, and our position and role in the integrity of the scholarly record.

To facilitate this, the event started with an introduction by Kora Korzec, Head of Community Engagement and Communication at Crossref, to our mission and vision and the importance of capturing the relationships between the objects, people and places involved in research through the Research Nexus. Amanda Bartell, Head of Member Experience, was next and she spoke about the scholarly record and the role that Crossref plays in preserving the record’s integrity. In her presentation, Amanda emphasised that Crossref’s role is not to assess the quality of content deposited by the members but rather to provide infrastructure that enables the community to provide and use metadata about the scholarly content produced by members. It’s important not to put up barriers to entry, but to work with all publishers to encourage best practices.

Dominika Tkaczyk, Head of Strategic Initiatives, shared details of a few Crossref projects that focus on monitoring and improving metadata completeness, thereby supporting ISR. These projects include improving the Participation Reports, using metadata matching to discover new relationships (e.g., preprint published as work, work supported by funder, etc), and importing more retractions and other updates from the Retraction Watch database that was acquired and made openly available by Crossref. Dominika used these examples to highlight the ways in which open and complete metadata can help in uncovering large scale trends and systemic concerns. The final speaker was Amanda French, ROR Technical Community Manager, who introduced the audience to the Research Organization Registry, or ROR.

To accomplish the primary aim of the event, which was to hear the community’s viewpoints, the participants were divided into breakout groups for discussions and given three prompts to answer. The rest of the blog is a summary of what we heard from the participants.

1. Is Crossref’s role what you expected? What surprised you? What are we missing?

An overarching sentiment from the academics in the audience was that Crossref does so much more than is known to researchers! They were surprised by the range of activities underway at Crossref. At the same time, there were calls for Crossref to play a bigger role. Suggestions included playing a leadership role in deciding which metadata elements are a priority, providing guidance on the main metadata components important for signalling trust, playing a greater role in connecting various identifiers to ensure that relationships between different content types are preserved well, and to coordinate the efforts being taken by institutions, publishers and service providers around research integrity, by virtue of Crossref’s unique position in the community. There was a broad agreement that by providing the essential infrastructure, Crossref acts as the base upon which other actors in the scholarly community can build.

2. What metadata elements do you consider important for signalling trust?

Many participants spoke about the various ways in which author identity and affiliation are important as trust signals. Being able to identify when an author has changed institutions, or being able to make a distinction between authors who have the same name is important. Author affiliations that are authentic and verified would go a long way in establishing trust.

Multiple assertions, e.g. for affiliations, would be welcome. The use cases for this could be when research starts at one institution and is carried over to another, or when researchers affiliated with an institution may perform part of the research overseas. Some of the participants, who actively investigate research data, shared that abstracts are valuable because they can be used for large scale analyses related to research integrity.

Other metadata elements that came up during this discussion were data on peer review, ethics approval, patient and donor consent in medical research, editorial boards (especially of special issues), pre-registration, funding metadata, datasets and programming scripts.

3. What value do you see in the integrity and completeness of the scholarly record in the way you operate? How do you contribute to it? How can it support you to achieve your own goals?

Participants acknowledged that integrity of the metadata and the scholarly record is essential. Ensuring this integrity is a dynamic process, much akin to the concept of organised scepticism which is the notion that all scientific work should be trusted subject to its verification. Several ideas were shared on how to progress the integrity and completeness of the scholarly record. One recommendation was to use multiple metadata trust markers as that can make it harder for bad actors to game the system, but this may run the risk of making things complicated. Another suggestion was to make metadata part of the onboarding procedure- by gathering staff ORCID iDs during the onboarding process and sharing the institutional ROR ID with staff to promote its use, institutions can ensure that this information is routinely made available. The metadata deposited with Crossref should be integrated with downstream workflows to better facilitate the use of this rich metadata. An example of this is to integrate Crossmark with other research tools such as reference management software.

The participants acknowledged that this discussion underlined for them the fact that having identifiers in itself is not an indicator of quality and that the underlying metadata records and wider context is key to understanding trustworthiness of the content.

This event was a good first step towards engaging researchers and academics in the conversation about ISR. It connected folks working in different parts of the world who are united by their interest in research integrity. There was good engagement among all and commitment to continue these conversations in the future, with many participants planning to connect at the World Conference on Research Integrity in June (I’ll be attending as well, for anyone who wants to continue the conversation - along with my colleagues Fabienne and Evans).

At Crossref, we plan on continuing these conversations with all segments of the community to understand their needs and perceptions around metadata. The greater the awareness about the importance of metadata and its applications, including for research integrity, the richer the metadata that we are able to collect together. This will lead to building a comprehensive Research Nexus and emergence of more relationships therein. Please write in response to this post on our Community Forum if you have any thoughts on this as we’d love to hear from you.

List of participants

*Note: name added 21-May-2024