Could we get ahead of research integrity issues?
Research integrity is a shifting goal. Every few years, new ways in which trust in research has been breached come to light, and this is speeding up as generative AI becomes a larger player in the research ecosystem. Is there a way we can get ahead of it, while maintaining a system based on trust that does not overly burden the majority of authors who are honest?
The value of trust
On my first visit to Tokyo almost two decades ago, I was taken by how the ticket barriers on the metro were open by default, unless a passenger fails to scan their pass. This allows for a smooth flow of traffic, even at busy times, and the assumption of trust has stuck with me. I think about it sometimes when waiting in a throng of people to get through barriers in cities where we have to prove our innocence, scanning our tickets before we pass.
Scholarly communication has been a high-trust industry, but faces significant challenges if it is to remain so. The peer review process is based on trust and retraction mechanisms exist in order that flawed work can be handled transparently. Reputation and careers depend on integrity, though retracted papers continue to be cited and authors might not be punished or shamed depending on the outcomes of institutional investigations, stretching trust issues further.
Between predatory publishers, paper mills, citation manipulation, fabricated data, image alteration and AI-generated content, the system is under strain.
The current picture
Pre-publication there are tools to detect plagiarism and image manipulation; and AI applications that spot anomalies in data or writing patterns are plentiful. The STM Research Integrity Hub is a coordinating effort that allows member publishers to check submitted articles for research integrity issues, share knowledge on threats and discuss solutions. These keep the system running smoothly for academics, and save contributing publishers time and money reviewing and retracting published papers later, so the tools aren’t just about keeping barriers open and workflows smooth for academics.
Other checks such as authorship contribution and conflict of interest statements, data sharing, clear methodology sections and statistical review carry an administrative burden for authors and publishers. If they are true, and checked, they can improve transparency and equity, but heavy administrative costs are a hallmark of low-trust societies: a sign that trust in publishing is being lost or is breaking down. As is often the case, those working with limited resources are affected more than others by these new author requests.
After publication there are a growing number of sleuths who flag issues. Retraction Watch reports on retractions and issues of integrity, publishing an article most weekdays, showing just how many problems there are to shine a light on. The site also hosts an open and freely searchable database that tracks over 60,000 retracted articles, expressions of concern and corrections. It was launched because these notes were missing from the sources researchers used such as PubMed, Web of Science and Scopus.
PubPeer was established in 2012 as a site where users could discuss published research, as they would in a journal club. Perhaps as a testament to growing trust issues in scholarly communications, it has also become a platform for whistleblowers and sleuths, who highlight problems in papers; most frequently image manipulation.
These accusations bring career consequences, which are severe even for co-authors who may not have been directly involved in the flagged issues, so we should be very careful with the processes for investigating problems. Does a fake citation in the reference list for example, mean that a whole paper cannot be trusted? Sceptics would say yes, but opinions differ. And how do we support researchers who have faced accusations about errors that eventually turn out to be innocent? Once an ‘Editorial Expression of Concern’ is listed it can be hard to shake.
Perhaps one change could be new categories of misdemeanours. The old labels served us well, but it could be helpful to add a citation label, an image manipulation one, an AI one (or several categories of AI labels), or even a papermill one, that would sit alongside a retraction notice. This would allow readers more insight into the issues that were flagged and led to a retraction.
On the front foot
While the majority of academics are not trying to manipulate the system, there will always be bad apples who will exploit loopholes or commit outright fraud for their own benefit. Admittedly a major driver of paper mills and AI generated papers is the pressure for many authors to get into high Impact Factor journals and get citations. This comes from the system itself, but the result is the same. And as the issues grow and evolve, we remain a step behind. As the use of AI increases, new challenges are arising and while we struggle to keep up, would it help to also contemplate what is around the corner?
United2Act brings together the STM Association, the Committee on Publication Ethics and a cohort of publishers to combat paper mills. And Digital Science has a growing research integrity focus, which I believe combines bibliometric analysis and product development to counter the issue.
Yet maybe part of the problem is that publishers have been trying to tackle this alone. Who wants to air their dirty laundry for all to see? Many initiatives are developed for select groups working together or to make money through paid for services.
So imagine if we broadened our reach beyond publishing and also resourced efforts to scan the horizon and predict what is coming down the line. Instead of scrambling on a paper-by-paper or issue-by-issue basis,could we get the jump on some of the issues that are coming our way? If the fraudsters can find the methods, so could a cross-industry group of focused, intelligent techies, publishers, research institutions and funders, even a couple of sleuths.
Any such effort would have to be agile and move faster than lots of initiatives do in order to get ahead on the issues. It would have to be resourced over and above efforts to weed out current issues. The people I know who work in research integrity are flat out, struggling to keep up and months behind the kind of speedy solutions that are needed. There seems to be no break from the challenges that are flagged and the papers that need investigating.
The World Conference on Research Integrity takes place in May and is open to ‘all stakeholders in research integrity’. This could be a good forum, among the interesting panels and discussions, for such a horizon scanning group to take shape, among the other topics on the programme.
The bigger picture
Trends like research assessment reform; the emphasis of research quality over quantity in hiring and promotion; and, building better open infrastructure for data sharing and verification are positive. But systemic change takes time and alone will not solve everything.
Whatever we do, we must remember that the vast majority of academics are not trying to manipulate the system. A recent retraction in Science shows exactly that: the authors were informed of issues in their analysis pipeline that influenced the results and requested that the study be retracted and are exploring what went wrong. Part of their statement reads: “This has been a humbling experience, but one that speaks to the self-correcting nature of the scientific endeavor. It is far better that these issues were detected than that they remained in the scientific record.”
The system was built on trust and it would be a shame if we closed barriers for everyone in order to stay on top of the issues created by a few.
I am extremely grateful to those who spent time talking to me about this topic. Many thanks for your knowledge and insights.