At Sense About Science we equip people to make sense of scientific and medical claims in public discussion. One way that we have helped people make sense of science stories is by encouraging them to ask the question “Is it peer reviewed?” It’s a great first question to ask about the status of science claims, and one that we encourage you to share.
Examining peer review’s strengths and weaknesses
As peer review plays such an important role in maintaining quality in science, we welcome initiatives which take a closer look at its strengths and weaknesses including a new white paper by Taylor & Francis (Peer review in 2015: a global view) on a large study of the academic community’s thoughts on peer review. Respondents overwhelmingly agreed that “Scholarly communication is greatly helped by peer review of published papers” – an opinion which hasn’t changed since our own peer review survey in 2009.
The need for peer review support and training
The white paper highlights the role of the publisher in providing support and training for peer review, which is something that resonates closely with our conversations with early career researchers (ECRs), who told us that they want to know more about peer review. One way we have tried to respond to this need is through running peer review workshops, partnered by Taylor & Francis, which are aimed at ECRs to address their concerns and answer their questions about peer review. Participants have since developed their own guide Peer Review: The nuts & bolts based on content from the workshops.
Highlighting (and helping with) the issues
Taylor & Francis’ white paper highlights many issues that come up in these workshops and also echoes the findings of our 2009 survey, such as the fact that peer review cannot detect fraud in the way many researchers expect it to:
“For STM respondents, the biggest shortfall between the ideal and the real world is in peer review’s ability to detect fraud, with a mean score of 7.9 saying it should achieve this, but only 6.3 saying it does.”
And that researchers don’t think peer review is perfect: it can be a long process, and it can be biased:
“Most researchers wait between one and six months for an article they’ve written to undergo peer review, yet authors (not reviewers / editors) think up to two months is reasonable.”
“Researchers rated regional and seniority bias highest, and suggest that double blind peer review is most capable of preventing any reviewer discrimination based on aspects of an author’s identity”
Can supporting ECRs through workshops help to address these concerns? I think so. The more engaged and equipped ECRs are, in peer review, the more robust and rigorous the system will be.
Sense About Science’s Peer Review: The nuts & bolts workshops are free to attend, and pare happening in London and Scotland in 2016. Find out more here.
Victoria Murphy is Programme Manager at Sense About Science. She coordinates the Voice of Young Science (VoYS) programme, working with over 20 partners to deliver four media workshops and two peer-review workshops each year for early career researchers. She is also working on a resource to help young people develop their critical thinking skills, as part of the Ask for Evidence campaign. Before joining Sense About Science in September 2011, Victoria worked in Australia in the mining industry and science outreach.