Peer Review in Science
Introduction
Every day, news is released about the latest and most exciting scientific discoveries. From new techniques to rejuvenate skin cells to breakthroughs in fighting drug resistance in cancer patients, we are constantly being inundated with information. But how can we tell if these stories are trustworthy? How can we know if a new technique works or if the data and conclusions made are reliable?
Reading the scientific article can be the best way to determine the accuracy of the information, but many laypeople and even scientists in different fields may not be able to properly evaluate the data and comment on the validity of the conclusions. This is where peer-reviewed journals come into play.
Peer-reviewed journals contain articles that are not only written by experts, but also reviewed by other experts in the same field. This peer-review system is vital for maintaining a level of rigor in scientific publications, as well as for building a sense of trust in the scientific community from the general public. However, there are several issues with the peer-review system that can undermine its credibility. To understand these issues, let's first look at how the peer-review process works.
The peer-review process
Different peer-review journals have different reviewing processes, but they usually follow a similar structure. After a paper is submitted to the journal, the editor sends the manuscript to two or more experts in the field who are unaffiliated with the authors. These reviewers then read and assess the article, providing comments and suggestions to the authors. The authors can then make changes to the manuscript, conduct more experiments, and address the reviewers' concerns. Once the authors have responded to the reviewers' comments, the journal editor makes the final decision of whether or not to accept or reject the manuscript for publication.
The journey to publication in a peer-reviewed journal like Science can be a lengthy one, with an average of 123 days elapsing between original manuscript submission and acceptance. However, a search of the Review Speed Database will reveal that this is a typical amount of time for manuscripts to be accepted, not accounting for the multitude of submissions that do not make the cut.
A complicated process for everyone involved
The reviewing process can be a difficult and time-consuming endeavor for both researchers and reviewers alike. A typical academic reviewer completes around 4.73 reviews per year, each taking up to five hours to complete. This equates to approximately 100 million hours of peer reviews globally in 2020, equivalent to 15 thousand years of work.
Although reviewers generally do not receive any payment for their work, they are still willing to put in the effort due to non-monetary rewards such as gaining free journal access, being acknowledged in journals, or hoping to receive favor in return when they submit their own papers. Journals themselves are also put under pressure, with an estimated 21 million articles submitted for review in 2020 and fluctuating submission rates throughout the year.
Potential problems of peer review
The review process of published articles is often inconsistent and unpredictable, with results varying between different journals and articles. The decisions of reviewers can be volatile, often rejecting a paper that had been previously accepted when resubmitted a few months later. Probability analysis has shown that it is almost impossible to get two reviewers to agree, with a similar chance of success to throwing a dice.
Furthermore, personal biases or conflicts of interest can lead to dubious research being published and even after a paper has been peer-reviewed it can still be retracted, with one of the most notable examples being the Lancet article linking autism to vaccines. This shows that even after a paper has been peer-reviewed, it does not necessarily mean that it is mistake-free and has good science.
Despite female researchers making up a significant portion of the research community, gender bias remains a significant problem in the selection of reviewers. A survey observed that authors, regardless of gender, tend to suggest mostly male peers. Another paper determined that female reviewers are less likely to be chosen by peers than if a reviewer was selected at random. This gender bias in the selection of reviewers may have far-reaching implications, such as the potential for further bias in the review process. Furthermore, this same study showed that fewer female authors are publishing than expected based on the population of female researchers, likely due to similar gender biases.
What can be done to improve?
The peer-review system is undergoing a number of changes in order to improve accuracy and timeliness. Journals are taking more steps to quickly retract incorrect papers, while there is ongoing discussion about paying reviewers to review papers, which could increase consistency in peer-reviewing. This could also lead to higher costs for publication and reading, which could reduce the number of papers a journal can publish.
With the advent of the internet, it has become much easier for the public to engage in discussion and review of scientific articles. Sites such as PubPeer and medRxiv/bioRxiv, for example, give scientists the opportunity to share their research with the public before it has been peer-reviewed, which has been beneficial in allowing for quicker feedback from the scientific community. However, this increase in preprint publication has had an effect on how news outlets report scientific discoveries, with many outlets not stating that the research is preprint. As we move forward, it is important that news outlets take the time to report on preprint articles appropriately and that consumers are aware of the fact that the research they are reading may not have been peer-reviewed yet.
When trying to assess the reliability of a new scientific discovery, it is important to look at whether or not the article has been peer-reviewed. While peer review is not fool proof, it is still the best way to ensure a certain level of academic rigor in research. However, it is important to remember that peer review does not guarantee fact-checking, bias-free analysis, or accuracy; therefore, it is wise to take the findings of these articles with a grain of salt.