The Problems With Peer Review Research

Executive Summary

• Peer review journals are typically held up as the gold standard of research while having many problems that tend to be overlooked.

Introduction

When some readers have a serious disagreement with an article at Brightwork Research & Analysis, sometimes the opposing and often offended individual will state that the articles at Brightwork are not “peer-reviewed.” People often make this claim with no academic background in research and typically no understanding of research, and those who have never worked for or generally read peer-reviewed journals.

These blind spots mean that the persons overstate the validity of the peer review system and use the lack of peer review against articles they simply disagree with.

In this article, I will highlight problems with the peer-review process.

Our References for This Article

If you want to see our references for this article and other related Brightwork articles, see this link.

Problem #1: Who Funds Large Scientific Studies?

The first part left out of the peer review discussion is who funds peer-reviewed studies. It usually is drug companies or medical device companies in the medical area. This is explained in the following quotation.

The bigger and the higher quality a scientific study has, the more expensive it is. This means that most big high quality studies are carried out by pharmaceutical companies. Obviously this is a problem because the companies have a vested interest in making their products look good. And when companies carry out studies, they don’t show their drugs in the best light that don’t show their drugs in the best light they will usually try to bury the data. This contributes to a problem known as publication bias. When a publication when publication bias, what publication bias means is that studies that show good effect are much more likely to get published and studies that show no effect. This is both because the people who conducted the study are more likely to push for it to be published. And because journals are more likely to accept studies that show benefit, because those studies get much more attention than studies that don’t show a benefit. And the more attention a journal gets, the more advertising money it gets. So one thing to be aware of before you start searching for scientific studies in a field is that the studies you can find on a topic often aren’t all the studies, you are most likely to find the studies that show the strongest effect. The effect of an intervention in the published literature is pretty much always bigger than the effects subsequently seen in the real world. – Why Most of What You Know About Covid is Wrong

Publication bias is a type of cherry-picking. And it dramatically benefits those companies with financial resources. A pharmaceutical company can run many studies and report those that support their commercial interests. If one runs seven studies, one of them will show a benefit just by luck. However, when the study is published, the pharmaceutical company or the individuals funded do not declare that one of only seven studies for that drug showed a benefit.

Secondly, many people in the peer review process also have financial conflicts and seek to protect and keep their relationships with pharmaceutical companies in good standing.

Problem #2: The Problem of Lack of Incentive on the Part of the Peer Reviewers

Peer review provides a sort of stamp of approval although it is questionable how much that stamp is worth. Basically peer review means that someone who was considered an expert on the subject of the article, but who wasn’t personally involved with it anyway in any way. Reach through and determines if it is sensible and worth publishing. Generally, the position of peer review of a peer reviewer is an unpaid position and the person engaging in peer review does it in his or her spare time, he or she might spend an hour or so going through the article before deciding whether it deserves to be published or not. Clearly, this is not a very high bar. Even the most respected journals have published plenty of bad studies containing manipulated to fake data, because they didn’t put much effort into making sure the data was correct. The early part of the COVID pandemic saw a ton of bad studies which had to be retracted just a few weeks or months after publication, because because the data weren’t properly fact checked before publication. – Why Most of What You Know About Covid is Wrong

The time estimation to review an article or study is far higher in this quotation.

It takes time – at least seven to eight hours per paper done properly, with no remuneration or recognition for the reviewer and hence rarely regarded as a priority in a busy academic schedule. As a result, scientific rigour can be lost when reviews become fast-tracked. – The Conversation

When I have been a peer reviewer, I have spent around 3 hours reviewing the article. However, the time-consuming part comes in when the article author responds to my comments, and I have to defend my initial observations.

When doing this, and in light of adding comments to a blog post, I have wondered why all of this has to be carried out in secret, without the readers seeing the differing views. Even YouTube has comment ability. This realization is dawning on many people and is explained in the following quotation.

Post-publication peer review in its most validated form, involves a journal such as Frontiers asking academics to perform a published interactive dialogue with authors during the review process, giving a level of accountability and responsibility.

Other journals such as Faculty of 1000 Research, Copernicus and PLOS ONE publish papers with minimal evaluation. This shifts the focus towards post-publication peer review – authors, reviewers and readers critique and comment on the paper to judge its scientific merit in the public domain. – The Conversation