Looking for reviewers - where did they go?

The scientific publishing industry is suffering from several problems to be resolved, some at least partially linked to the continued pressure to publish an increasing number of articles, preferably in so-called high-impact journals (e.g., Kellner 2021a), a trend that does not appear to be going away anytime soon. Yes, financial issues remain at the top of the list (e.g., Kellner 2021b) and the discussion about open science, the harm caused by abusive publication fees, and predatory journals are just some of the issues that have not yet been satisfactorily addressed. In recent years, however, another challenge has emerged that, in my opinion, is reaching pandemic proportions: the lack of reviewers!

The challenges of selecting appropriate referees for grants, articles, or otherwise in academia have long been known and have been considered by some to be the Reviewer Assignment Problem (RAP; e.g., Aksoy et al. 2023). In relation to scientific journals that have relied on the Peer Review System (PRS) for decades, it is essential to have researchers willing to dedicate their time to analyze a study by another scientist. As far as I know, in the vast majority of cases, this is done free of charge, with no particular reward other than the satisfaction of knowing that we are helping science advance. In some journals that charge for publication, there is the possibility of compensating reviewers with reduced publication fees if the scientist who carried out a review (or a certain number of reviews) chooses this journal for the outcome of his research. In fact, this is a very interesting way for journals to get closer contact with potential authors, which could be interesting and beneficial for both parties. Others sent gifts - I have some very nice cup-holders that I treasure very much!

Nowadays, however, finding reviewers has become an increasingly greater challenge and has been identified as the number one difficulty faced by editors. During much of my academic life, including when I started at Annals of the Brazilian Academy of Sciences (AABC) as editor-in-chief, it was quite common to select just two reviewers to examine an article. If one did not accept the invitation, a third was immediately invited. In my personal experience, getting reviewers has never been a real problem. At some point, as some invitations went unanswered, delaying the entire publication process, I started sending out invitations to review a manuscript to five referees, waiting for two or three to return. I recommended all other editors of the AABC to proceed in the same fashion. Unfortunately, this strategy simply does not work anymore and the cases where I do not select at least a dozen experts to get two reviews have become exceedingly infrequent! My personal record - I hate to admit it - happened recently: 35 invitations! Talking to other editors, some claim to be approaching 50 invitations in some extreme cases! This is not only becoming ridiculous, but also unfeasible. Although perhaps higher profile journals may have fewer problems, since reviewing articles for these periodicals may be considered attractive and a sort of status symbol, this problem is increasing. Recently, a high-level journal that charges a lot of money for publication justified the long delay in responding to the evaluation of one of my articles due to the difficulty of obtaining two referees! My perception talking with other editors, is that this situation of a large number of multiple invitations to evaluate articles is starting to turn from the exception to the rule. To be very honest, in the few cases where I have been late for some reviews, instead of being “honorably discharged” from my reviewer duties - a common practice of journals not so much time ago-, editors continue extending deadlines and sending me “nice” reminders!

There is no need to delve into the important role a reviewer plays in the publishing system (e.g., Fox 2017). When reviewing an article, he or she can point out problems and different approaches to particular aspects of the study, offer contributions that could improve the article (scientifically or in its readability), and draw the author’s and editor’s attention to problems that can be solved or indicate that the study is flawed. Although reviewers can also be important in detecting ethical issues, the latter is more the responsibility of the editors. Yes, sometimes reviewers can overstep their function, trying to force authors to express their opinion on the subject, or “suggest important articles” published by themselves or their study group to be cited. However, these and other situations can be handled by an experienced editor without compromising the contribution of the review.

It is clear that to elaborate usable reviews takes time. What is in for the scientists to perform this task with the attention that it needs? If advancement of science is not convincing, editors expect that the notion that someone else is doing the same for papers submitted by the reviewer when he or she is on the other side of the bench should work. It might have for some time, but this is no longer happening...

Although I did not elaborate detailed statistics, my perception is that between 30 and 50% of invited reviewers do not even bother to respond to the invitations. This also applies to reviewers appointed by the author. Those who respond negatively typically do not suggest other researchers who might be willing to do the job, and even when editors kindly ask them to do so, few respond. Note that here I am not even addressing the issue of getting superficial reviews that tend to accept manuscripts without any meaningful comment!

Why are reviewers not responding? Among the various reasons that can explain this situation, which can be defined as the lack of reviewers pandemic (LRP), is the brutal time constraint that scientists have to face. Writing grants, advising students, administrative activities, teaching, and the tens of meetings that need to be attended are skyrocketing. The obvious result is that researchers are having a hard time to find time to do research and write papers - their prime reason for starting a scientific career. This lack of time is definitely a discouraging factor for anyone to dedicate the little time they have left to produce a detailed and thorough review of someone else’s study. But this, in my opinion, is not all.

There is growing frustration in a multimillion-dollar industry where publishers are making fortunes while scientists are expected to do extensive reviews for free and pay increasingly higher publication fees when they take on the role of author. Moreover, in some cases they have to pay to have access to papers, including final versions of the ones they reviewed.

There does not seem to be an easy solution to this problem. The trend of the pressure leading to a constant necessity of more published articles (the Bakery Effect - see Kellner & Ponciano 2008) continues, despite criticisms of different nature (McManus et al. 2021) that occasionally get entangled with broader issues (e.g., Oliveira et al. 2021). If publication fees are the way forward, something that is increasingly widespread (e.g., Alencar & Barbosa 2021), the importance of the reviewer must be recognized more effectively. Perhaps some kind of reward should be considered, such as a financial gesture, as it seems that a full consultancy payment that would be fair and equivalent to the work performed is simply not viable. Valuing reviews for career advancement, ensuring that these can somehow at least be partially equivalent to the importance given to publications, could be another path to be taken. One way or another, the LRP problem is here, it is serious, and, if a definitive solution is not viable, serious discussions about how to mitigate it should be conducted.

REFERENCES

• Aksoy M, Yanik S & Mehmet MF. 2023. Reviewer Assignment Problem: A Systematic Review of the Literature. Journal of Artificial Intelligence Research 76: 761-827.
• ALENCAR BN & BARBOSA MC. 2021. Open Access Publications with Article Processing Charge (APC) Payment: a Brazilian Scenario Analysis. An Acad Bras Cienc 93: e20201984. DOI 10.1590/0001-3765202120201984.
• FOX CW. 2017. Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution. Scientometrics 113: 465-477. DOI 10.1007/s11192-017-2489-5.
• KELLNER AWA. 2021a. Brief comments on the 2020 Impact Factor of the AABC released by Journal Citation Reports. An Acad Bras Cienc 93: e2021934. DOI 10.1590/0001-376520212021934.
• KELLNER AWA. 2021b. What lies ahead for periodicals produced in Brazil? An Acad Bras Cienc 93: e202193s4. DOI 10.1590/0001-37652021202193s4.
• KELLNER AWA & PONCIANO LCMO. 2008. H-index in the Brazilian Academy of Sciences - comments and concerns. An Acad Bras Cienc 80: 771-781. (http://dx.doi.org/10.1590/S0001-37652008000400016).
  » https://doi.org/10.1590/S0001-37652008000400016).
• MCMANUS C, BAETA NEVES AA, DINIZ FILHO JA, MARANHÃO AQ & SOUZA FILHO AG. 2021. Profiles not metrics: the case of Brazilian universities. An Acad Bras Cienc 93: e29290261. DOI 10.1590/0001-3765202120200261.
• OLIVEIRA L, REICHERT F, ZANDONÀ E, SOLETTI RC & STANISCUASKI F. 2021. The 100,000 most influential scientists rank: the underrepresentation of Brazilian women in academia. An Acad Bras Cienc 93: e20201952. DOI 10.1590/0001-3765202120201952.

Publication Dates

• Publication in this collection
  15 Dec 2023
• Date of issue
  2023