Acessibilidade / Reportar erro

Peer review, science, young investigators feelings and frustrations

Revisão por pares, ciência, sentimentos e frustrações de jovens investigadores

14 - POINT OF VIEW

Peer review, science, young investigators feelings and frustrations

Revisão por pares, ciência, sentimentos e frustrações de jovens investigadores

Paulo Roberto Barbosa EvoraI, Andrea Carla CelottoII

IFull Professor, Head of the Department of Surgery and Anatomy, Ribeirao Preto Faculty of Medicine, University of Sao Paulo (USP), Ribeirao Preto-SP, Brazil.

IIPhD, Postdoctoral, Department of Surgery and Anatomy, Ribeirao Preto Faculty of Medicine, USP, Ribeirao Preto-SP, Brazil.

Correspondence Correspondence: Paulo Roberto B. Evora Rua Rui Barbosa, 367/15 14015-120 Ribeirão Preto _ SP Brasil Phone/Fax: (55 16)3602-2497 prbevora@fmrp.usp.br

Brief history

David Flaxbart Chemistry Librarian from University of Texas, Austin, commenting about the book "Editorial Peer Review: Its Strength and Weaknesses,"1 wrote by Ann Weller2, that peer review can be defined simply as the process by which journal editors solicit evaluations of submitted articles from outside experts who remain anonymous to the authors. The role of journals as gatekeepers to the scientific record dates to the 17th century, when the Royal Society's council was instructed to review submissions to its Philosophical Transactions. But the modern process of "blind" peer review is much more recent. Until the mid-20th century many papers were approved solely by a journal's editors rather than independent reviewers, and for some major journals this is still the case. The explosion of scientific output after World War II strained the review process, significantly extending the lag times between submission and publication. Peer review has always had to strike a balance between speed and quality. Each end of this continuum has its champions. How the question plays out depends largely on the "culture" of a particular scientific discipline. In fast-moving fields like physics, today's breakthrough is tomorrow's old news, and speed is preferred over editorial thoroughness. This led physicists to create the preprint system, whereby pre-publication copies of papers under review were traded among scientists, essentially relegating their journals to archival repositories rather than front-line communication mechanisms. This preprint culture was quickly and enthusiastically converted to an "e-print" system pioneered by Paul Ginsparg at Los Alamos in the early 90s. Yet in other fields, such as chemistry, preprints did not catch on and journals remain supreme, although rapid-publication "letters" journals stepped in to speed up the process.

Peer review methods

In open peer review, the identities of both authors and reviewers are revealed, affording the authors the ability to identify the reviewers' comments to a person. Even though this might be an equitable strategy to prevent unfair rejections, this process has no safeguard against unfair acceptance of papers - reviewers, and especially newcomers, may feel pressured into accepting a mediocre paper from a more established lab in fear of future reprisals. Single-blind peer review (SBPR), in which the reviewer knows the identity of the author but not vice versa, is the currently accepted practice. Because SBPR can be vulnerable to sexism and nepotism1 its ethical foundations have come under criticism; the method is frequently recognized to be biased against new ideas, women, young scientists, career changers, and scholars from less prestigious universities and/or from developing countries3. Generally, two policies have been proposed to eliminate bias from the peer-review process: open peer review and doubleblind peer review (DBPR)4.

Effects of editorial peer review

Editorial peer review is widely used to select submissions to journals for publication and is presumed to improve their usefulness. Sufficient research on peer review has been published to consider a synthesis of its effects. To examine the evidence of the effects of editorial peer-review processes in biomedical journals, Jefferson et al.5 conducted electronic and full-text searches of private and public databases to June 2000 and corresponded with the World Association of Medical Editors, European Association of Science Editors, Council of Science Editors, and researchers in the field to locate comparative studies assessing the effects of any stage of the peer-review process that made some attempt to control for confounding. Nineteen of 135 identified studies fulfilled our criteria. Because of the diversity of study questions, methods, and outcomes, they did not pool results. The review resulted in nine studies considered the effects of concealing reviewer/author identity. Four studies suggested that concealing reviewer or author identity affected review quality (mostly positively); however, methodological limitations make their findings ambiguous, and other studies' results were either negative or inconclusive. One study suggested that a statistical checklist can improve report quality, but another failed to find an effect of publishing another checklist. One study found no evidence that training referees improves performance and another showed increased interrater reliability; both used open designs, making interpretation difficult. Two studies of how journals communicate with reviewers did not demonstrate any effect on review quality. One study failed to show reviewer bias, but the findings may not be generalizable. One nonrandomized study compared the quality of articles published in peer-reviewed vs other journals. Two studies showed that editorial processes make articles more readable and improve the quality of reporting, but the findings may have limited generalizability to other journals. The authors concluded that editorial peer review, although widely used, is largely untested and its effects are uncertain6.

Five years later the same author, trough a wide methanalisys concluded that little empirical evidence is available to support the use of editorial peer review as a mechanism to ensure quality of biomedical research. However, the methodological problems in studying peer review are many and complex. At present, the absence of evidence on efficacy and effectiveness cannot be interpreted as evidence of their absence. A large, well-funded programme of research on the effects of editorial peer review should be urgently launched5.

Peer review and young investigators feelings

Most trainees and young scientists experience their first encounters with the peer review process in the role of an author. Non-acceptance of manuscripts is unfortunate, but occurs frequently, and is occasionally perceived as a personal rejection. If trainees understand the tasks and roles of authors, editors, and reviewers, they may be able to appreciate the feedback mechanism of peer review. In addition, they can recognize that the primary goal of the feedback is to assess the research and not the researcher. As trainees have limited exposure as authors, editors often do not know the trainees and their area of expertise7-9.

"As far as we know this is the first study demonstrating the " This kind of phrase became a kind of jargon, it is dangerous because it may suggest ignorance, scientific immaturity, or even sound like pretentious. Once again I had this reaction when reviewing a paper on a quiet Sunday. But I was struck by doubt about young researchers. With this romantic doubt turned to Google and MEDLINE. A Google search showed a strong correlation between peer review, and young researcher grants. A MEDLINE search showed no paper and I had the desire to write for the first time endeavored to correlate the peer review system with the frustration of young researchers. But I'm not getting any younger...

Many years ago, as a young researcher, I wrote this kind of concept and one reviewer taught me a great lesson: "it would be a first study, but is it important or relevant?" I am writing this comment because I am sure that it is a great kind of scientific advice. I incorporated it to my basic directions on research. It's a pity I have not known this reviewer to say thanks. This is a great peculiarity of science and of the peer review system for paper evaluations.

Approving a paper is a pleasurable activity, but rejection implies some degree of emotional distress. However, when the revised paper is published and indexed the reviewer get a little pride in his anonymous contribution. This is one of the best feelings that a scientist can have. For this reason is that the activity of peer review, often a hard, is gratifying, but I needed to express my concern, once again, with the paper rejection frustrations. With all the criticism possible peer review is the best, most democratic and honest evaluation system. The reviewers, in addition to their anonymous participation in publications, have a great opportunity to confront their opinions with other "anonymous" colleagues. Surely, this is one of the most rewarding ways of doing science.

This text is a tribute to every young researcher and why not to every researcher and his sadness at the difficulties in publishing a paper? It is a particular tribute to our students of graduation and post-graduation many times that I share their frustrations when one paper, the result of hard work, is rejected.

  • 1. Weller Ann C. Editorial peer review: its strength and weaknesses. (ASIST monograph series) Medford NJ: American Society for Information Science and Technology, 2001. ISBN 1-57387-100-1.
  • 2. Flaxbart D. Book Reviews Weller, Ann C. Editorial peer review: its strength and weaknesses. Issues in Science and Technology Librarianship, Summer. 2001. http://www.istl.org/01-summer/review.html
  • 3. Wenneras C, Wold A. Nepotism and sexism in peer-review. Nature. 1997;387:341_3.
  • 4. Abate T. What's the verdict on peer review? Available from http://www.columbia.edu/cu/21stC/issue-1.1/peer.htm 1996. Accessed 20 July 2005.
  • 5. Jefferson T, Alderson P, Wager E, Davidoff F. Effects of editorial peer review: a systematic review. JAMA. 2002;287:2784-6.
  • 6. Mainguy G, Motamedi MR, Mietchen D. Peer review-the newcomers' perspective. PLoS Biol. 2005;3:e326.
  • 7. Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev. 2007;2:MR000016.
  • 8. Pernetta AP. Mentors could support a student reviewer database (Comment). Nature. 2007;448(7150):129.
  • 9. Aronsky D, Haux R, Leong TY, McCray A. The student editorial board of methods of information in medicine-an opportunity to educate tomorrow's peer reviewers. Methods Inf Med. 2007;46:623-4.
  • Correspondence:
    Paulo Roberto B. Evora
    Rua Rui Barbosa, 367/15
    14015-120 Ribeirão Preto _ SP Brasil
    Phone/Fax: (55 16)3602-2497
  • Publication Dates

    • Publication in this collection
      20 Jan 2011
    • Date of issue
      Feb 2011
    Sociedade Brasileira para o Desenvolvimento da Pesquisa em Cirurgia https://actacirbras.com.br/ - São Paulo - SP - Brazil
    E-mail: actacirbras@gmail.com