Peer review: a constantly-evolving scientific process

POINT OF VIEW

Programa de Pós-Graduação em Ciências do Exercício e do Esporte - Universidade Gama Filho; CLINIMEX - Clínica de Medicina do Exercício - Rio de Janeiro, Brasil

Mailing Address

Keywords: Peer Review, Research; Bibliometrics; Periodicals as Topic.

This article reviews some aspects and contextualizes the process of evaluating scientific merit by peer review, with emphasis on the absence of a more systematic method and lack of formal training to perform the review. Subsequently, based on the experience of the author as scientific reviewer for journals and funding agencies, 25 practical suggestions are presented for a more fruitful and efficient peer review of the Brazilian Archives of Cardiology.

The advance of science is essential to society. For new concepts, information and technology to be incorporated, it is crucial that science progresses on a solid, consistent and reliable basis. While in the past the book was the primary form of dissemination of knowledge for scientists, in recent centuries, the scientific journals have been occupying this space. Structured primarily as organs of dissemination of scientific societies, the journals, which by their periodic nature were named "periodicals", made the information, results and conclusions published therein to become more valued.

In parallel, whereas in the past only a few researchers, often dissociated from everyday practice, published articles and books, over the years this attitude gradually changed. Initially, the number of professionals increased with the involvement of the university due to the association between teaching, research and extension, creating a growing demand for higher-education teachers, regardless of their field of knowledge, to engage in research. More recently, with the increase in training opportunities and differentiated academic formation - Master and Doctorate degrees - the number of graduates from these courses who have a differentiated, scientifically based professional activity, often dissociated from the university career, but who remain motivated and engaged in the production of knowledge is increasing.

It is therefore natural that there has been a considerable increase in production of knowledge, resulting in strong pressure on scientific journals. In turn, the editorial possibilities of increasing the number of articles published by a scientific society in a given period of time, either for economic or operational reasons, tend to be objectively limited. Within this context, there has been a progressive imbalance in the demand-supply association, forcing the journals to be more selective when choosing what to publish.

The academic-scientific scenario has always had the practice of evaluating the merit in thesis or dissertation presentations, in studies sent to Congresses, when requesting financial support for research funding or when reviewing articles submitted for publication in journals. This process, generally called peer-review, has been granted an increasing decision-making responsibility^1,2. While, in past, an editor with occasional support from some members of the editorial board decided to accept or reject an article, more recently, peer-review has become more sophisticated and consolidated as a fundamental step in the scientific process³.

Regardless of academic discussions about merits and shortcomings of this process^3-10, including those recently published in Brazilian medical journals^11,12, peer-review is now universal in higher-quality journals. Interestingly, this process is rarely taught, practiced or discussed during the higher-level academic formation^13-19 or even by the researcher or scientist, and, it is, in fact, very little scientifically validated^1,5,6. Similarly, scientific journals publish little on the subject and offer scant resources for the peers to perform this function.

This lack of information and knowledge is being addressed, albeit still incipiently, by literature^15,20 and, more recently, by a specific forum of a publishing company responsible for several journals in different fields of knowledge²¹. It seems clear that the process of peer-review is presently in frank development and evolution.

Similarly to the leading high-quality journals in the healthcare area, the Brazilian Archives of Cardiology have been consolidating a peer-review policy. When submitting a manuscript, authors may suggest reviewers from a long list of names of peers. However, it is possible that many of them have not had specific training or lack the experience to act as reviewers. Based on a long history in the process of peer review - on both sides, as author and reviewer - I intend to present in this text, 25 considerations and practical suggestions on how to act efficiently, for the journal, the editor and the author (s), as reviewer.

1. Being invited to work in peer-review is an act of implicit recognition of your competence in the area of knowledge and your expertise. The editor or associate editor who invited you is delegating the responsibility to advise them on the merits of publishing or not publishing a given article.

2. Carefully read the submitted abstract and only accept to review it if you have the proper technical and scientific expertise and the assurance of time availability to do so within the deadline proposed by the journal.

3. Confirm receipt and acceptance of the invitation to review it as soon as possible.

4. Carefully record this invitation, either by using a specific marking for the e-mail or putting it in a specific folder, and, ideally, record at least the following items: journal name, article title, date of the invitation and deadline for response.

5. Prioritize your activities as not to exceed the proposed deadline. If you have to do it, inform the editor as soon as possible, allowing him/her to decide whether to cancel the request made to you or to extend the deadline. As good practice, once you accepted the invitation to assess the manuscript, it should be done within a maximum of 15 days. This period should be even lower for the second review, when it takes less time to check the authors' answers and the new version of the text.

6. If you have too many manuscripts to assess, try to schedule a time in your agenda for this action. Ideally, start and complete the task of assessing a manuscript on the same day and without major disruptions.

7. Do not discuss with professors, students or colleagues about the article you are to review or the assessment you are to make, even after the article has been published. Maintaining the secrecy of the reviewer and the author is critical to the quality and fairness of the process.

8. Always consider that the article must have been written by serious and competent researchers. YOU ARE NOT THE AUTHOR! Thus, respect the authors' style, ideas and conceptions, even if they are not exactly of your preference. Be open to healthy disagreements to your status quo.

9. Carry out a complete and preliminary reading of the article for an overview and as a reader who knows the topic. Form a preliminary view on the degree of originality, overall quality of text and contribution to the body of knowledge.

10. With the text on your computer screen, open a window of your word processor and insert two titles - general points and specific points -. Below the general points, write a brief paragraph describing what the article is about.

11. Following, write a paragraph, scoring: adequacy of the article to the journal scope and probable degree of interest to readers, general aspects of the article formatting in relation to the journal style - number of words, organization into sections, number of figures and tables, keywords etc. - and clarity of writing. If there are problems with the grammar and/or typing errors, just specify the need for a revision in this regard.

12. Carefully analyze the list of references. Be stringent regarding secondary references (it is necessary to cite the original source) or others that are not very accessible, such as theses and congress abstracts. It is convenient to do a quick literature search with the most appropriate keywords, checking if any recent and relevant article has been omitted. Make a general comment on the appropriateness of the number, degree of updating and formatting of references. Make sure the cited articles have, in fact, the information described in the article undergoing review. As the theme is your area of expertise, an incorrect statement will probably be easier to be observed and checked. This is one of the best possible contributions made by the reviewer. If you identify or remember some important article that has been omitted, make a note below the caption "specific points". Do not be offended if the authors did not mention you "famous and classic" article in their reference list. If the number of references is already long, you should indicate those that seem less relevant or important.

13. Analyze the title and summary. Are they self-explanatory? Both the title and summary must be well written, rousing the interest of potential readers. For original articles, the abstract should contain numerical results and an explanatory conclusion.

14. Review the various sections of IMRAD (Introduction, Methods, Results And Discussion), noting down questions, identifying omissions or making suggestions for text improvement, sequentially at specific points. More relevant and potentially serious questions should be still placed in general points. These points must be carefully and thoroughly answered and/or reviewed by the authors, if the editor gives the authors a chance for a new submission. Be careful and objective in your writing. Avoid arguing, devaluing or using negative adjectives. As you write, think about how the writer will react to your assessment.

15. Those who regularly publish in the area and about the topic of the article are in serious danger of feeling as if they "owned" the subject and wanting to be too detailed. Resist the most to this perverse trend. Always remember, you are not the author of the article!

16. Only make comments about the statistical analysis if you have the appropriate expertise. In case of doubt or otherwise, suggest the need for an expert's opinion.

17. Comment on the need and appropriateness of tables and figures, especially in terms of readability, duplications and clarity of the information shown in them.

18. Be critical of the use of scientific units and technical terms throughout the text. Errors in these aspects are very harmful to the journal, if the article turns out to be effectively published.

19. Get acquainted with it, if necessary, and always keep in mind the typical level of scientific articles regularly published by that journal. This perception is crucial to properly adjust your opinion. There are no 100% perfect or imperfect articles. Try to stratify and weight your opinion according to this perspective. For the same article, the degree of advice on its publishability should be different when it comes to a journal of high impact factor or another still in the consolidation phase, of a more local or regional character.

20. Recommend rejection if the article has serious problems that cannot be solved by text review or data reanalysis, such as incomplete collection or poor quality of information, excessively small sample size, lack of control groups when this was crucial to analysis, inappropriate scientific hypothesis or one that has already been well studied (lack of originality), inadequate experimental design or method to answer the formulated question. Even in this situation, always try to offer suggestions for improving the text and the performed analysis.

21. It is almost always possible to improve an article somehow. Feel free to constructively suggest changes and/or details that seem the most appropriate and the author (s), the editor and readers will thank you for it.

22. Make no objective mention about accepting or rejecting the article in the space destined for comments for the authors; this must be done in space reserved for confidential comments to the editor. Avoid using adjectives that are very direct in favor of or against the article in your assessment.

23. If the article is sent to you for reassessment after the authors submitted a new version, limit yourself to check if your recommendations have been incorporated. Be tolerant and carefully evaluate the situations in which the authors disagree with your recommendations. Only insist on demanding that something else be revised if it is of utmost relevance and importance and also potentially damaging to the readers and the scientific knowledge, which, let us face it, is very rare.

24. Keep in mind that your role as reviewer is to advise the editor or associate editor, based on best available evidence, and not to make the final decision, which is their sole responsibility. Your role is that of an expert, not a judge!

25. In the final comments to the editor, thank for the opportunity to have reviewed the article and try to synthesize your assessment, concluding it in an objective manner with a final recommendation, signaling, in your opinion, what degree of priority and what the potential citability of the article is, in case it is published.

Working with peer review means voluntarily allocating a significant amount of time and energy into an assignment of great responsibility, which involves different interested parts. In fact, a recent editorial pointed out that for every article submitted, an author should be available to review two or three articles of their peers⁹. It is also a unique opportunity to know new articles before they are published to learn with different peers how to deal with the scientific process¹². Especially rich in learning is the chance to read the comments of other reviewers and editors, answers and new versions of the articles submitted by authors after assessments and the process operationalization by the journal back-office, which will certainly help you in your next submission as an author.

I hope this article contributes to the improvement of peer review of the Brazilian Archives of Cardiology; one must not fail to recognize that the process of peer review still needs to be much improved and that its consistency - measured by comparing the opinions of different reviewers for the same article - is still alarmingly low, causing dissatisfaction among the authors, especially due to the long waiting periods²², and the fact that, even when there is rejection by a journal, most articles end up being published, even if in journals with lower impact factors^5,23,24 .

Acknowledgments

The author's scientific activities are partially financed by research productivity grant from CNPq - Level 1A - and the Scientist of Our State - FAPERJ program.

References

• 1. Barrio JR. Consensus science and the peer review. Mol Imaging Biol. 2009;11(5):293.
• 2. Triaridis S, Kyrgidis A. Peer review and journal impact factor: the two pillars of contemporary medical publishing. Hippokratia. 2010;14(Suppl 1):5-12.
• 3. Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer review. JAMA. 2002;287(21):2786-90.
• 4. Hazlewood CF. Current peer review systems: a problem for innovative approaches in science. Cardiovasc Dis. 1974;1(2):67-9.
• 5. Horrobin DF. Something rotten at the core of science? Trends Pharmacol Sci. 2001;22(2):51-2.
• 6. van der Wall EE. Peer review under review room for improvemenerlands Heart J. 2009;17(5):187.
• 7. Walbot V. Are we training pit bulls to review our manuscripts? J Biol. 2009;8(3):24.
• 8. Yaffe MB. Re-reviewing peer review. Sci Signal. 2009;2(85):eg11.
• 9. Peer review--do unto others. Am J Epidemiol. 2010;171(12):1249.
• 10. Henderson M. Problems with peer review. BMJ. 2010;340:c1409.
• 11. Moraes Jr HV, Rocha EM, Chamon W. Funcionamento e desempenho do sistema de revisão por pares. Arq Bras Oftalmol. 2010;73(6):487-8.
• 12. Evora PRB, Celotto AC. Peer review, science, young investigators feelings and frustrations. Acta Cir Bras. 2011;26(1):77-8.
• 13. Peterson P. Teaching peer review. JAMA. 1973;224(6):884-5.
• 14. Gough NR. Training for peer review. Sci Signal. 2009;2(85):tr2.
• 15. Rangachari PK. Teaching undergraduates the process of peer review: learning by doing. Adv Physiol Educ. 2010;34(3):137-44.
• 16. Paice E. How to write a peer review. Hosp Med. 2001;62(3):172-5.
• 17. Christensen NB, Yokomizo A. How to peer review. Int J Urol. 2010;17(9):754.
• 18. Abbott A. A how-to for peer review. Nature. 2011;473(7345):17.
• 19. Zucker RS. A peer review how-to. Science. 2008;319(5859):32.
• 20. Navalta JW, Lyons TS. Student peer review decisions on submitted manuscripts are as stringent as faculty peer reviewers. Adv Physiol Educ. 2010;34(4):170-3.
• 21. van Wezenbeck E, Lardee J, Brumfield G. How editors find and choose reviewers - the find reviewers tool. Elsevier - Reviewers'Update: Elsevier; 2011.
• 22. Weber EJ, Katz PP, Waeckerle JF, Callaham ML. Author perception of peer review: impact of review quality and acceptance on satisfaction. JAMA. 2002;287(21):2790-3.
• 23. Ray J, Berkwits M, Davidoff F. The fate of manuscripts rejected by a general medical journal. Am J Med. 2000;109(2):131-5.
• 24. White C. Little evidence for effectiveness of scientific peer review. BMJ. 2003;326(7383):241.

Peer review: a constantly-evolving scientific process

Claudio Gil Soares de Araújo

Publication Dates

• Publication in this collection
  27 Feb 2012
• Date of issue
  Feb 2012