IF PUBLISHING IS NECESSARY, SO IS REVIEWING!

FIRST WORDS

The subject of “reviewing articles” is a relevant part of the daily life of every researcher. However, my experience as an editor and reviewer for Brazilian and international journals indicates that the level of commitment of researchers, in general, is much less when they are reviewing as compared to when they are authoring an article. The prestige of being published in a leading journal is not the same as being an outstanding reviewer in the same journal. Such asymmetry is contradictory, since peer review is the primary system that gauges the quality of scientific output and is as important for the survival of a journal as the submission of articles. The quality of the reviewers and their reviews must be equal or higher than the published articles in order to ensure the system’s credibility.

I aim to reflect on the challenges that arise in reviewing scientific articles on Administration in Brazil, as well as contribute some ideas to improve the training of reviewers. I utilize two aspects that I believe are important to carrying out the proposed objective. The first is to discuss the reviewer’s role during the review of articles. The second is to reflect on the technical skills necessary to conduct a good review. In addition to these two points, I conclude with a reflection on how the current criteria for evaluating master’s and doctoral programs, as well as the granting of research assistance in Brazil, have hindered the training of reviewers and devalued the importance of conducting reviews.

A BRIEF SUMMARY OF THE BRAZILIAN CONTEXT

In recent years, the field of Administration in Brazil has attempted to broaden its international scope as well as the impact of its scientific production. In a “tropicalized mimetic” process (i.e., a decontextualized copy), the Brazilian field of Administration has the practices from the countries of the North (particularly the United States and the United Kingdom) as the references of scientific excellence (Faria, 2011). The principal practice of scientific measurement adopted in these countries is the number of articles published in high impact journals. To publish articles in notable journals is the primary means of defining superior quality in knowledge production.

This situation has led to a significant increase in the number of journals in Brazil, as well as in Brazilian researchers seeking to publish articles. Consequently, there has been an expansion in the demand for people capable of reviewing all that scientific production. The article review process seeks to ensure the quality of published content through rigorous editorial processes (Chrisman, Sharma, & Chua, 2017). However, when there is a scarcity of reviewers due to their lack of availability or lack of knowledge in conducting a quality review, the editorial process is compromised in terms of the time it takes to revert to authors and regarding the quality of the published content.

The process of reviewing scientific articles is rather complex as it involves both behavioral and technical elements. In the Brazilian context, the method and technical aspects of evaluative reviewing need to be discussed and incorporated into the curricula of graduate programs in Administration with the same consideration currently provided to the production of scientific articles. The desire to “publish” is turning master’s and doctoral programs into laboratories for article production, instead of educating superior master’s and doctoral candidates. Thus, the objective or goal of a graduate course has become to generate final papers that can be transformed into “articles” (Bispo & Costa, 2016).

BEHAVIORAL ASPECTS OF THE ARTICLE REVIEW

It is common to conceptualize the review of scientific articles solely from a technical perspective on how to perform this task (Shigaki & Patrus, 2016). However, the basis for high-level reviews depends on the behavior of the reviewer. The principal components of appropriate behavior for a review are: a) an awareness of the importance of the evaluation process, and b) a constructive, developmental approach by the evaluator in relation to the authors of the evaluated text. Although it may seem obvious or banal, it is common for researchers not to realize that being available to review articles - in proportion to the number of articles that are produced - is of equal importance to writing and publishing articles.

In order to ensure equilibrium in the review system, a fundamental requirement is an awareness to be as willing to evaluate the writing of colleagues, as researchers are to write their own articles. It is important to remember that every submitted article requires at least two other individuals (not including the editors) to review it. Therefore, it is essential that each author who submits an article should be available to review at least two manuscripts by colleagues. The lack of willingness to review the necessary proportion of articles leads to an overloaded evaluation system. In times of “academic productivism” (Alcadipani, 2011; Faria, 2011; Rigo, 2017), it is common to find researchers with numerous published works who have few or even no reviews listed on their résumés. This attitude signals a level of egotism and lack of engagement with the field and with peers.

Another important aspect regarding awareness of the importance of the review process is compliance with the deadlines to complete reviews. In general, many researchers consider review activity to be secondary in their agendas. Hence, it is common for articles to take months to be evaluated, and authors do not receive the first response to their article until one or two years have elapsed. Deprioritizing the completion of reviews contributes to the tardiness of the editorial process. Reviews need to be given the same priority as article writing.

Regarding the importance of a constructive stance, it is necessary to emphasize that reviewing is a process wherein there should be no hierarchy between the person who evaluates and the person whose work is being evaluated. It is customary for someone who is reviewing an article one day to be authoring an article the next. Thus, evaluating articles that are assigned to us for review should be accorded the same cordiality and respect that we would expect from the reviewers of our own articles. It is not uncommon to get feedback that demonstrates arrogance or even disdain for the work under review as well as for its authors (Clair, 2015; Gondim, 2004; Rigo, 2017). A good reviewer and a good review always employ a cordial and constructive attitude. Reviewers who adopt this perspective, seek to help authors improve their work rather than only criticize them.

A constructive stance is always appropriate, regardless of the quality of the work under review. Even if a reviewer has been designated to review “the worst article in the world”, the reviewer must remain respectful and seek to assist the author in developing his or her ideas. It is important to note that in the Brazilian publishing system, there are numerous people in training (students) who are pressured (in my view, erroneously) to publish. They are compelled to publish in order to obtain their diplomas (which is absurd). Student production of articles is included in assessments by the Brazilian government as a criterion of the “quality” of master’s and doctoral courses (which is also illogical), leading professors to force their students to publish articles. In these cases, in addition to being cordial, the review process should be didactic and educational; in other words, it should not be traumatic for young authors. It is common to find senior researchers engaging in angry debates and harshly criticizing students’ work in a tone that they would not use when debating with peers. Essentially, a good reviewer must recognize the author’s level of maturity when reading the manuscript.

Another important point is that the reviewer must not outsource the task, especially to his or her students. Many reviewers, on the pretext of initiating their students into the world of article reviewing, delegate evaluation responsibility to their mentees who are still discovering the academic world. These reviewers argue that conducting an actual review is an excellent method to learn evaluating. They also claim that after a student reviews the text, the reviewer will “revise” the completed review. Any experienced editor can identify a review conducted by someone who does not understand the process well. This form of outsourcing to students compromises the quality of the review and demonstrates a lack of ethics on the part of the reviewer. A simple method to initiate students into reviewing is to first instruct them on how the process works and then ask them to revise texts that senior reviewers have already reviewed and submitted to journals.

TECHNICAL ASPECTS OF THE REVIEW OF ARTICLES

Most graduate programs on Administration in Brazil do not include the topic of reviewing articles in the curriculum. In general, it is a heuristic learning process wherein people are invited to review articles without any prior preparation. This causes reviewers to evaluate the text according to what they think is appropriate, which is undesirable as it results in fluctuating evaluation standards without any assurance that reviews will actually assist the authors in improving their work.

Despite the lack of a single model for conducting reviews of scientific articles, there are certain guidelines that should be adhered to in a good review (Clair, 2015; Gondim, 2004; Shigaki & Patrus, 2016). In my opinion, the most important aspects of a review are: a) to evaluate what the author has accomplished, not what the reviewer would have done; b) to focus on the text’s contribution; c) to adopt a critical stance while offering constructive suggestions; d) to list actions in sequence; e) to refer the author to relevant sources in the literature; and f) to ask questions when unclear or doubtful about anything in the article.

The first key point in a review is to conduct the process while respecting the author’s choices. That means reviewing the article by examining what the author has accomplished, not how the reviewer would have done the work. This point relates primarily to epistemological, theoretical, and methodological choices, as well as the author’s style. Although the author’s choices are not the reviewer’s, it is the reviewer’s duty to respect them and conduct the evaluation following the author’s line of thought. This includes respecting the author’s choices regarding the text’s structure and its use of language (for example, the adoption of the first person singular tense to compose the manuscript). Thus, besides providing the opportunity to learn about new approaches, the reviewer maintains the author’s autonomy and originality in the production of the text.

The second point is to focus on the contribution of the manuscript to the body of knowledge in that field. There are many reviewers who, rather than focu sing on the article’s content, pay more attention to indicating grammatical and formatting inconsistencies. Although this should be done at some point in the review process, the most important aspect is to evaluate the actual contribution of the work to the field and whether, in that sense, the content is credible. Notice that I use the word “contribution” instead of “innovation.” This word choice is to indicate that a scientific text can be relevant even without presenting an “innovation.” A text can make an excellent “contribution” by focusing on practical or didactic issues or offering new interpretations about a particular subject, theory, or method.

The third point, about the importance of a critical-constructive stance, establishes the entire foundation of any review. The reviewer must articulate criticisms that he or she considers necessary, but they should always be accompanied by suggestions that guide the author to overcome weaknesses encountered in the text (Clair, 2015; Gondim, 2004). This attitude ensures that the author not only understands the criticism of the work but also which path to follow in order to improve the manuscript. It also facilitates the revision of the article by the authors as well as the editors, and indicates distinct ways to develop the work being evaluated.

The fourth aspect is to list actions in sequence by organizing the evaluative assessment into topics and a sequence of actions. Thus, the reviewer clarifies the weaknesses to the author, point by point. Simultaneously, the reviewer offers sequential suggestions for improving the text. This technique facilitates the author’s understanding of exactly what is required, and makes it easier for the author to prepare a response letter to the editor during the review process.

The fifth point concerns referring the author to pertinent studies in the literature. Although it is not mandatory for a review, suggesting complementary reading to the author that relates to a particular point of criticism provides depth to the discussion and improves the author’s understanding of the criticism that has been made. However, many evaluators take advantage of the opportunity to refer their own texts to the author, thus increasing the number of citations to their own work. While referring an author to the reviewer’s own work is not prohibited, it should be undertaken only when the cited article significantly improves the author’s work.

Finally, asking the author questions when the reviewer lacks knowledge or is in doubt is an excellent means of establishing a cordial dialogue and provides the author opportunities to explain certain topics more clearly. Even if the reviewer is an expert in the field, such a position does not imply that he or she “owns the truth” or even possesses all knowledge on a certain topic. Asking questions significantly contributes to the author’s ability to improve the explanations of concepts and ideas that may still be unclear in the current version of the text.

WHAT SHOULD TO BE DONE TO IMPROVE REVIEWS?

Appending the behavioral aspects of an article’s development to the technical aspects, make the review of a scientific article a process of mutual learning for both reviewers and authors, thereby significantly facilitating the advancement of scientific knowledge. Rather than being merely a “publication,” a scientific article should be instrumental in promoting new knowledge. In this final section of the text, I will present some suggestions on training new reviewers in doctoral programs in Administration. In addition, I will comment on the way researchers race to publish that directly impacts the article review system and other academic texts such as research projects.

Regarding the training of new reviewers, I argue that it is essential to include this topic in doctoral programs in Administration. This is not to suggest that a specific course should be created on reviewing articles, but rather that it should be included as a topic in courses that involve the subject. Creating workshops that address the review process through discussions and hands-on activities is an excellent method of introducing doctoral students to the world of article review.

One relevant observation is that good evaluators inevitably significantly improve their own skills as authors. This occurs through the exercise of seeking to understand the nuances of a good scientific article, as well as identifying means to improve an article. Moreover, understanding the behind-the-scenes editorial processes of journals broadens an individual’s comprehension regarding the requirements for publishing an article successfully, especially in reputed journals. In addition, those who frequently evaluate articles, especially for renowned journals, obtain access to studies that are at the vanguard of research. The process also provides reviewers with opportunities for self-assessment of their own research and work.

In conclusion, I would like to discuss a final point that I believe is extremely important. Further dialogue is required on the degree to which the grading and evaluation systems in Brazilian graduate programs-under the responsibility of the Office for the Improvement of Higher Education Personnel (CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior)-as well as the process for determining scholarships and research grants by the National Council for Scientific and Technological Development (CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico), culminate in being complicit in questionable practices that lead to academic productivism.

This is evident when CAPES, in the evaluation processes of graduate programs, and CNPq, in its procedures to grant financial aid to researchers, adopt the number of published articles as a principal criterion for assessment. Such practices also discount the value of researchers’ contributions through other activities to the teaching and research system as a whole. By exclusively focusing on the publication of articles, there are currently some researchers with more than one publication per month (I wonder how it is possible to write that much given all the daily activities that are demanded of researchers). The fundamental point in this discussion is the fact that individuals with questionable academic practices are being rewarded. The excessive number of articles published, as well as the number of authors in each article, and the position of authorship listed in texts raises questions. The exclusive focus on publications means some researchers do not contribute to article evaluation, or coordinate research projects, or other even other responsibilities at the expected degree of quantity and quality.

For instance, all the recent minutes of the Consulting Committee in the field of Administration and Accounting of the CNPq regarding the evaluation of researchers who received research grants, mentioned that there is always a designated contingent of projects that are not evaluated by researchers. In the minutes, the Committee repeatedly requested that the CNPq leadership decisively act on the matter. This example highlights the necessity of improving the rules for evaluating graduate programs and for awarding research grants in order to value the review process, especially regarding scientific article reviews.

Adopting a criterion that strikes a balance between the number of submissions and the number of publications by researchers in the field of Administration in Brazil is urgently necessary. Such action is essential for two reasons. First, to ensure the sustainability of the system in terms of submissions and evaluations by providing appropriate timeframes for authors to receive responses and quality feedback. Second, to ensure that resources invested in graduate programs as well as in journals and research aid, are transformed efficiently and effectively into productive knowledge. This should be knowledge that improves the quality of life of society as a whole, rather than merely becoming a videogame where the player with the most points wins.

REFERÊNCIAS

• Alcadipani, R. (2011). Resistir ao produtivismo: Uma ode à perturbação acadêmica.Cadernos EBAPE.BR, 9(4), 1174-1178.
• Bispo, M. S., & Costa, F. J. (2016). Artigos como avaliação discente em disciplinas de pós-graduação: Instrumento educativo ou subsistema de linha de montagem?Cadernos EBAPE.BR, 14(4), 1001-1010. doi: 10.1590/1679-395141927
  » https://doi.org/10.1590/1679-395141927
• Chrisman, J. J., Sharma, P., & Chua, J. (2017). The mind-set of editors and reviewers. Family Business Review, 30(3), 211-218. doi:10.1177/0894486517708377
  » https://doi.org/10.1177/0894486517708377
• Clair, J. A. (2015). Procedural injustice in the system of peer review and scientific misconduct. Academy of Management Learning & Ed­ucation, 14(2), 159-172. 10.5465/amle.2013.0243
  » https://doi.org/10.5465/amle.2013.0243
• Faria, A. (2011). Repensando produtivismo em gestão no (e a partir do) Brasil.Cadernos EBAPE.BR, 9(4), 1164-1173.
• Gondim, S. (2004). A face oculta do parecerista: Discussões éticas so­bre o processo de avaliação de mérito de trabalhos científicos. Orga­nizações e Sociedade, 11(31), 195-199.
• Rigo, A. S. (2017). Comunidade acadêmica, produtivismo e avaliação por pares.RAE-Revista de Administração de Empresas, 57(5), 510-514. doi: 10.1590/s0034-759020170508
  » https://doi.org/10.1590/s0034-759020170508
• Shigaki, H. B., & Patrus, R. (2016). Avaliação de artigos científicos em administração: Critérios e modelos de avaliadores experientes.Teoria e Prática em Administração, 6(2), 107-135. doi:10.21714/2016_v6i228445
  » https://doi.org/10.21714/2016_v6i228445

Publication Dates

• Publication in this collection
  Jul-Aug 2018