Halo Effect in Peer Review: Exploring the Possibility of Bias Associated with the Feeling of Belonging to a Group

Introduction Analyses Conclusion

In this study, we explore the possibility of bias in the article evaluation process during peer review, due to homogeneous groups of reviewers, most of whom belong to the same professional field. The conditions of the context of journals considered for the manifestation of bias are: a) articles written in a regional language other than English, which restricts the group of reviewers; b) specific theme, which requires reviewers who specialize in the theme and predominantly belong to the same professional group; and c) publication of articles associated with the interests of the professional group to which the reviewers belong. Our proposition is that under these conditions, articles containing messages in keeping with the concerns and demands of the group to which the reviewers belong, could be viewed more favorably reviewers, thereby benefiting from the halo effect. We use the case study as a research strategy, identifying of a Exploramos nesta pesquisa a possibilidade de viés no processo de avaliação de artigos, durante a revisão por pares, decorrente de grupo homogêneo de avaliadores pertencentes em sua maioria a uma mesma categoria profissional. As condições de contexto da revista consideradas para manifestação do viés são: a) artigos redigidos em idioma regional, distinto do inglês, que restrinja o grupo de avaliadores; b) temática específica, que demande avaliadores especializados no tema e pertencentes predominantemente a mesma categoria profissional; e c) publicação de artigos associados aos interesses da categoria profissional de pertencimento dos avaliadores. Nossa proposição é que nessas condições, artigos contendo mensagens alinhadas aos anseios e reivindicações do grupo de pertencimento dos avaliadores, podem obter maior empatia dos avaliadores, beneficiando-se assim do efeito halo. Utilizamos o estudo de caso como estratégia de pesquisa a partir da identificação de uma revista que atendesse as condições de contexto descritas. Aplicaram-se as técnicas de análise de conteúdo, aos textos de artigos pertinentes ao contexto da pesquisa. As análises indicam que interesses de grupos profissionais podem prevalecer sobre os

Introduction
The rejection rate of articles submitted to top tier journals is over 90% (DAY, 2011). This rejection rate has risen in recent decades (STRAUB, 2008), creating major stress and concerns among authors involved in the review of scientific articles. The most widespread review process comprises two phases: the editor review and peer reviews. Biases may occur in any decision-making process (TVERSKY; KAHNEMAN, 1974;WHITE;POLDRACK, 2014). Therefore, there is also the risk of wrong decisions in double-blind peer reviews (LEE; SUGIMOTO; ZHANG; CRONIN, 2013). The literature indicates the occurrence of bias in article reviews especially in the editor review due to the perception of the quality of the article (halo effect) due to the characteristics of the authors, especially the author's name and affiliation (PETERS; CECI, 1982). This study discusses the possibility of occurrence of the halo effect in the subsequent phase of the process, the peer review phase. This study is based on the assumption that articles aligned with the concerns and demands of the group to which the reviewers belong are viewed more favorably by the reviewers, encouraging the halo effect.
The possible analysis bias investigated in this study is highly specific in terms of the necessary characteristics of the context of the environment for it to occur, making it very rare and, thus, difficult to observe. A combination of the following characteristics of the journal is required: a) texts of articles written in a regional language other than English, restricting the group of reviewers to those that master the language; b) a highly specific theme, requiring reviewers who specialize in the theme and predominantly belong to the same professional field; and c) the journal publishing articles that address topics associated with the interests of the professional field to which most of the reviewers belong. The overlapping of these characteristics could mean that reviewers assess articles addressing themes closely linked to the interests of their own professional field. This situation could result in bias in favor, in other words, the halo effect in the review of these articles. Identifying a journal with all the characteristics of context described, we structured the present study with the following objective: to analyze articles with content closely linked to the interests of the professional group of the reviewers, verifying whether this content was developed within the precepts of scientific research; in other words, whether they respect the canons of scientific research.
The importance attributed to journals and their articles for the evolution of science and society is demonstrative of the relevance of the double-blind review (BORNMANN;2012;THYER, 2008) and serves as the justification for the present study. The interference of the halo effect in peer reviewing, even sporadically, interfering in the publication of few articles, can be very harmful to the image, reputation and credibility of the journal. The work of many decades and hundreds of articles may be called into question if the community perceives that articles are published that do not respect the canons of scientific research to serve the interests of a certain professional group.

Halo Effect in article reviews
Studies on double-blind peer review have grown for three reasons: a) the perception of their importance to the production, evaluation, and consumption of science; b) the results of the studies remain inconclusive; and c) because they are a source of multiple criticisms that have encouraged debate among researchers (SABAJ MERUANE; GONZÁLEZ VERGARA; PINA-STRANGER, 2016). One of the aspects that have come under debate is the elimination of bias and the drive for impartiality in the review process. "Impartiality is a recognized source of procedural justice in meritocratic systems because it demonstrates that decision-making processes are objective and unbiased" (CLAIR, 2015, p. 162). As the peer review is a decision-making process, it falls to journal editors and managers to ensure impartiality in the process, combatting the various forms of bias that emerge in double-blind peer review.
Among the most discussed and accepted biases on the part of academia in the double-blind peer review are those associated with the halo effect and the horn effect during the editor review (PETERS; CECI, 1982). The halo effect is a cognitive decision-making bias, defined as an aspect of the individual or entity under analysis, perceived by the reviewer, as something positive that can have a positive effect on the judgment of all the other important aspects to be analyzed, thus distorting the result of the decision. The horn effect is similar, but involves an aspect perceived by the reviewer as something negative, having a negative impact on the result. Thus, the halo effect can aid an article with little scientific merit due to the authorship being associated with a prestigious author or institution, while the horn effect can harm the chances of an article that is scientifically valuable because of the authorship being associated with an unknown author or institution (SMITH, 2006). As the metadata of the authors is required upon submission, the influence of the author's name or affiliation affects only the editors or the editor review phase.
Several other forms of bias in the double-blind peer review of articles have been discussed in academic circles in recent decades. Lee, Sugimoto, Zhang and Cronin (2013) developed a taxonomy of these biases:

Category
"Content-Based Bias", broken down into: "Confirmation bias", reviewer bias against manuscripts that describe results that are not consistent with his theoretical perspective (JELICIC; MERCKELBACH, 2002); "Conservatism bias", bias resulting from the conservatism of the reviewer who tends to be against groundbreaking and innovative research (BRABEN; 2004); "Publication bias", bias associated with the tendency for journals to publish research demonstrating a positive rather than negative outcome (BARDY; 1998); and "Bias against interdisciplinary research", bias associated with the understanding that disciplinary reviewers prefer mainstream research (TRAVIS; COLLINS, 1991); Category "Bias as Deviation from 'True Quality' Value"showing that reviewers can fail to interpret and apply evaluative criteria in consistent ways (BLACKBURN;HAKAL, 2006). Broken down into: "Bias as low inter-rater reliability", agreement between reviewers is very low, the inter-rater reliability for two reviewers on a single submission should be about 0.8 − 0.9 (MARSH; JAYASINGHE; BOND., 2008); and "Bias as deviation from proxy measures for true quality", encompassing studies that seek to assess the construct validity of peer review as a test/process by comparing its outcomes to proxy measures for manuscript quality, e.g., studies have investigated the citation patterns of "rejectedthen-published-elsewhere" articles (BORNMANN; DANIEL, 2008); Category "Bias as a Function of Author Characteristics", broken down into prestige, affiliation, nationality, language and gender; Category: "Bias as a Function of Reviewer Characteristics", the authors do not divide the content into subcategories, but cite possible biases regarding gender, nationality and disciplinary affiliation, e.g., philosophers or psychologists (LEE; SCHUNN, 2011).
In this study, a new subcategory of bias associated with "Bias as a Function of Reviewer Characteristics" is presented and discussed: the professional group to which reviewers belong. The bias associated with the characteristics of a group, in the case of this study the group of reviewers, is based on two theories: Social Identity Theory (SIT) and Selfcategorization Theory (SCT). Social identification is the central element of SIT and refers to the degree to which an individual perceives himself as a member of a social group or category (TAJFEl, 1981). According to SIT, too strong a social identification can have a considerable impact on actions and beliefs, which may provoke favoritism towards those within the group and derogation towards those outside the group (TAJFEL, 1982). Similarly, SCT states that positive feelings and attitudes toward a group are likely to manifest when the social categorization process is activated (TURNER, 1999). This study discusses the possibility that the categorization process of a group of reviewers of a journal, composed predominantly by public servants, is activated when analyzing the content of articles directly associated with topics of interest of this group.
To determine whether an article might have been unduly favored during the peer review process, we needed to verify whether the content most attractive to the group of reviewers, in other words, the demands associated with the group to which they belong, are duly advocated in accordance with scientific research canons. "Scientific canons include significance, theory-observation compatibility, generalizability, consistency, reproducibility, precision, and verification." (CORBIN; STRAUSS, 1990, p. 4). In this study, we will use two canons as analysis units for the articles in the sample, generalizability and consistency. Definitions of these two canons are given below: generalizability is "partly achieved through a process of abstraction that takes place over the entire course of the research. The more abstract the concepts, especially the core category, the wider the theory's applicability" (CORBIN; STRAUSS, 1990, p. 15); "consistency in data collection, the investigator should watch for indications of all important concepts in every observation -ones carried over from previous analyses as well as ones that emerge in the situation. All of the observations would be qualified by noting the conditions under which the phenomena occur, the action/interactional form they take, the consequences that result, and so forth. Careful noting of qualifiers gives specificity to concepts" (CORBIN; STRAUSS, 1990, p. 9).

Research method
As it is difficult to find journals combining all the characteristics of the context of the environment necessary for the occurrence of this possible event, as described in the introduction, we will work with the single case study as a research strategy, a journal, and with the identification of an intentional sample of extreme cases (YIN 2003). Extreme cases are understood as articles in a context with great potential for conflict of interest due to the direct interests of the author and reviewers with regard to the object of study. In this subsection, we will first describe the criteria that characterize the selected journal within the context of the environment and the criteria used to identify articles with great potential for conflict of interest. We will then describe the dimensions (registration units) used in the content analysis technique to verify possible violations of canons associated with good scientific research practices in messages supporting the demands of the group to which both the authors and reviewers belong.

Case selection: the thematic field and journals chosen for analysis
Selection of the thematic field of the articles. To analyze the halo effect in question, we had to identify a thematic field whose journal have a high probability of homogeneity, with the authors and reviewers belonging to a common professional group. The purpose was to analyze articles with a greater potential for conflicts of interest and higher risks of biased research due to the halo effect associated with the interests of the professional group of both the authors and reviewers. Therefore, we opted to analyze journal in the field of Public Administration, as their authors and reviewers are concentrated in public research and teaching institutions. In other words, most of the researchers and authors in this field are public servants. Another advantage of this field is the possibility of easily identifying themes that are prone to conflicts of interest. Every article on the subtheme of people management in public organizations has this potential. This subtheme involves discussing subjects such as wages and salaries, benefits, pensions, quality of life at work, training, evaluation of performance, laws that outline the group's rights and duties and other themes of interest to public servants.

Selection of journals.
Due to the need for a journal with language restrictions, we selected a Brazilian journal in the field of public administration, which has been publishing articles since 1967. Brazil has 36 institutions that offer Master's or Doctorate Degree courses in Public Administration. Of these, 32 are in the public sector (Registration Data, 2016). The predominance of texts in Portuguese (92.5%) and Brazilian reviewers (93.5%) is shown in Appendix A, with a chart showing the numbers regarding the language of the published articles and the origins of the authors and reviewers. To demonstrate the heavy concentration of authors and reviewers in the regional journal, using a language other than English, we also show in Appendix A the tabulation of these data, for the same period, from one of the most prominent international journals in the field, the Public Administration Review. From here on, we will identify the articles of Brazilian journal as BR-J (acronym for Brazilian journal) and the articles of Public Administration Review as PAR-J (acronym for Public Administration Review journal).
Appendix A shows that the vast majority of articles (92.5%) published in the BR-J are in Portuguese, the official language of Brazil, and most reviewers and authors are associated with Brazilian institutions, 98.9% and 93.5%, respectively. Of the 443 reviewers that analyzed articles in the period in question, only 5 did not belong to Brazilian institutions. In the case of the PAR-J, there is a better distribution in terms of the country of origin of the first authors of the articles and the reviewers. The USA is the country with the highest concentration for the PAR-J: 64.7% of authors and 74.2% of reviewers. Using principles of probability, we can state that of the 201 articles published by the Brazilian Journal from 2013 to 2015, there is a 92.47% (0.935*0.989) chance of an article having been written by a first author with links to a Brazilian institution and of this article being evaluated by a reviewer who is also associated with a Brazilian institution. The chance of an overlap between a group of reviewers and authors from the same country in the case of the PAR-J is 48.01% (0.647*0.742), considering the data of authors and reviewers from the USA. Applying the two-binomial proportion test, these differences are statistically significant, at the level of significance of 0.05, for the difference in concentration of country associated with the institution of the first author, 64.7% American for the PAR-J and 93.5% Brazilian for the BR-J, and also for the concentration of country associated with the institution of the reviewer, 74.2% American for the PAR-J and 98.9% Brazilian for the BR-J.
The concentration of the BR-J's reviewers linked to the same group is characterized in Table 1, which shows the origin of the reviewers identified in the annual lists of reviewers (nominees) of the articles published in the specialized journal between 2010 and 2015. As shown, 64% of the reviewers are public servants. Of the other reviewers, we should highlight those connected to foundations that specialize in the management of public companies. These foundations provide consultancy services to public agencies, often without requiring a bidding process due to their competence in the sector. Therefore, this group is also naturally motivated to be more sensitive to messages associated with the problems and needs of public servants. Considering both groups of reviewers subject to the bias of perception provoked by the halo effect, public servants and employees of foundations that specialize in public administration account for 81.5% of the total number of the reviewers of journal from 2010 to 2015, and prone to the halo effect. Fonte: dados da pesquisa.

Identification of articles with a high potential for conflicts of interest
In February 2016, a search was conducted on the website of both journals and in repositories that publish the journals. As the PAR-J and BR-J have been publishing articles since the 1940s and 1960s, respectively, some articles are digitally stored as images. Therefore, the search mechanisms were replicated in repositories that divulge the journals' articles in order to ensure a more accurate and wide-ranging search. In the first stage of identification, articles on the theme of people management were sought. For this purpose, a search was made in the strings of texts associated with the Title and Abstract metadata. At least one of these had to include one of the following terms: "public servant(s)", "public employee(s)", for the PAR-J; "servidor(es) público(s)", "funcionário(s) público(s)" for the BR-J. The abstracts of the articles resulting from this first activity were read in order to gauge their suitability for the study with some of the themes of people management. In the second part of the process of identifying articles, it was necessary to determine whether at least one of the authors was a public servant operating in the same context of the public sphere (local, state or federal) involved in the analyses of the article. For this analysis of whether the authors were connected to public agencies, the content of two metadata of all the authors was analyzed: name of the institution and the supplied e-mail address. In the third stage of the article identification process, the content analysis technique was used to determine whether the texts of the article contained any demands for the public servant community. Table 2 shows the number of articles that resulted from each of the three stages of the article identification process. Thirteen articles with a potential conflict of interests were identified, with 12 in the BR-J and 1 from the PAR-J.
1ª) Title or Abstract: containing the term "public servant(s)" or "public employee(s)" for the PAR-J, "servidor(es) público(s)" or "funcionário(s) público(s)" for the BR-J; in addition to one of the themes of people management 26 32 2ª) Authors: at least one must be a public servant working in the same context of the public sphere (local, state or federal) involved in the analyses of the article 14 16 3ª) Content: presenting a demand for public servants in the context debated in the article [a] Research conducted on the journal website and in the SciELO repository [b] Research conducted on the journal website and in the EBSCO repository [c] References of the articles described in Appendix B [d] References of the articles described in Appendix C Fonte: dados da pesquisa.

Content analysis procedures and techniques for the articles in question
Content analysis (MILES; HUBERMAN 1994) was the main technique applied in this study. In the process of identifying articles with potential conflicts of interest, it was used to identify articles that presented an explicit demand in favor of the community of public servants addressed in the study. For this purpose, two dimensions were used: report and demand. A report includes texts that describe a problem or potential problem. Texts associated with a demand present the action recommended by the authors of the article in the sample to resolve the problem or avoid the occurrence of a problem identified in the report. The results of this stage of the selection process of articles with a potential conflict of interest are shown in Figure 1. Each article has two lines of description, the first presenting the texts of the report and the second with the texts of the demand. In the last column of Figure 1, we present an interpretative summary, with a more direct version of the essence of the texts of the articles in terms of Report and Demand.
To determine whether the demands identified in the articles respected the canons of scientific research, the content analysis technique was then applied, working with two types of dimensions: generalizability and consistency. Thus, the aim was to gauge: a) whether the specific issues of a public agency or public location were not being improperly generalized for a wider range of public servants, in other words, respect for the canon of generalizability; b) whether the data collected and analyzed hold up the main claims that support the demand, in other words, respect for the canon of consistency. The logical foundations for defining these aspects to be analyzed were respectively: a) the wider the scope of public servants that benefit from the demand, the greater the possibility of the reviewers feeling that they are included in the demand and will thus benefit from the demands made in the article; b) favorable demands are always likely to be appreciated by those set to benefit from them, and the anxiety to achieve the demand to benefit their own group will lead authors to write messages making demands that do not necessarily reflect the results of their fieldwork. . However, when the questions are about compensation and the possibility of growth, private employees report significantly higher levels of job satisfaction (see also Rainey, 1983;Buelens and Broeck, 2007;Karl and Sutton, 1998

A5
Public Servant Report (p. 53): the data show that most public servants are quite satisfied, with job stability, followed by social integration, supervision, social relevance and of their work and working conditions (health and safety). On the other hand, most respondents are dissatisfied with the reward system and their chances of professional growth. The state reward system (salary and benefits) and the career plan are recurring sources of dissatisfaction in among public servants.
Public servants are not satisfied with their rewards and career plans.
Demand (p. 53): therefore, it is the job of public administration in general to attempt to modernize the management to offer more attractive career plans and reward schemes.
Demands a more attractive salary and career plan.

A6
Public Servant Report (p. 62): Public opinion in general has almost always focused on the fact that public servants are technically unprepared and do not show enough professionalism when it comes to the range of "perks" and privileges and especially the "excessive" amount of workers, who are the people most responsible for a series of woes in public service, which especially include the high public payroll. It should be registered in this busy plan reported by the media, people's common sense, party ideology, ideological conditions in general, all the negative image accumulated by public servants in the public eye, the bare face of a complicated process of "delegitimization" of the state, whose shenanigans seem to have a long way to go. Meanwhile, among public servants, the focus is split in the less visible spaces of the public apparatus, occupied by the great mass of unseen workers there is skepticism and dissatisfaction, with a clear and uncontested predominance of the issue of salaries. Here there is a climate of a lack of commitment with the State, of despair and ideological confusion, of alienation regarding substantive issues and legal devices in general. On the other hand, in the spaces where it is easier to project and make their voices heard, marked by the operations of trade union vanguards, an unquenchable spirit of corporation rules, directed to trivialize the understanding of everything, to condition the analytical perspective towards explaining an endless agenda of demands and reduce each and every discussion to the dimension of rights, with the consequent reduction of weight relative to obligations, responsibilities and effective possibilities. One way or another, human resources ends up being pushed into the background, and the stage is occupied by an active block of attitudes that are against change. Among government technicians and researchers, emphasis has shifted once again, this time to the inexistence, in the Brazilian state, of an effective human resources policy that is capable of being Public servants with improper stance and mentality.
something more than a mere "logistic" of ... Demand (p. 80): A consistent managerial training program needs to be designed and put into practice with extreme urgency to update the theoretical and political agenda of public servants, diffuse new managerial modalities and new organizational technologies, make administrative action more flexible and take it out of its routine and give it a boost in terms of a culture that is superior to specialized technical consciousness.
Demands a managerial training program.

A7
Public Servant Report (p. 1189): In general, the quality of life of the interviewees is at a satisfactory level. A joint analysis shows that the reality is stressful… Public servants with stress and low levels of quality of life due to the productive restructuring promoted by the government.
Demand (p. 1189): ... the structuring of a broader range of people management policies and practices is required to identify, demystify and monitor occupational strength among company managers, with through research with more specific diagnose.
(p. 1208): In light of the results of this study, it is suggested that the company should structure and develop people management policies and practices to minimize the bad effects caused by "management shock" in individuals. This is the case for managers and non-managers and encompasses actions mainly based the treatment and development of staff, functional evaluation (performance, results and potential), and jobs and careers.
Demands the adoption of people management policies and practices to identify and reduce stress. (p. 702): according to the interviewees, the influence of internal and external policies has negative effects on the perception of individuals regarding their relationship with the organization, affecting their psychological contract, since it reduces their expectations regarding recognition of their professional merit. This negative view is further strengthened by the feeling that some changes in the structure are used only as a pretext to create opportunities for "close acquaintances". Another important factor is that in some of the change the company opted to hire external consultants rather than create opportunities for its own employees.
Public servants with problems in terms of performance and self-esteem due to loss of identity caused by the restructuring of their public company.
Demand (p. 702): ... to aid the development of future studies that propose to observe change processes in the context of public organizations, and provide managers of these organizations with the opportunity to handle the inherent difficulties of these processes better ... Public servants with fewer rights than other workers in the private sector.

Demands
Demand (p. 115): The times of democratic openness will have to be seized as an opportunity to define (or redefine) the rights of groups that have been overlooked on the national scenario for a long time, such as public servants.
Demands review of public servants' rights. (p. 41): ... although merit ("professional performance"and technical qualifications ("professional training") are aspired and incorporated values, the opinion prevails that in practice, those who have political patronage end up being Public servants demotivated due to dominant culture of patronage, which destroys the good managerial proposals of a meritocracy. more successful. Although public servant have universalist attitudes and aspirations, this does not correspond to the belief that the rules of the game are the same for all, which probably has negative implications from the viewpoint of training in an ethos of involvement and commitment to the public thing.
(p. 45): Although people join the public sector with beliefs and aspirations of performance and professional and social recognition, the macro-historical dimension of social processes is configured as a more wide-ranging structure that produces certain relationship patterns that escape individual control.
Demand (p. 46): The success of generalist proposals should not only be based on the public servants who sign up for them, which depends on the filtering of shared beliefs and meanings. It should also be because one of the reforming goals is to make them produce new meanings implying another perception concerning rights, duties, policy and the whole public thing in Brazil. But this certainly cannot forgo a policy of valuing public servants, which will depend on political behavior and the direction that the reform of the State will eventually take.
Demands a policy of valuing public servants that includes meritocracy and the end of political patronage.

A12
Public Servant Report (p. 8): It is hoped that new managerial stances will emerge, a public and non-egocentric stance of interpreting and not being a "know-all", a stance that represents the organization and concerns over the participation of public servants, in a process of organizational democratization through democratic behavior at the levels of human relationships within working organizations.

Findings
We begin this section by presenting the findings of the content analysis of the 13 articles in the sample with regard to two scientific research canons: a) consistency in communication of the "public servant report" and communicating the "demand" in terms of the collection or availability of information to support these declarations; and b) the generalizability of the demand in relation to the collectivity studied in each of the articles.

Generalizability of the demand
The more generalizable the context of the demand in terms of public servants who will benefit from it, the more interesting and attractive the article will be to the whole public servant community, including the reviewers and authors that are members of this professional group. This situation is more likely to attract the interest of reviewers and, consequently, manifest the halo effect. It is also the most laborious and arduous task for the researchers. A scientifically integral item of information in this broad context would require many efforts for collecting information from the three public spheres (federal, state and municipal) and different segments in which public servants are employed (public safety, education, health, transport, etc.). Non-compliance with this requirement could create two situations of risk associated with the quality of the study, with penalties for the generalizability canon of scientific knowledge: Risk A -Excessive generalizability from limited samples: the authors generalize and include all public servants of the country in question, despite the restrictions on data collection in terms of the sphere and/or segment of the public employees in the sample. In this situation, findings are overestimated and the results achieved configure a loss of focus and the aims of the study; Risk B -Idiosyncratic findings due to limited samples: discussions are very restricted, e.g., involve only one public company, which makes the results largely unattractive to most readers of the journal and diminishes the scientific importance because of their non-generalizability.
The content analysis technique was applied to the 13 texts in the sample to verify the occurrence of the two situations of risk. For this purpose, the units of analysis were defined as "public institution" analyzed, the "operational segment" of the institution and its "level within the public sphere". The values observed for these units of analysis are described, respectively, in the third, fourth and fifth columns of Figure 2. Another important aspect was the term used to highlight the group associated with the demand stated in each article (as indicated in the underlined excerpts of the transcript in Figure  1). This was configured as another unit of analysis, shown in the sixth column of Figure 2, "term used for demand".  [1] Term identified and underlined in the line of Demand of each of the articles described in Figure 1 Fonte: dados da pesquisa.
Of the 12 articles of BR-J, 10 showed problems related to one of the two risks associated with the generalizability canon. Risk A was identified in 8 articles (A1, A2, A3, A5, A8, A9, A11, and A12) and Risk B in 2 articles (A4 and A7). The two articles that showed no problems in terms of generalizability were A6 and A10. For the only PAR-J article in the sample (B1) we did not find problems regarding the risks associated with the generalizability canon.
As shown in the second column of Figure 2, the most used research strategy in the thirteen articles in the sample was Case Study Research. This strategy has several purposes that are useful to science. However, it is not recommended when the purpose is the generalizability of theories, axioms and other levels of abstractions, as is the case in articles that present demands. The risk of idiosyncrasy associated with the application of the case study research strategy is highlighted by Eisenhardt (1989, p.547): Another weakness is that building theory from cases may result in narrow and idiosyncratic theory. Case study theory building is a bottom up approach such that the specifics of data produce the generalizations of theory. The risks are that the theory describes a very idiosyncratic phenomenon or that the theorist is unable to raise the level of generality of the theory.

Consistency of the report and demand
The consistency canon became a theme for the content analysis of the thirteen articles in the sample. The origin of the argumentative logic employed by the authors to issue their report to public servants and present their demand was observed. The analysis unit "method" was created, to which the following values were assigned:

Referenced -if based on texts from other authors or sources;
Implied -from the analysis of the data collected in the field; Opinion -if only the opinion of the authors is given without being based on field data or external references. Figure 3 shows the classification attributed to the method employed to draft the report and demand. The two columns headed "Analysis of evidence in the text" (third and fifth columns) contain excerpts from the texts of the article, with commentary and analysis, explaining the assigned classification, respectively, to the Report and Demand. The most critical situation in terms of the consistency of the statement is when there is no reflection of the literature and when there is no basis in the field data, in other words, it is merely an opinion. For the Report unit of analysis, the classification "Opinion" was assigned to 3 articles (A4, A6 and A8). For the Demand unit of analysis, the classification "Opinion" was assigned to 12 articles of BR-J.
In a cross analysis of the 13 articles in the sample regarding the two canons (generalizability and consistency), problems related to both canons were identified in 10 articles. Articles A6 and A10 had problems restricted to the consistency canon, A6 in relation to the texts of the Report and Demand, and A10 only in relation to the Demand. In short, all the articles of the BR-J sample had problems in relation to the texts of the Report and Demand. The reviewers should have recommended the exclusion of these texts or greater detail and depth of the studies to support their claims. On the other hand, no problems were detected in the article from the PAR-J. Drafting of the Report Preparation of Demand Nature of Demand Method [1] Analysis of evidence in the text Method [1] Analysis of evidence in the text

A1
Implied and Referenced The authors tested hypotheses ("In order to answer the research question and test the hypotheses, we conducted a survey research", (p. 1484)). They also cited literature that supported the report ("... display higher levels of work satisfaction than private employees

Opinion
Despite proving the differences in terms of satisfaction between public and private sector workers, there is no test or analysis to indicate that the competitive practices of private organizations are responsible for their employees being less satisfied with their working environment and job security, i.e., the demand is not supported by the study.
Preventive A2 Implied Observed by the authors in the field: "The dominating climate during the reports is that of an outburst, with a feeling of impunity, as they think that nothing will happen to the person guilty of harassment" (p. 1080).

Opinion
The demand is a proposal from the authors to attempt to curb the problem highlighted in their study.
The study does not prove that the laws and practices proposed by the authors will curb the problem in question. The demand is not the central object of the study and does not characterize the scientific knowledge created within. The demand is merely a proposal by the authors inspired by the problem stated in the research.
Corrective A3 Implied Section "4. Forms of provision and indicators of the use of services" (p. 396)) tabulates the field data that support the implication by the authors that the services available do not meet their demand.
Opinion "Differentiated medical assistance as an integral part of an active human resources policy" is a suggestion by the authors to address the problem of insufficient services to meet demand.

A4 Opinion
The characterization of the knowledge of public servants, in some situations, as "limited" (p. 57) is not supported by references to the literature or even the field data used

Opinion
The authors suggest legislation and policies to improve the knowledge of public servants. The recommended agencies are suggested only because Corrective in the study. they are not identified at the institution in question: "At the time of writing, there was no policy formally put in place for HR, according to information obtained from the human resources director", (p. 58).

A5
Implied Section "4.1 Descriptive analysis of the data" (p. 49)) tabulates the field data in support of the implication of the authors regarding the report in question: employees dissatisfied with the reward system and possible professional growth

Opinion
The proposal to offer more attractive rewards and career plans is only the opinion of the authors. This is not reflected in the analyses, comparisons or any other form of reasoning that might be used to justify this statement.

A6 Opinion
Despite the wide range of adjectives and texts associated with the report prepared by the author in the introduction to the article (p. 62), no quotes are provided in the two pages of this section, nor is there any mention of field data.

Opinion
As stated in the title, the article deals with "notes on State reform and human resources". There is no declared method or even data collection. The article merely reports the opinions of the author on the theme in question.

A7 Implied
In Section "4. Presentation and analysis of the data" (p. 1201), it can be observed that the report was implied from the field data. It should be highlighted that the data were obtained from interviews only with managers who had been identified as stressful, i.e., there was no triangulation with data from other sources.

Opinion
The authors state in their text that they are addressing a pragmatic suggestion: "in light of the results of the study, it is suggested that the company should structure and develop managerial policies and practices for people focused on..." (p. 1208).

A8 Opinion
The authors state that their report is an opinion; in this case, not of the authors but of the public servants who were interviewed: "In the opinion of the interviewees, the influence of internal and external policies has a negative effect … affecting their psychological contract, since ..." (p. 702).

Opinion
The authors indicate that this is something to be developed in future studies: "and aid the development of future research that proposes to observe change processes in the context of public organizations and provide mangers with a chance to help them deal with the inherent difficulties of Corrective these processes" (p. 702).

A9 Implied
In Section "2.4 Information on public servants" the authors present the official Brazilian government data used in their calculation: "The results obtained for the base scenario suggest that for the average public servant, change is unfavorable, as this would mean a reduction of 17% (men) and 15% (women)", (p. 1501).

Opinion
The authors state in the abstract that "These results illustrate the need for measures to increase the level of information of public servants" (p. 1479). At no time does the study address the knowledge level of public servants regarding the new social security regime and its impacts.

A10 Implied
The report is developed from an analysis of the texts of laws and versions of the Brazilian constitution: "Nevertheless, the Constitution of 1937 that addresses workers in the country for the first time as ... From the Constitution of 1946, workers received the freedom to join a trade union and go on strike, ..." (p. 110)).

Opinion
The use of the future tense indicates that this is a suggestion, something to be achieved: "The days of democratic openness will have to be seen as an opportunity to define (or redefine?) the rights of certain groups that have been ignored for a long time on the national scene, such as public servants" (p. 115).

A11 Implied
In Section "4. Representations, values and expectations among public healthcare servants" (p. 38), it can be seen that the report was implied from the field data. The data were obtained through interviews only with public servants, i.e., the data were not triangulated with data from other sources.

Opinion
The incorporation of policies for valuing public servants is a proposition of the authors. It is not reflected in analysis, comparison or any other form of logical reasoning that shows that this statement was the result of the research in question.

Implied and Referenced
The authors based the report on field data presented in Section "3. Analysis and treatment of data" (p. 6), and cited research that supports their report: "This leads us to infer that the type of predominant leadership is in keeping with the theory of Schon, ..." (p. 7).

Opinion
The proposal for a system of networks in which the largest number of members of an organization can take part in the decision-making process is a proposal by the authors. It is not reflected in analysis, comparisons or any other form of logical reasoning that shows that this statement was a result of the research in question.

Further analyses of the demands identified in the articles
The 13 articles identified as having a potential conflict of interest were submitted to the second stage of the content analysis: Repackaging and aggregating the data (MILES;HUBERMAN 1994). In accordance with the precepts of content analysis, during the content analysis, all 13 articles from the sample were read and analyzed by more than one researcher. During the reading, an effort was made to "identify themes and trends in the data overall" and "find out where the emphases and gaps in the data are" (MILES; HUBERMAN 1994, p. 92). For this purpose, during the reading, the researchers introduced labels identifying excerpts of texts associated with emphases: characteristics of communication that emphasize and highlight the demand associated with the interests and concerns of the public servants, making it more attractive to this community, including the reviewers. The analysis and consolidation of the set of excerpts of texts marked with labels for emphasis, led us to find two more analysis dimensions: "Diffusion and use of the demand throughout the article", and "Discussion centered on public servants".

Diffusion and use of the demand throughout the article
To gauge the level of importance and emphasis given by the authors to the Demand, as well as the probability of the reviewers having noted this content, we conducted analyses to locate the Demand within the structure of the text. Figure 4 shows the location of the texts in the article in which the authors present their demands. In three articles, the demand is already highlighted in the abstract. This is the case of articles A4, A7 and A9. Three articles state the demand in more than one section of the text (A1, A4 and A7). Considering that there are two articles common to both these groups, it is shown that four articles used at least one of the resources to highlight and emphasize their demands.
[1] Despite being called "Analyses", this refers to the final section of the article, similar to those that declared the demand in the Conclusions Fonte: dados da pesquisa Highlighting the demand, including declaring it in the abstract, would be justified, for instance, in a participant study in which demands were made as this would be the major objective of the study. This is not the case of the articles in the sample, as shown in the second column of Figure 2, which indicates the type of research strategy. Highlighting and emphasizing the demand may be considered a means of ensuring the perception of the message and, consequently, increasing the chances of sensitizing the reviewers the occurrence of the halo effect when they analyze the article.

Discussion centered on public servants
"The primary role of the public servant is to help citizens articulate and meet their shared interests" (DENHARDT; DENHARDT 2000, p.549). Articles with discussions that focus exclusively on the interests of public servants within the context of people management in public organizations have another characteristic resulting from this bias. They discuss only the rights of public servants in a way that is not associated with their duties. In practice, this means discussing aspects of people management independent of their objectives and goals. In the case of public servants, this discussion should include the beneficiaries of their actions: society and citizens.
One way of identifying egocentric articles, i.e., articles focused on public servants, was to seek articles that do not include the terms "citizen" and the term "client". For this purpose, the Word Cruncher function of Atlas.Ti qualitative analysis support was used to count the frequency of all the words in each of the thirteen articles. This was followed by a reflective reading of the articles to gauge the context of the use of the terms citizen and client, discarding the occurrence of words out of context. For a better understanding of the discourse that was strongly focused on public servants, we included in Figure 5 the total for the word "servant" in the article. Figure 5 shows the final result of these actions and indicates that seven of the articles in the sample have egocentric characteristics.   Figure 6 summarizes the results of the seven aspects analyzed for the 13 articles in the sample. The most critical dimensions in terms of characterizing biased analysis are those associated with the canons of good practices in scientific research, in other words, the four items associated with the two topics of "generalizability of demand" and the two topics of "consistency of the report and demand". All of the 12 articles of BR-J violate one of the four aspects associated with these two topics. When all the other aspects are considered, focusing on the contextualization of the message to the interests of public servants, five articles of BR-J sample make use of four or more of the resources analyzed. The only one article of PAR-J making demands in favor of public servants did not violate the canons associated with good scientific research practices.

Discussion
The results of the analyses indicate that the halo effect could occur in a peer review in the context of language restrictions as observed in BR-J. The high probability of texts in the BR-J written by authors associated with Brazilian public institutions also being evaluated by reviewers associated with Brazilian public institutions is explained by the language restriction of the articles. We can say nothing about the horn effect because we did not have access to the articles on public servant management rejected by the journals. Thus, what we can highlight is the possible occurrence of bias, in other words, a moderate risk of articles being favored in peer reviews due to the feeling of belonging in the group of reviewers. This possibility implies a need for precautions in terms of the accountability and quality assurance in the peer review process of their articles. Issues also arise here regarding the authors, the submission of articles and the reviewers concerning the acceptance of articles for evaluation, possible personal conflicts of interest due to the theme addressed in the article and the professional career of the reviewer. When no reviewers are available from other professional fields other than the one addressed in or related to the article, the journal should expand its team of reviewers. If it is not possible to do this with professionals from other fields, the subtheme should be excluded from the themes addressed by the journal.
Regional journal editors with limitations in terms of language and devoted to a very specific theme, as in the case of the present study, are exposed to many difficulties. To help overcome these problems, researchers should reflect on how important it would be to the evolution of academia and science in countries with less scientific tradition to aid the evaluation process of these peripheral journals. It is necessary to sensitize researchers to commit themselves not only to major journals that confer prestige and status, but also to cooperate with journals with less prestige and recognition. The creation and valuing of social performance indicators for researchers, encompassing these collaborative actions, could be the way to sensitize and act to meet this demand.
In terms of supraeditorial oversight, associated with the metadata of articles from the journals and the repositories of journals often used as sources of information for studies in the field of epistemology, library science, scientometrics and information science, it would be important to add one more category of metadata in addition to language of publication: the language of evaluation. This is relevant because many local and national journals allow peer analysis of articles written in the local language despite the fact that they will later be published in English. This mechanism is used by these journals to increase their readership and to ensure that investments are made in translation only for authors whose texts are approved following peer review. This practice was adopted recently by a number of journals in Brazil and other countries that do not use English as an official language. With all articles published in English, it is more difficult to perceive bias associated with the restrictions of less widespread languages, as observed in this study. This accounts for the importance of registering the metadata protocols of articles, such as the Dublin Core Metadata Initiative (2009), which adds another term before "language", resulting, for instance, in "publication language" and "peerreview language".
Limitations and continuity of the study. This study did not analyze the bias of homogeneity in the group of researchers regarding the analysis of texts that carry messages contrary to the concerns demands of the group. In the theoretical perspective, the horn effect in the context in question was not considered, i.e., the rejection of quality scientific articles because they include information that is contrary to the interests of the group of reviewers. This type of bias in the double-blind peer review of articles is as important and critical as the context analyzed in this study. The result of the halo effect is a tendency to publish articles that violate scientific precepts. The result of the horn effect is that, because of the homogeneity of the reviewers regarding a characteristic of the group, there is a risk that important scientific knowledge will not be published in order to preserve the interests of the group. These two possible biases on the part of reviewers of scientific articles, due to group interests, are portrayed in Figure 7: Type 1, bias due to the halo effect, addressed in the present study; Type 2, bias due to the horn effect, not addressed in this study. Thus, the Type 2 bias is an opportunity for continuing the present research project.