The myth of meritocracy: academicism and methodological flaws in Brazilian public recruitment

El mito de la meritocracia: academicismo y fallas metodológicas en las licitaciones públicas brasileñas

Abstract

Is there meritocracy when 58% of administrative assistants and 87% of physicians employed in Brazilian federal higher education institutions (HEI) were selected through public exams based solely on theoretical multiple-choice tests? Is there meritocracy in selection processes that do not assess the candidates’ experience or apply discursive or practical exams? This article addresses a historically constructed assumption assimilated as a kind of myth in Brazil: the idea that the current model for filling government positions is meritocratic. The extreme objectivity of these public exams reduces the Brazilian tradition of nepotism but presents serious flaws. One is academicism, which overvalues educational titles and theoretical knowledge at the expense of essential skills for many government positions. Calls for civil service entrance exams to fill administrative and medical positions in federal HEI throughout Brazil were verified using the descriptive statistics method (via frequency analysis), tabulating the evaluation types used in each selection, and generating tables and graphs. The results suggest adopting assessments more broadly connected with the competencies the positions require and prioritizing skills and practical behaviors instead of theoretical knowledge of little or no applicability. The study points out alternatives for a nation with more than 200 million inhabitants, the vast majority of which do not realize the incalculable damage of real academicism and illusory meritocracy.

Keywords:
meritocracy; public exams; public management; people selection; academicism

Resumen

Cuando el 58% de los trabajadores administrativos y el 87% de los médicos fueron seleccionados únicamente sobre la base de pruebas teóricas de opción múltiple, ¿existe la meritocracia? Selecciones sin requisito de experiencia, ni evaluaciones discursivas o prácticas. Solo marque una “x”... Este artículo busca responder a un supuesto construido históricamente y asimilado como una especie de mito en Brasil: la idea de que el actual modelo de licitación pública sería meritocrático. La extrema objetividad de estos concursos tiene el mérito de reducir la tradición brasileña de nepotismo, pero el modelo de evaluación tiene serias fallas. Una de las disfunciones es el academicismo, que sobrevalora los títulos educativos y los conocimientos teóricos en detrimento de competencias que son simplemente imprescindibles para muchos cargos públicos. Las convocatorias a concursos públicos para ocupar cargos administrativos y médicos en instituciones federales de educación superior en todo Brasil fueron verificadas utilizando el método de estadística descriptiva (mediante análisis de frecuencia), tabulando los tipos evaluativos utilizados en cada selección y generando tablas y gráficos relacionados. Con base en los resultados, el estudio sugiere la adopción de evaluaciones más ampliamente conectadas con las competencias de los cargos y que, en lugar de privilegiar conocimientos teóricos de poca o nula aplicabilidad, prioricen habilidades y comportamientos prácticos inherentes a los cargos. Esta investigación señala alternativas para una nación de más de 200 millones de habitantes, la gran mayoría de los cuales no se da cuenta del daño incalculable del academicismo real y una meritocracia ilusoria.

Palabras clave:
meritocracia; licitaciones públicas; gestión pública; selección de personas; academicismo

Resumo

Quando 58% de trabalhadores administrativos e 87% dos médicos foram selecionados apenas com base em provas teóricas de múltipla escolha, existe meritocracia? Seleções sem exigência de experiência, nem avaliações discursivas ou práticas. Basta apenas marcar um “x”... Este artigo busca responder um pressuposto historicamente construído e assimilado como uma espécie de mito no Brasil: a ideia de que o atual modelo de concurso público seria meritocrático. A extrema objetividade dos concursos tem o mérito de reduzir a tradição brasileira de nepotismos e apadrinhamentos, mas o modelo de avaliação apresenta graves falhas. Uma das disfunções é o academicismo, que supervaloriza títulos educacionais e conhecimentos teóricos em detrimento de habilidades simplesmente essenciais de muitos cargos. Foram verificados sob método de estatística descritiva (via análise de frequência) editais de concursos para preenchimento de cargos administrativos e médicos em instituições federais de ensino superior em todo o Brasil, tabulando os tipos avaliativos empregados em cada seleção e gerando tabelas e gráficos conexos. A partir dos resultados, o estudo sugere a adoção de avaliações que se conectem mais amplamente com as competências dos cargos e que, em lugar de privilegiar conhecimentos teóricos de baixa ou nula aplicabilidade, priorizem habilidades e comportamentos práticos inerentes aos cargos. Esta pesquisa aponta alternativas para uma nação com mais de 200 milhões de habitantes, cuja grande maioria não percebe os danos incalculáveis de um academicismo real e de uma meritocracia ilusória.

Palavras-chave:
meritocracia; concursos públicos; gestão pública; seleção de pessoas; academicismo

1. INTRODUCTION: A MYTH BORN OUT OF A VICTORY THAT TOOK HALF A MILLENNIUM TO HAPPEN

If a national public opinion poll were carried out today in Brazil on meritocracy, it is very likely that Brazilian public notices to fill government positions would be singled out as the main example of the triumph of merit. In fact, “a large portion of the population sees public exams as the best option to get a job in the country” (Caldeira & Vilarinho, 2021, p. 4). The model has remained basically the same - and dysfunctional - for decades (Coelho & Menon, 2019) and the address of public exams in the national imagination is honorable, though needing further clarification (Barbosa, 1996a).

But, speaking of merit, what is meritocracy? In this study, one of Barbosa’s concepts (1996a) will be adopted, for whom meritocracy is a “fundamental criterion ... of social ordering, especially with regard to the socioeconomic position of people...”, and its effects is to create “legitimate and desirable hierarchies” based on “selecting the best”.

In this sense, this study aims to map the methodological structure of some Brazilian civil service exams in order to verify if they offer or not conditions to promote meritocracy. For that, a simple frequency analysis was carried out regarding the types of evaluation used in order to verify eventual prevalence, patterns, and/or distortions.

The Brazilian model of civil service exams (of hyperbolic objectivity) results from a historical process focused on qualification in the first half of the 20^th century to the detriment of the competence approach, meaning academicism. It dates back to the reform of the Administrative Department of Public Services (Dasp) in the Vargas Government in the 1930s. As the “earning of this history is highly known” (Coelho & Menon, 2019, p. 169) and is not the focus of this study, such historical construction will not be the object of further detail.

That said, the classic (and fair) argument of “inequality of opportunity” will not be explored here, which would tarnish the idea of meritocracy. The idea is to address less recurrent aspects in the literature such as the evaluation models adopted. In fact, even when there is equality of opportunity, it needs to be methodologically adequate so that equality, when attempted, does not become useless. One example is the public exam for judges. They are among the exams with the greatest refinement and variety of stages, making public exams for surgeons (and many others) seem like playful pastimes of early childhood. Surprisingly, many are open to any citizen, so even those without training in the area can often participate. Even so, public exams for judges show a “limited” assessment of the competencies needed for the position (Passos, 2018). And worst of all, though democratically open to the participation of all, the Brazilian judiciary exams still have a “strong tendency towards the social reproduction of members of an economic elite who seek the exam as an instrument for acquiring symbolic capital” (Passos, 2018, p. 260).

As for the good reputation of public exams, part of this is perhaps due to the fact that the adopting them was a kind of unprecedented victory (J. B. A. Oliveira, 1986) against the personalism, nepotism, and patrimonialism that have dominated Brazil from its birth to the present. By setting up exams for entering public service positions, for the first time in national history after about half a millennium, it would be possible to access public positions based on personal merit without having to resort to the influence of a government official, for example. For those who did have a genuine desire to serve the public and even for those just looking for a job, it is certainly not a trivial victory.

Such victory is enshrined in the 1988 Constitution, Art. 37, item II (Constituição da República Federativa do Brasil de 1988), which defined public exams as the means of accessing a public service position. But human history demonstrates that not every victory has all the value it appears to have. Gaps, excesses, or inaccuracies in guidelines and practices can lead to failures in the result considered victorious, to say the least. One example is that access to public service positions in Brazil remains unregulated since 1988 (Caldeira & Vilarinho, 2021; Fontainha et al., 2014). The pending issue is more than 30 years old and has received fragmented approaches, generally dealing with deadlines, quotas, and bureaucratic procedures, but with little or no attention to the evaluative dimension, which is the obvious heart of a selection worth its salt. As a result, public exams have serious flaws not noticed by the majority of the population and almost undocumented by the national literature, as we will try to demonstrate.

2. LITERATURE REVIEW

The theme of civil service entrance exams is recurrent in Brazilian literature, but in a curious way. The studies bring interesting and varied approaches such as reflections on racism and public exams (Alencar, 2021), study techniques (Maia, 2021), absence of regulations (Caldeira, 2020; Caldeira & Vilarinho, 2021), lack of specific training for certain positions (Prata & Romão, 2019), tourist experience of those traveling to take part in a public exam (Lobato & Alberto, 2019), mismatch between the college degree bibliography and the themes covered in the public exam (Holstein & Rockembach, 2017).

However, the literature points out that the theme of civil service entrance examinations is predominantly linked in Brazilian research to legal approaches (Caldeira & Vilarinho, 2021; Coelho & Menon, 2019; A. B. S. Oliveira, 2017). Less attention is given to other relevant dimensions of public exams, especially the evaluative and managerial ones. However, even with so much legal reflection, some relevant legislative theoretical and empirical distortions keep happening for years on end and almost without academic record. One example is law 11,091/2005 (Lei nº 11.091, de 12 de janeiro de 2005) that prescribes more requirements for admission to dozens of simpler positions, administrative assistant for example, than for more complex positions such as physicians and surgeons.

Meanwhile, the Major Law (Constitution) expressly prescribes the opposite (Constituição da República Federativa do Brasil de 1988, Art. 37, Subsection II) of what occurs in this and innumerable other public exams: “the investiture in a public position or employment depends on prior approval in a public exam ... according to the nature and complexity of the position or employment”. It is not by chance that the judicialization for incidental unconstitutionality is recurrent (ACP 000156635.2012.4.05.8100, Tribunal Regional Federal da 5ª Região [TRF5], 2017; AC 6205420124058200, TRF5, 2014, etc.) or other failures in parts of Law 11,091/2005 (Lei nº 11.091, de 12 de janeiro de 2005), sometimes with the victory going against experience requirement. There are institutions that even discard in their notices some of the abusive requirements, although no one fixes the problem definitively as in this case the problem is not in the notices, but in the law itself. Interestingly, this and many other oddities involving thousands of public service positions are almost unrecorded in the national literature (A. B. S. Oliveira, 2017).

Academic literature and common-sense feed back into public exams and with rare exceptions (A. B. S. Oliveira, 2017; Fontainha et al., 2014) end up prioritizing, whether intentionally or not, less scientific and methodological accuracy of the public exams. Thus, simplism and academicism remain in most exams with a strong social acceptance and also with the tacit or express endorsement of the literature.

The Brazilian context for selecting millions of people for the public sector has been a noble concern for decades, especially toward fighting personalisms, but it overshadows other important concerns. Little or no attention has been paid, for example, to the predictive capacity of selections and to a greater incremental validity of examinations in disagreement with relevant international literature on such subjects (Hunter, Schmidt, & Judiesch, 1990; Joseph, Jinh, Newman, & O’Boyle, 2015; Schmidt & Hunter, 1998, 2015; Schmidt, Oh, & Shaffer, 2016).

Interestingly, while both the international literature (Hunter et al., 1990, for example) and the Brazilian Constitution (Constituição da República Federativa do Brasil de 1988, Art. 37, II) expressly cite the complexity of positions as a factor to be considered, this criterion seems solemnly absent from thousands of public notices published in Brazil, including all the notices analyzed in this study, as will be seen below.

In addition, the international literature also considers other factors ranging from the refinement of self-reported emotional intelligence (Joseph et al., 2015) to the relevance and validity of structured interviews, as mapped by Schmidt and Hunter (1998) with theoretical and empirical data during a period of 85 years, extended to 100 years (Schmidt et al., 2016). Such aspects seem almost unthinkable in the exam model that has dominated Brazil for more than half a century, although a small minority of positions (prosecutors, federal professors, public defenders, etc.) have procedures similar to interviews.

The literature points out another gap in the Brazilian selection model: the strong absence of a minimum notion of “aptitude” or “vocation” (Fontainha et al., 2014), a notion that is extremely valued in some nations with a strong tradition in civil service entrance exams, such as this is the case of France, for example (Fontainha et al., 2014; Desforges et al., 2011).

In Brazil, the literature records that more than half of the candidates (51.21%) are against taking practical tests in public exams (Caldeira & Vilarinho, 2021), suggesting a relevant cultural distortion around what should be a minimally efficient selection process for professionals. It is not clear why engineers, surgeons, diplomats, nurses, and hundreds of other positions would not require passing the minimum scrutiny of a practical test, simulation, or the like.

Furthermore, even among the few exams that have a slightly more refined evaluative structure (higher education teachers, judges, and others), there are relevant dysfunctions. Exams for judges are inadequate and limited (Passos, 2018) and the high judicialization and repeated recommendations of the Federal Public Ministry (Ministério Público Federal [MPF], 2015) for teaching exams to comply with absurdly simple procedures are some of the symptoms little reported by society and literature.

3. METHOD

Due to the characteristics of the phenomenon and the scarcity of empirical studies on the subject, a descriptive and exploratory research design was chosen operationalized through a multi-case study in terms of collection and descriptive statistics (frequency analysis) for treating the data collected.

Due to their social and administrative relevance, two very different positions were chosen for the sample with different attributions, natures, and educational levels in order to allow possible comparisons between them. The positions are physicians/surgeons of all human medicine specialties, which necessarily excludes veterinary medicine, and clerks (administrative assistants, excluding other administrative positions). Only public exams for positions linked to Federal Institutions of Higher Education were considered, which involves an initial universe of 69 federal universities and 38 federal institutes due to the capacity that such institutions would have to carry out richer and more adequate selections. Thus, at least in theory, the possible lack of structure or expertise in preparing the exams is previously ruled out, although institutions without expertise often hire external organizing boards. In addition, several federal universities are among the country’s leading public exam organizing boards, often being hired to be responsible for many of the largest and most important national public exams in various areas. In 2021, during the COVID-19 pandemic, 25 public notices from 22 different institutions were identified with some institutions releasing more than one notice in the year. It should be noted that there are many other notices from various other institutions, but they did not include the positions chosen for this study and therefore are not part of the sample.

Data collection consisted of capturing and analyzing notices for public examinations that offered either of the two positions (physician/surgeon and/or clerk/administrative assistant). By law, notices must be published with free access to anyone. The time frame involved notices published between January 1 and December 31, 2021.

Additionally, the 2021 data were compared to data obtained by the same authors in 2016 when they studied the same positions and evaluation types long before the COVID-19 pandemic, an extremely important time marker for the study.

Five evaluation methods or requirements were analyzed, as described below:

1) Multiple-choice objective theoretical test

Description: written theoretical test where generally a single correct option must be marked among different alternatives (a list of answers with “A”, “B”, “C”, “D”, and “E”, “right” and “wrong” or something similar). It assesses theoretical knowledge through the sum of the number of questions that the candidate got right.

Advantages: allows evaluating large populations with up to millions of candidates at the same time in a few hours with an objective character, reducing the risk of favoritism and personalism.

Limitations: it is easier to be defrauded such as improper transmission of answers during the exam, and it only assesses theoretical aspects without practical, behavioral dimensions, etc. It usually lacks mechanisms against casual hits.

Context: it almost always has an eliminatory and classificatory character. This test model under the multiple-choice objective theory is by far the dominant one in Brazil and is present in almost all civil service entrance exams. It is often the main and even the only type of test used for different positions, regardless of education, nature, complexity, or attributions.

Application recommendation: ideal for leveling minimal and only theoretical knowledge, especially in large populations. It can be used as an initial stage in order to rationalize the number of candidates to be evaluated in later stages. Ideally, it could just be eliminatory and be followed by stages that contemplate practical dimensions and other variables of each position, such as simulations.

2) Discursive test (writing)

Description: theoretical test where the candidate must usually write a text, essay, or similar production of their own authorship on some topic. The size of the essay, the correction criteria, and the topics chosen to vary enormously depending on the position, exam, organizing company, available budget, etc. It evaluates the theoretical knowledge in a more detailed way and the candidate’s argumentative capacity based on the text written by him.

Advantages: it can go far beyond the objective multiple exams when the intention is to assess the candidate’s argumentative capacity and theoretical knowledge. In terms of improper transmission of answers during the exam, it is more difficult to rig this type of test. Casual hits are also often much more difficult.

Limitations: may require more time and a greater budget for proofreading. Less objective than the multiple-choice test, although there may be objective, public, and pre-established criteria for correction. Although “applied” questions can be asked such as case studies and problem situations, it is still only a theoretical assessment.

Context: although there seems to be some recent growth of this type of test in Brazil, it still clearly corresponds to a small minority of civil service entrance exams. Generally, only the discursive tests of those who obtained the best placements in the multiple-choice test are corrected. It usually has an eliminatory and qualifying character. It often scores lower than the multiple-choice test, although it allows for a much more in-depth assessment.

Application recommendation: ideal for evaluating not only argumentative skills, but especially specific and more crucial knowledge of each position. Ideally it would be applied to a smaller number of candidates after a multiple-choice elimination stage, for example. It could be an intermediate stage before evaluations that contemplate practical dimensions and other variables of each position, such as simulations.

3) Practical test

Description: practical assessments usually involve simulation of job activities. Evaluators monitor the execution and assign marks for the skills demonstrated by each candidate (vehicle driving tests, class simulation, burial, translation, etc.). There are cases of practical or simulation tests in which the performance of humans as evaluators is less (typing/formatting test, for example, can be monitored by software for characters typed per minute, characters different from what was requested, etc.).

Advantages: it can go far beyond theoretical assessments and analyze practical aspects of each position, connecting it to skills, competencies, behavioral aspects, attitudes, etc. Even for positions with a higher incidence of theoretical content (teachers, prosecutors, magistrates, diplomats, etc.), the simulation of practical dimensions is much broader and more complete. Depending on its configuration, it can assess from decisive behavioral aspects to the candidate’s theoretical preparation level.

Limitations: it usually requires more time and budget to run the tests and may require more complex logistics and infrastructure in some cases. Less objective than the multiple-choice test, although there may be objective, public, and pre-established correction criteria, in addition to the possibility of audiovisual recording, for example.

Context: it is a type of test applied to a discrete minority of positions in Brazil. Although there are cases without any complications, there are situations that need better organization in terms of regulation, infrastructure, more effective protection against cross nepotism in some careers, etc. Generally, this type of test composes an intermediate stage of the examination process. It usually has an eliminatory and qualifying character. Although it is not an absolute rule, it often has greater weight than the other steps.

Application recommendation: with proper planning, high applicability, and clear and objective evaluation criteria, it is an ideal type of test for all positions, even those with a greater theoretical aspect. Ideal to be applied to a discrete number of candidates after previous steps. It can be the most decisive evaluative element for hiring in some cases.

4) Experience requirements

Description: consists of requiring the candidate to prove previous experience in the activities of the position he intends to occupy via the civil service entrance examination. It intends to grant a greater margin of safety in hiring by proving similar competences. When a legal requirement, it tends to be eliminatory only. When associated with proof of titles, it is usually only qualifying.

Advantages: if the veracity and convergence of the experience is observed, it far exceeds the mere theoretical assessment and may be the only evaluative method capable of eventually overcoming the practical and simulation tests. Along with proof of titles, it tends to be the assessment with the lowest cost among all analyzed in this study.

Limitations: it is the assessment that, in theory, tends to be the most subject to fraud, especially via spurious statements, misrepresentation, etc. Experience in other contexts is relevant and even necessary in many cases, but it can still prove to be insufficient in some situations.

Context: this type of requirement involves few positions in Brazil but generates many distortions. For example, there are mid-level positions that require experience and higher-level positions in the same career that do not. It can be objected that higher education would dispense with the need for experience, but such an argument, in addition to disregarding the idea of complexity, leaves aside a curiosity: absence of regulation of civil service entrance exams, existence of hundreds of careers, and managerial distortions produce disparities in demand of experience even between the same jobs and education. Interestingly, the lack of experience requirement is the only difference between many civil service positions with stability for complex positions compared to selections for the same positions, but without stability (temporary, for example). Several public selections without tenure sometimes evaluate only the curriculum/experience, while many civil service positions with tenure for the same placement sometimes evaluate only via a multiple-choice test. The health area in Brazil is one of those with many examples of this type of distortion.

Application recommendation: with proper planning and clear evaluation criteria, it is a type of requirement that could be decisive in many positions and contexts, reducing significant costs with basic training, for example, and investing in professionals with duly proven skills. Ideally it would be applied for all civil service positions with an eliminatory character for more complex positions and qualifying for less complex positions.

5) Correction for guessing

Description: when the candidate is insecure or simply recognizes that he/she does not know the answer, but answers randomly or artificially to try to obtain a score anyway.

Advantages: N/A.

Limitations: one can question the meritocratic dimension of this practice.

Context: in Brazil there are many examples of content that teach “how to get a question right even without knowing”. Furthermore, only one out of 286 organizers in the country have a more effective and systematized mechanism that inhibits casual hits. Although some companies copy or adapt the method, exams using this type of mechanism are still a tiny minority.

Application recommendation: there should be an effective mechanism against casual hits in all exams.

The mapping of public notices, the central element of this study, followed qualitative and quantitative aspects. Qualitatively, content analysis was carried out (Bardin, 2011), which consisted of a prior reading of all public notices, exploration for establishing categories, identifying synonymy, etc., and categorization for reaching a systematized distribution according to categories of selective test methods as presented in Box 1. Quantitatively, a frequency analysis was performed (Barbetta et al., 2004) to count the absolute incidence of the categories, as well as the relative frequency and distribution of the evaluative type categories among the public notices through the respective percentages. The statistical frequency is systematized and presented especially in graphs 1 to 8.

Data analysis presents the description of the positions and classifies the main evaluation mechanisms and types of each notice (multiple-choice objective tests, discursive tests, practical tests, proof of titles, experience requirements, correction for guessing, etc.). In addition to categorizing the evaluation types (qualitative dimension), the analysis also involved simple frequency (quantitative dimension) or basic statistics of recurrence of each evaluation type and identified convergences or divergences.

4. RESULTS

Are Brazilian civil service entrance exams an effective example of meritocracy or is it possible to be faced with a myth in the worst sense of the word? To what extent would overcoming personalism not have been a Pyrrhic victory with the mere exchange of personalist dysfunctions for other dysfunctions?

The results are presented and discussed below and may help to answer these questions. Box 1 presents a synthetic description of the public positions and evaluation types used in the respective exams (multiple-choice tests, discursive tests, proof of titles, psychological evaluations, practical tests, etc.). The requirement of experience is also considered since at times this is part of the competencies for the positions, and it appears that in some notices this dimension can actually eliminate candidates.

Box 1
Evaluation structure of public notices for exams to fill positions in Federal Institutions of Higher Education in Brazil: Administrative Assistants and Physicians

Box 1
Continuation

Box 1
Continuation

Box 1
Continuation

Box1 presents each public exam, position, institution, and evaluation categories used in each event, but the following graphs can help summarize and better illustrate the contents of Box 1 and the evaluation situation of the public exams.

Graph 1shows that 58% of the exams for Administrative Assistants (AA) are carried out solely through a multiple-choice objective theoretical test without any other type of evaluation of competencies for the position. Graph 2, in turn, shows that the proportion of admissions for doctors based solely on objective theoretical tests of multiple choice is even greater and corresponds to 87% of the public exams.

GRAPH 1
PUBLIC EXAMINATIONS FOR ADMINISTRATION ASSISTANT BASED SOLELY ON MULTIPLE-CHOICE OBJECTIVE THEORETICAL TESTS

Graph 2
Medical examinations based solely on multiple-choice objective theoretical tests

In addition to what is shown in Graphs 1 and 2, which suggests the phenomenon of an objectivist academicism (overvaluation of objective theoretical evaluation) that dominates Brazilian public exams and is little analyzed in the literature, the related results below aggravate the context. Graph 3 and Graph 4 show an indicator that could at least work as a complement to objective tests that do not assess the minimum competencies for the positions: an analysis of the candidate’s experience. The sample from this study corresponds to 100% of public exams for administrative assistants and physicians at federal institutions of higher education in Brazil in 2021. In this context, Graph 3shows that 31.57% of assistant certificates required proof of experience and were all an eliminatory criterion. Among the selections for physicians, Graph 4 shows that none required proof of experience. In fact, only one (FURG) among 15 institutions considered that experience could possibly be interesting in a selection process for government labor but placed experience only as part of the proof of titles and gave the experience a mere classifying value.

Graph 3
Experience requirement in public exams for administrative assistants

Graph 4
Experience requirement in public exams for medical positions

A clarification should be made regarding Graphs 3 and 4. The requirement of experience as an eliminatory factor for several simpler positions (including assistant, in this study) and especially for dozens of more complex positions (including physician, in this study) is mandatory according to Law 11,091/2005 (Lei nº 11.091, de 12 de janeiro de 2005). It is because of this legislation that some institutions require experience for simpler positions. To not require experience for dozens of more complex positions is a blatant managerial and even legal affront. However, despite the law clearly and expressly imposing the experience requirement, 68.43% of the institutions dared (or were forced by Justice to) not follow this obligation regarding the assistant, while no institution dared (nor did the court order) demand experience from doctors and from many other medical positions.

Practical tests, simulations, and the like would be one more opportunity to complement the objective tests and would allow the assessment of several skills that are impossible to be analyzed with an “x” on an answer sheet. But, in this sample, absolutely no public exams for neither assistants nor physicians held practical tests.

Another interesting evaluation type, at least as a complement to other types, would be proof of titles (Barbosa, 1996b). Graph 5 shows that none of the exams for assistant positions presented evidence of titles and Graph 6 shows that only 1 in 15 selections (or 6.66%) for physicians used this evaluation type.

Graph 5
Proof of titles in assistant positions

Graph 6
Proof of titles for medical positions

Another rich evaluation type that can far surpass competency evaluation compared to a multiple-choice test are Discursive Tests. According to Graph 7, they were only applied in 15.78% of the public exams for Assistants (3 out of 19). Graph 8, on the other hand, shows an even smaller proportion for doctors as only 1 in 15 public exams (or 6.66%) used this type of assessment.

Graph 7
Discursive test in Assistant positions

Graph 8
Discursive test in medical positions

Given the almost generalized absence of almost all types of assessment and the unjustified predominance of multiple-choice objective theoretical tests, one measure would be even more important: adopting correction for guessing. However, none of the exams (assistant and medical positions) included this type of mechanism.

5. DISCUSSIONS

The data obtained here were taken directly from published official documents and present a series of inconsistencies and dysfunctions that deserve attention. At first, it can be objected that the notices took on these configurations due to the COVID-19 pandemic, but this author did a similar survey with notices from the same sample (federal higher education institutions in Brazil) and basically with the same positions: a clerk with another education but with administrative responsibilities very similar to the assistant and the same positions as doctors. The survey was carried out in 2016, was not published in a peer-reviewed journal, but has been available in a university repository at least since 2017 and therefore long before the aforementioned pandemic. This study is produced for the purpose of updating the data and now for a broader discussion via peer review.

So that the effects of the COVID-19 pandemic are not mis-dimensioned, a synthetic comparison of the 2016 and 2021 results seems opportune, as shown in Table 1:

Table 1
Evaluative structure notices for exams to fill positions in Federal Institutions of Higher Education in Brazil: Clerks and Doctors (comparatives between 2016 and 2021)

Study	2016 (A. B. S. Oliveira, 2017)	2021 (this study)
Notices	49	25
Positions	Administrative Assistants (AA) and Physicians (M)	Administrative Assistants (AA) and Physicians (M)
Objective proof as a single assessment (AA)	82%	58%
Objective proof as a single assessment (M)	64%	87%
Correction for guessing (AA)	4%	0%
Correction for guessing (M)	4%	0%
Presence of discursive tests (AA)	14%	15.78%
Presence of discursive tests (M)	17%	6.66%
Presence of proof of titles (AA)	0%	0%
Presence of proof of titles (M)	25%	6.66%
Presence of practical tests (AA)	0%	0%
Presence of practical tests (M)	5%	0%
Experience required (AA)	46%	31.57%
Experience required (M)	2.5%	0%

• Source: Elaborated by the authors.

From Table 1 and the context of the period (2016-2021), it can be inferred, and it should be noted that:

The publication of notices fell by about 48.97% for these positions, possibly due to the pandemic (health risk and economic impact), although some positions in the health area have seen an increase in the number of hirings precisely due to the increase in demand (hospital admissions, ICU, etc.) caused by the pandemic. The reduction in the offer of public exams, however, was already being defended and practiced by the federal government even before the pandemic (Maia & Souza, 2019).

The position of Administrative Clerk of elementary school level was suspended (Decreto nº 9.262, de 9 de janeiro de 2018) and his functions were basically absorbed by the Administrative Assistant of high school, who always had a similar function. For administrative positions, the context of the pandemic involved even a greater diversity of assessment types. In 2016, 82% of administrative staff had objective evidence as the only means of assessment against 58% in 2021, which is positive and is directly contrary to the thesis that the pandemic harmed the selection model.

Conversely, though discreet, there was an evaluative impoverishment of public examinations for doctors, going from 64% with only objective evidence in 2016 to 87% in 2021.

Correction for guessing, which was already very low in 2016 (4%), reached zero in 2021.

Discursive tests for administrative staff increased from 14% (2016) to 15.78% (2021), a slightly positive sign. Meanwhile, they dropped from 17% (2016) to 6.66% (2021) for doctors.

Proof of titles was 0% for administrative positions (2016) and continued to be so (2021). For physicians, such tests reduced from 25% (2016) to 6.66% (2021).

Practical tests were 0% for administrative personnel (2016) and continued to be so (2021). For physicians, such tests reduced from the low margin of 5% (2016) to 0% (2021).

Experience requirement for administrative departments dropped from 46% (2016) to 31.57% (2021), which may be more related to the strong judicialization around the subject, which is judicially declared as incidental unconstitutionality with some frequency. For physicians, the requirement was almost nil (2.5%) in 2016 and reached zero in 2021.

The evaluative flexibility for doctors, who are professionals who have never been the target of really refined selections, can be partially explained by the need to hire them more quickly during the pandemic (Proposta de Lei nº 3252, de 2020), which does not apply so dramatically to administrative staff, who received more evaluation requirements during the pandemic period. Still, the 2016 data show that the dysfunctions of the examinations were basically the same as they are today, which include objective and rigid academicism, low evaluation diversity, excessive theorization of evaluation types, wide disconnect between evaluation types and the essential competencies of the positions, very little investment in practical assessments, inadequate legislation, etc.

Furthermore, the public exam model in Brazil has been basically the same without major variations for more than half a century. In fact, the pandemic does not significantly change the model, so much so that the Administrative Reform project via the Constitutional Amendment (Proposta de Emenda à Constituição nº 32, de 2020),which has been in progress in the National Congress throughout the pandemic, deals with stability of employees, performance evaluation, and several other themes, but practically without touching the public exams, causing one to assume this could be another reform that will leave “the teleology of reforms and the public machine untouched” (Costa, 2008, p. 869).

On the other hand, there are many affronts to the meritocracy of public examinations, generally unknown to the public and almost untouched by literature that is produced mostly by professionals who have passed a public examination.

The affronts range from random guessing, which produces doubtful approvals as few public exams adopt any mechanism to combat this practice, to the model analyzed here that allows among other dysfunctions for a surgeon to be selected based on theoretical evidence to mark an “x” and nothing else. The population confuses meritocracy with the fact that everyone can participate in public exams with the simple fact that they are objective. It seems not to notice the relationship between poor selections from a methodological-evaluative point of view and the illusion of meritocracy, not to mention the failures of the deficient public service resulting from poor selections.

Guessing techniques and tips are taught to exhaustion in videos on the internet, in books, and in preparatory courses, which can be quite expensive. But there are those who confuse the cunning of guessing with meritocracy without realizing the conceptual, cultural, ethical, and social abyss involved. One example is that the public sector’s islands of excellence coexist with uncomfortable statistics. A study by the Fundação Getulio Vargas (FGV, 2017) in the two largest cities in Brazil (São Paulo and Rio de Janeiro) indicated on a scale from 0 to 10 that the population’s perception of quality in relation to some public services. Some of the results: health (3.2), safety (3.6), transportation (4.7), universities (4.6), and schools & day care centers (4.6). If the majority, and sometimes the entirety, of the staff are employees who have passed examinations that capture the pinnacle of intelligence and professionalism, how can we explain such low scores? Are budget and infrastructure, for example, capable of explaining on their own these poor assessments of public services? Would random guessing, poor selections, and content memorized by those approved in public exams have any relevant participation in this context?

However, one can relativize the study by Instituto Brasileiro de Economia (FGV, 2017) due to the lack of structure or budget that would lead to people’s dissatisfaction, but it is not clear what the real difference is, positive or not, of what human presence could do in this context, which opens up more complex and partially overlooked discussions such as digitization/automation, use of artificial intelligence, or other issues in both public services and notices).

In fact, a public exam that preserves impersonality without admitting the best cadres is dysfunctional by definition (Fontainha et al., 2014) in addition to producing injustices both towards candidates in good faith and towards those who will pay salaries to obtain public services of low quality due to incapacity or inadequacy of those who were inappropriately selected.

Fontainha (Laporta, 2014) is incisive: “The public notice is a machine of social injustice.... Public exams ... serve to select those who were most prepared for the tests and not the most competent. This reflects on the quality of public services.” And he reinforces that the “poor selection of candidates explains the poor quality of the public service” (Bibiano, 2014). Despite any shortcomings, it is worth remembering that in the mind of Brazilians, the idea prevails that through public exams, “for the first time ... the democratization of access to public office through comparative merit was achieved” (J. B. A. Oliveira, 1986, p. 50). Thus, public exams and democracy are intertwined. Barbosa (1996a, p. 74) asserts that in Brazil “...the public exam is a paradigmatic element in the legitimization of meritocracy. In reality, in terms of representations in our society, there is an overlap between democratic instruments such as public exams and meritocratic systems”. One of the problems in this context, however, is that the links between democracy and public exams seem consistent but are incredibly fragile and sometimes non-existent. Now, “the discourses ... of meritocracy together with vocational logic and training for competitiveness ... have subjected education to processes of erosion of democracy” (Lima, 2021, p. 2). Certainly, there is an erosion of democracy in the last case when there is a systematic and repeated falsification of merit through an apparent neutrality such as that of public exams that enjoy a social legitimacy based on an illusory meritocracy considering all the results presented here that uncover dysfunctions in this process.

6. CONCLUSIONS

The correction of the many dysfunctions reported here requires a broad and deep re-discussion and reformulation of each one of them with the involvement of several interested parties and not just the candidates, as the guidelines around the theme of public exams usually are. The most crucial improvement is to maintain impersonality, but not confuse it with the extreme objectification that dominates and stifles most public exams. Therefore, if one wants to select the best professionals, it seems reasonable for the assessment to be less academic and as connected as possible with the attributions and competencies of the positions to be filled. This would make sense from a managerial (hire the best professionals) and ethical (effectiveness of merit and purpose) point of view.

In this sense, considering the literature, the results of this study, and the Brazilian context, there are some proposals to discuss possible improvements in the model for filling civil service positions. Note that Box 2presents transition proposals as the broad improvement as to how to fill civil service positions will require even more profound changes in the short, medium, and long-term path.

BOX 2
PROPOSALS FOR IMPROVING THE BRAZILIAN MODEL FOR FILLING CIVIL SERVICE POSITIONS

Initial suggestions include to apply practical tests and/or simulation in the intermediate or final phases of all processes for filling civil service positions, limiting the valuation of multiple-choice tests to a maximum of 50% of the total score in this process; include qualifying/non-eliminating evidence of titles in all exams (titles related to education, experience, etc.); adopt correction for guessing in all public exams; use discursive tests when appropriate, which almost always is; create broad and unified regulations in these processes, including the evaluative dimension, to be discussed in advance with all stakeholders, especially with the least heard and sometimes most forgotten by many public servants-the one who pays the salary of those who pass the public exam.

7. LIMITATIONS OF THIS STUDY AND PROPOSALS FOR FUTURE STUDIES

This work, like any other research effort, has limitations: 1) one of the weaknesses of this study derives from a limitation of the researched object itself as the article analyzes a small number of positions. But, while countries such as the United States have a single federal executive career, Brazil has more than 300 federal careers in the Executive Branch (Máximo, 2020) with some careers having, in isolation, hundreds of positions in their composition (example of a career, out of 300, with hundreds of different positions: Brazil, 2005). 2) Another limitation is the temporal arc, which could be more longitudinal. This was mitigated when using and comparing pre-pandemic (2016) and intra-pandemic COVID-19 (2021) data. 3) A limitation generated by the focus of the paper was the smaller space for historical, cultural, legal discussions, etc.

Due to the thematic complexity and little literature, there are vast possibilities for future studies, but the ones that stand out are as follows: 1) considering positions in numbers, natures, and other varieties to increase the power of generalization on the subject; 2) studies that confront a greater range of historical, sociological, and/or legal data such as on reforms, legislation, social constructions, institutionalization of examinations, etc.); 3) a study could be developed with a focus on the programmatic content of the notices, contrasting it with job descriptions, competencies, evaluation methods, etc.

Reviewers:

Patrícia Vendramini (Universidade do Estado de Santa Catarina, Florianópolis / SC - Brazil) https://orcid.org/0000-0001-6500-4799

Fernando de Souza Coelho (Universidade de São Paulo, São Paulo / SP - Brazil) https://orcid.org/0000-0003-2803-0722

Bruno Sendra de Assis (Universidade de São Paulo, São Paulo / SP - Brazil) https://orcid.org/0000-0001-9925-3080

Peer review report: the peer review report is available at this https://bibliotecadigital.fgv.br/ojs/index.php/rap/article/view/87929/82702.

ACKNOWLEDGMENTS

We thank the anonymous reviewers for the relevant suggested improvements. And we honor Joel de Lima Pereira Castro Junior (in memoriam), co-author of this article, who died shortly after the conclusion of this work - he was a friend, master’s advisor of the first author and left a legacy of dedication and contributions to the field of Administration in Brazil and abroad.

REFERENCES

• Alencar, A. E. V. (2021). Re-existências: notas de uma antropóloga negra em meio a concursos públicos para o cargo de magistério superior. Revista De Antropologia, 64(3), e189647. Retrieved from https://doi.org/10.11606/1678-9857.ra.2020.189647
  » https://doi.org/10.11606/1678-9857.ra.2020.189647
• Barbosa, L. (1996a). Meritocracia à brasileira: o que é desempenho no Brasil? Revista do Serviço Público, 74(3), 58-102. Retrieved from https://doi.org/10.21874/rsp.v47i3.396
  » https://doi.org/10.21874/rsp.v47i3.396
• Barbosa, L. (1996b). Igualdade e meritocracia: a ética do desempenho nas sociedades modernas Rio de Janeiro, RJ: FGV.
• Bardin, L. (2011). Análise de conteúdo São Paulo, SP: Edições 70.
• Barbetta, P. A., Reis, M. M., & Bornia, A. C. (2004). Estatística para cursos de Engenharia e Informática São Paulo, SP: Atlas.
• Bibiano, B. (2014, November 02). Seleção ruim de candidatos explica má qualidade do serviço público, diz pesquisador. Veja Retrieved from https://veja.abril.com.br/educacao/selecao-ruim-de-candidatos-explica-ma-qualidade-do-servico-publico-diz-pesquisador/
  » https://veja.abril.com.br/educacao/selecao-ruim-de-candidatos-explica-ma-qualidade-do-servico-publico-diz-pesquisador/
• Caldeira, L. F. S. (2020). Proposta de estatuto do concurso público fluminense: um estudo avaliativo (Master Thesis). Fundação Cesgranrio, Rio de Janeiro, RJ.
• Caldeira, L. F. S., & Vilarinho, L. R. G. (2021). Estatuto do Concurso Público Fluminense: uma Avaliação de Concurseiros e Gestores Públicos. Examen: Política, Gestão E Avaliação Da Educação, 4(4), 1-39. Retrieved from https://examen.emnuvens.com.br/rev/article/view/126/65
  » https://examen.emnuvens.com.br/rev/article/view/126/65
• Coelho, F. S., & Menon, I. O. (2019). A quantas anda a gestão de recursos humanos no setor público brasileiro? Um ensaio a partir das (dis)funções do processo de recrutamento e seleção - os concursos públicos. Revista do Serviço Público, 69, 151-180. Retrieved from https://doi.org/10.21874/rsp.v69i0.3497
  » https://doi.org/10.21874/rsp.v69i0.3497
• Constituição da República Federativa do Brasil de 1988 (1988). Brasília, DF. Retrieved from http://www.planalto.gov.br/ccivil_03/constituicao/constituicao.htm
  » http://www.planalto.gov.br/ccivil_03/constituicao/constituicao.htm
• Costa, F. J. L. (2008). Brasil: 200 anos de Estado; 200 anos de administração pública; 200 anos de reformas. Revista de Administração Pública, 42(5), 829-874. Retrieved from https://doi.org/10.1590/S0034-76122008000500003
  » https://doi.org/10.1590/S0034-76122008000500003
• Decreto nº 9.262, de 9 de janeiro de 2018 (2018). Extingue cargos efetivos vagos e que vierem a vagar dos quadros de pessoal da administração pública federal, e veda abertura de concurso público e provimento de vagas adicionais para os cargos que especifica. Brasília, DF. Retrieved from http://www.planalto.gov.br/ccivil_03/_ato2015-2018/2018/decreto/D9262.htm
  » http://www.planalto.gov.br/ccivil_03/_ato2015-2018/2018/decreto/D9262.htm
• Fundação Getulio Vargas. (2017, August 02). Saúde e segurança são serviços públicos com pior avaliação de cariocas e paulistanos Retrieved from https://portal.fgv.br/noticias/saude-e-seguranca-sao-servicos-publicos-pior-avaliacao-cariocas-e-paulistanos
  » https://portal.fgv.br/noticias/saude-e-seguranca-sao-servicos-publicos-pior-avaliacao-cariocas-e-paulistanos
• Fontainha, F. C., Geraldo, P. H. B., Veronese, A., Alves, C. S., Figueiredo, B. H., & Waldburger, J. (2014). Processos seletivos para a contratação de servidores públicos: Brasil, o país dos concursos? Rio de Janeiro, RJ: Fundação Getulio Vargas.
• Holstein, K. X., & Rockembach, M. (2017). Um estudo sobre os concursos públicos para arquivista no Brasil. Biblionline, 13(2), 69-81. Retrieved from https://doi.org/10.22478/ufpb.1809-4775.2017v13n2.34242
  » https://doi.org/10.22478/ufpb.1809-4775.2017v13n2.34242
• Hunter, J. E., Schmidt, F. L., & Judiesch, M. K. (1990). Individual differences in output variability as a function of job complexity. Journal of Applied Psychology, 75(1), 28-42. Retrieved from https://doi.org/10.1037/0021-9010.75.1.28
  » https://doi.org/10.1037/0021-9010.75.1.28
• Desforges, C., Angel, N., Debart, M. H., Renaud-Boulesteix, B., Paulard-Lanapats, C., Tessier, A., … Berlioz, J. M. (2011, November). Rapport Sur La Formation Initiale Des Fonctionnaires De L’etat Dans Les Écoles Du Service Public Retrieved from https://www.vie-publique.fr/sites/default/files/rapport/pdf/124000076.pdf
  » https://www.vie-publique.fr/sites/default/files/rapport/pdf/124000076.pdf
• Joseph, D. L., Jinh, J., Newman, D. A., & O’Boyle, E. H. (2015). Why does self-reported emotional intelligence predict job performance? A meta-analytic investigation of mixed EI. Journal of Applied Psychology, 100(2), 298-342. Retrieved from https://doi.org/10.1037/a0037681
  » https://doi.org/10.1037/a0037681
• Laporta, T. (2014, September 15). Concurso público é uma máquina de injustiça social. IG Retrieved from https://economia.ig.com.br/carreiras/2014-09-15/concurso-publico-e-uma-maquina-de-injustica-social.html
  » https://economia.ig.com.br/carreiras/2014-09-15/concurso-publico-e-uma-maquina-de-injustica-social.html
• Lei nº 11.091, de 12 de janeiro de 2005 (2005). Dispõe sobre a estruturação do Plano de Carreira dos Cargos Técnico-Administrativos em Educação, no âmbito das Instituições Federais de Ensino vinculadas ao Ministério da Educação e outras providências. Brasília, DF. Retrieved from http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2005/lei/L11091compilado.htm
  » http://www.planalto.gov.br/ccivil_03/_ato2004-2006/2005/lei/L11091compilado.htm
• Lobato, F. H. S., & Alberto, D. P. S. (2019). “O estudo é com você, a viagem é com a gente”: as agências de turismo especializadas em viagens a concursos públicos no brasil. Revista Turismo, Visão e Ação, 21(2), 82-101. Retrieved from https://doi.org/10.14210/rtva.v21n2.p82-101
  » https://doi.org/10.14210/rtva.v21n2.p82-101
• Lima, L. C. (2021). Democracia e educação: Dewey em tempos de crise da educação democrática. Arquivos Analíticos de Políticas Educativas, 29(154), 1-16. Retrieved from https://doi.org/10.14507/epaa.29.5881
  » https://doi.org/10.14507/epaa.29.5881
• Maia, B. (2021). O adestramento para o estado: tradição de conhecimento e técnicas de aprendizado entre candidatos ao serviço público no Brasil. Anuário Antropológico, 46(1), 253-271. Retrieved from https://doi.org/10.4000/aa.7691
  » https://doi.org/10.4000/aa.7691
• Maia, G., & Souza, A. (2019, June 23). ‘Dificilmente teremos concurso no Brasil nos próximos poucos anos’, diz Bolsonaro. Economia. O Globo Retrieved from https://oglobo.globo.com/economia/dificilmente-teremos-concurso-no-brasil-nos-proximos-poucos-anos-diz-bolsonaro-23757508
  » https://oglobo.globo.com/economia/dificilmente-teremos-concurso-no-brasil-nos-proximos-poucos-anos-diz-bolsonaro-23757508
• Máximo, W. (2020, February 23). Reforma administrativa pretende revisar salários e número de carreiras. Política. Agência Brasil Retrieved from https://agenciabrasil.ebc.com.br/politica/noticia/2020-02/reforma-administrativa-pretende-revisar-salarios-e-numero-de-carreiras
  » https://agenciabrasil.ebc.com.br/politica/noticia/2020-02/reforma-administrativa-pretende-revisar-salarios-e-numero-de-carreiras
• Ministério Público Federal. (2015, November 09). MPF recomenda mais transparência em concursos para professores universitários Retrieved from https://www.mpf.mp.br/pgr/noticias-pgr/mpf-encaminha-recomendacao-ao-mec-para-dar-transparencia-aos-concursos-para-professores-universitarios
  » https://www.mpf.mp.br/pgr/noticias-pgr/mpf-encaminha-recomendacao-ao-mec-para-dar-transparencia-aos-concursos-para-professores-universitarios
• Oliveira, A. B. S. (2017). Seleção de pessoas para o setor público brasileiro: competências, legislações e visão estratégica (Master Thesis). Universidade Federal Fluminense, Niterói, RJ.
• Oliveira, J. B. A. (1986). Síndicos ou Estadistas: quem prepara as elites? Revista de Administração Pública, 20(2), 49-70. Retrieved from http://bibliotecadigital.fgv.br/ojs/index.php/rap/article/view/10025/9026
  » http://bibliotecadigital.fgv.br/ojs/index.php/rap/article/view/10025/9026
• Passos, D. V. S. (2018). Concurso público para a magistratura: repensando o modelo de seleção e o papel dos juízes na democracia brasileira (Doctoral Dissertation). Universidade de Fortaleza, Fortaleza, CE.
• Prata, A. B. A., & Romão, T. L. C. (2019). Formação de intérpretes versus ofício de tradutores públicos e intérpretes comerciais no Brasil: um estudo de caso no Estado do Ceará. Mutatis Mutandis Revista Latinoamericana De Traducción, 12(1), 209-229. Retrieved from https://doi.org/10.17533/udea.mut.v12n1a08
  » https://doi.org/10.17533/udea.mut.v12n1a08
• Proposta de Emenda à Constituição nº 32, de 2020 (2020). Altera disposições sobre servidores, empregados públicos e organização administrativa. Brasília, DF: Câmara Federal. Retrieved from https://www.camara.leg.br/propostas-legislativas/2262083
  » https://www.camara.leg.br/propostas-legislativas/2262083
• Proposta de Lei nº 3252, de 2020 (2020). Acrescenta o art. 2º-A à Lei nº 13.959, de 18 de dezembro de 2019, que institui o Exame Nacional de Revalidação de Diplomas Médicos Expedidos por Instituição de Educação Superior Estrangeira (Revalida). Brasília, DF: Câmara Federal. Retrieved from https://www.camara.leg.br/proposicoesWeb/fichadetramitacao?idProposicao=2254990
  » https://www.camara.leg.br/proposicoesWeb/fichadetramitacao?idProposicao=2254990
• Schmidt, F. L., & Hunter, J. E. (2015). Methods of Meta-Analysis Corrected Error and Bias in Research Findings Thousand Oaks, CA: Sage publications.
• Schmidt, F. L., & Hunter, J. E. (1998). The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings. Psychological Bulletin, 124(2), 262-274. Retrieved from https://doi.org/10.1037/0033-2909.124.2.262
  » https://doi.org/10.1037/0033-2909.124.2.262
• Schmidt, F. L., Oh, I. S., & Shaffer, J. A. (2016, October). The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research Findings (Working Paper). Retrieved from https://www.researchgate.net/publication/309203898
  » https://www.researchgate.net/publication/309203898
• Tribunal Regional Federal da 5ª Região. (2014). AC (Apelação Cível) AC: 6205420124058200. Relator: Desembargador Federal Geraldo Apoliano, Data de Julgamento: 20/02/2014, Terceira Turma, Data de Publicação: 11/03/2014Retrieved from https://trf-5.jusbrasil.com.br/jurisprudencia/25012301/ac-apelacao-civel-ac-6205420124058200-trf5
  » https://trf-5.jusbrasil.com.br/jurisprudencia/25012301/ac-apelacao-civel-ac-6205420124058200-trf5
• Tribunal Regional Federal da 5ª Região. (2017). ACP (Ação Civil Pública) 000156635.2012.4.05.8100. Apelação Cível. Relator: Desembargador Federal Lazaro Guimarães, Data de Julgamento: 11/04/2017, Quarta Turma, Data de Publicação: DJE - Data: 02/05/2017 Retrieved from https://trf-5.jusbrasil.com.br/jurisprudencia/927771129/ac-apelacao-civel-15663520124058100/inteiro-teor-927771207
  » https://trf-5.jusbrasil.com.br/jurisprudencia/927771129/ac-apelacao-civel-15663520124058100/inteiro-teor-927771207

• [Original version]

APPENDIX

BOX A1
CONTINUATION

BOX A1
LINKS TO ACCESS THE FULL TEXT OF THE PUBLIC NOTICES ANALYZED IN THIS STUDY

Edited by

Editor-in-chief: - Alketa Peci (Fundação Getulio Vargas, Rio de Janeiro / RJ - Brazil) https://orcid.org/0000-0002-0488-1744

Associate editor: - Mauricio Dussauge Laguna (Centro de Investigación y Docencia Económicas, Ciudad de México - México) https://orcid.org/0000-0001-7630-1879

Publication Dates

• Publication in this collection
  16 Jan 2023
• Date of issue
  Nov-Dec 2022

History

• Received
  11 May 2022
• Accepted
  17 Nov 2022