Evaluating the impact of the selection process of principal in Brazilian public schools

Educational quality in Brazil has improved at a slow pace despite continued public policy efforts. The international literature has noted the importance of school management and the principal’s proﬁ le on students’ performance, but this discussion is recent in Brazil. In this study, we investigate the effects of the selection process of principals on students’ performance from public state schools in Brazil. We conclude that while the “selection process” channel is not clear, the “principal quality effect” indicates that the more democratic and technical selection methods tend to select most qualiﬁ ed principals in terms of leadership and managerial abilities. The principal who has time to implement changes, who adopts management practices to encourage teachers and has experience being the leader of the school present better results in terms of student’s performance. Therefore, it is important that the principal selection process choose professionals with the skills mentioned.


Introduction
In recent decades, several countries have advanced more in educational quality (academic achievement, students' performance, etc.) than Brazil, despite continued efforts of public policies, such as promoting access to education (93.8% of 6-to-14-year-olds attend elementary school and 81.2% of 15-to-17-year-olds are enrolled in high school) 1 and high levels of public spending in terms of GDP, higher than many countries participating in the OECD Program for International Student Assessment (PISA) 2 . In 2011 while Brazil spent 6.1% of GDP (or 19% of public expenditure), the OECD countries spent an average of 5.6% of GDP 3 (Fernandes et al. 2014). However, the performance of Brazilian schools in the PISA standardized testing the country ranked: 57 th (among 65 countries) in mathematics, 55 th in reading (Portuguese) and 59 th in sciences, while Singapore was third and South Korea ranked seventh in the overall standings.
Thus, the major challenge now facing Brazilian education, especially in public schools, is to raise quality, particularly in basic education (preschool, elementary and secondary schools) due to its importance for the future performance of individuals. It is well known that several factors related to the characteristics of the family, student and school infl uence students' performance. Besides that, a crucial factor affecting school success that has received little attention in the literature is the role played by the school principal and its methods of selection.
The school principal is the legal head of the educational unit and the person responsible for organizing and planning the school activities as well as creating conditions to enhance the educational process. Therefore, the importance of school principals is centered on their ability to create better working conditions for teachers (motivation and attendance of teachers, encouraging training, etc.), ensure the basic quality of infrastructure (ventilation, lighting, restrooms, security against violence, etc.), provide conditions for the academic life of students (motivation, disciplinary matters, etc.), organize and plan school activities, assist and enhance the educational/instructional process and encourage the participation of parents and the school community.
The most common methods to select principals are different types of appointment (technical, political or others) referral, election (including or not the community participation), examination ranking, public exam (for public schools) 4 , national tests, interview, resume submission, certifi cation, or a combination thereof. The ideal selection process should choose the most qualifi ed candidate, according to reasonable selection criteria, to meet the above-mentioned technical and managerial qualifi cations. Beyond that, the selection process per se can create favorable conditions (or not) for principals conduct their activities. Therefore, we can distinguish between two possible channels through which the different principal selection methods might impact school performance: the "selection process effect" -related to the environment for principals to develop their activities -and the "principal quality effect" -related to the principals' personal/managerial characteristics. For instance, depending on the principal selection method, the principal can stay a longer or shorter length of time in the position, or can have greater or lesser access to funds, or even greater or lesser political articulation for creating partnerships, or dialoguing. In turn, the "quality" of the principal can impact students' learning and/or achievement by means of his/her personal traits, such as academic background, experience in the position and management skills (leadership, attainment of goals, motivation of staff and students, etc.), because different processes select principals with different qualities/characteristics.
Within this context, we state the following assumptions concerning the "selection process effect": a) usually appointment methods or election are characterized by higher principal turnover resulting in shorter time periods to implement the actions planned, while "public exam" method can generate an overly extended tenure of the same principal in the position hindering the implementation of more modern school administration practices, since in the case of public schools this method of selecting principals ensures stability in the position; b) it is possible that certain methods, such as "political appointment" or "election" work better to select principals who are more at ease with raising funds, due to greater political articulation. Related to the "principal quality effect": c) "public exam", "certifi cation", "national tests", or "appointment by technical staff" methods are usually based on technical criteria (academic background, experience, etc.), while the "election" and "political appointment" methods tend to favor the candidate's management project to the school, merit criteria and/or leadership characteristics.
To understand the potential effects of the principal selection process, it is worth scrutinizing the economic incentives behind such choices. Suppose that the principal is chosen by a central planner 5 . If the performance and characteristics of candidates for principal are known by planner (or can be inferred through observed performance) and if the objective of the planner is compatible with the well-being of society, the theoretical results show that the planner's choice (political appointment) would be effective. Nevertheless, the legitimacy of any of these assumptions can be questioned, particularly with regard to a political appointment by local educational authorities which could work toward interests other than those aligned with effi cient selection of the principal.
Another interesting example arises when employees and members of the community choose the principal by election. If the information available to voters (staff and community members) allows them to assess the ability of the candidates more effectively than the assessment made by a public planner and if the purpose of the voting group is compatible with that of the society, the economic results indicate that it is possible to choose the best possible principal for that community using this process. Again, these conditions can be violated if employees and teachers have self-interests that confl ict with the interests of society. This process can also fail if the community is not well informed about the candidates' performance.
For cases in which the principal is selected by means public exam or other testing, such incentives are more complex since the defi nition of subjects and test formats are prepared by specialists based on the expected performance of the candidates.
From these variations, a major discussion has developed on the best manner to select school principals. Therefore, in this study we aim to understand how the different ways of selecting principals infl uences the performance of public state school students in their 5 th and 9 th grades in Brazil. These grades have more impact on future educational outcomes and refl ect intermediate (5 th ) and fi nal (9 th ) evaluations from basic education. We examine the impact of both "selection process" and "principal quality" channels in Portuguese and mathematics scores, IDEB 6 rate and dropout rate. We use panel data of state public schools 7 and two different approaches to identify the effects: full sample panel analysis and a case study where the selection process of the principal changed during the period analyzed.

Principal selection in Brazil
In Brazil, there are different methods to select principals of public schools, such as appointment (technical, political or others), election (including or not community participation), state or municipal public exam, national tests, certifi cation, or a combination thereof. The method by which a principal is chosen for the position is different in each state (or municipality) and has been changing in recent decades. In the past, the principal was traditionally selected based on political criteria (chosen by political appointment). With redemocratization in Brazil (from the 1980s forward), the demand for a more democratic and transparent process increased, and other methods were gradually introduced such as public exam, election (by staff, parents and/or students), election combined with public exam, or certifi cation or appointment by technical staff on the basis of minimum requirements (educational level, experience in school administration, length of teaching practice). The selection methods have also changed with respect to the tenure and monitoring of principals' performance. Moreover, such innovations in the processes of principal selection differ from state to state in the way they take place and the amount of time they take 8 (Xavier et al., 1994). In many cases, different methods are combined to legitimize the selection process.
6 Basic Education Development Index calculated by the Ministry of Education. 7 Brazil has (often overlapping) public school systems at the "fundamental" level (through ninth grade) and secondary level run by the state and municipal governments. 8 More recently, the approval of the National Educational Plan in 2014 (Law 13,005, target 19) states that the selection of school principals should consider merit and performance criteria and also consultation with the local community (teachers, students and parents). It also states that a national test should be adopted to select the candidates, therefore combining different selection methods to create a more democratic environment.
Besides that, it is not clear in the (scarce) process selection literature which method selects the "best" principal candidate in terms of improving school performance, either through the "selection process effect" or "principal quality effect" (or both). A survey conducted by Fundação Victor Civita (2011) with 24 (of 27) Brazilian state education authorities indicated the main advantages and disadvantages of each method of principal selection. Concerning the "political appointment" method, although it can follow technical criteria and occur according to the school demand and tailored characteristics, in many cases it is based exclusively on political issues and the person chosen may not be accepted by the school community. The "election" method, in turn, tends to select a principal that is accepted as legitimate by the school community since the school staff and parents usually vote in the election. The disadvantage of this type of selection is that it may favor clientelistic practices or antagonistic groups, generating fragmentation in the school. Concerning the "public exam" method, the main issue raised is the possibility of non-acceptance of the principal within the school community, although the selection was made according to criteria established by experts.

Literature review
We present the studies by grouping them as follows: a) studies related to the "selection process effect" (via turnover of principals, stability in the offi ce or access to fi nancial resources), and b) studies that focus on the "principal quality effect" (principals' personal traits and school management characteristics). There are few studies on the impact of the principal selection method on school performance.
With regard to the "selection process effect", Coelli and Green (2012) noted a negative effect of short principal tenure on the English score (among other performance measures) of high school students in British Columbia, Canada, from 1995 to 2004, while Béteille et al. (2012), using longitudinal data for Miami schools, showed that principal turnover is detrimental to school performance. Likewise, Miranda and Pazello (2014) analyzed data from Prova Brasil 9 between 2005 and 2011 and used the political discontinuity in elections for the mayor as an instrument for the principal turnover variable in order to avoid potential problems of endogeneity. The authors concluded that the political appointment of principals impacts the IDEB negatively by causing an 11-point decline on average in the SAEB 10 score, since on average the "political appointment" method was associated with a higher turnover.
In turn, Barros and Mendonça (1998) examined the changes made in the selection methods in the redemocratization period (from 1982 to 1993), using a state panel, which expanded that selection process including the "election" method The authors found a negative and statistically significant impact of the "election" method on students' performance, though the impact was modest.
Concerning the evidence on the "principal quality effect", Dobbie and Fryer (2013) analyzed a set of managerial practices adopted in high performing charter schools 11 in New York and concluded that the fi ve best practices 12 -increased institutional time, more effective teachers and administrators, high dosage tutoring, data driven instructions and culture of high expectations -were strongly correlated with students' achievement and explained approximately 45% of the variation in school effectiveness. Interestingly, they also found that the traditional input measures analyzed -class size, expenditure per student, teacher fraction with no certifi cation and teacher fraction with advanced degree -were not correlated with school performance. In the same direction, Fryer (2014) used experimental methods and showed that, on average, the adoption of the fi ve index managerial practices in low performing public schools in the U.S. signifi cantly increased students' achievement in mathematics in the treated schools by 0.15-0.18 standard deviation. In turn, Bloom et al. (2015) analyzed the quality of management in four broad areas -performance monitoring, operations management, target setting and talent management in day to day activities -in 1,800 schools in eight countries (including Brazil). They found that higher management quality is strongly positively correlated with students' performance and had a greater effect than class size, 10 National System of Evaluation of Brazilian Basic Education, conducted by Ministry of Education. 11 A charter school is a public school managed independently, ensuring operational fl exibility. The admission is by lottery, so authors use this characteristic to form the control group (students not admitted). 12 The fi ve index practices were suggested by over 40 years of qualitative research. competition or teacher quality. Therefore, they stated that the role of the principal should focus on school management either through standardized management procedures or through the principal's existing characteristics as a manager. Another study that sought to identify good management practices, was conducted in several countries, including Brazil, in four dimensions within the school: operating and monitoring, setting goals to improve learning, personnel management, and leadership (World Management Survey Research, 2014). Based on data from interviews, the study showed that Brazil ranked penultimate (only better than India) under the general management practices indicator, with its worst dimension being personnel management.
Besides managerial practices and/or principals' managerial skills, other empirical studies have tried to understand how the learning process is affected by principals' personal traits, such as educational background, years of schooling, job experience, leadership characteristics, etc. In particular, leadership impact can work through a variety of mechanisms, such as supervision and retention of teachers and students, educational coordination of the courses, articulation of school vision and goals, resource allocation or openness to the community during tenure. In this context, Eberts and Stone (1988) and Béteille et al. (2012) examined American schools and found that the principal's teaching experience had a positive effect on students' performance. Béteille et al. (2012) noted the infl uence of the principal on recruiting and retaining the best teachers while dismissing the worst ones and promoting the development of teachers already hired. On the other hand, Eberts and Stone (1988) found evidence of a negative impact on students' performance when dealing with a lack of consensus between the principal and the teachers and the exercise of leadership that is too forceful by the principal in situations of confl ict resolution and guidance for teachers. In turn, Branch et al. (2012) and Clark et al. (2009) found a positive relationship between principals' experience and students' test scores.
Addressing Brazil, Marioni et al. (2014) used panel micro-data from a longitudinal study of the 2005 school generation (GERES) from 2005 to 2008 to evaluate different effects of principals on students' performance. They found evidence of a positive effect for the following variables: teachers' confi dence in the principal, principals' attention given to administrative rules, and principals' encouragement of innovative activities.
We use a panel of schools from Prova Brasil and the school censuses from 2007 and 2011 13 . Prova Brasil is a census-based evaluation conducted every two years encompassing students in the 5 th grade and 9 th grade at state schools 14 . The fi nal sample, after excluding outliers and processing the data, was 28,906 state schools in 2007 and 26,973 state schools 15 in 2011 across all Brazilian states.
The school performance indicators (proxies for student learning) were the IDEB score, mathematics and Portuguese scores and dropout rate. When it comes to the "selection of principal" 16 variable, we investigate the following methods of principal selection: a) "public exam"), b) "election", c) "public exam and election", d) "appointment by technical staff", d) "political appointment", e) "appointment by others", and f) "other methods 17 ".
With regard to school performance (Table 1), all the indicators (Portuguese and mathematics score, IDEB and dropout rate) showed an improvement in 2011 compared to 2007 (for both 5 th and 9 th grades) and the 5 th grade registered a higher increase in all the indicators (or in the case of the dropout rate, a higher decrease) in comparison to the 9 th grade, mainly due to policies oriented to literacy and early childhood education. Figure 1 presents the breakdown of the different methods of principal selection adopted in state schools in 2007 and 2011. The most frequent methods were "election" (28% in 2007 and 30% in 2011), followed by "public exam and election" (22% in 2007 and 24% in 2011) and "public exam" (15% in 2007 and 14% in 2011). The incidences of "political ap-13 The 2009 data could not be used due to a high rate of missing values in the questions under "Questionnaire of the Principal" in most states. 14 This evaluation was carried out in schools with at least 20 students enrolled in the grades evaluated. 15 In the period under analysis, there was a process of municipalization of the fundamental education schools. In 2007, there were 12,114 state schools that provided at least 5 th grade schooling, and in 2011, this number dropped to 10,026. On the other hand, the number of municipal schools providing the 5 th grade went from 23,928 in 2007 to 30,355 in 2011. With regard to the 9 th grade, the number of state schools remained almost the same between 2007 and 2011, while municipal schools jumped from 9,372 in 2007 to 13,873 in 2011. 16 Question from the Principals' Questionnaire: "You took over as principal of this school by: (A) Selection, (B) Election only, (C) Selection and Election, (D) Appointment by Technical Staff, (D) Appointment by Politicians, (F) Other Referrals, (G) Otherwise." It is worth pointing out that the questionnaire is self-populated by school principals. 17 The "other methods" are not specifi ed, therefore, we did not analyze it. pointment", "appointment by technical staff" and "appointment by others" are more stable and less frequent: 10%, 6% and 10% respectively in 2011. Considering the different geographic regions in Brazil, Table 2 presents the incidence of each principal selection method in state schools by Brazilian states in 2011. The incidence of principal selection through "political appointment" is higher in the northern and northeastern states and in the state of Santa Catarina in the south region (Table 2). On the other hand, the political appointment method is not used in any of the midwest states and in some the states of the southeast and is the lowest in the south and southeast regions There is predominance of "election" (8 states: PA, PI, RN, AL, RJ, PR, RS and MT) and "public exam and election" (8 states: AC, CE, PE, BA, MG, MS, GO and DF), the fi rst being observed across all the regions in the country.  The state schools in Sao Paulo and Espírito Santo use "public exam" (45% and 34%, respectively) and there is a predominance of "appointment by technical staff" in three states, all of them in the north region (RO, AM and RR).

Southeast region
Minas Many state schools changed their method of selecting principals between 2007 and 2011. Table 3 shows the variation in the frequency (in percentage points) of each method by state: A lighter shade of gray indicates a positive variation in the method adopted in that state and a darker shade of gray means a negative variation. The cases that draw more attention are: a) the states which switched to the adoption of "public exam and election" -Goiás, Distrito Federal, Mato Grosso do Sul and Bahia; b) the states in which "political appointment" was abandoned 18 -Espírito Santo, which switched to the "public exam" method, Roraima, which switched to the "election" and "appointment by others" processes, Amapá, which switched to the "appointment by technical staff" method; and c) Tocantins, which discontinued the use of the "public exam" and switched to "political appointment" and Pará, which switched from "technical appointment" to "election". We use the changes introduced in some states as robustness tests of the results 19 . The principals' characteristics (both personal and managerial) seems to differ according to the way the principal was selected. Table 4 indicates that the principals chosen by "public exam" and "public exam and election" show better managerial characteristics than those appointed by technical staff or politicians. The managerial skills that we examine include: principal participation in continuing education programs, development of tutoring education programs, exchange of information with other principals, promotion of continuing education programs for teachers, development of programs to reduce dropout and failure rates, and promoting meetings of the teaching advisory and deliberative council (conselho de classe) more than three times a year.   Finally, with respect to the principal's term of offi ce according to the method of selection, analysis of the 2011 data (Table 5) reveals that in state schools, the selection methods "appointment by technical staff", "political appointment" and "appointment by others " are associated with a higher turnover of principals: 50% of politically appointed principals, 48% selected by "appointment by others" and 47% appointed by "technical staff" remain in offi ce less than two years, while 34% of the principals selected by "public exam", 27% selected by "public exam and election" and 25% selected by "election" remain in offi ce between 5 to 10 years and 16% selected by "public exam and election" stay in the position between 15 to 20 years.

Methodology
We aim to understand the impact of the selection process of public school principals on students' learning and achievement. Possible confounders are unobserved variables of the principals or schools correlated with the selection process. For example, part of the negative relation observed between politically appointed principals and students' learning can be explained by the higher frequency of this selection process in the northern and northeastern states, where students' learning rates are lower, on average, than in the southern and southeastern states (Table 6). To mitigate those problems, we test two approaches. The fi rst considers that possible confounders are specifi c to the schools and remain constant in time (fi xed effects), so panel data is used to identify these effects. In this case, unobserved variables, which are believed to be controlled by this approach, would be unobserved characteristics of students, parents, teachers and principals, as well as programs and actions effectively implemented in the school and not identifi ed by the observed variables and the difference in raising funding, among others. In this fi rst approach we use a sample from Brazilian state public schools. The equation we estimate is: in which: represents the performance measure d for school i in year t; d identifi es the performance measure used (IDEB, scores in mathematics and Portuguese and dropout rate); is a binary variable that identifi es the selection process ( j = "election", "public exam", "public exam and selection", "political appointment", "appointment by technical staff" and "appointment by others", apart from "other methods") in school i in year t; denote the matrix of characteristics of the students (A), the school (E) and the principal (D) of school i in year t; are the fi xed effects of school and year, respectively, for each performance equation; τ, β are parameters to be estimated (the comparison of τ brings about the effect intended to estimate in this study); and, ε is the idiosyncratic error of the equation.
In this approach, the average effect of the selection process is representative for Brazil and is given by (being { -j } the set complementary to { j } of selection processes): We estimate Equation (1) by fi xed effect for linear regressions when the outcome variable is continuous, and by logit fi xed effects when the outcome variable is binary.
The second approach bases its identifi cation on case studies (improving the internal validity but impairing the external validity of the results). We propose a propensity score analysis with difference-in-differences approach, in which the treatment group is composed of public schools in states in which the principal selection method changed between 2007 and 2011 (from process A to process B, for example) and the control group encompasses state schools in states in which there was no change in the selection process in the same period (states that maintained process A). (1) The problem of endogeneity in this analysis (selection bias) emerges from the fact that schools from states that changed their selection process may have different characteristics from those that did not change and that are related to students' performance/learning. Therefore, we need to control for those differences to compare "similar" schools within the states that are part of the control and treatment group. We analyze the following cases: a) Case 1: The effect of switching from "election" to "public exam and election" in the period between 2007 and 2011. The treatment group is state schools in Goiás 20 and Mato Grosso do Sul 21 and the control group is state schools from Mato Grosso (which maintained the "election" selection process from 2007 to 2011). b) Case 2: The effect of switching from "public exam" to "political appointment" between 2007 and 2011. The treatment group is state schools in Tocantins 22 and the control group is state schools in Ceará and Amazonas (whose selection process was the same: "public exam"). c) Case 3: The effect of switching from "political appointment" to "public exam and election", with the treatment group of schools in Bahia 23 and a control group composed of schools in Sergipe, Paraíba and Maranhão (which maintained "political appointment" as the selection process). We exclude from the sample schools whose principal was the same in the period for both comparison groups.
We use the propensity score estimation to weight the schools from the control group in order to improve the estimations, in a procedure called propensity score weighting. The propensity score is defi ned by a binarychoice model in which the dependent variable refers to the principal selection process in the state chosen. in which: refers to the binary variable that identifi es the new selection process in the treatment state in schools i during the pretreatment period for the pair of states p (only the schools that employed the same selection process in the pretreatment period and the schools located in one of the states in which the change occurred and intervention took place); denotes the matrix of observed variables that determine the selection process in the pretreatment period for schools i; δ is a vector of parameters to be estimated; and is the idiosyncratic error of the equation.
Therefore, the average effect of the change in the selection process can be written by comparing the performance of paired schools in both pretreatment (t 0 ) and post-treatment (t 1 ) periods:

Results
We divide this section into: a) estimation of the "selection process effect" using panel data methods for the full sample of schools (POLS and fi xed effects) and for a selected sample of schools (propensity score analysis with difference-in-differences) and b) verifi cation of the "principal quality effect" by using the panel data method of logit fi xed effects 24 .
24 At this stage, the results were estimated in order to identify the characteristics of principals with the strongest impact on students' learning. Based on these characteristics, it was possible to see how different principal selection methods impact the selection of better or worse principals.
(3) Table 7 presents the estimation results of the "selection process effect" on the IDEB rate, and Table 8 does the same for mathematics scores 25 . Tables  5 and 6 show the results under the POLS 26 (columns 1, 3 and 5) and FE (columns 2, 4 and 6) methods as well as the results broken down as follows: a) state schools (columns 1 and 2), b) state schools -5 th grade (columns 3 and 4), and c) state schools -9 th grade (columns 5 and 6). From Table 7 we observe that: a) in the state schools and 9 th grade in which the principal was selected by "public exam", the average IDEB tends to be higher compared to the "political appointment" method (base method) in the POLS results. However, for the 5 th grade the result is the opposite, indicating that the "public exam" method is related to lower IDEB rates when compared to "political appointment" process, also using POLS estimation. Besides the mixed signs with POLS estimation, the fi xed effect method did not present signifi cant results, making it diffi cult to reach conclusions about the relation between the IDEB rate and the "public exam" selection method in comparison to "political appointment"; b) the same happens to the "public exam and election" method -it is positively correlated to IDEB rates with POLS estimation and negatively correlated to IDEB rates with FE estimation to state schools and 9 th grade, when compared to "political appointment"; c) the "election" method showed signifi cant results for state schools, 9 th and 5 th grades for both POLS and FE estimations. In all the cases, the average impact on IDEB rate was worse in schools that selected their principals by "election" when compared to schools whose principal were politically appointed; d) the average impact of using the "appointment by technical staff" selection process signifi cantly decreases the IDEB rate in state schools and 5 th grade (both under FE estimation) and increases it in the 9 th grade (POLS estimation), in comparison to schools with politically appointed principals; and e) the "appointment by others" method also showed signifi cant (positive) results only with POLS estimation for state schools and 9 th grade.

Selection process effect
25 Portuguese score and dropout rate were also scrutinized, and their results were similar to the IDEB rates and mathematics scores. These results are available upon request. 26 We present the POLS estimates because they demonstrate the need for control by other covariates and by specifi c effects of schools in order to identify the desired effect.   Robust Standard Errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1.
[1] Students per class, students TDI, percentage of women, percentage of black, percentage of students who own computer and internet, live with their mother and/or father, parents' educational level, encouragement from the parents, percentage of students who work, percentage of students who started studying at preschool (2-6 yr olds), students' absenteeism.
[2] Condition of classrooms and restrooms, safety conditions of school, infl ow and outfl ow of students, signs of deliberate damage at school, existence of a library in the school, logarithm of the total employees, surveillance service in the school and time frame of classes (full-time, part-time and morning or evening classes).
[3] Teachers characteristics: race, education, taken continuing education, teachers' salary, have other jobs (related or not to teaching practice), experience in teaching, experience in the school, excessive absenteeism, whether teachers have been assaulted.
[4] Sex, educational level and ethnicity of the principal, principal's experience in that position and in the school, principals motivation of teachers, whether they encourage innovation, continuing education, and students' learning, information on whether the principal helps with the textbook in the school, foreign interference in the management and number of meetings of the teaching advisory and deliberative council (Conselho de Classe) and School Council.
Source: Elaborated by authors.
Therefore, we observe that although the "election" process registered a negative average impact on IDEB rate in all specifi cations, no assertive conclusion can be drawn with regard to the direction of the impact of the various selection methods. In relation to students' performance in mathematics (Table 8), the results are mixed: depending on the estimation method, the signs of the impact alternate for all selection processes except "appointment by others", which shows a positive average effect on mathematics scores (with POLS estimation) in state schools, 5 th and 9 th grades in comparison to "political appointment". It is noteworthy that even the "election" method, which registered negative correlation with the IDEB rate, presents an unclear impact on mathematics score, being positively correlated to mathematics score with POLS estimation and negatively correlated with FE estimation, in all specifi cations (state schools, 5 th and 9 th grades).
Therefore, the evidence suggests that the "selection process effect" on students' performance is not conclusive, since it is possible to fi nd better and/or worse results (for IDEB and mathematics) by selection method compared to political appointment for all specifi cations examined. In a complementary analysis, we study the states where the legislation was amended and apply a differences-in-differences model using the propensity score.   Robust Standard Errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1.
Same notes as Table 7.
Source: Elaborated by authors.
Tables 7, 8 and 9 summarize the estimations of the effect of the change on the treated group. In relation to students' average performance in Case 1 (Table 9): a) The results for the 5 th grade are quite similar to the estimates for the state schools in Goiás and Mato Grosso do Sul, i.e., they show statistically signifi cant positive results of the selection process switching from "election" to "public exam and election", as observed through Portuguese and mathematics scores. The results for IDEB and dropout rate also indicate the same in both analyses, although signifi cance was swapped. b) The results of the "public exam" process change for the Portuguese and mathematics scores, which are now signifi cant in the differences-in-differences (DID) analysis for the 9 th grade, but lower in magnitude when compared to the results for the 5 th grade. As for dropout rate, the results of the fi xed effects model are statistically equal to zero, while for the DID effect the results for the change are negative (positive for dropout rate). We fi nd the following evidence in Case 2 (Table 10): a) The results observed for the 5 th grade are quite similar to the estimated results for the state schools in Tocantins. The selection process negatively affects the students' average performance. b) The results of the selection process changing to "political appointment" are not signifi cant for all specifi cations with the exception of the score in Portuguese for students in the 5 th grade. By estimating using DID, we identify that the change has a positive impact on the students' performance on the Portuguese exam (different from the results observed by the fi xed effects model, although the latter was statistically non-signifi cant). [2] 126 schools in the treatment group and 71 in the control group.
[3] Covariates: number of employees, teacher absence, library at school, number of school council meetings, year dummy, and school fi xed effects.
Source: Elaborated by authors.
The results for the students' average performance in Case 3 (Table 11) in comparison with the regressions by state show the following. a) The results of the selection process changing from "political appointment" to "public exam and election" are positive and signifi cant for mathematics scores in 5 th grade in both specifi cations. b) With respect to the dropout rate, the results show a reduction when the political appointment is replaced by public exam and election. [1] 28 schools in the treatment group and 36 in the control group.
[2] 30 schools in the treatment group and 38 in the control group.
[3] Covariates: number of employees, teacher absence, library at school, number of school council meetings, year dummy, and school fi xed effects.
Source: Elaborated by authors. [2] 7 schools in the treatment group and 13 in the control group.
[3] Covariates: number of employees, teacher absence, library at school, number of school council meetings, year dummy, and school fi xed effects.

Source: Elaborated by authors.
Although some of the results are consistent with the estimated effects by the fi xed effects model, there appears to be little consensus on the direction of the "selection process effect" in state schools (some evidence of the negative effect of political appointments and positive effects of election with public exam). In this sense, we run some tests to check if the effecttransmission mechanism is given indirectly, i.e., by selecting a principal with different characteristics.

Principal quality effect
We use fi xed effects model to evaluate the "principal quality effect", or to identify which characteristics of the principal (both personal and managerial) infl uence (positively or negatively) the students' performance. Thus, we present in Table 12 the average effect of principal characteristics (that are statistically relevant to school performance 27 ) on IDEB rate (column 1), mathematics and Portuguese scores (columns 2 and 3 respectively) and dropout rate (column 4).
27 In addition to the principals' characteristics in Table 12 (and Table 13), tests were run for the following characteristics, which showed no (or low) statistical relevance: principal holding degree in pedagogy (educational theory and practice) or other teacher education programs, principals' participation in continuing education, exchange of information with principals from other schools, textbooks selected by the principal, meetings of the teaching advisory and deliberative council (Conselho de Classe) or school council, promotion of remedial education, a principal that encourages the introduction of innovations and so forth.  Robust Standard Errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1.
The results in Table 12 show that the principals' characteristics with the strongest impact on the students' average performance are as follows. a) Principal who works in the same school between 11 to 15 years has a very signifi cant positive effect in all the specifi cations analyzed 28 (IDEB, mathematics score, Portuguese score and dropout rate). b) Principal who promotes teachers' continuing training programs also has a very signifi -cant positive impact on the average student performance in all the aspects mentioned (IDEB, mathematics score, Portuguese score and dropout rate). c) The characteristics that also have a positive impact but with lesser relevance include holding a diploma from a high school teacher training program (mathematics and Portuguese scores) and principal who works in the position (not in the same school) for more than 15 years (mathematics and Portuguese scores). d) The principal with high school diploma and teacher educational program (college degree) has a positive impact on Portuguese scores. On the basis of the statistically signifi cant characteristics of the principal profi le listed above (works in the same school between 11 to 15 years, promotes teachers' continuing training programs, high school teacher training program and is principal for more than 15 years (not in the same school), we estimate the relationship between the different principal selection methods and such characteristics. To do so, we use the logit fi xed effects model (Table 13) broken down by state school (columns 1, 4, 7 and 10), 5 th grade (columns 2, 5, 8 and 11) and 9 th grade (columns 3, 6, 9 and 12). Table 13 shows. a) The methods of "election" and "public exam and election" are the ones with the highest probability of selecting stable principals (who stay in the position in the same school between 11 and 15 years). Principals who are appointed politically remain in their schools for less time (thus impairing the students' performance) compared to the "election" and "public exam and election" methods. b) The methods of "election", "public exam and election", "public exam" and "appointment by technical staff" are the ones with the highest probability of selecting principals who promote continuing educational training programs for teachers. Politically appointed principals give less encouragement to continuing the training of their teachers compared to all the previous methods of principal selection. c) The method of "public exam and election" presents a higher probability to select principals that remain in the position for more than 15 years (not in the same school) when compared to the "political appointment" method. On the other hand, the probability of a politically appointed principal staying in the position for more than 15 years is higher compared to the "appointment by others" method. d) On average, the probability that a politically appointed principal has graduated from a high school teacher training program is higher when compared to the "election" and "public exam and election" methods. These results show that different selection methods seem to choose principals with different average characteristics/qualities (such as experience -practice and time, incentive to continuing training). Moreover, we can conclude that the "principal quality effect' is favored when the principal is selected through more democratic/technical methods than when using "political appointment".

Final remarks
Brazil is still moving at a slow pace as far as quality of education is concerned. Thus, the major current challenge of Brazilian education, especially in public schools, is how to improve quality, particularly by improving basic education, due to its importance for the future performance of individuals. The discussion of the role played by principals on educational quality is still incipient in the country. There are few studies in the international literature that discuss the importance of principals' personal traits on students' achievement. Considering the potential of the principal and the potential of this low-cost public policy, continued research on the topic can provide evidence to educational leaders and policymakers to improve the principal selection methods and ultimately raise student achievement in public schools.
In this study, we investigated how the different methods of selecting principals impact the performance of students in the 5 th and 9 th grades enrolled in state public schools in Brazil. We tested two assumptions: the "selection process effect" and the "principal quality effect". We also tested two models to identify the fi rst effect and results were mixed (5 th and 9 th grades). There was weak evidence that politically appointed or elected principals negatively infl uence students' performance. This evidence might be a result of the high turnover observed for these selection methods. Although specifi c results seem interesting, we cannot conclude that any of the different selection methods analyzed is the best, as we are analyzing more correlations than causal effects. Therefore, we conducted the second analysis to check how the principals' characteristics infl uence students' achievements, or the "principal quality effect".
By evaluating the results of the impact of the principals' characteristics on performance, we found that the one positive impact is when the principal works in the position in the same school between 11 and 15 years. In this case, we understand that it takes time for the principal become acquainted with the reality of the school, to gain experience in the management approach and to implement major measures to improve the students' performance (goal-based actions, other forms of incentives). In this sense, the average time of service seems to be an extremely positive factor in the students' school performance.
Another relevant characteristic is the promotion of teachers' continuing training programs. Although the educational literature does not indicate any consensus on the necessary training and skills to be a good teacher or on how to develop such skills, the fact that the principal encourages teachers seems to reveal important management characteristics to improve students' performance, either due to more active leadership or to a continuous concern with teacher training.
Finally, the information about principals' academic background shows that the principals who have graduated from a high school teacher training program positively infl uence the performance of students in schools. This result, although surprising, is corroborated by several managers of public education, owing to the closeness between this sort of background and the hands-on and instructional issues related to learning. There is a debate in the educational literature about the excessive formalism in the educational theory and practice courses in the country, which may not be as effective as the high school teacher training programs.
When verifying whether the principal selection methods affect the selection of principals with the positive characteristics mentioned above, we found consistent results. More transparent/technical selection processes (linked to "public exam" and/or "election" and "appointment by technical staff") relate positively with most positive characteristics: principals who remain much longer in the schools and have positive leadership characteristics (as identifi ed by the encouragement of teachers' continuing education). In contrast, the principals chosen by "political appointment" do not have these characteristics. Just as a side comment on principals' academic background, the selection-by-appointment processes gains an advantage when generally choosing more principals graduated from high school teacher training programs, which positively impacts the students' performance.
We conclude that while the "selection process effect" is not clear in terms of defi ning which selection process creates the best environment for principals to develop their activities, the "principal quality effect" indicates that the more democratic and technical selection methods tend to select most qualifi ed principals in terms of leadership and managerial abilities. Moreover, our results corroborate the international literature on the importance of school management for students' performance: the ideal principal is one who adopts management practices in order to encourage teachers and has experience being the leader of the school. Therefore, in spite of the few existing incentives in Brazil for the position of principal (low wages, overload of duties and so forth), it is of the utmost importance for the principal selection process to choose professionals with the skills mentioned.