Better Incentives, Better Marks: A Synthetic Control Evaluation of the Educational Policies in Ceará, Brazil

This article evaluates the effects of two educational policies implemented in the Brazilian state of Ceará. The first was a tax incentive (TI) for mayors to improve municipal education. Under this policy, municipal tax transfers were conditioned on educational achievement. The second was a program to offer educational technical assistance (TA) to municipalities. The impact of these policies was estimated by employing the synthetic control method to create a synthetic Ceará not affected by TI and TA. When the two policies were combined, the results were consistent with a 12 percent increase in Portuguese test scores in primary education and a 6.5 percent increase in lower secondary education. There were similar increases in mathematics test scores; however, these were not statistically significant. This study also investigates the impact of educational interventions on upper secondary schools, which, despite not being directly affected by the new policies, received better-prepared students from lower secondary schools. The findings show no effect on this level of education, highlighting the need for debate on how to extend the benefits of educational policies to upper secondary schools, as well as to other Brazilian states. This research is the first to analyze the impacts of the policies in Ceará on primary, lower secondary, and upper secondary schools using data from 1995 to 2019.

Considering this literature into account, I assume that the policies implemented in Ceará increased the scores of local students in mathematics and Portuguese tests.Furthermore, I assume that, although not directly affected by the new policies, upper secondary schools also experienced improvements because they received better-prepared students from lower levels schools.
To gauge the effect of the interventions in Ceará, I employ the synthetic control method (henceforth referred to as SCM).In the context of this investigation, SCM is an algorithm that selects a set of Brazilian states not affected by the educational interventions to create a control unit.Each selected control state contributes to the synthetic control unit according to a specific weight.Simply put, SCM estimates a synthetic Ceará whose performance in education is a weighted average of the performance of a set of chosen control states.This method provides transparency and a data-driven tool to select an adequate control unit (ABADIE, 2021).
When TI is combined with TA, the findings are consistent with increases of 12 and 6.5 percent in Portuguese test scores in primary and lower secondary schools, respectively.Regarding mathematics, the effects were similar, but not statistically significant.There was no evidence of impact of the new policies on upper secondary education in Ceará.
These findings are in line with the literature highlighting the positive impact of technical assistance and incentives on educational outcomes (ANGRIST and LAVY, 2001;BRANDÃO, 2014;BRESSOUX et al., 2009;CARNEIRO and IFFI, 2018;FREDRIKSEN et al., 2015;FUJE and TANDON, 2015;LAUTHARTE et al., 2021;McEWAN, 2015).They also seem to support the model of educational production proposed by Bishop and Woessmann (2004), which links higher political priority for education with better student performance.In the final section, I highlight my main conclusions.

The new educational policies devised in Ceará
Ceará is located in the northeastern region of Brazil and had an estimated population of approximately 9.2 million inhabitants in 2020.Its area is 148,895 km2 , which is slightly larger than Greece.In 2019, the state had a monthly 'per capita' In this article, TI and TA are regarded as incentives to increase educational quality.TI is clearly an incentive, since mayors receive higher revenues as a reward for improving education.On the other hand, PAIC's nature as an incentive is less obvious.Bonamino et al. (2019) see PAIC as a complex arrangement with a high capacity to articulate the cooperation between the state government and municipalities.In a way, TA offers a set of incentives for municipal governments to improve learning quality.Training for teachers and civil servants working in school management can be understood as incentives for improving their teaching and management skills.
Moreover, the Escola Nota Dez prize, an initiative related to PAIC, awards the best schools in learning achievement.These schools are granted financial resources, but only receive the complete prize if they offer support to a lowerperforming school (CRUZ et al., 2020;SUMIYA et al., 2017).Thus, Escola Nota Dez is an incentive for schools to achieve better results and to cooperate with other schools.
The following sections explore in detail each of the new educational policies implemented in Ceará.

Programa Alfabetização na Idade Certa (TA)
In Brazil, federated states and municipalities collaboratively organize their educational systems (BRASIL, 1996).As shown in Table 01, both municipalities and states are responsible for providing primary and lower secondary education.This leads to an overlap of responsibilities and ambiguity in the role of each sphere of the federation.Source: Created by the author, based on Brasil -LDB (1996).
Unlike other Brazilian states, Ceará started to address this problem decades Brazil.In the same year, 96 percent of lower secondary schools were managed by municipalities in Ceará, compared to only 50.5 percent in Brazil (LOUREIRO et al., 2020).
With clear-cut competencies for municipalities and for the state, the collaboration between the two spheres became smoother.By not providing primary and lower secondary education, the state could focus on offering TA to the municipalities.The collaboration between the state and municipalities was institutionalized by Law 14.026, from 2007, which established PAIC (SEGATTO, 2015).
PAIC resulted from the articulation of several organizations and actors (SEGATTO and ABRUCIO, 2018).In 2004, the state parliament of Ceará created the committee for the elimination of illiteracy in Ceará, with the aim of investigating the quality of education in the state.This initiative received support from the United Nations International Children's Emergency Fund (UNICEF), the association of mayors of Ceará, the union of municipal educational leaders (Undime), the state and federal universities of Ceará, private universities, and specialized civil servants working in education (BONAMINO et al., 2019;SEGATTO, 2015).
The committee's investigation showed that only 40 percent of the students in the analytical sample were literate.To change this reality, PAIC was implemented as a state program starting in 2007 (SEGATTO, 2015).Although the program was optional, it was adopted by all of the state's 184 municipalities since its beginning (SUMIYA et al., 2017).PAIC established technical and instrumental standards that defined the responsibilities of each stakeholder in the educational process (CRUZ et al., 2020).
The main actions of the program were: the training of teachers focused on classroom practice; the provision of literacy materials to schools; the promotion of workshops to disseminate best practices; the strengthening of the state system of evaluation of primary and lower secondary education (SPAECE 5 ); and the training of municipal civil servants with a focus on the management of school systems (LAUTHARTE et al., 2021).These activities were carried out through agreements between municipalities and the State Secretariat of Education (SEDUC).
To facilitate the cooperation between the state government and  The tax incentive (TI) The Brazilian Constitution states that revenues from the 'Imposto sobre Circulação de Mercadorias e Serviços' (ICMS), the state consumption tax, shall be divided between states (75 percent) and municipalities (25 percent).Furthermore, from the 25 percent of revenues reserved for municipalities, 75 percent should be distributed based on the added benefit criteria, which means that municipalities producing and selling more will receive more resources.The Constitution grants the federated states discretion to define how to distribute the remaining 25 percent among their municipalities.  is the education quality index in municipality m;   is the health quality index in municipality m; and   is a dummy indicating whether municipality m has an operational solid waste management system (LOUREIRO et al., 2020).
The methodology to calculate   is shown in Equation 02. 8 It was reformulated in 2011 to focus on the lower tail of the distribution of performance, that is, municipalities that improve the outcomes for students lagging behind are benefited more than others (LAUTHARTE et al., 2021).  is the Literacy Quality Index for municipality m.   is the index that measures the quality of so-called 'fundamental schools' in municipality m -in Brazil, fundamental schools comprise primary and lower secondary schools.Finally,   is the average passing rate in primary school for municipality m.Appendix A presents how each component of Equation 02 is calculated.One caveat of the EQI computation is that it considers current and past educational performance.Thus, in the first years of his or her term, a given mayor's incentives are dependent on the past administration.They will only receive incentives integrally dependent on their own performance near the end of their term.

Incentives for the actors involved in the educational process
A growing literature explores the impact of incentives on the quality of education.Incentives might target several actors involved in the educational process.Regarding incentives for teachers, results are mixed since effectiveness relies on appropriate incentive design (IMBERMAN, 2015).In a randomized controlled experiment in Tanzania, teacher salary bonuses dependent on student performance increased schooling quality (MBITI et al., 2019).Similar incentives implemented in schools in India and Israel also appear to have positively impacted student performance (LAVY, 2009;MURALIDHARAN and SUNDARARAMAN, 2011).
However, not all studies found statistically significant impacts; some even found negative effects of financial incentives for teachers (FRYER, 2013;FRYER et al., 2012).
Incentives for students have also been a topic of research.Although some interventions had a positive effect on student attendance, the effects on performance are less clear (BARRERA-OSORIO et al., 2011;GALIANI and McEWAN, 2013).One intervention in the United States provided cash transfers for students who successfully completed standardized tests.Results showed improvements in mathematics, but no impact on reading and science scores (BETTINGER, 2011).

Bruno Gasparotto Ponne
( 2023) 17 (1)  e0005 -11/44 Studies have also shown that central exams incentivize students to increase their performance since scores might be seen by future employers or by educational institutions (BISHOP, 1997;WOESSMANN, 2018).However, there is also evidence that excessive focus on central exam contents might negatively impact student achievement (COLLIER, 2012).Portuguese test scores in primary education.Brandão (2014) and Petterini and Irffi (2013) employed the same methodology to analyze the policy and found positive impacts in mathematics and Portuguese scores.
In a more theoretical approach, Bishop and Woessmann (2004) devised a basic model of educational production.According to their model, giving political priority to education has positive effects on student achievement.

Technical assistance in education
Several works have examined the effect of TA on academic achievement.In a randomized control trial in Mongolia, researchers found that the provision of textbooks increased student scores, and that this improvement was intensified when textbooks were combined with teacher training (FUJE and TANDON, 2015).There is also evidence that the provision of textbooks improves student achievement (FREDRIKSEN et al., 2015;McEWAN, 2015).However, a randomized trial in Kenya showed that textbooks had a positive impact only for the bestperforming students (GLEWWE et al., 2009).Textbook choice also appears to have an impact on student scores.More engaging and demanding textbooks seem to increase scores more than less challenging ones (HADAR, 2017;HAM, 2018).

Methods and analytical sample
The synthetic control method The fundamental problem of causal inference is that once a policy intervention is implemented in a particular space and time, one cannot no longer assess how the outcome of interest would have developed in the absence of that intervention.SCM is employed in this investigation to overcome this limitation.
SCM is a causal inference method that has gained popularity over the last two decades.It has been called "arguably the most important innovation in the evaluation literature in the last fifteen years" (ATHEY and IMBENS, 2017, p. 09).This method was developed to estimate causal effects when there are few aggregate units, with one unit being treated while the others are not.In this context, a combination of non-treated units provides a better control than any single nontreated unit (ABADIE, 2021).
To understand how SCM is estimated, let us consider that we have data for J + 1 units and j = 1, 2, ..., J + 1.In this research, j varies from 01 to 27, since Brazil has 27 federative units, 26 of which are states and the other of which is the Federal District.j = 1 is the treated unit, Ceará.The non-treated units constitute the donor pool, that is, all the candidate control states, j = 2, ..., J + 1.For each time t and unit j, we observe the outcome of interest, student performance.Considering that we have T periods and that T0 refers to pre-intervention periods, the effect of the reforms in Ceará when t > T0 is given by: The method is concerned with how to optimize the choice of these weights.
Suppose  = ( 2 , … ,  +1 ) is a vector of weights assigned to each donor pool unit and 'V' is a vector of weights assigned to each predictor k. 'W' is defined, dependent on 'V', so that the mean squared prediction error (MSPE) between the treated unit and its synthetic version is minimized in the pre-intervention period9 .
The weights are chosen so that the synthetic control most closely resembles     The synthetic control findings In this section, SCM findings will be analyzed to further investigate the indications provided by the exploratory plots.I estimated 15 one synthetic control for each level of education and subject, resulting in six models.The donor pool comprised all Brazilian states, except for Ceará.The predictors are the state's characteristics, as presented in Table 02.
Table 03 shows the 'W' vector of each synthetic control.It indicates the four states, which contributed to the models: Bahia, Pernambuco, Piauí, and Rio Grande do Sul.The first three states are situated in the same region and share economic, social, and historical characteristics with Ceará.Therefore, it seems reasonable that these states contributed the most to the models.Rio Grande do Sul has distinct socioeconomic characteristics, but it only contributed substantially to the models for upper secondary education, where no statistically significant effects were found.In Figure 06  The graphs in Figure 07 show the difference between the score of Ceará and its respective synthetic version.Table 06 shows the results of TI alone and TI combined with TA.Even in lower secondary education, where TI had more time to develop its effect, the effects only increased substantially when TI was combined with TA.In primary education, over the period between 2011 and 2019, scores increased 16.9 and 18.5 points on average in mathematics and Portuguese,

Robustness checks
The in-time placebo synthetic control tests the robustness of the findings provided by SCM (Abadie 2021).In this test, an intervention starting in 1999 is ______________________________________________________________________________________________ 16 They were in the 9 th grade in 2008 (TI start) and reached the 12 th grade (Saeb exam) in 2011, assuming they did not fail any grade. 17artificially created to estimate whether SCM still shows any effect from this 'false' intervention.If this were the case, the validity of the results could be put into question.Figure 10 shows no significant difference between the trends after the artificial intervention.Moreover, even when the intervention is artificially backdated by nine years, the effects appear shortly after 2008 with very similar magnitudes as the ones presented in Figure 06.A second recommended test is a leave-one-out re-analysis to test whether the results are sensitive to any units selected to create the synthetic control (ABADIE, 2021;ABADIE et al., 2015).Table 03 showed that SCM selected four Brazilian states: Bahia, Pernambuco, Piauí, and Rio Grande do Sul.To check whether eliminating one of these states affects my results, I estimate four synthetic controls by selecting one of the contributing states from the sample one at a time.In Figure 11, the leave-one-out synthetic controls are shown in gray.They are very similar to the synthetic control estimated using the complete donor pool.All of  Finally, the generalized synthetic control method (GSCM) was employed to test the robustness of the findings.This method unifies the SCM with linear fixed effects models to improve efficiency and interpretability.It also avoids specification searches and provides p-values for inference (XU, 2017).The results are presented in Appendix C. Average effects provided by traditional SCM are within the confidence interval of effects provided by GSCM in primary and lower secondary education.However, point estimates and the statistical significance of GSCM findings in primary education are sensitive to the choice of predictors.Moreover, the method suggests statistically significant effects in mathematics, which are further analyzed in the results discussion.The method does not provide statistically significant improvements in upper secondary education.Inference for the SCM findings Abadie (2021) proposes a mode of inference based on permutation methods to assess inferential aspects of synthetic control estimates.In this mode, a permutation distribution is obtained by reassigning treatment to each unit in the donor pool one at a time.Each of these estimated synthetic controls produces a 'placebo effect'.All the placebo effects can then be compared to the effect estimated for the truly treated unit.The effect is only considered significant if it is extreme relative to the permutation distribution (ABADIE, 2021).
In Figure 12, the effects obtained with the treatment artificially reassigned to each donor pool unit are presented.The effect in Ceará is highlighted in green, while the effect in control states is shown in gray.The effects for Ceará are always  Many of the synthetic controls do not fit the pre-intervention data as well as the control estimated for Ceará.Therefore, to compare my synthetic control only with the ones that had similar pre-intervention mean squared prediction errors (MSPE), I excluded cases in which the MSPE was more than twice the MSPE of the synthetic control for Ceará.The results of this procedure can be seen in Figure 13.
For Portuguese scores, the findings are unusually large compared to the estimations for other states.For mathematics scores, however, the rarity of the effects remains unclear.To further investigate the significance of my findings for mathematics, I carried out a post/pre-intervention MSPE test suggested by Abadie et al. (2010).In this test, the post/pre-MSPE ratio distribution is plotted for all placebo gaps.This approach eliminates the need to choose an MSPE cut -off for evaluation.The idea is that a good synthetic control has a low error before the intervention because it closely fits the data.On the other hand, for the treated unit, the error is large after the intervention because there is an intervention effect.Therefore, I expect my estimated synthetic control post/pre ratio to be unusually high compared to the controls of non-treated states.
As Figure 14 shows, mathematics ratios are not sufficiently rare in the distributions.If one were to assign the intervention randomly in this data, the probability of obtaining a post/pre-intervention MSPE as large as the one for Ceará in primary education -mathematics -would be 8/27, or 0.3.For mathematics in lower secondary school, this probability is 5/27 = 0.19.Thus, the results are not statistically significant for mathematics.For Portuguese, the probability is 0.037 for both levels of education, and the estimates are statistically significant.In primary education, TI was implemented alone for a short period, and perhaps its effect would increase even without TA.However, in lower secondary education, TI had more time to develop its effects (2008)(2009)(2010)(2011)(2012)(2013)(2014), but still did not reach half of the effect observed when TI is combined with TA.This suggests that TA is a significant driver of the effects and that the policies produce better results when implemented together.

Results in mathematics
SCM indicates that improvements in mathematics scores are not statistically significant, as highlighted in Figure 14.On the other hand, robustness checks (Appendix C) and previous investigation suggests statistically significant effects also in mathematics (LAUTHARTE et al., 2021).It is important to highlight, however, that both the robustness check and Lautharte et al. (2021) employ ______________________________________________________________________________________________ 18 To make this comparison, the score increase of students between the 5 th and 9 th grades was calculated.Only students in states other than Ceará were considered.It was assumed that the students in 5 th grade in 2007 reached the 9 th grade in 2011.The same procedure was performed for 2011 and 2015; 2015 and 2019.These increases were then averaged and resulted in a 14.9 increase on average per year.Since one year of schooling in Brazil consists of 10 months (200 days) of effective schooling, a 14.9 increase was associated with 10 months of effective schooling. Better

Results in upper secondary schools
The absence of statistically significant impacts on upper secondary education is plausible because, while TI functions as an incentive for mayors, upper as well as on the later life of students.

Further limitations
One concern in this study is a pre-intervention upward trend in scores between 2005 and 2007 (Figure 01).This trend could indicate that a third factor, originating in 2005, could be impacting education in Ceará.However, this upward trend is also seen in the control states.A possible explanation for it is the rise in investment in education which occurred between 2005 and 2011 in all of Brazil.
This should not bias the results since factors that impact both treated and untread units are already accounted for in the synthetic control.Moreover, investment in education is included as a predictor in the models.
An additional issue is that TI might have led mayors to exert pressure over  the effects described here would not indicate real improvement in schooling.
However, I argue that this is not the case because this study employed Saeb scores (national assessment) to estimate the causal effects.In contrast, the government of Ceará employs SPAECE (a state assessment) to grant TI.
The mechanisms driving performance increase TI and TA appear to provide better results when implemented together.
Impacts of TI more than doubled when it was implemented combined with TA.This suggests TA drives a substantial share of the observed effects.Possible mechanisms behind TA are the trainings for teachers and school civil servants, the collaboration between schools, and the provision of textbooks.Further research is needed to confirm to what extent each of TA's actions contributed to the effects.
Regarding TI, it is plausible that the program's incentives increased the level of political priority accorded to education because mayors wanted to maximize their tax revenues.Local politicians might thus have felt more encouraged to improve school infrastructure, including libraries, science labs, and sports facilities.
These amenities, in turn, led to better performance.SPAECE, the state system of evaluation of primary and secondary education, understood as an annual central exam, could also drive part of the effects by increasing students' reward for studying and by strengthening the monitoring of schools (BISHOP, 1997;WOESSMANN, 2018).Finally, Figure 04 shows that Ceará is not among the states with the highest average spending on education, suggesting that higher spending is not a critical channel driving these effects.

Conclusion
This study provides evidence that incentives and technical assistance can This study contributes to the literature on the connection between technical assistance, notably the provision of textbooks and teacher training, and the quality of education.Furthermore, it provided empirical evidence of the effect of political priority for education on the level of schooling offered to students.
Recently, Brazil's Congress passed a constitutional amendment 20 , demanding that all Brazilian states condition tax transfers on educational outcomes.
The results presented here appear to support this constitutional change.
However, the lower municipalization rate observed in other Brazilian states might jeopardize the positive effects of TI, since fewer schools are under municipal administration than in Ceará.Another challenge is to overcome the resistance of mayors who might be wary of losing municipal revenues.
From a policymaking perspective, this study raises a relevant issue regarding the absence of improvements at the level of upper secondary schools.
Even though students had a better quality of schooling in primary and lower secondary schools, they did not experience improvements in upper secondary schools.Policymakers should debate incentives directed to governors, the authorities responsible for managing upper secondary education in Brazil.
The distribution of a federal tax could be conditioned to the educational outcomes of states in the same way that the distribution of the state tax, ICMS, is conditioned on the educational outcomes of municipalities.
Regarding the national system of education 21 , technical assistance and collaboration between the federal government, states, and municipalities could be tools to replicate the successful experience of Ceará.It is important to note, however, that most Brazilian states do not have a history of collaboration between the state government and municipalities comparable to that of Ceará.
This article is organized as follows.First, I discuss the interventions implemented in Ceará and analyze the existing literature on education.Then, I present my methodology, along with information on data sources and summary statistics.In the following section, I present, analyze, and contextualize my results.

Figure 01 .
Figure 01.Scores of students in primary and lower secondary schools from 1995 to 2019.
1995, the state passed the municipalization law.Its objective was to transfer the responsibility for primary and lower secondary schools to municipalities, thus clarifying the role of each sphere of the federation(SEGATTO, 2015).Moreover, Ceará established a forum on education and a permanent program for assistance to municipalities.These initiatives aimed to promote the democratization of access to education, cooperation, and the municipalization of primary and lower secondary education (NASPOLINI, 2001).Ceará's new policies produced remarkable results.Between 1995 and 2000, enrollment in primary and lower secondary schools increased by 35.4 percent in the state, whereas the increase in Brazil as a whole was of only 9.3 percent(NASPOLINI, 2001).Regarding municipalization, in 2018, 99.3 percent of the primary schools in Ceará were under municipal administration, compared to 83.5 percent in all of municipalities, Ceará established the 'Coordenadoria de Cooperação com os Municípios' (COPEM), the 'Coordination for Cooperation with Municipalities'.Figure 02 provides an overview of COPEM.At the state level, experts were hired to train 'specialist teachers', the name given to teachers responsible for disseminating skills and good practices in their municipalities.Each municipality had one local manager and several specialist teachers.The local manager was responsible for managing the actions and establishing communication with the SEDUC.Both local managers and specialist teachers could apply for financial support to improve their qualifications and skills (CRUZ et al.,2020).

Figure 02 .
Figure 02.Collaborative arrangements between the State government and municipalities

______________________________________________________________________________________________6
In 2020, Constitutional Amendment 108, from 2020, increased the proportion of ICMS over which states have discretion from 25 percent to 35 percent.The Amendment also requires that states condition at least 10 percentage points to performance in education.7After the passing of Constitutional Amendment 108, from 2020, Ceará approved the following new criteria: education (18 percent), health (15 percent), and environment (02 percent).This change does not affect the period studied in this investigation.8For details about   and   , consultLOUREIRO et al. (2020).Better Incentives, Better Marks: A Synthetic Control Evaluation of the Educational Policies in With regard to incentives for city mayors, LAUTHARTE et al. (2021) studied the interventions in Ceará, employing a regression with year and city fixed-effects.They restricted their sample to schools located at the border between Ceará and neighboring states to make control and treatment groups more similar.The findings showed that TI combined with TA improved student scores in mathematics and Portuguese tests.Carneiro and Irffi (2018) employ a difference-in-differences model to investigate the impact of TI in Ceará between 2007 and 2009.The findings are consistent with an increase of approximately 04 percent in mathematics and Angrist and Lavy (2001) found positive effects of teachers' in-service training on student scores in reading and mathematics tests.In line with these findings are those ofBressoux et al. (2009), who studied the effects of teacher training in French schools.Their estimates showed an increase of 0.25 standard deviations in mathematics scores, but no improvement in reading.
Ceará's outcome before intervention.The ability of SCM to estimate the counterfactual depends on how well it predicts the outcome of interest of the treated unit before the intervention.Importantly, SCM provides a transparent and datadriven methodology for choosing the control unit while avoiding specification search(ABADIE, 2021).In a different context, this method was employed to estimate the effects of homicide prevention measures in the Brazilian state of São Paulo(FREIRE, 2018).Analytical sample: summary and descriptive statisticsSaeb is the main source of data for this research.It consists of Portuguese and mathematics exams, and assesses students every two years: at the end of primary school (5 th grade), lower secondary school (9 th grade), and upper secondary school (12 th grade).The exams are carried out by the 'Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira' (INEP), an institute linked to Brazil's Ministry of Education.Saeb was created in 1990 and, in 1995, started to employ the Item Response Theory (IRT) to allow comparisons across time.Therefore, this study ______________________________________________________________________________________________ Better Incentives, Better Marks: A Synthetic Control Evaluation of the Educational Policies in Ceará, Brazil (2023) 17 (1) e0005 -14/44 uses data from 1995 until 2019 (INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA, 2020b) 10 .During this period, a random sample of the population was assessed (INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA, 2006).From 2007 onwards, all public schools with at least 30 students were tested (LAUTHARTE et al., 2021).Only 0.8 percent of basic education is managed by Brazil's federal government, most of which consists of upper secondary schools (INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA, 2020a).Since these schools receive greater financial and technical support, and since their teachers are usually better qualified than those working in state and municipal schools, they were not included in the sample.For the same reasons, private schools were also excluded.The sample thus only consists of schools run by states and municipalities.Each observation in the data set contains the average score of a state in a particular year, grade, and subject.I also created a dataset to control for Brazilian states' social, demographic, and economic characteristics.These indicators come from different sources.The population estimates are from the Ministry of Health 11 .Data on investment in education and industrial electricity consumption 12 are provided by the 'Instituto de Pesquisa Econômica Aplicada' (IPEA) 13 .Real values for investment in education and culture were determined by deflating nominal values against the consumer price index, using 2020 as the base year 14 .The real values were then divided by the population to generate a figure for investment per capita for each state.Homicides per 100,000 inhabitants were compiled with data from IPEA and Brazil's Ministry of Health.Finally, the unemployment rate was compiled with data from IPEA and the 'Instituto Brasileiro de Geografia e Estatística' (IBGE).Linear interpolation was used ______________________________________________________________________________________________ (2023) 17 (1) e0005 -15/44 to impute two missing values in electricity consumption and investment in education and culture.Table 02 presents the analytical sample statistics with standard deviations in parentheses.Before synthetic control findings are presented, exploratory plots will provide an overview of the analytical sample.Figure 03 presents the distribution of scores in mathematics and Portuguese tests before and after the intervention.Other states were more likely to have the best scores before the intervention.However, Ceará had better scores in the post-intervention period for both primary and lower secondary education.The same improvements were not observed in upper secondary education.

Figure 03 .
Figure 03.Density plots of the scores in mathematics and Portuguese

Figure 04
Figure 04 presents the change in test scores versus the average per capita investment in education and culture between 2007 and 2019.In primary and lower secondary education, Ceará achieves the highest score increase with a relatively low level of investment per capita.The plot suggests that Ceará was more efficient than other states.Once again, upper secondary education does not exhibit the same positive results.

Figure 04 .Figure 05 .
Figure 04.Score change by average investment between 2007 and 2019 , mathematics and Portuguese scores for Ceará and synthetic Ceará are presented.Before the intervention (left of the dashed lines), the synthetic control can emulate the performance of Ceará quite well.For all models, in the post-intervention period, a gap between Ceará and its synthetic version becomes progressively larger, indicating that the intervention had an effect.The yellow lines indicate the performance Ceará would have had in the absence of the reforms, while the green lines indicate the actual performance attained by the state.

Figure 06 .
Figure 06.Performance of Ceará vs. synthetic Ceará in primary and lower secondary education.

Figure 07 .
Figure 07.Gap between Ceará and synthetic Ceará in primary and lower secondary education

Figure 08 .
Figure 08.Performance of Ceará vs. synthetic Ceará in upper secondary education

Figure 09 .
Figure 09.Gap between Ceará and synthetic Ceará in upper secondary education They were in the 8 th grade in 2015 (TA starts in lower secondary education) and reached the 12 th grade in 2019.Better Incentives, Better Marks: A Synthetic Control Evaluation of the Educational Policies in

Figure 10 .
Figure 10.In-time placebo test with artificial intervention in 1999 a positive effect of the reforms; in some cases, they point to even larger effects.

Figure 11 .
Figure 11.Leave-one-out Test positive and continuously increase between 2008 and 2015.By contrast, most of the other models move randomly and show smaller effects compared to the ones observed in Ceará.

Figure 12 .
Figure 12.Score gaps in Ceará and placebo gaps in the 26 control states

Figure 13 .
Figure 13.Score gaps comparison with MSPE up to twice the MSPE for Ceará

Figure 14 .
Figure 14.Post/pre-intervention MSPE ratio Additionally, Lautharte et al. (2021) employ a reduced geographic and time span (2007 to 2017) compared to this study.One explanation for this divergence is that SCM fails to precisely fit Ceará's performance in mathematics before the intervention, which leads to a lower post/pre-intervention MSPE that is not sufficiently unlikely in the placebo post/preintervention MSPE distribution (Figure 14) 19 .
teachers to train students specifically to perform well in Saeb tests.If this happened, ______________________________________________________________________________________________19 Data on education performance in Brazil is available only since the beginning of the 1990s.SCM might not perform well with a small number of pre -intervention periods.Classic applications of SCM, like the estimation of the effect of California's tobacco control program(ABADIE et al., 2010), employ around 18 pre-intervention periods while this study had only seven periods, from 1995 to 2007, every two years.For a comprehensive discussion about this issue, please consult Abadie (2021).
effectively improve educational outcomes.It has been shown that TI and TA have led Ceará to experience substantial and robust improvements in test scores for Portuguese.Compared to baseline scores, there was an increase of around 12 and 6.5 percent in Portuguese scores in primary and lower secondary education, respectively.These findings present a promising alternative for other Brazilian states pursuing a better quality of education.The new educational policies combined Better Incentives, Better Marks: A Synthetic Control Evaluation of the Educational Policies in Ceará, Brazil (2023) 17 (1) e0005 -32/44provided a performance gain equivalent to approximately 12.4 and 9.7 months of effective schooling in primary and lower secondary education, respectively.These improvements were achieved without increasing public spending on education relative to other Brazilian states.

______________________________________________________________________________________________ 20 Constitutional
Amendment 108, 2020, available at ˂http://www.planalto.gov.br/ccivil_03/constituicao/emendas/emc/emc108.htm˃. 21PLP 235, 2019 is a bill currently being discussed in the Brazilian Congress.It aims to establish a national system of education to improve governance and collaboration between the federal government, states and municipalities.

Table 01 .
Responsibility for education in Brazil according to the national educational guidelines law.

Table 02 .
Summary statistics of the analytical sample Created by the author, based on data provided by IBGE, INEP, IPEA and SUS.

Table 03 .
Vector W showing the contribution of each state to each synthetic control Source: Created by the author, based on data provided by IBGE, INEP, IPEA and SUS.

Table 04
shows the 'V' vector.It indicates to what extent each predictor contributed to defining the synthetic controls.Industrial electricity consumption, homicides, and population seem to be the predictors with the strongest influence in the models.______________________________________________________________________________________________ 15 SCM was estimated with the R library 'Synth' (ABADIE et al., 2011).Better Incentives, Better Marks: A Synthetic Control Evaluation of the Educational Policies in Ceará, Brazil (2023) 17 (1) e0005 -20/44

Table 04 .
Vector V showing the contribution of each predictor to each synthetic control Source: Created by the author, based on data provided by IBGE, INEP, IPEA and SUS.

Table 06 .
Results by policy, level, and subject Source: Created by the author, based on data provided by IBGE, INEP, IPEA and SUS.
secondary education is under the state governor's responsibility.Moreover, the first students affected by TA only reached upper secondary examinations in 2019 and were only partially affected by TA.It is also reasonable that the effects dissipate over the 03 years of upper secondary education.Further research is needed to evaluate the long-term effects of the interventions on upper secondary education,