Poverty and municipalization of education: the analysis of ideb results (2005-2009)

Abstracts

O artigo apresenta resultados de pesquisa empírica que buscou verificar o impacto da pobreza sobre o Índice de Desenvolvimento da Educação Básica em escolas de ensino fundamental em mais de 5.500 municípios brasileiros, comparando o desempenho de escolas estaduais e municipais. Foram analisados os dados de 2005, 2007 e 2009 por meio de regressões lineares simples, resíduos de regressões e de correlações parciais. Os resultados mostram que a pobreza tem um forte impacto no desempenho escolar, chegando a explicar até 60% da variação. Os autores discutem os resultados obtidos vis-à-vis às atuais tendências de políticas educacionais, como as políticas fundamentadas em fatores de input e na responsabilização de professores, apontando suas limitações.

Políticas Educacionais; Municipalização Do Ensino; Pobreza; Brasil

Poverty and the municipalization of education: the analysis of IDEB results (2005-2009)

Christina W. Andrews; Michiel S. de Vries

^IAdjunct Professor II of the São Paulo Federal University - Unifesp -, Guarulhos Campus. E-mail: [e-mail]

^IIFull Professor, Radboud Universiteit Nijmegen - Holanda. E-mail: [e-mail]

ABSTRACT

This paper presents the results of an empirical research that sought to estimate the impact of poverty on the Brazilian School Performance Index - IDEB - for elementary schools in more than 5,500 Brazilian municipalities, comparing the performance of state and municipal schools. The investigation examines IDEB data for 2005, 2007, and 2009 through simple linear regressions, residue analysis, and partial correlation analysis. The results showed that poverty has a strong impact in students' performance, explaining up to 60 percent of the variation of IDEB scores. The authors discuss the results vis-à-vis the current trend in educational policies, such as those based on input factors and teachers' accountability, pointing out their shortcomings.

Keywords: educational policies; municipalities; poverty; brazil

One issue within contemporary debates on education seems to have achieved consensus both among researchers and policy makers: the problem of access to basic education has virtually been solved in most developing countries, and the challenge now is to enhance the quality of the teaching and learning process (WINKLER, GERSHBERG, 2000; HANUSHEK, WÖßMANN, 2007). This new consensus has had a profound impact on policies devoted to Education worldwide, and not only among developing or "emerging" countries, since central to their policies have become the evaluation of school performance, understood as the main indicator of quality of education. In the last 20 years, the United Kingdom and the United States have adopted strategies associating the accountability of teachers and managers with market mechanisms, leading to competition between public and private teaching institutions for government resources (RAVITCH, 2010; TOMLINSON, 2005). In Brazil, although we have only recently made access to primary and middle school universal, the issue of quality is increasingly driving the debates on educational policies.

This change in emphasis has also been reflected in educational research. In the last 20 years, hundreds of studies have been conducted in order to identify which school-related factors have impact school performance, but the results have been inconclusive (HANUSHEK, 2003, 1995). The effect of so-called input factors - teacher qualification, class size, infrastructure and the like - on school performance has not proven consistent, but varies according to the context in which the studies were conducted and the degree of aggregation of the variables that are taken into consideration. However, one factor has proven consistent in its impact on school performance: the socio-economic conditions of students' families. One of the best-known studies of its impact on school performance was the Coleman Report in the 1960s, which published the results of a comprehensive and detailed study into factors affecting performance in American schools (COLEMAN et al., 1966). The researchers applied statistical controls to eliminate the effect of families' social and economic characteristics and concluded that school features had a very small impact on students' performance. In other words, the family's socio-economic background proved to be the factor best explaining school performance, while other factors such as the number of students per class, teacher qualifications, school equipment, and so on, proved not to be significant. Since then several studies have confirmed the results of the Coleman Report. Strangely, however, educational policies in the USA - as well as the United Kingdom -have sought to promote school performance through interventions in input factors, above all through the accountability of schools and teachers. The situation is scarcely different in other countries. In the case of Brazil, after the emergence of performance indicators, several states and municipalities adopted accountability policies; however, very little is known about the effect of these policies in the medium and long-term (BROOKE, 2008). In any case, the impact of families' socio-economic background has not been taken into consideration in such policies.

Another issue with implications for the current debate on educational policies in Brazil is the role of decentralization. Some analysts have stated that decentralization has a positive impact on the quality of education. These authors feel that decentralization may increase parents' participation in, and control over, educational services, particularly when combined with market mechanisms (HANUSHEK, 2003; WEST, 1997; WINKLER, GERSHBERG, 2000). However, critics of this perspective argue that the decentralization of education - above all in the developing world - has merely been a corollary of structural adjustment programs promoted by the World Bank, often clashing with its declared goal of combating poverty (BONAL, 2004; CARNOY, 1997; KAMAT, 2002). In turn, critics of the educational reforms in the United Kingdom and the USA have argued that in these countries, they have entailed greater central government control over schools, despite the rhetoric on the promotion of autonomy (TOMLINSON, 2005). Winkler and Gershberg (2000) believe that decentralization can enhance the quality of teaching in developing countries but admit that it is hard to check whether there has actually been a positive impact. The difficulty is threefold: firstly, developing countries often lack information on school performance and indicators of school quality; secondly, the results of educational policies appear slowly in response to interventions; and thirdly, it is not easy to control the effects of "external shocks" (natural catastrophes, teachers' strikes, financial crises, leadership changes, and so on). For this reason, the authors have decided to assess the impact of decentralization on education in some countries by analyzing factors that characterize high-performance schools, such as strong leadership, highly qualified teachers, staff dedication and a focus on results. However, this indirect approach is not satisfactory since, as we have noted, there is no established consensus as to which features of schools have a significant impact on the quality of education.

The issue of decentralization in education in Brazil must therefore be addressed using different assumptions. First of all, central government has played a small role in delivering basic education in Brazil since the twentieth century, and this responsibility has been taken on by the states. Federal regulation has always been present, but since the enactment of the Lei de Diretrizes e Bases da Educação Nacional - LDB [National Education Foundations and Guidelines Act] (BRASIL, 1996a), states and municipalities have gained greater autonomy in preparing curricula. Decentralization in education in Brazil may thus be best understood as "municipalization". Universal access to education, promoted since 1997 by means of the Fundo de Manutenção e Desenvolvimento do Ensino Fundamental e de Valorização do Magistério - FUNDEF [Primary Education Funding System], has created strong incentives for the municipalization of educação fundamental [primary and middle school] (BRASIL, 1996a, 1996b). As we shall see in detail below, despite the FUNDEF incentives, the provisions of the Constitution (BRASIL, 1988) and the LDB, the municipalizing process of fundamental education is not yet complete.

In this article, we seek to assess the process of municipalization of primary and middle school from the point of view of the impact of poverty on school performance. We pose the following questions: (a) what is the impact of poverty on the performance of the municipal schools vis-à-vis State schools? (b) Are there municipalities that stand out because of their ability to overcome the constraints imposed by poverty and thus achieve good school performance? (c) What input factors can explain the school performance of Brazilian municipalities? We have adopted three statistical techniques to answer these questions: regression analysis, residual analysis, and partial correlation analysis. These questions, however, are not the whole story: results must be evaluated in the light of the contemporary debate on educational policies. In the next section, we therefore present and discuss the results of the statistical analyses; in the following one, we reflect on the implications of these results for educational policy in Brazil and, lastly, we conclude with some final remarks.

POVERTY, MUNICIPALIZATION AND SCHOOL PERFORMANCE

FUNDEF, which came into force in 1997, proved an efficient mechanism for promoting enhanced enrollments in fundamental education, as data before and after its introduction show. In 1, 86.6% of 7-to-14-year-old children were enrolled in school; in 2007, this rose to 97% (HENRIQUES et al., 2009; SILVA, ALCÂNTARA, 2009). In the north-east of Brazil, the country's poorest region, enrollments for this age group rose from 75% in 1991 to 96% in 2002 (DE MELLO, HOPPE, 2005). However, although Brazil has virtually made access to school universal, it is estimated that only 60% of students enrolling in the first year complete the eight years of compulsory education (fundamental education") (HENRIQUES, GIAMBIAGI, VELOSO, 2009).

FUNDEF, however, has not had a positive impact on students' school performance, as Sistema de Avaliação da Educação Básica - SAEB [Basic Education Evaluation System] results show. As it can be seen in the graph shown in figure 1, the performance of students completing the 4th and 8th grades fell after the introduction of FUNDEF, with a trend toward recovery beginning in 2001. It should be noted that in the smaller and poorer municipalities, FUNDEF has led to a more rapid increase in enrollments than observed in medium and large municipalities (DE MELLO, HOPPE, 2005). In itself, this information shows that the fall in school performance is due to the impact of poverty. Increased enrollments in small towns brought millions of children from low-income families into the school system for the first time. As we shall see below, the fall in school performance was greater in municipal than in state schools, precisely because the former received the larger contingent of students from poor families. This backs up the argument over the strong impact of socio-economic background on school performance found in studies carried out in other countries (ERMISCH, FRANCESCONI, 2001; LEE, BARRO, 2001; HANUSHEK, 1995; COLEMAN et al., 1966).

Thanks to the availability of data for the Índice de Desenvolvimento da Educação Básica - IDEB [Basic Education Development Index] from 2005, educational performance can be consulted for virtually all of Brazil's municipalities. IDEB is a compound index calculated for three periods: from the 1st to the 4th year, from the 5th to the 8th year and at the end of the 3rd year of secondary education. It ranges from 0 to 10, and includes two indicators: standardized grades for tests of mathematics and reading (Prova Brasil), and the average pass rate (FERNANDES, 2007). Linking these two components, IDEB expresses a compensating mechanism between exam performance and pass rate. A 10% fall in the average pass rate must thus be counterbalanced by a 10% rise in the average examination performance for IDEB to remain constant.

Table 1 shows IDEB results for fundamental education in Brazil for the three data-gathering waves: 2005, 2007 and 2009, as well as projections by INEP (see above), an agency of the Ministério da Educação - MEC [Ministry of Education]. INEP's performance goal is to achieve an overall Brazilian average score for IDEB of 6.0 at the end of the 4th year, which is considered equivalent to the average performance of students in developed countries as measured by PISA (FERNANDES, 2008). Actual results both for state and municipal schools, except for private schools, exceed the goals established for 2007 and 2009.

Table 2, in turn, shows the number of municipalities in the IDEB database for the 2009 data-gathering exercise. As it can be seen, although municipalization has advanced by the 4th year, it has not advanced significantly for the 5th to the 8th-year period. For education up to the fourth year, municipalization has been completed in nearly 40% of the municipalities; for 5th to 8th-year education, municipalization has been completed in only 12.9% of municipalities. Most municipalities continue to deliver fundamental education through municipal and state schools both for the 1st to 4th year group and for the 5th to 8th-year group - 52.6% and 47.1%, respectively.

SIMPLE LINEAR REGRESSION ANALYSIS

To measure the impact of poverty on school performance, we carried out simple regression analyses in which IDEB scores were the dependent variables, and the level of poverty in the municipality, as measured by the percentage of inhabitants of the municipality living on incomes lower than half of one minimum salary, was the independent ones. We also tested other 13 social variables that might have an impact on school performance in a major component analysis in order to arrive at one and single independent variable. This analysis resulted in two factors: variables expressing wealth/poverty and Theil's inequality index. However, there is no clear linear correlation between IDEB scores and values for Theil's index, as can be seen on analysis of the scatter chart. This is explained by the fact that thousands of Brazilian municipalities show low rates of inequality because they are uniformly poor. Simple regression with a single variable - the level of poverty in the municipality - thus proved most suitable for the purposes of this study. We can express the impact of poverty on school performance using this equation:

Ideb_i (m₄, m₈, e₄, e₈) = β₀ β₁P_i (m₄, m₈, e₄, e₈) + ε_i

where:

• Ideb_i (m₄, m₈, e₄, e₈) is the predicted IDEB score for both types of school (m = municipal and e = state), and the two years considered (4^th and 8^th);

• P_i (m₄, m₈, e₄, e₈) is the percentage of impoverished individuals in each municipality, for the schools and years studied;

• β₀ is the value of the constant, corresponding to the intercept of the straight line;

• β₁ is the coefficient expressing the linear regression gradient;

• ε_i is the estimated error.

Table 3 shows values for R2, Beta (standardized coefficient) and t-test scores for simple linear regression corresponding to each type of school and year data. We can see that in all cases, there is a significant negative linear association between IDEB scores and the level of poverty in municipalities, in other words, the larger the number of individuals living below the poverty level in the municipality, the lower is the school performance. R2 and Beta values show that there is a greater impact of poverty on school performance in municipal schools up to the 4th year, and in 2005 more than 60% of variation in IDEB can be explained by the variation in the level of poverty in the municipality. Although this impact diminished after 2005, it was still strong in 2009 (54%). In general, school performance in municipal schools is more strongly affected by poverty than performance in state schools. This result bears out the argument put forward above that the fall in school performance after 1997 occurred because of the inclusion of students from poorer families in the education system.

RESIDUAL ANALYSIS

Are the municipalities able to overcome the limits imposed by poverty on school performance? In order to answer this question we analyzed the residues of linear regression for municipal schools using the IDEB results for the 4th year. This year was chosen not only because it was the most strongly impacted by the level of poverty, but also because in many places, municipalization of primary and secondary education only took place up to this year. Residual analysis may be used to identify out-performers or under-performers in a given context (DE VRIES, 2009; DE VRIES, LAKO, 2009). The technique is based on the attributes of a normally distributed population, where 99% of the standardized residues - ZRE - are located between 2.5 and -2.5; thus, within this range, variations observed are deemed random. Cases with standardized residues above 3 and below -3, however, are considered outliers; the former indicates the out-performers and the latter the under-performer's ones.

Figure 2 shows the final results of residual analysis; only municipalities presenting ZRE scores above 3 and below -3 in at least two waves of data have been taken into consideration. By this criterion, seven municipalities may be classified as outperformers: Barra do Chapéu (São Paulo [Sao Paulo State]), Boa Vista do Tupim (Bahia), Cajuru (São Paulo), Itápolis (São Paulo), Lajinha (Minas Gerais), Ouro Verde (São Paulo) and Santa Rita d'Oeste (São Paulo). The only municipality consistently showing a worse-than-expected performance was Toritama (State of Pernambuco). It should be pointed out that the number of out-performers and under-performers varied greatly year-on-year. In 2005, 16 municipalities had ZREs above 3, while nine had ZREs below-3; in 2007, 35 municipalities had better-than-expected performances, with ZREs above 3, and only four had ZREs below -3; finally, in 2009, 31 municipalities had ZREs above 3, and three had ZREs below -3. It should also be noted that six of the seven out-performers are in Brazil's most highly-developed region, the south-east.

Table 4 presents a comparison of school performance and demographic and socio-economic parameters for the seven out-performers. Except for Barra do Chapéu (São Paulo) and Boa Vista do Tupim (Bahia), the remainder has fewer than 36% of their residents classed as poor; furthermore, all out-performers are small or very small municipalities. The relatively low level of poverty found among most out-performers bolsters the argument that poverty is indeed a very important factor in determining school performance.

Since few municipalities showed a consistent performance, this shows that IDEB scores are also subject to the temporary fluctuations seen in the performance tests of other countries. Kane and Staiger (2002) note that the results of school performance through evaluation tests, apart from showing fluctuations due to sampling variations, have several other sources of variation. These variations "may cause temporary fluctuations in the performance of an entire school, such as a dog barking in the school parking lot, bad weather on exam day or curriculum differences that interact in different ways with the types of exam used" (p. 95). Since nearly 55% of Brazil's municipalities have no more than three municipal schools, momentary fluctuations in a single school may affect the IDEB score for the whole municipality.

ANALYSIS OF PARTIAL CORRELATIONS

How far can input factors - in other words, school-relevant factors - explain school performance in Brazil? In order to assess this question, we used partial correlations to examine the impact of the number of primary education teachers in a municipality with university diplomas (BRAZIL, s. d.[e]) on IDEB, controlling statistically for the correlation, in order to discount the effect of poverty. The results are presented in table 5, and show that teacher's qualifications - at least qualifications corresponding to university diplomas - explain less than 1% of the variation in IDEB scores when the effect of poverty is discounted. This suggests that the impact of poverty is strong enough to neutralize the effect of input factors commonly associated with educational quality. A discussion of the so-called Heyneman-Loxley effect is highly relevant for this issue.

Heyneman and Loxley (1983), using data gathered in 1970, found that the quality of schools - measured by input factors - was more important in explaining students' performance in low-income countries than in high-income countries. The authors argued that in those countries the effect of families' social and economic conditions would be partially counterweighted by school quality factors. The so-called Heyneman-Loxley effect is therefore, typical of less-developed countries and may be explained by the relative scarcity of supply of education: if education is a service with low levels of supply, students would be more motivated, whatever their socio-economic level, resulting in a better-than-expected performance for the socio-economic standard of the country. Baker, Goesling and Letendre (2002) used a mix of high and low-income countries similar to that used in the Heyneman and Loxley study, seeking to ascertain whether the conclusions of the latter authors were still valid in the 1990s. Their results showed that the Heyneman-Loxley effect had disappeared, and the effect of families' socio-economic background was similar to what was seen in the classic study by Coleman et al. (1966). The authors observed that "for all the countries in the sample, variables pertaining to families' socio-economic circumstances are much more significant in predicting school performance than variables pertaining to the school's resources" (2002, p. 303-304). Furthermore, the authors stated that the effect observed in the 1970s probably reflected inequalities in access to education. In fact, when places in schools are scarce, only students in better socio-economic circumstances have access to these places. Once the scarcity is eliminated, the impact of this variable becomes apparent. For this reason, the idea that public schools were much better in the past should be treated with caution: the scarcity of places masked the impact of socio-economic conditions, which is no longer possible after access to primary education has become universal.

IMPLICATIONS FOR EDUCATIONAL POLICIES

In this context - Brazil has almost achieved universal access to primary education and aims to improve school performance - what are the most suitable policies? As it has been mentioned, practically all educational policies have ignored the impact of the socio-economic factor on school performance. While the United States and the United Kingdom are among the few developed countries that have adopted accountability policies associated with market mechanisms, the World Bank has advocated universal adoption of these policies. In a study available on the institution's website, Hanushek and Wö mann (2007) propose that educational policies must combine three inter-related strategies in order to improve the quality of school learning: to favor competition between schools; to give schools greater autonomy and to introduce an accountability system.

It is impossible within the scope of this study to present an exhaustive review of the ongoing controversy as to the use of market mechanisms in educational policies; for our purposes, it is enough to point out two aspects of the argument. Firstly, if there is no empirical evidence that a school's characteristics can improve school performance- and in fact, this is the conclusion of a wide-ranging review of studies carried out by Hanushek (2003) - competition between schools does not make sense. For there to be competition one must presuppose that certain schools have superior characteristics to others. However, as the Coleman Report showed, differences between schools are reflections of differences in students' socio-economic status rather than of schools' intrinsic features (COLEMAN et al., 1966). In the case of Brazil and other large middle-income countries with thousands of small impoverished municipalities, promoting competition between schools is simply unfeasible owing to the opportunity costs involved. In Brazil 54.7% of municipalities have up to three municipal schools; these municipalities obviously do not have the human, financial or administrative resources for competition between schools to make the slightest sense.

The two other recommendations made by Hanushek and Wö mann - autonomy and setting up an accountability system - are hard to argue against, since the terms have acquired positive connotations in public opinion and even among certain specialists, shielding them against criticism. However, it should be noted that in the proposal put forward by Hanushek and Wö mann, autonomy merely means "the autonomy to compete"; not the autonomy to adopt teaching practices and curriculum content suitable to the social setting in which the school is located. Within the proposal grounded on market mechanisms, autonomy presupposes competition since it only exists in order for "the best school to win".

Accountability policies - whether associated with market mechanisms or otherwise - have begun to be criticized by specialists, particularly those assessing the U.S. No child left behind - NCLB program. Ravitch (2010) argues that one of the consequences of NCLB has been a narrowing of curriculum content since school performance evaluations have focused on reading skills and mathematics; studies have shown that several schools have cut the hours dedicated to other subjects such as social studies and sciences in order to include training classes for the state and federal evaluation examinations. Another negative consequence has been fraud in evaluation exams. In the NCLB program, the consequences of poor school performance may be dramatic, including not only the loss of federal resources, but also the closure of schools and the firing of teachers; for this reason, the policy is being called high-stakes testing. Therefore, fraud has become "attractive'. According to Ravitch (2010), manipulation of performance test results has ranged from refusing to enroll students from families with low socio-economic indicators, through the prior selection of those students who will take evaluation tests, to fraud pure and simple. In July 2011, The Economist published an article on the fraud scandal in evaluation tests in schools in Atlanta, Georgia. Several types of fraud were identified according to the article.

"Some teachers gave pupils answers. Some teatchers filled in answers themselves. Some pointed to answers wilhe standing over pupils' desks. Others let low-scoring children sit near - and copy from - higher-scoring ones. One group of teachers had a test--changing party over the weekend" (ATLANTA'S, 2011, p. 32)

This fraud has far-reaching consequences since it erodes the credibility of the performance indicators. As "Campbell's Law" puts it: "The more a quantitative social indicator is adopted for decision-making in public policy, the greater will be the pressure toward corruption and the more this indicator will be used to distort and corrupt the social processes it should monitor" (CAMPBELL, in RAVITCH, 2010, location 2947-2955).

The experience of public accountability policies in Brazil is still too recent for any conclusive evaluation of its results. However, the adoption of certain elements that are typical of high-stakes testing may entail consequences similar to those previously observed in NCLB. IDESP, the school performance indicator used to assess schools in the State of São Paulo, is already being used to distribute monetary incentives to teachers and employees through the Bonificação por resultados program [Bonuses for results] (SÃO PAULO, 2008). According to the program's guidelines, each state school is given a yearly target, which is calculated assuming the expected performance level of all schools in 2030 (SÃO PAULO, 2011a). The schools most highly impacted by poverty - in other words, those with the lowest indicators - are precisely those that will have to show the biggest annual progress (SÃO PAULO, 2011b). Like other accountability policies, the bonuses for the results of a given program ignore the impact of poverty and the variation in the results of evaluation examinations, which may occur owing to factors beyond schools' control. That is why the program runs the risk of confusing teachers and discouraging the adoption of pedagogical practices that can only produce aimed results in the long-term.

Despite the redistributing component that characterized FUNDEF - which was maintained in FUNDEB, the fund that succeeded it in 2007 (BRASIL, 2007) - the federal government's educational policies have also given scant attention to the impact of poverty on school performance. Still, it has undertaken to invest R$1 billion annually in a program to provide undergraduate-level distance-education courses to primary teachers (MEC, 2009). Traditional educational programs, however, were not expanded. The Programa nacional do livro didático - PNLD [National Textbook Program], begun in 1929, Brazil's oldest program to enhance school performance, spent R$577.6 million on books and R$112.8 million to distribute them to public schools (BRASIL, s.d.[c]). Not much is known about the impact of this program on student performance throughout Brazil, but evidence from studies conducted in other countries suggests that the availability of books is an important factor in school performance (HEYNENMAN, FARREL, SEPULVEDA-STUARDO, 1981). Fuchs and Woessmann (2004) showed that there is a positive association between the presence of books at home and school performance. However, PNLD rules demand that students - except for those enrolled in the first year of primary education - return used books at the end of the school year; after three years the school is authorized to purchase new books and the old are discarded. This evidence, albeit incipient, suggests that PNLD should be expanded to allow students to keep their textbooks. Such a program could be tested on a small-scale and if results are positive could then be extended to all public schools. Another recurring problem is that educational policies in Brazil are usually "all or nothing" and, also, rarely introduced after pilot projects to test the impact of specific policies.

In the context of this discussion, Programas Condicionais de Transferência de Renda - CTR [conditional income transfer programs], which became widespread in Brazil in the mid-1990s, are worth mentioning (ANDREWS, 2004). The Bolsa Família [Family Allowance], launched in 2003 based on the Bolsa Escola [School Allowance] experience, is currently the world's largest income transfer program, giving monetary subsidies to more than 12.5 million families (BRASIL, s.d.[d]). The program standards lay down that beneficiaries must keep their children at school until the age of 15; they must also meet other health-related demands. However, the impact of the Bolsa Família on dropout rates has been small. Glewwe and Kassouf (2008) concluded that the Bolsa Família accounted for a 0.5 percentage point fall in dropout rates in the first four years of elementary school and 0.4 percentage points for the last four years of primary schooling. The authors argue that the effect of the program on the dropout rate among children of beneficiary families is up to three times greater than the observed overall impact, since only one-third of Brazilian children are served by the program. Nonetheless, the Bolsa Família has an unimportant impact on school dropout rates: 1.5 percentage points for the 4th year and 1.2 for the 8th year. This would suggest that municipalities are not enforcing the conditions of the program as the federal government requires. The Bolsa Família has been considered highly efficient in reducing poverty and inequality, achieving important results in the North-east (HOFFMANN, 2006).

Even if an improvement in inspecting the program led to a significant fall in school dropout rates, one cannot expect a similar impact on school performance measured by standardized tests, since such programs do not affect the cultural conditions associated with poverty, above all low levels of schooling among students' parents. CTR programs could have a favorable impact in the long run, since one expects that increased schooling would have a positive impact on the next generation, creating a virtuous cycle. The Bolsa Família would need to be able to have a beneficial impact on school dropout rates for this to occur, which does not seem to have been the case to date. As we have noted, 60% of children are currently not completing the eight years of basic education.

We have seen that municipal school performances have improved since 2005, and some of the goals established for 2009 were achieved earlier than expected. Some hypotheses might explain these results. From 2003 to 2009, poverty fell by 45% in Brazil, above all due to increased formal employment (DANTAS, NERI, 2010). One initial hypothesis, therefore, is that the reduction of poverty had a relatively rapid impact on educational performance. However, this estimate was based on data for metropolitan regions (NERI, 2008, 2009); it is not yet known how much poverty fell in small municipalities. Another hypothesis capable of explaining improved IDEB scores is that schools have become familiar with the structure of the Prova Brasil (multiple-choice tests) and are coaching their students before applying the tests. A third hypothesis has to do with the same phenomenon as was observed in the United States - adulteration of evaluation test results. Finally, there is the hypothesis that teaching quality is actually improving and that this is being reflected in the results of school performance evaluation. In this case, these educational improvements - we do not know which ones they are - could be managing to break down the barrier of students' socio-economic backgrounds. It would be reasonable to suppose that all four hypotheses are partially true, but we do not know which of them has had the greatest impact on the results observed since 2005. Without further studies to identify the reasons behind the improvements seen in recent years, interpretation of the IDEB results and other performance indicators becomes increasingly difficult. The creation of IDEB and its dissemination throughout almost all of Brazil's schools is fundamental to enable educational policies to be prepared, but educational authorities must remain attentive to inherent constraints and risks in the use of indicators in creating public policies, above all with regard to high-stakes testing-based strategies. Be that as it may, the impact of these socio-economic factors on school performance cannot be ignored.

CONCLUSIONS AND FINAL REMARKS

In this study, we have analyzed the impact of poverty on school performance in Brazilian municipalities, considering the context of the municipalization of basic education in Brazil after the introduction of FUNDEF. Analysis of the data has shown that municipal schools, accounting for most of the increase in enrollments in basic education for the period under consideration, are more seriously affected by poverty than state schools. This result should not be ascribed only to their own characteristics, but rather to the fact that FUNDEF created incentives for municipalities to open new schools, increasing the number of enrollments and bringing children from poorer families into the school system. The universal access to education promoted by FUNDEF established enhanced education quality as the next step, laying the foundations of a goal that its intrinsic strategy - increased enrollments - will be unable to attain. This has implications for the municipalization of education: in itself, it cannot promote improvements in school performance.

Given the results of the present study, we can perceive the importance of the conclusions of the Coleman Report (1966) for Brazil and other emerging economies. Brazil can no longer be deemed a "developing" country, but it is still a country marked by poverty and inequality. Brazil's socio-economic context is closer today to what is found in the USA and other developed countries because these countries themselves have become poorer and more unequal in recent decades. So, the socio-economic background, both in these countries and in Brazil, is still the main explanatory factor for school performance levels. For 30 years, the theory of human resources supported the thesis that poverty could simply be overcome through mass education systems (EASTERLIN, 1981). Today, its advocates have begun to argue that poverty can only be overcome by improving the quality of education (HANUSHEK, WÖßMANN, 2007). However, it should be recognized that improved quality of education also entails overcoming poverty.

The impact of poverty on school performance is a great challenge for the Brazilian political class. On the one hand, current education policies are geared towards improving input factors or - what is more troubling - to assume that teachers' accountability is the key to promoting quality in education. Nevertheless, if the impact of poverty is not taken into consideration, educational policies in Brazil run the risk of adhering to typical "symbolic policy" practices, in other words, they will be increasingly driven by public opinion expectations rather than by the objective fundamentals of social reality (DE VRIES, 2010). This will be very likely limit alternatives for public policies. Local economic development programs could possibly have a greater impact on school performance than educational policies based on input factors or on the accountability of schools and teachers. If the belief that governments should resort only to the latter type of policies persists, the array of possible interventions will remain limited. At a certain moment of the debate, an expectation was created that educational policies should produce short-term results. This led politicians to rapidly "cycle through" educational policies, discarding supposedly inefficient policies for novelties. However, "novelties" wear off and the next will soon need to be invented.

Let us be clear: improvement in school performance is a desirable goal, but quality schools are an even more desirable goal. Perhaps what has been missing in this debate is a consideration of education as a social right, in its most basic meaning. All Brazilian students should have the right to qualified teachers and access to textbooks, to study in pleasant settings, and to enjoy other features associated with good schools. These characteristics must be promoted by policy-makers as a way of providing every child and adolescent with opportunities. Educating is a complex activity and must not be carried out merely by manipulating a dozen or so inputs as a production function, or by holding schools and teachers accountable. In brief, performance indicators must be seen as ancillary instruments of educational policies and not as parameters to which the policies must unthinkingly submit.

REFERENCES

ANDREWS, Christina W. Anti-poverty policies in Brazil: reviewing the past ten years. International Review of Administrative Sciences, v. 70, n. 3, p. 477-488, 2004.

ATLANTA'S Public schools: low marks all round. The Economist, p. 32, 16-22 July 2001.

BAKER, David P.; GOESLING, Brian; LETENDRE, Gerald K. Socioeconomic status, school quality, and national economic development: a cross-national analysis of the "Heyneman-Loxley Effect" on mathematics and science achievement. Comparative Education Review, v. 46, n. 3, p. 291-312, 2002.

BONAL, Xavier. Is the World Bank education policy adequate for fighting poverty? Some evidence from Latin America. International Journal of Educational Development, n. 24, p. 649-666, 2004.

BRASIL. Constituição (1988).Constituição da República Federativa do Brasil. Diário Oficial [da] República Federativa do Brasil,5 out.de 1988. Disponível em: <http://www.planalto.gov.br/ccivil_03/constituicao/constituicao.htm>. Acesso em: 3 ago. 2011.

______. Emenda constitucional n. 14, de 12 setembro de 1996. Modifica os arts. 34, 208, 211 e 212 da Constituição Federal e dá nova redação ao art. 60 do ato das disposições constitucionais transitórias. Diário Oficial [da] República Federativa do Brasil, 13 set. 1996a. Disponível em: < http://www.planalto.gov.br/ccivil_03/constituicao/emendas/emc/emc14.htm>. Acesso em: 3 ago. 2011.

______. Lei n. 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Diário Oficial [da] República Federativa do Brasil, 20 dez. 1996b. Disponível em: <http://www.planalto.gov.br/ccivil_03/leis/l9394.htm>. Acesso em: 3 ago. 2011.

______. Lei n. 9.424, de 24 de dezembro de 1996. Dispõe sobre o Fundo de Manutenção e Desenvolvimento do Ensino Fundamental e de Valorização do Magistério, na forma prevista no art. 60, § 7º, do Ato das Disposições Constitucionais Transitórias, e dá outras providências. Diário Oficial [da] República Federativa do Brasil, 26 dez. 1996c. Disponível em: <http://www.planalto.gov.br/ccivil_03/leis/l9424.htm>. Acesso em: 3 ago. 2011

______. Lei n. 11.494, de 20 de junho de 2007. Regulamenta o Fundo de Manutenção e Desenvolvimento da Educação Básica e de Valorização dos Profissionais da Educação - Fundeb, de que trata o art. 60 do Ato das Disposições Constitucionais Transitórias; altera a Lei n. 10.195, de 14 de fevereiro de 2001; revoga dispositivos das leis n. 9.424, de 24 de dezembro de 1996, 10.880, de 9 de junho de 2004, e 10.845, de 5 de março de 2004; e dá outras providências. Diário Diário Oficial [da] República Federativa do Brasil, 21 jun. 2007. Disponível em: <http://www.planalto.gov.br/ccivil_03/_ato2007-2010/2007/Lei/L11494.htm>. Acesso em: 3 ago. 2011.

BRASIL. Ministério da Educação. Plano nacional de formação de professores. Brasília, s.d. [b]. Disponível em: <http://portal.mec.gov.br/index.php?option=com_content&view=arcle&id=13583&Itemid=970>. Acesso em: 18 set. 2009.

______. Programas livro-didático. Brasília: FNDE, s.d.[c] Disponível em: <http://www.fnde.gov.br/index.php/programas-livro-didatico>. Acesso em: 31 maio 2010.

BRASIL. Ministério da Educação. Instituto Nacional de Pesquisas Educacionais Anísio Teixeira. Ideb: índice da educação básica. Brasília, s.d.[a]. Disponível em: <http://portalideb.inep.gov.br/index.php>. Acesso em: 12 set. 2011.

______. Saeb 2005: primeiros resultados: médias de desempenho do Saeb/2005 em perspectiva comparada. Brasília, 2007. Disponível em: <http://www.inep.gov.br/download/saeb/2005/SAEB1995_2005.pdf>. Acesso em: 12 abr. 2009.

BRASIL. Ministério do Desenvolvimento Social. Relatórios e estatística: sistema de informações da Senarc. Brasília: MDS, s.d.[d]. Disponível em: <http://www.mds.gov.br/bolsafamilia>. Acesso em: 1^o jun. 2010.

BRASIL. Ministério do Planejamento. Instituto de Pesquisas Econômicas Aplicadas. Ipeadata. Brasília, s.d. [e]. Disponível em: <http://www.ipeadata.gov.br>. Acesso em: 19 set. 2011.

BROOKE, Nigel. Responsabilização educacional no Brasil. Revista Iberoamericana de Evaluación Educativa, v. 1, n. 1, p. 93-109, 2008.

CARNOY, Martin. Is privatization through education vouchers really the answer? A comment on West. The World Bank Research Observer, v. 2, n. 1, p. 105-115, 1997.

COLEMAN, James Samuel et al. Equality of educational opportunity: summary report. Washington: Department of Health, Education and Welfare, Office of Education, 1966.

DANTAS, Fernando; NERI, Marcelo. Demos os pobres ao mercado. É hora de dar o mercado aos pobres: entrevista com Marcelo Neri. O Estado de S. Paulo, 27 set. 2010. Disponível em: <http://www.estadao.com.br/estadaodehoje/20100927/not_imp615802,0.php>. Acesso em: 28 set. 2010.

DE MELLO, Luiz; HOPPE, Mombert. Education attainment in Brazil: the experience of Fundef. Paris: OECD, 2005. (OECD Economics Department Working Papers, n. 424). Disponível em: <http://www.oecdilibrary.org/oecd/content/workingpaper/ 556807485718>. Acesso em: 12 abr. 2010.

DE VRIES, Michiel S. Between symbolic and evidence-based policies: the Brazilian efforts to increase the quality of basic education. In: INTERNATIONAL CONGRESS OF ADMINISTRATIVE SCIENCES, 28., July 2010, Bali. Bali: IIAS, 2010.

______. Performance measurement and the search for best practices. In: INTERNATIONAL SPECIALISED CONFERENCE OF THE INTERNATIONAL INSTITUTE OF ADMINISTRATIVE SCIENCES, 5., 2009, Helsinki. Helsinki: International Institute of Administrative Sciences, 2009.

DE VRIES, Michiel S.; LAKO, Christiaan J. Adapted performance measures and the performance of Dutch municipalities in reducing crime nates. Journal of Regional Studies and Development, n. 18, v. 1, p. 29-57, 2009.

EASTERLIN, Richard A. Why isn't the whole world developed? The Journal of Economic History, v. 41, n. 1, p. 1-19, 1981.

ERMISCH, John; FRANCESCONI, Marco. Family matters: impacts of family background on educational attainments. Economica, n. 68, p. 137-156, 2001.

FERNANDES, Reynaldo. Índice de desenvolvimento da educação básica (Ideb). Brasília: Inep, 2007. (Textos para Discussão n. 26). Disponível em: <http://www.publicacoes.inep.gov.br/detalhes.asp?pub=4121#>. Acesso em: 3 mar. 2009.

______. Índice de desenvolvimento da educação básica (Ideb): metas intermediárias para a sua trajetória no Brasil, estados, municípios e escolas. Brasília: Inep, 2008. Disponível em: <http://www.inep.gov.br/ download/Ideb/Artigo_ projecoes.pdf>. Acesse em: 8 maio 2008.

FUCHS, Thomas; WOESSMANN, Ludger. What accounts for international differences in student performance? A re-examination using PISA data. Munich: CES-IFO, 2004.(CES-IFO Working Parper, n. 1235). Disponível em: <http://www.ifo.de/portal/pls/portal/docs/1/1189152.PDF>. Acesso em: 23 set. 2011.

FUNDAÇÃO IBGE. Censo demográfico 2010. Rio de Janeiro, 2010. Disponível em: <http://www.ibge.gov.br/home/estatistica/populacao/censo2010/default_sinopse.shtm>. Acesso em: 30 set. 2011.

GLEWWE, Paul; KASSOUF, Ana Lucia. The Impact of the bolsa escola/família conditional cash transfer program on enrollment, grade promotion and drop-out rates in Brazil. In: ENCONTRO NACIONAL DE ECONOMIA, 36., 9-12 dez. 2008, Salvador. Disponível em: <http://www.anpec.org.br/encontro2008/artigos/200807211140170-pdf>. Acesso em: 10 jun. 2010.

HANUSHEK, Eric A. Interpreting recent research on schooling in developing countries. World Bank Research Observer, v. 10, n. 2, p. 227-246, 1995.

______. The Failure of input-based schooling policies. The Economic Journal, n. 133, p. F64-F98, 2003.

HANUSHEK, Eric A.; WÖβMANN, Ludger. The Role of education quality in economic growth. Washington: World Bank, 2007. (World Bank Policy Research Working Paper, n. 4122). Disponível em: <http://library1.nida.ac.th/worldbankf/fulltext/wps04122.pdf>. Acesso em: 15 set. 2009.

HENRIQUES, Ricardo; GIAMBIAGI, Fábio; VELOSO, Fernando. Educação básica no Brasil: construindo o país do futuro. São Paulo: Campus, 2009.

HEYNEMAN, Stephen; FARREL, Joseph; SEPULVEDA-STUARDO, Manuel. Textbooks and achievement in developing countries: what we know. Journal of Curriculum Studies, v. 13, n. 3, p. 226-246, 1981.

HEYNEMAN, Stephen; LOXLEY, William A. The Effect of primary-school quality on academic achievement across twenty-nine high-and low-income countries. American Journal of Sociology, v. 88, n. 6, p. 1162-1194, 1983.

HOFFMANN, Rodolfo. Transferência de renda e redução da desigualdade no Brasil e cinco regiões entre 1997 e 2004. Econômica, v. 8, n. 1, p. 55‑81, 2006.

KAMAT, Sangeeta. Deconstructing the rhetoric of decentralization: the state in education reform. Current Issues in Comparative Education, v. 2, n. 2, p. 110-119, 2002.

KANE, Thomas J.; STAIGER, Douglas O. The Promise and pitfalls of using imprecise school accontability measures. Journal of Economic Perspectives, v. 16, n. 4, p. 91-114, 2002.

LEE, Jong-Wha; BARRO, Robert J. Schooling quality in a cross-section of countries. Economica, v. 68, n. 272, p. 465-488, 2001.

NERI, Marcelo. A Nova classe média. Rio de Janeiro: FGV/IBRE, CPS, 2008. Disponível em: <http://www3.fgv.br/ibrecps/M3/M3_ANovaClasseMedia_Port_2.pdf>. Acesso em: 1 jun. 2010.

_____. Crônica de uma crise anunciada: choques externos e a nova classe média. Rio de Janeiro: FGV/IBRE, CPS, 2009. Disponível em: <http://www3.fgv.br/ibrecps/C2008/C2008_CronicaTexto2.0Fim.pdf>. Acesso em: 1 jun. 2010.

RAVITCH, Diane. The Death and life of the great American school system: how testing and choice are undermining education. New York: Basic Books, 2010. Kindle edition.

SÃO PAULO. (Estado). Lei complementar n. 1.078 de 17 de dezembro de 2008. Institui Bonificação por resultados - BR, no âmbito da Secretaria da Educação, e dá providências correlatas. Diário Oficial do Estado de São Paulo, São Paulo, v. 118, n. 239, seção I, p. 1-17, dez. 2008.

SÃO PAULO. (Estado). Secretaria de Educação. Bonificação por resultado: escolas públicas do estado de São Paulo: premiar quem alcança as metas definidas para a melhoria da qualidade de ensino. São Paulo, 2011a.

______. Programa de qualidade da escola: nota técnica. São Paulo, 2011b. Disponível em: <http://idesp.edunet.sp.gov.br/Arquivos/Nota_tecnica_2010.pdf>. Acesso em: 30 set. 2011.

SILVA, Maria de Salete; ALCÂNTARA, Pedro Ivo. O Direito de aprender: potencializar avanços e reduzir desigualdades. Brasília: Unicef, 2009.

TOMLINSON, Sally. Education in a post-welfare society. 2. ed. Buckingham: Open University, 2005.

WEST, Edwin G. Education vouchers in principle and practice: a survey. The World Bank research observer, v. 12, n. 1, p. 83-103, 1997.

WINKLER, Donald R.; GERSHBERG, Alec Ian. Education decentralization in Latin America: the effects on the quality of schooling. In: BURKI, Shahid et al. (Ed.). Annual World Bank conference on development and accountability of the public sector. Whashington, DC: World Bank, 2000. p. 203-225.

Received: OCTOBER 2011

Approved for publication on: MAY 2012

Translated by David Coles

This article presents the initial results of surveys carried out within the Center for Studies in Centro de Estudos da Cultura Contemporânea - CEDEC [Contemporary Culture], funded by the Brazilian Ministry of Education's Anísio Teixeira Institute for Educational Studies and Research - INEP.

1

O Saeb é calculado com base em uma amostra representativa das escolas brasileiras.

2

O Programme for International Student Assessment - Pisa -, criado pela Organização para Cooperação e Desenvolvimento Econômico - OCDE -, avalia o desempenho de jovens aos 15 anos em 65 países. Informações adicionais podem ser encontradas em: <

http://www.pisa.oecd.org>.

3

Os dados pertinentes ao Ideb foram obtidos do sítio do Inep (BRASIL, s.d.[a]) e os dados sobre o nível de pobreza nos municípios foram obtidos do sítio do Ipea (BRASIL, s.d.[e]).

4

O teste

t indica se os valores para os coeficientes β são significativamente diferentes de zero.

5

Estamos agora realizando uma investigação qualitativa, visitando todos os oito municípios identificados por meio da análise de resíduos com o objetivo de verificar quais práticas poderiam explicar o bom desempenho observado. Os resultados, especialmente os de Barra do Chapéu (SP) e Boa Vista do Tupim (BA), poderão ajudar os municípios com alto nível de pobreza a melhorem o desempenho escolar.

6

Uma professora da 5

^a série da rede estadual de educação do estado de São Paulo informou-nos que em 2010 ela recebeu mais de R$ 3 mil como bônus pelo desempenho de sua escola, mas que em 2011 o valor do bônus caiu para R$ 35,00. Ela ou seus colegas não alteraram suas práticas pedagógicas de 2010 para 2011, mesmo porque o que estavam fazendo foi recompensado inicialmente com o bônus. Diante desse quadro, o que os professores devem fazer para receber o bônus em 2012? Continuar com suas atuais práticas pedagógicas que foram premiadas em 2010? Ou mudar tudo o que faziam antes, porque "fracassaram" em 2011? A mensagem da política de bônus baseada no desempenho anual é altamente dúbia.

7

Segundo dados do censo de 2000, 88,9% dos municípios brasileiros têm até 50 mil habitantes (IBGE, 2011).

Publication Dates

• Publication in this collection
  13 Mar 2013
• Date of issue
  Dec 2012

History

• Received
  Oct 2011
• Accepted
  May 2012