The effects of early childhood education on literacy scores using data from a new Brazilian assessment tool

The effects of early childhood education on literacy scores using data from a new Brazilian assessment tool

Fabiana de Felício^I; Rafael Terra^II; Ana Carolina Zoghbi^III

^IMETAS - Avaliação e Proposição de Políticas Sociais, Endereço para contato: Rua Santa Justina, 464 - Vila Olímpia - CEP: 04545-042, e-mail: [e-mail]

^IIDoutorando em Economia, Escola de Economia de São Paulo (EESP-FGV), Endereço: Rua Itapeva 474 - Bela Vista - Cep: 01332-000, e-mail: [e-mail]

^IIIDoutora em Administração Pública e Governo - (EAESP-FGV), Endereço para contato: Rua Itapeva 474 - 12º andar - Bela Vista - São Paulo - CEP: 01332-000, e-mail: [e-mail]

ABSTRACT

The aim of this study was to identify the effects of early childhood education (ECE) on literacy scores of 2nd grade students in elementary school. To do that, the Provinha Brasil was administered in Sertãozinho-SP, in conjunction with a socioeconomic questionnaire. Despite external validity problems, the evaluation of the effects of ECE in one municipality is advantageous, as we can estimate the effects of one kind of treatment. Other studies ignore this fact. Often, they estimate an average effect of various treatments effects (not just one), as they use data from different municipalities where ECE programs have different levels of quality. The OLS and Propensity Score Matching results show that students who started school at the ages of 5, 4, and 3 years had literacy scores between 12.22 and 19.54 points higher than those who began school at the age of 6 years or later.

Keywords: early childhood education, literacy scores, Propensity Score Matching

JEL Classification

A29, I21, C21

RESUMO

O objetivo deste trabalho foi identificar os efeitos da Educação Infantil (EI) sobre os escores de alfabetização dos alunos da 2º ano do Ensino Fundamental. Para isso, a Provinha Brasil foi aplicada em Sertãozinho-SP, juntamente com um questionário socioeconômico. Apesar dos problemas de validade externa, a avaliação dos efeitos da EI em um único município é vantajosa, já que se podem estimar os efeitos de um determinado tratamento. Vários estudos ignoram esse fato. Muitas vezes é estimado um efeito médio de vários efeitos de tratamento e não apenas um, já que usam dados de diferentes municípios onde a EI tem diferentes níveis de qualidade. Os resultados por OLS e Propensity Score Matching mostram que alunos que ingressaram com 5, 4, e 3 anos de idade, obtiveram escores de alfabetização cerca de 6% maiores do que os obtidos pelos que ingressaram na escola com 6 anos ou mais.

1. Introduction

With the advent of the Human Capital Theory – especially with the work conducted by Becker (1964) – we observed the consolidation of a theoretical framework where education played a major role in the determination of poverty, long-term growth, per capita income, and income inequality within and between countries. Nevertheless, the identification of this relationship between education and income per se was not enough for the implementation of public policies targeted at the improvement of people’s living conditions. In this respect, we have the Economics of Education literature, whose main goal is to identify the most important factors for the development of people’s skills.¹ Among these factors, family background plays a crucial role. According to these studies, the influence of family on the development of children’s skills is so important that the room for public sector action seems quite restricted. This perspective is particularly problematic for Brazil, since most adults have a low educational level and live in precarious conditions, which hinders their children’s skill development and perpetuates the cycle of poverty.

Despite the limited scope of public policies aimed at improving education (in terms of quality and quantity), some factors indirectly related to family are important, as evinced by empirical studies. Improvement of school infrastructure, qualification of teachers and principals, and accountability, in addition to other activities offered at schools, are some examples.²,³,⁴ But another specific factor has been given special attention in the literature: Early Childhood Education (ECE).

In the last two decades, there has been a debate about the effects of early childhood education on future educational outcomes. A considerable number of scientific studies find evidence that educational intervention early on during childhood yields significant and long-lasting results on future school performance, and even on adult life successes. Based on this literature, Cunha et al. (2005) introduce a model for skill development that reflects the necessity that investments in education be made in early childhood. The model also considers that later investments are important to maintain the skill level developed in early childhood. If investments are not made in early childhood, the ability to acquire knowledge in the future will be impaired, and this cannot be offset by further investments in the future. Therefore, the authors refer to early childhood as the "critical period".

Low scores by Brazilian students on standardized national exams (Prova Brasil and SAEB –National Basic Education Assessment System) and on standardized international exams (Programme for International Student Assessment – PISA), high grade repetition rates, and high dropout rates before graduating from high school all underscore the need for urgent action by the public sector, and intervention in early childhood is an important alternative for Brazilian public education.

Some actions have already been carried on by the Brazilian federal government. To raise children’s educational levels and to encourage early school admission, the minimum time period for completing fundamental education has recently been extended from 8 to 9 years, which decreased the mandatory age for school admission to 6 years. Moreover, an attempt has been made to boost openings at day care centers and preschools and to stimulate school enrollments of children aged 0 to 5 years with the inclusion of early childhood education in FUNDEB (Fund for the Maintenance and Development of Basic Education and Teaching Improvement).⁵ This implies the guarantee of a minimum common grant value in all states and municipalities for every child enrolled in an ECE program through high school. Also, there is a congressional bill in discussion that, if made into law, will make these two stages mandatory, as is Fundamental Education school now.

Another measure that is important for the improvement of Brazil’s educational system is to warrant that children be taught to read and write at the appropriate age so that they will be ready to develop the skills that are required throughout their school life. In this regard, the Brazilian Ministry of Education developed Provinha Brasil in 2007, an exam that assesses the literacy of children aged 6 to 8 years.

This exam is an innovative tool used to measure the development of children’s reading and writing skills, when it is still possible to correct learning deficiencies in a more effective way (at least if we compare it with the possibilities of educational intervention for older children) and to adjust, whenever necessary, the process that precedes the assessment. It can also be used to assess the effects of public policies adopted in response to deficiencies revealed by the exam results.

As Provinha Brasil assesses students at the very beginning of the fundamental education, the results should help formulating educational policies targeted at early childhood education and the first years of fundamental education. This is consistent with the model put forward by Cunha et al. Students with low literacy scores on Provinha Brasil might not be able to achieve the potential they would be able to if they had been properly taught to read and write. Therefore, interventions should occur when children are still young, i.e., during early childhood education years, so that students in the subsequent cohorts can be better prepared when they are assessed.

The aim of this essay is to empirically determine the relationship between the age at school admission (which also reflects the fact that a student attended or did not attended an ECE program) and a child’s literacy score based on the results obtained on Provinha Brasil. Provinha Brasil differs from other tests applied by the Ministry of Education. Its administration and grading requires the active participation of the local school systems because although the Ministry of Education designs the test and the manuals for its administration and provides the public school systems all over Brazil with the exam and respective manuals in digital format, the exam is administered and graded by the local school systems.

There is neither a national database with the exam results (as the administration of the exam is decentralized), nor a socioeconomic questionnaire to be answered by the children, as occurs with SAEB, since the children who take this exam are too young to answer such a questionnaire in an accurate way. Thus, the solution we found to use the exam results in an econometric analysis was to closely follow the administration of Provinha Brasil in a Brazilian municipality where early childhood education was well structured and where we could also administer socioeconomic questionnaires directly to the parents. With these factors in mind, we chose Sertãozinho, a municipality located in the state of São Paulo.

As this is a medium-sized town (with a projection of 109,565 inhabitants according to the Brazilian Institute of Geography and Statistics – IBGE – in 2008 and 2,081 students attending the 2nd grade of Fundamental Education school in 2007), it was possible to standardize and inspect universal administration properly. This means that it was also possible to administer the exam in private schools, which allowed us to draw generalized conclusions about the econometric results we obtained. Note that the cooperative attitude of private and public schools was also important in the selection of this municipality.

The assessment of a single municipality, even though it has some problems relative to the external validity of the results, is desirable in an analysis using Propensity Score Matching like the one we carried out in this study. Friedlander & Robins (1995) and Michalopoulos et al. (2004) find evidence that when treatment and control groups belong to the same geographic location, the matching procedure yields better results. Indeed, if we consider early childhood education an intervention, we cannot use treatment units from various localities because each place has one quality of education and thus we would have a number of different treatments instead of just one.

Finally, this essay is important because it verifies the effectiveness of policies aimed at expanding early childhood education as a way to increase students’ learning. Accordingly, we sought to identify the relationship between early childhood education and the literacy score attained in the 2nd grade of Fundamental Education, using a new assessment tool that measures an especially important set of skills that affects directly the learning capacity in any area of knowledge.

This essay is organized in six sections, including this introduction. Section 2 reviews the literature on the effects of early childhood education on children’s skill development. Section 3 describes Provinha Brasil in more detail, as well as the administration of this exam and the questionnaire in the selected municipality, and also presents some descriptive statistics. Section 4 explains the whys and wherefores of using the Propensity Score Matching as estimation method. Section 5 describes the estimation results and, finally, Section 6 brings the concluding remarks.

2. Literature Review

The literature dealing with the effects of early childhood education on children’s skill development is quite rich and comprehensive. Most of the reported results are based on random experiments, but important studies that use non-experimental methods are also included. Some experiments conducted in the United States, which are especially noteworthy in the literature, include the following: the High/Scope Perry Preschool Project, the Carolina Abecedarian Project, and the Early Training Project (Currie, 2001).

The High/Scope Perry Preschool Project was an experiment undertaken between 1962 and 1967 which assessed 123 children aged 3 to 4 years (all of whom had a vulnerable socioeconomic background). The treatment consisted of preschool attendance for one shift (morning or afternoon) every day plus a 90-minute weekly home visit for 8 months a year during 2 years. Students usually left the program at the age of 5 years. All of the teachers had a Master’s degree and the teacher-to-student ratio was 1:6. According to Schweinhart et al. (1993), the results obtained with this program were quite positive: better performances on skill tests (at 9 and 14 years), better performances in high school, higher high school completion rates, lower rates of imprisonment (at 27 years), higher salaries (at 27 years), and lesser use of government support (at 27 years).

The Carolina Abecedarian Project was an experiment targeted at children with vulnerable socioeconomic backgrounds and with risk for mental retardation due to the precarious living situations (Currie, 2001). Admission to the experiment occurred at 6 to 13 weeks of age. The treatment consisted of an intensive child care and language development program for 8 hours a day, 5 days a week, 50 weeks a year, from birth to the age of 5 years. After admission to the program, the treatment group was randomized into two groups, one with a tutor who provided additional instruction at home, and another one that did not have any additional intervention. The program was concluded up to the age of 8 years. The teacher-to-student ratio was initially 1:3, rising to 1:6 as children grew older. The results were also quite positive. The treatment group had better results on proficiency tests, higher averages of school performance in high school, lower incidences of grade repetition (at 15 years), and lower dropout rates (at 21 years), in addition to lesser need for special education (at 15 years) and higher probability of attending college (at 21 years).

The Early Training Project was a less intensive program, targeted at children aged 4 to 5 years. It consisted of weekly home visits for 1 year, in addition to a single-shift 10-week course for 2 or 3 summers for the treatment group. The treatment was concluded at the age of 6 years. According to Gray et al. (1983), the results showed a reduction in the need for special education for children in the treatment group.

These three experiments share a common feature: they are all small-scale projects. This may have important implications, since it is impossible to determine the external validity of these experiments. The results of these experiments if they were to be carried out on a large scale cannot be predicted. An important and more comprehensive experiment is the Head Start program, created by the U.S. government in 1964 and whose target public is composed of children aged 3 to 5 years. Intervention consists of providing health care, meals, snacks, and child care with a higher quality standard than that which low-income parents can provide for their children. Currie & Thomas (1995) conducted a quasi-experimental study of the effects of this program on performances on the Peabody Picture Vocabulary Test (PPVT) and on the probability of never having to repeat a grade. The authors found positive effects on the performance of white children who participated in this program, whereas at first the results for Afro-American children who participated in the program were not statistically different from those who did not participate. According to the authors, this finding may be linked to the possibility of children from this ethnic group facing more hostile environments and/or fewer opportunities after they finish the program. Thus, in order for the benefits to last longer for Afro-American children, as occurs among white children, more investments, even after the program has finished, should be made in those children enrolled in the program.

Recently, important experiments and quasi-experimental studies have been conducted in Latin America. Schady (2006) carried out a literature survey on the assessment of the impact of investment programs in childhood on cognitive and non-cognitive development. Gertler & Fernald (2004, apud Schady, 2006), for instance, find evidence that transfers made by Mexico’s Opportunities Program had a positive impact on motor skills and socioemotional behavior. Behrman, Parker & Todd (2004, apud Schady, 2006), on the other hand, gather evidence that these transfers had a positive impact on the probability of children enrolled in the program (aged 0 to 6 years) starting school at an earlier age. They also find evidence of higher promotion rates and higher expected schooling years among children in the treatment group.

Behrman, Cheng & Todd (2004) analyze the results of a Bolivian preschool program called Proyecto Integral de Desarrollo Infantil (PIDI) using Propensity Score Matching. The program consists of intensive child care, such as a full-time daycare center and nutritional and educational services for children aged between 6 months and 6 years from low-income families. The authors encountered evidence of improved motor and psychosocial skills and improved language acquisition. These results were more significant among children older than 3 years and among those who attended the program for a longer period.

Berlinski et al. (2009) assess the effects of an exogenous variation in the supply of preschool openings due to a school construction program in Argentinians students’ performance. The authors collected evidence that the performances in Spanish and mathematics of cohorts and regions subjected to the treatment (construction of schools) was significantly higher than those of children who were not exposed to this exogenous variation. They also found evidence of positive impacts on non-cognitive skills such as attention, participation, and discipline in the classroom.

In Brazil, a few studies were developed to assess the effects of an intervention (i.e., early childhood education) in early childhood. Curi&Menezes-Filho (2006), for instance, demonstrate that students who attend a preschool or a daycare center are more likely to finish primary education (1st to 4th grades of Fundamental Education), middle school (5th to 8th grades), high school, and college. The authors also observe that because individuals attend a daycare center or preschool, they have a higher average of schooling years (measured from the first year of primary education) and higher salaries as well. Finally, the authors find evidence that early school admission has positive effects on math proficiency.

This result regarding the effects of early childhood education in Brazil on students’ performance is corroborated by Felicio & Vasconcellos (2007). The authors use methods for the correction of endogeneity and self-selection bias and find positive and statistically significant effects of preschool attendance on the performance of 4th graders on SAEB. Depending on the region, these effects ranged from 9 to 19%.

Given the evidence of these studies and the model proposed by Cunha et al. (2005) described in the introduction of the present study, it is fundamental to address the problem of liquidity constraints faced by socioeconomically underprivileged families. Following this line of research, the simulations of a general equilibrium model run by Restuccia & Urrutia (2004) confirm the existence of an intergenerational persistence of earnings and schooling. According to the authors, this problem arises mainly from low investments in the earliest childhood stages combined with liquidity constraints by poorer parents. Therefore, children whose parents cannot afford to invest in this initial stage will be condemned to have poorer skills in the subsequent stage and, consequently, lower earnings in adult life.

These statistics are particularly worrying as children with the worst family backgrounds are exactly those who would benefit the most from these programs in early childhood (Currie, 2001). These programs would be useful to minimize adverse family conditions. This way, there would be room for the government to act in order to provide equal opportunities to the children, i.e., to lessen the large disparities observed between the initial skills of economically privileged and underprivileged groups.

3. Dataset

The database used in this study was built using the results obtained from the first administration of Provinha Brasil in Sertãozinho,⁶ in the state of São Paulo, in May 2008. The exam was administered universally (to public and private education systems) with the aim of assessing literacy measured as the reading and writing skills of students attending the 2nd grade of a 9-year-long elementary school education. The choice of Sertãozinho was based in part on the fact that this town had a not-so-large number of children enrolled in this grade (2,081 students according to the 2007 School Census), thus allowing us to follow up with the administration of the exam and the socioeconomic questionnaires in an appropriate fashion. The presence of a well-structured early childhood education system is another highlight of this municipality.⁷ Finally, the availability of public and private schools to participate in the study was also decisive for the selection of this municipality.

It should be underscored that Provinha Brasil differs from other Brazilian exams in some important aspects, besides the fact that it assesses literacy. The first aspect concerns the responsibilities attributed to each phase of the exam. For other Brazilian exams developed by the Ministry of Education (Prova Brasil and SAEB), an external institution is hired to administer the exams, organize the data, and hand the database over to the Ministry of Education. In the case of Provinha Brasil, the National Institute for Educational Studies and Research (INEP) is in charge of devising the material and making it available online so that schools can print it. The use of Provinha Brasil by municipal Departments of Education is optional. Therefore, the administration, grading and publication of results are locally assigned responsibilities.⁸

Another difference lies in the assessment method. The score of Provinha Brasil ranges from 217.4 to 665.0 points, as opposed to the scores of Prova Brasil and SAEB, which range from 0 to 500 points. As Provinha Brasil aims to assess children’s literacy, its maximum score (665.0) can be achieved by children only when they are completely literate. The other exams, however, focus on determining the skill levels developed by each student during K-12 education, so in practice this means that students rarely achieve the maximum score. Nevertheless, Provinha Brasil, SAEB, and Prova Brasil are all based on the Item Response Theory, which allows the results to be placed on the same scale and compared between assessments, between grades for the same exam, and over time. Therefore, the results of these exams permit monitoring the quality of education in Brazil.

As stated in the first section of the present study, another aspect that distinguishes Provinha Brasil from other Brazilian exams is the absence of questionnaires for children for the collection of relevant information to explain student performance. Children who take part in Provinha Brasil are very young, which does not allow them to answer a questionnaire in a consistent manner.

For the administration of Provinha Brasil in public schools, the municipal Departments of Education offer examiners a training course. To carry out this study in Sertãozinho, this was no different. We only followed up the process to guarantee that both the administration and the grading were standardized.

In private schools, we endeavored to reproduce the same standard adopted for public schools. In order to get the schools ready, we relied on help from the Board of Education of the State Department of Education. We held a meeting with the coordinators of each school to explain the objective and importance of the study and then we trained the teachers in charge of administering the exam. The training also included instructions on how to administer the socioeconomic questionnaires to be answered by students’ parents or surrogates, the same guidelines public schools were provided with.

Altogether, 14 public schools and 9 private schools were assessed in Sertãozinho. Among those students enrolled in the 2nd grade of Fundamental Education, 1,986 took Provinha Brasil, which is a significant share considering the estimate of approximately 2,100 students enrolled in 2008. The exam was administered on the same day in both the public and private schools. With regard to the questionnaires, they were administered in order to collect information about the socioeconomic characteristics and school history of students.⁹ Parents were asked to fill out the questionnaires at parent teacher conferences, or the questionnaires were sent to those parents who did not come to the conferences so that they could answer and return them later.

The information obtained this way is more reliable than that obtained from 4^th graders on SAEB or Prova Brasil. Out of the 1,986 students who participated in Provinha Brasil in Sertãozinho, 1,850 questionnaires were returned. This represents a loss of only 6.85%, which is much lower than that of the Prova Brasil questionnaires in 2007, where 18.71% of the 4th graders did not answer any of questions. Regarding the question about the mother’s level of education, for example, only 1% of the Provinha Brasil questionnaires filled out by the parents held "I don’t know" as the answer. Conversely, in the questionnaires related to Prova Brasil in 2007, this rate was as high as 30.43%.

According to the descriptive statistics of the variables used in the estimations in the present study (Table 1), the average score on Provinha Brasil in 2008 in Sertãozinho was 547.9 points. According to the Brazilian Ministry of Education, this score is lower than what is desirable in terms of literacy (563.9 points or more). When we assess the results considering children’s age at school admission, we note that the younger the children begin attending school, the higher their literacy score. Recall that after the implementation of the 9-year Fundamental Education, the age for school admission became 6 years. However, we verified that 84.2% of the students from Sertãozinho began attending school at the age of 5 years or less, allowing us to conclude that a significant share of the students had attended an ECE program.

In regard to family arrangement, those children who lived with their father and mother achieved an average score of 554.2 points. Among those who did not have such a family arrangement, the average score corresponded to 540.2 points. Family size was also important. Children from big families (5 people or more) had a lower average than those from smaller families (4 people or less), with an average difference of 20.7 points on literacy scores. As for the participation of parents in their children’s school life, we found a difference of 41.9 points between students whose parents attended the parent teacher conferences and those whose parents did not do so. Finally, as expected, the higher the mother’s level of education, the higher the student’s score.

Another interesting piece of information concerns the time children devoted to studying and reading. Those who spent more time studying or reading had better scores than those who studied or read less often (less than 3 days). The differences in scores corresponded to 43.2 and 26.0 points, respectively.

As far as the households where children live are concerned, we perceived that higher scores were associated with better socioeconomic conditions. Thus, students who lived in a house with a larger number of bedrooms, Internet access, a DVD player and a car, had better scores than those students from socioeconomically underprivileged families. In addition, children whose households had a quiet place for studying had on average a score of 32.5 points higher than those who did not.

4. Estimation Strategy on the Presence of Self-Selection Bias

Average performance (in terms of literacy scores on Provinha Brasil) of students who started school at an earlier age is certainly higher than that of those who entered school later (Table 1). Notwithstanding, there is a consensus agreement in the literature that children with a better family background tend to start school earlier than those with less favorable social conditions. They self-select to receive treatment S, which in this case refers to earlier school admission.

Therefore, to obtain consistent estimates of the effects of earlier school admission on the literacy scores on Provinha Brasil of children aged 7 to 8 years, it is necessary to find an estimation strategy that corrects the self-selection bias inherent to the difference in averages between treatment and control groups.

Ideally, the real average effect of treatment on literacy could be known if it were possible to observe the same children in two distinct situations, one in which they had been enrolled in school at age s (treatment designated by S ), and another one in which they had started school when they were l years old (control designated by L), where. Thus, the Average Treatment Effect (ATE) could be obtained by:

where is the potential result of child i on Provinha Brasil if she belongs to treatment S, and is the potential result of this same child if she belongs to control group.¹⁰

Though it is not possible to observe the same individual in these two states simultaneously, one can observe:

Adding and subtracting counterfactual in this equation, we have

rearranging

The first term in (4) is the average treatment effect on treated subjects (ATT), and the two subsequent terms stand for the self-selection bias. The interest lies in ATT, but to estimate it, it is necessary to use an estimation method that rules out the self-selection bias. A very frequent solution described in the literature consists of social experiments based on treatment randomization for a selected set of individuals, resulting in a group of treated and untreated (control) subjects. By obtaining a perfect randomization of treated individuals, potential outcomes will be independent from the treatment status, and the self-selection bias will be null, i.e. .This way, Equation 4 can be rewritten as

Therefore, with a perfect randomization, it is possible to estimate the ATT by comparing the average outcomes of treated and untreated groups. However, in many cases, it is in the individual’s interest to receive treatment, so then it might be difficult to prevent him/her from self-selecting to participate in the treatment, especially in social experiments. In the case proposed in the present study, this difficulty is even more evident, as the age at which children are enrolled in school is determined by the characteristics (or preferences) of their families. Additionally, besides other difficulties related to the conduct of experiments,¹² follow-up time of observation units is also a hindrance. For example, if treatment refers to school admission at the age of 3 years, while control refers to school admission at a later age, it would take us about 4 years before we could assess the effects of treatment on literacy. This time period is too long, considering the lack of Brazilian studies on the topic and also the urgent necessity for novel studies that may contribute to the recent debate about the mandatory requirement of early childhood education in Brazil.

In view of these arguments, a more appealing alternative for estimating the effects of early childhood education on Provinha Brasil literacy scores is the use of non-experimental methods based on the hypothesis of selection according to observable characteristics. This can be done if the following assumption holds true: in a given set of observable characteristics X that determine the selection for treatment, potential outcomes do not depend on treatment status (Rubin, 1977), that is

In fact, this is a strong assumption, but we regard it as valid for the present study.

Nevertheless, note that if there are many covariates, it might be difficult to obtain cells with treatment and control groups in a sufficient amount to estimate the treatment effect.¹⁴ An alternative proposed by Rosenbaum & Rubin (1983) to circumvent the dimensionality problem is the use of the propensity score (p(X_i)), which consists of a measure that combines individual characteristics into a single indicator with the same independence property between potential outcomes and treatment assignment:

where p(X_i) gives each individual i the probability of receiving treatment based on her characteristics X_i and allows treated and untreated individuals with similar indicators to be compared. Another requirement is that observable characteristics should not fully determine treatment status, that is

In this study, the propensity score was obtained using a probit regression where the dependent variable S_i is equal to 1 if the individual is treated, i.e., if she started school at age s, and 0 if she started school at age 1. Explanatory variables X_i that determine treatment were chosen based on two procedures: 1) statistical significance; and 2) the "hit or miss" method.¹⁵ The first procedure consists in selecting covariates from a larger set whose coefficients are statistically significant. The second one consists in developing an indicator equal to >p, and 0 otherwise, where is the estimated probability of receiving treatment and p is the percentage of treated individuals. The larger the number of correct predictions obtained with the indicator developed in relation to dependent variable S_i which designates treatment, the better the model.

The subsequent step consists of choosing a matching algorithm based on the predicted Propensity Score Matching (PSM) to estimate the effect of treatment on treated subjects (ATT_X). In line with Heckman, Ichimura& Todd. (1997),¹⁶ the method used as a benchmark among the different options available was the Kernel Matching (with a bandwidth of 0.06 and Epanechnikov weighting function). Using PSM has an advantage compared to Ordinary Least Squares (OLS) conditioned on covariates: it does not suppose a linear additive functional form. For the sake of comparison and robustness check of the results, ATT_X were also estimated by 1) OLS with covariates; 2) OLS with the reciprocal of the Propensity Score as weights;¹⁷ 3)OLS with the Propensity Score as covariate;¹⁸ 4) Nearest Neighbor PSM with replacement; 5) Nearest Neighbor PSM without replacement; 6) Nearest 10 Neighbors PSM with replacement; 7) Radius PSM with caliper of 0.1; 8) Radius PSM with caliper of 0.001; 9) Radius PSM with caliper of 0.0001;¹⁹ 10) PSM within Strata with 5 strata; and 11) PSM within Strata with 10 strata.²⁰

Concomitantly with the PSM estimations, we ran tests to check whether the covariates were balanced between the treatment and control groups (i.e., to check whether both groups were alike). Two tests proposed by Rosenbaum and Rubin (1985) have such purpose. The first one consists of a t test to determine the differences between the averages of treatment and control groups for each covariate before and after the matching. The second test is based on the calculation of standardized biases of a given covariate, also before and after the matching. This test is obtained by the ratio between the difference of covariate means of the treatment and control groups and the squared root of the average of the variances of the same covariate for the treatment and control groups. A significant reduction in the bias, such that the bias indicator after the matching is lower than 5%, indicates that the explanatory variable was properly balanced.²¹

Another way to improve covariate balance was by using a trimming rule. This rule guarantees that observation units outside the common-support region will be excluded, as well as the treatment or control units within the common-support region located on a given interval (bin) of the histogram with a frequency lower than q%.²²

With the results obtained from Provinha Brasil and the answers to the socioeconomic questionnaires administered to students’ parents, nine groups were constructed for PSM implementation, each one of them referring to different treatments and controls. This is well illustrated in Table 2. In Group 1, for instance, children who started school at the age of 5 years or before were the treated subjects, while those who started school at the age of 6 or later were the controls. As previously mentioned, given that the correct age for admission to Fundamental Education in Sertãozinho is 6 years, treatment in Group 1 refers to the attendance of an ECE program for at least 1 year, i.e., having started school at the age of 5 years or earlier, while control refers to not attending an early childhood education program, i.e., having started school at the age of 6 years or later.

From Groups 2 through 7, treatment S is always associated with a specific age at admission lower than the age at admission of children from control group L. All possible combinations based on this rule were employed. Differently, in Group 8, treatment refers to those students who started school at the age of 5 years or earlier in another town, whereas controls are those students who started school at the age of 6 years or later in Sertãozinho. This enabled the identification of the average difference of literacy scores between students who attended an ECE program outside Sertãozinho and those who started Fundamental Education there. This strategy allows assessing the role of the quality of early childhood education on students’ literacy scores.

In Group 9, on the other hand, we tried to consider treatment and control units in such a manner as to distinguish the results between those who attended an ECE program in public schools and those who attended one in private schools. Treatment in this case consisted of starting school at the age of 5 years or earlier in private schools, while the control group included those who started school at the same age but in public schools. This econometric exercise is important to complement the estimation of the effect of earlier school admission on literacy. As occurred in Group 8, this exercise goes beyond the idea that earlier school admission alone is sufficient for the improvement of future school performance, as the objective is to gather evidence of the role of early childhood education quality on literacy.

5. Results

In this section, we present the econometric results obtained to assess the effect of early childhood education on children’s literacy. The variables that define the treatment and control groups used to capture this effect refer to the ages at which children were enrolled in school. These variables are more suitable to achieve the intended target, as they allow assessing whether there are striking differences between children who attended an ECE program for 1 year and those who did it for more than 1 year.

First, in Table 3, we present the OLS estimates (with robust standard errors in parenthesis) of the effects of earlier school admission. As proposed by Rubin (1977), conditional on a set of covariates that define treatment, treatment variables are independent from potential outcomes, and an OLS estimation should produce unbiased estimates. We checked whether the fact that a child attended an ECE program, i.e., if she started early childhood education at the age of 5 years or earlier, has a positive effect on the literacy score (OLS1, OLS2, and OLS3 specifications), compared to children who only started school at the age of 6 years or later. The OLS1 estimation coefficient represents the result that is not conditional on the child’s observable characteristics, and therefore, it should be biased. In this case, we observed a literacy score 31.45 points greater than that obtained by students who did not attend an ECE program.

In OLS2 specification, it was possible to assess the effect of early childhood education when we controlled for the set of covariates used to obtain the propensity score related to the probability of each student’s receiving treatment, which we call a "smaller set." Note that the magnitude of the coefficient of the variable "children who entered school at age 5 or less" was lower than in OLS1 specification, producing a positive effect of 13.65 points on the literacy score compared to the control group.

The magnitudes of dummy coefficients related to the age at which a child started school changed when we inserted the covariates used to obtain the propensity score (smaller set). In the OLS5 specification, we observed that children who started school at the age of 5, 4,or 3 years presented literacy scores 10.91, 11.33, and 17.66 points higher than those who entered school at the age of 6 years or later, respectively. The same was observed in the OLS6 specification, in which we used more covariates (full set).

The estimates based on Propensity Score Matching are shown in Table 4.²³ As mentioned in the previous section, our main results are based on Kernel Matching, which are highlighted in gray. The other matching methodologies were implemented to check the robustness of the results.

By analyzing Group 1 (treatment and control), we observed that treatment effect is positive and significant. Children who started school at the age of 5 years or less presented literacy scores 17.33 points higher compared to those who only started school at the age of 6 years or later.

In Group 2, which is composed of children enrolled in school at the age of 3 years or less (treatment group) and children enrolled in school at the age of 6 years (control group), we found a difference of 19.54 points in the literacy scores between the treatment and control groups. If we consider the same treatment individuals and the control units as children who were enrolled in school at the age of 5 years (Group 5), the difference was 10.09 points, slightly lower than in Group 1. However, there was no statistically significant difference in the literacy scores between the children who entered school at the age of 3 or less with those who entered at the age of 4 (Group 7).

On the other hand, if treatment refers to school admission at the age of 4 years, while control refers to school admission at the age of 6 years (Group 3), we observed a positive and significant effect of 18.25 points in the literacy scores of treated children. However, this effect is not perceived in Group 6, in which the treated children are exactly the same students in Group 3, and control subjects are those students enrolled in school at the age of 5 years.

Group 4 refers to students who started school at the age of 5 years (treatment) and those who entered school at the age of 6 years (control). In this case, we observed that treated children had a literacy score 17.89 points higher.

It is important to underscore that we carried out two additional exercises: one to evaluate the effect of a child starting school at the age of 5 years or less (i.e., attended an ECE program) in another municipality; and another one to measure the effect of a child’s having attended a private school at the same age. First, Group 8 presents the comparison between students who enrolled in school at the age of 5 years or less in another municipality (treatment) and those who enrolled in school at the age of 6 years in Sertãozinho (control). Note that treated children had a literacy score 5.65 points lower, but this difference was not statistically significant. This result indicates that the literacy scores obtained by students in each of these groups are similar. It also indicates the fact that students had early childhood education does not guarantee that they will have higher literacy scores, and thus it favors the choice of a municipality in which early childhood education is well structured.

Group 9 included children admitted to a private school at the age of 5 years or less (treatment), and those who had early childhood education in a public school (control). The results do not show significant differences in literacy scores between the two groups. This result confirms that early childhood education provided by public schools is as good as that offered at private schools, suggesting that public schools, with the largest number of openings in Sertãozinho, have a good structure and fulfill the purpose of educating rather than just taking care of the children of worker parents (considering that private schools offer appropriate quality standard).

Generally, the other Propensity Score Matching procedures produce very similar results to those of Kernel Matching. The exceptions are the estimates generated by Nearest Neighbor and Nearest Neighbor Matching without reposition, whose results were mostly insignificant. However, both methods work more adequately when the amount of control units is much larger than treated ones, which is not the case in this paper.

Overall, the results obtained show that children who attended at least 1 year of an ECE program (i.e., enrolled in school at the age of 5 years or less) presented a higher literacy score than those who did not attend this stage. Furthermore, the sooner the student entered the ECE program, compared with those who did not attend this type of educational program at all, the greater the literacy scores. But the returns (in terms of literacy scores) of an additional year of education at an early age seems to be diminishing, as one can note by the statistically significant difference in mean literacy scores between children who entered school at the ages of 5 and 6 years (the last referring to those that did not enrolled in an ECE program) together with a smaller and statistically significant difference in scores between those who were enrolled at the age of 5 years and those who were enrolled at the age of 3 years or less. This leads to the conclusion that 1 year of early childhood education (just before entering the K-12 educational system), offered by a well-structured educational system such as that in Sertãozinho, may be enough for children to be practically literate by the age of 7 or 8 years. This does not mean that there will not be skill differences between those who attended an ECE program for 1 year and those who did it for a longer period of time. In this paper, only a limited set of skills was assessed: those skills that determine literacy. Enrollment in school at the age of 3 years or less, for instance, may contribute to the development of other important skills not contemplated by Provinha Brasil. Thus, assessing a broader set of skills may well be consistent with constant or even increasing returns to Early Childhood Education.

The tests described in the "Estimation Strategy Section" were performed in order to assess the quality of the matching procedures. In general, as it can be seen in Tables A.1 and ^A.2 of the appendix, the test results suggest a good matching, such that there are no systematic differences on observable characteristics between the treatment and control groups.²⁴

Finally, some limitations regarding the identification strategy of this study are noteworthy. If the selection into treatment, i.e., the school starting age, is based on non-observable characteristics, then the results would be biased. In this case, it is possible that students who started school early differed systematically from students who started school at the age of 6 in ways that are difficult to observe. For example, perhaps starting school at an early age is related to parental expectations. Hopefully, the direction of the bias is negative, attenuating the coefficients, or selection into treatment is based only on observables variables. Unfortunately, with the dataset in hand, it is not possible to find evident exogenous sources of variation.

6. Final Remarks

This paper aims to identify the effects of early childhood education on children’s literacy scores. To do that, we used data from Provinha Brasil administered in Sertãozinho to students attending the 2nd grade of Fundamental School and from a socioeconomic questionnaire answered by the parents.The main contribution of this paper to the literature is to explore this new assessment tool proposed by the Brazilian Ministry of Education. In addition, to our knowledge, there are no studies in the literature, at least not in Brazil, that evaluate the relationship between earlier school admission and literacy scores.

The results obtained in this paper with Propensity Score Matching (and also with OLS) demonstrate that students who started school at the age of 5 years or less had higher literacy scores than those who started school at the age of 6 or later. In general, students who started school at the ages of 5, 4, and 3 years or less obtained literacy scores between 12.22 and 19.54 points higher than those who started school at the age of 6 (or later). The results reasonably suggest that the returns in terms of literacy scores are diminishing in relation to the number of years of early childhood education as the effect of attending school at the age of 3 or less (i.e.,3 or more years of Early Childhood Education) is less than 3times greater than that found among students that entered school at the age of 5 years (i.e., 1 year of Early Childhood Education).

Indeed, there might be a problem with external validity of these results. Although the study used a non-experimental method to estimate the treatment effect, it has the same problems as experiments or non-experimental studies with treatment and control groups that belong to one locality. The effect of early childhood education on the literacy of students all over Brazil couldbe even more pronounced. But we believe that the results obtained in this work constitute good guidance for the implementation of public policies.

Finally, we underscore that this discussion is very relevant to the Brazilian case. If investments in education in early childhood are essential to skill development later in life, it is necessary to invest more heavily in early childhood education (before the age of 6 years) in order to improve the overall quality of education in Brazil. However, school attendance rates in Brazil are not universalized yet, and expenditures per student are much lower than those verified in developed countries. This stresses that a lot more effort should be put into expanding the coverage and improving the quality of this early stage of education.

Recebido em 28 de janeiro de 2011.

Aceito em: 12 de dezembro de 2011

APPENDIX

• Abadie, A; Imbens, G. W. 2006. On the Failure of the Bootstrap for Matching Estimators Issued in June NBER Technical Working Paper Nş. 325
• Albernaz, A.; Ferreira, F. H. G.; Franco, C. 2002. Qualidade e Eqüidade na Educação Fundamental Brasileiro. Pesquisa e PlanejamentoEconômico, v. 33 Nş 3.
• Angrist, J., 1998. Estimating the labor market impact of voluntary military service using Social Security data on military applicants. Econometrica 66, 24988.
• Becker, G. 1964. Human Capital: a theoretical and empirical analysis, with special reference to education. New York: NBER/Columbia University Press.
• Behrman, J.; Cheng, Y.; Todd; P. 2004. Evaluating Pre-school Programs when Length of Exposure to the Program Varies: A Nonparametric Approach.Review of Economics and Statisticsvol.86, n.1, p.108-32.
• Berlinski, S., Galiani, S., Gertler, P. 2009. The effect of pre-primary education on primary school performance.Journal of Public Economics, Volume 93, Issues 1-2, February 2009, Pages 219-234.
• Caliendo, M.; Kopeinig, S. 2005. Some practical guidance for the implementation of propensity score matching. Bonn, Germany: Institute for the Study of Labor (IZA), (IZA Discussion Papers, 1588)
• Carnoy, M.; Loeb, S. 2002. Does external accountability affect student outcomes? A Cross-State Analysis.Educational Evaluation and Policy Analysis, 24(4), 305331.
• Coleman, J. S. et al 1966. Equality of educational opportunity. Washington: Government Printing Office.
• Cunha, F.; Heckman, J.; Lochner, L.; Masterov, D. 2005.Interpreting the Evidence on Life Cycle Skill Formation.NBER Working Paper 11331, Cambridge, MA.
• Curi, A., Menezes-Filho, N. 2006. Os efeitos da pré-escola sobre os salários, a escolaridade e a proficiência escolar. Anais do XXXIV Encontro da ANPEC, Salvador, Bahia, p. 1-21.
• Currie, J.; Thomas, D. 1995.Does Head Start Make a Difference? The American Economic Review vol. 85, n. 3 (Jun., 1995), p. 341-364.
• Currie, J. 2001.Early Childhood Education Programs.Journal of Economic Perspectives, vol.15, n.2, p.213-238.
• Dehejia, R., and Sadek W. (2002).Propensity Score Matching Methods for Nonexperimental Causal Studies, National Bureau of Economics Research Working Paper Nş. 6829, forthcoming Review of Economics and Statistics
• Duflo, E.; Glennerster, R.; Kremer, M. 2006.Using Randomization in Development Economics Research: A Toolkit.MIT Department of Economics Working Paper Nş. 06-36.
• Felicio, F.; Fernandes, R. 2005. O Efeito da Qualidade da Escola sobre o Desempenho Escolar: uma avaliação do ensino fundamental no Estado de São Paulo. In: ENCONTRO NACIONAL DE ECONOMIA, 33, 2005, Salvador.
• Felício, F.; Vasconcellos, L. 2007. O Efeito da Educação Infantil sobre o desempenho escolar medido em exames padronizados. Anais do XXXV Encontro da ANPEC, Recife, Pernambuco, p. 1-22.
• Friedlander, D.; Phil, R. 1995. Evaluating Program Evaluations: New Evidence on Commonly Used Non Experimental Methods. American Economic Review, Vol. 85: 923-937.
• Gray, S.W.; Ramsey, B.; Klaus, R. 1983. From 3 to 20: The Early Training Project in As the Twig is BentLasting Effects of Preschool Programs. Consortium for Longitudinal Studies, ed. Hillsdale NJ.: Erlbaum, p. 171-200.
• Heckman, J. J.; Ichimura, H.; Todd, P. E, 1997. Matching an Econometric Evaluation Estimator: Evidence fron Evaluating a Job Training Programme, Review of Economic Studies, Blackwell Publishing, vol. 64(4), pages 605-54, October.
• Imbens, G. 2004. Nonparametric estimation of average treatment effects under exogeneity: a review. The Review of Economics and Statistics, 86 (1) fev. 2004.
• Jacob, B. A., 2005. Accountability, incentives and behavior: the impact of high-stakes testing in the Chicago public Schools, Journal of Public Economics.Elsevier, vol. 89(5-6), pages 761-796, June.
• Michalopoulos, C.; Bloom, H.; Hill, C., 2004. Can Propensity Score Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-to-Work Programs? Review of Economics & StatisticsVol.86 (1):156-179.
• Restuccia D.; Urrutia C. 2004. Intergenerational Persistence of Earnings: The Role of Early and College Education. The American Economic Review, Vol. 94, Nş. 5 (Dec., 2004), p. 1354-1378.
• Rivkin, S. G., E. A. Hanushek, And J. F. Kain, 2005. Variable Definitions, Data, and Programs for Teachers, Students, and Academic Achievement. Econometrica Supplementary Material, 73, 2.
• Rosenbaum, P., Rubin, D. 1983. The Central Role of the Propensity Score in Observational Studies for Casual Effects. Biometrika n.70, p.41-55.
• Rubin, D. 1977. Assignment to a Treatment group on the basis of a Covariate. Journal of Educational Statistics n. 2, p.1-26.
• Schady, N. 2006. Early childhood development in Latin America and the Caribbean.World Bank Policy Research Working Paper 3869, March.

1

Coleman

et al.(1966) conducted a seminal study on the determinants of school success.

2

For educational accountability results, see Jacob (2005) and Carnoy & Loeb (2002).

3

For further information about the effects of teachers’ characteristics on school success, see Rivkin, Hanushek & Kain (2005).

4

For the effects of some variables related to school infrastructure on students’ performance on standardized exams, see Albernaz, Ferreira & Franco (2002) and Felício&Fernandes (2005).

5

Before FUNDEB, the federal fund for the transfer of grants for education was FUNDEF (Fund for the Maintenance and Development of Elementary Education and Teaching Improvement – implemented by Amendment nº. 14 in September 1996), which guaranteed minimum resources for students enrolled in Fundamental Education. Early childhood education (for children aged 0 to 6 years at the time) and high school education (for teenagers aged 15 to 17 years) were financially supported only by states and municipalities. Given the evidence of the efficiency of investments in Early Childhood Education, this strategy of preferring investments in elementary education, with support from FUNDEF, has been deemed misguided.

6

Sertãozinho is a countryside town in the state of São Paulo whose major economic activity is the sugarcane industry.

7

By comparing preschool attendance between public and private schools (using data from the Basic Education Census), we noted that the average rate between the total number of enrollments in public and private schools in Sertãozinho was 6.29 between 2005 and 2006. When we made the same calculation for Brazil, the average rate was 2.69. In terms of municipal expenditures for Early Childhood Education per student (based on data from the National Treasury Department), we perceived that Sertãozinho spent on average R$1,761 between 2005 and 2006, while the average expenditure at the national level was R$1,196.

8

Actually, some well-known state level exams (e.g., SARESP in the state of São Paulo) are administered and evaluated by the teachers themselves, instead of an external institution.

9

With respect to school history, we collected information about the age at which children entered school and which school they attended at each age. The questions about socioeconomic characteristics were based on the

Prova Brasil questionnaire from 2005. The questionnaire administered in this study may be obtained from the authors upon request.

10

The result observed can be denoted as , where D is a variable that assumes value equal to 1 if individual

i was submitted to treatment, and 0 otherwise.

11

This last equation requires the "Stable Unit Treatment Value Assumption" (SUTVA) to be true. This means that the potential outcome of one unit cannot be related to the treatment status of another unit.

12

Duflo, Glennerster& Kremer (2006) describe several setbacks related to the con duct of random social experiments. They highlight the difficulty in avoiding the contamination of the control sample by possible treatment externalities and by social interactions. They also mention that costs may be remarkably high depending on the study design.

S

13 Α The notation used in (6) – for treatment and

L for control – is not the same one used by Rubin (1977), as it seeks to maintain the notation that was previously employed in this study.

14

See Angrist (1998) for an application of a non-parametric matching.

15

See Breiman

et al. (1984;

apud Heckman, Ichimura & Todd, 1997).

16

In fact, the authors use a Biweight weighting function.

17

See Imbens (2004).

18

See Imbens (2004).

19

See Dehejia&Wahba (2002) for an application of this method.

20

See Rosenbaum & Rubin (1983) and Dehejia & Wahba (2002).

21

Caliendo & Kopeinig (2005) provide the exact formula for the test. However, the maximum acceptable percentage of bias after the matching is not precisely known (the authors assert that 5% should suffice). So, it is important to analyze the two tests jointly.

22

See Caliendo&Kopeinig (2005).

23

The standard errors of these estimates were calculated conventionally and not by bootstrapping. Abadie&Imbens (2006) show that standard errors are not valid if calculated by the bootstrapping method when Nearest Neighbor Matching is implemented. For the other matching algorithms it is not clear whether it is possible or not to apply this technique. We calculated these standard errors, which can be obtained from the authors upon request. However, we highlight that no important difference was verified.

24

The

Table A.1 in the APPENDIX shows the complete results of the two tests administered and

^{Table A.2} shows the number of treatment and control units on and off common support. The discussion of these tests is not reported here because of space limitation, but can be obtained on request.

Publication Dates

• Publication in this collection
  09 Apr 2012
• Date of issue
  Mar 2012

History

• Received
  28 Jan 2011
• Accepted
  12 Dec 2011