SciELO - Scientific Electronic Library Online

 
vol.44 número1Emotions in Mathematics Education: a view based on complex thoughtNotas para uma Pedagogia do Movente: sobre vontade e educação em Henri Bergson índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Indicadores

Links relacionados

Compartilhar


Educação & Realidade

versão impressa ISSN 0100-3143versão On-line ISSN 2175-6236

Educ. Real. vol.44 no.1 Porto Alegre  2019  Epub 25-Fev-2019

http://dx.doi.org/10.1590/2175-623677006 

OTHER THEMES

Assessment of Educational Systems and Educational Management: theoretical contributions

IFundação Carlos Chagas (FCC), São Paulo/SP - Brasil

IIUniversidade de São Paulo (USP), São Paulo/SP - Brasil

ABSTRACT

This paper aims to discuss how different academic studies have analyzed and reported the existing relationships between external large-scale assessment and educational management, both at the level of educational districts as well as at the school level. For this, it initially analyzed the expansion of initiatives for this type of evaluation, in the Brazilian context, at all federated levels. Second, the research from which the present text resulted is explained, and the criteria used for the selection of the studies and the general characteristics of this production are described. Finally, the analysis of the contributions to this production follows, emphasizing the themes discussed and the relations between assessment and management reported in the doctoral studies.

Keywords Large-Scale Assessment; Educational Management; Bibliographic Study

Expansion of Large-Scale Assessments and their Influence on Educational Management

Since the last quarter of the 20th century, the development and expansion of external assessments of educational systems have been observed in Brazil. This is not only in the federal area, but also in the state and municipal spheres. Some of the mechanisms responsible for this expansion are: the concern with improving the educational performance indicators, such as the Index of Basic Education Development (IDEB), and the dropout and retention rates; the need to improve the quality of the educational services offered and; the efficient management of the resources available for education; keeping in mind the enhancement of the pedagogic work conducted within the schools; as well as the effectiveness of the programs and/or actions taken; through the production of information that supports decision-making in the educational area (Bauer, 2012; Freitas, 2013). Another factor to be considered is the introduction of new public management mechanisms and strategies for the management of educational systems, giving the assessment of instructional systems a prominent role.

It is worth mentioning that the interest in forms of evaluation and assessment that could assist in the administration and management of educational systems is not a recent phenomenon. It was triggered at the end of the 20th century under the aegis of the so-called neoliberalism. Coelho (2008) recalls that, within the scope of the Brazilian State, the discussion about the need for evaluations that could serve as an instrument for the administration of the educational system dates back to the 1930s; and, such initiatives are usually associated with the scientific construction of factors of quality, efficiency and productivity of the system.

Coelho’s analysis is echoed in the work of Freitas (2005), which points out that the state’s justifications for evaluating were distinct in the period from the beginning of the second quartile of the twentieth century until the last decade of the century, when evaluation was established in the country through large-scale testing. Aiming initially at verifying whether the objectives of national education were being met, and with a view to expansion and improvement, the various forms of evaluation were justified over the years by the discussion of the need to conduct diagnoses and to know the national education better. As the aim was also to modernize and rationalize public administration and, more recently, to subsidize state regulation and to serve as a parameter for management decisions.

It was from the 1990s that an evaluation model implemented through large-scale testing was expanded began to gain centrality in the administration and management of the educational systems of Brazilian states and municipalities.

This centrality of evaluation for management is also expressed by its presence in the most recent national education plans. Werle (2011) underscores that, in the 2001 National Education Plan, assessment appears linked to the progressive increase in the level of student performance through the use of the National System of Evaluation of Basic Education (SAEB) and its monitoring indicators.

The 2014-2024 PNE (National Education Plan) keeps the central role of external assessement, also referring to the National System of Evaluation of Basic Education as a guide for educational public policies. According to Bandeira (2015), IDEB, the purported quality of education indicator, which provides information about student proficiency in large scale tests and flow, used for monitoring the achievement of the goals proposed, plays a central role. The author emphasizes that the PNE text suggests a close interaction between the curricular proposition and external evaluation:

Goal 7 refers to raising the quality of basic education to the national averages, for the Basic Education Development Index (IDEB). Achieving the intended IDEB goals is related, according to the document, to the establishment of pedagogical directives and to the creation of a common, national curricular base with definitions of learning objectives for each level / grade that will be measured as long as the PNE is in effect. Thus, the text suggests a relationship between attaining the learning objectives stipulated by the common national curricular base and improving the IDEB. For this to happen, the large-scale tests would have to dialogue with the common national curricular base; that is, the processes would have to be thought of in an articulated way, and not in isolation (Bandeira, 2015, p. 2).

In 2007, the advent of the IDEB introduced a new way of relating assessment and management, in which the former became a powerful tool for accomplishing the latter. Based on this argument, Blasis (2013) recalls the role of the IDEB and the projections of bi-annual goals in calling the attention of educational managers to educational indicators and external assessments.

As a result, there is an expansion of local evaluation models for monitoring the school´s systems, allowing forecasting the possible results achieved by the federated entities in the indicator, and making interventions focused on improving the results.

To illustrate this movement, it can be said that, in a survey conducted in 2016, 21 of the 27 Brazilian states had their own proposals for external evaluation, as shown in Table 1. The data illustrate the growing adherence of the states to these large-scale assessment systems, based on the understanding that they provide information for the administrative and pedagogical management of networks.

Table 1 External Evaluations of Educational Networks in the Brazilian States 

State Abbreviation Year System Name Components evaluated Grades evaluated
AC SEAPE 2009 State System for Evaluation of School Learning Portuguese Language and Mathematics 5th and 9th Basic Education (BE); 3rd High-School (HS)
AL AREAL 2009 Evaluation of Student Learning in the State Education Network of Alagoas Portuguese Language, Mathematics and Text Production 5th, 9th BE; 3rd HS
AM SADEAM 2011 Evaluation System of Educational Performance of Amazonas Portuguese Language and Mathematics in BE For HS and Youth and Adult Education (YAE): Human Sciences, Natural Sciences, Text Production 3rd, 7th BE; 3rd HS YAE
BA SABE 2011 Evaluation System of Education of Bahia Avalie Alfa (Alpha Evaluation) and Avalie EM (HS Evaluation) Avalie Alfa - PL and Math and Avalie EM - Languages, Codes and their Technologies; Mathematics and their Technologies; Natural Sciences and their Technologies; and, Human Sciences and their Technologies 2nd BE All years of HS
CE SPAECE 1992 Permanent Evaluation System of Basic Education of Ceará Portuguese Language and Mathematics Evaluation of Literacy - SPAECE Alpha - 2nd grade 2th, 5th and 9th BE All years of HS
DF SIADE 2008 Evaluation System of the Performance of Educational Institutions of the Educational System of the Federal District Portuguese Language and text production, Mathematics, Sciences, Physics, Chemistry and Biology (only in 2008) History, Geography, Philosophy and Sociology (2009 and 2010) 3rd, 5th, 7th, 9th BE 3rd HS End of the 1st, 2nd and 3rd grades of YAE
ES PAEBES 2004 Evaluation Program of Basic Education of Espírito Santo Portuguese Language and Mathematics; Natural, Biological, Physical and Chemical Sciences 5th, 9th BE; 3rd HS
GO SAEGO 2011 Educational Evaluation System of Goiás Portuguese Language and Mathematics 2nd, 5th, 9th BE; 3rd HS
MA ___ 2015 Evaluate Maranhão Portuguese Language and Mathematics 5th, 9th BE; 3rd HS
MG SIMAVE- Proeb 2000 Evaluation System of Public Education / Evaluation Program of the Public Network of Basic Education of Minas Gerais Portuguese Language and Mathematics 5th, 9th BE; 3rd HS
MS SAEMS 2011 Evaluation System of Basic Education of Mato Grosso do Sul Portuguese Language/Text Production and Mathematics 1st, 2nd, 3rd and 4th HS
PA SIPAVE/ SisPAE 2011 2013 Educational Evaluation System of Pará Portuguese Language and Mathematics 4th and 8th BE 1st, 2nd, 3rd HS
PB ___ 2012 Evaluation Paraíba Portuguese Language and Mathematics 5th and 9th BE 3rd HS
PE SAEPE 2000 Educational Evaluation System of Pernambuco Portuguese Language and Mathematics 3rd, 5th and 9th BE; 3rd HS
PI SAEPI 2011 Educational Evaluation System of Piauí Portuguese Language and Mathematics 5th and 9th BE; 3rd HS
PR SAEP 2012 Evaluation System of Basic Education of Paraná Portuguese Language and Mathematics 9th BE; 3rd HS
RJ SAERJ 2008 Educational Evaluation System of the State of Rio de Janeiro Portuguese Language and Mathematics 4th BE; 3rd HS
RO SAERO 2012 Educational Evaluation System of Rondonia Portuguese Language and Mathematics 2nd, 5th, 6th, 9th BE; All Grades of HS
RS SAERS 2007 System of Evaluation of Scholastic Performance of Rio Grande do Sul Portuguese Language and Mathematics 3rd BE and 1st HS 3rd and 6th BE
SP SARESP 1996 System of Evaluation of Scholastic Performance of the State of São Paulo Portuguese Language and Mathematics - all grades. History and Geography - 7th and 9th BE; 3rd HS Text Production - 5th, 7th and 9th years of BE; and 3rd grade of HS. 2nd, 3rd, 5th, 7th, 9th BE; 3rd HS
TO SALTO 2011 Evaluation System of Tocantins Portuguese Language, Mathematics and Sciences 5th and 9th BE; 3rd HS

Source: Developed by the author based on information collected from the sites of states’ Departments of Education and from the CAED/UFJF.

This expansion does not occur, however, only in the subnational states. It is possible to notice that several municipalities have adopted systematic strategies for using the results of the federal assessments, or even proposed their own initiatives for external assessments of their schools’ systems (Bauer; Sousa; Horta Neto; Valle; Pimenta, 2017). This seems to exemplify the recognition, in different spheres of government, of the importance of supporting management actions with information and data collected in a systematic and organized way.

Other studies have sought to understand the concrete influence of large-scale assessments on educational management. Initially, two studies that illustrate the changes in the characteristics of the relationship between assessment and management stand out. These are the studies of Sousa and Oliveira (2007) and of Brooke and Cunha (2011).

During the period from 2005 to 2007, Sousa and Oliveira (2007) coordinated a study that analyzed the assessment systems implemented in five states (Bahia, Ceará, Minas Gerais, Paraná and São Paulo). One of the aspects investigated, in addition to the design of the assessment proposal in these states, was if and how the results produced by them guided the formulation and implementation of educational policies that intervened in the management of educational systems and schools. The study revealed a disparity between the official discourse and reality. If the official documents tended to announce that the assessments supported the decision making and the schools management, influencing the curriculum, the pedagogical practices and the better application of resources, the empirical evidence pointed out that such influences were tenuous or nonexistent. However, in interviews with managers, they seem to expect that large-scale assessments would play an important role in management.

Subsequent research, conducted by Nigel Brooke and Maria Amália Cunha (2011), placed the relationship between external assessment and management at another level. The authors see the increased application of the results of external assessments in what they call a new generation of management policies. For them, the evaluation results were being used as management tools having several objectives: to evaluate and guide educational policies; to propose policies for salary incentives and teacher evaluation; to inform schools about student learning and define continuing education strategies; to inform the public about the results obtained; to allocate resources; and, to certify students and schools.

In the addition to the researches mentioned above, other academic studies were conducted to contribute to the understanding of these relationships. These studies constitute the body of this article1, which seeks to conduct a descriptive analysis of works, selected from a database built with theses and dissertations focusing on their general characteristics (area of concentration, focus of interest, temporal evolution, institutions in which they were produced, advisors, methodological procedures used), theoretical-methodological trends and discussion gaps, in order to understand convergences in this production. The questions that guided the preparation of this text are:

  • What are the characteristics of Brazilian scientific production in the assessments of educational systems between 1988 and 2011?

  • What themes are discussed over time? What themes do not appear as the object of study?

  • What methodologies are used in these studies?

  • What theoretical references have supported the analyses found in the studies?

Finally, from the doctoral studies, we sought to understand the relationships that are established between large-scale assessment and the management of educational systems and schools. The purpose is to perceive the vicissitudes and controversies highlighted by their authors.

Methodological Notes

This study is the result of a bibliographical survey made on Coordination of Superior Level Staff Improvement (CAPES) Theses and Dissertations Database covering studies produced from 1988 to 20112. It sought to include academic studies conducted on the topic of external assessment of educational systems in Brazil during the period, generating a database containing 294 papers3.

To write this article, only studies that deal with the topic of management were selected from this database. To make this selection, searches were conducted using the descriptor management in the fields title, summary and keywords. The selection resulted in 44 studies4: 9 doctoral, 1 professional master’s and 34 academic master’s. Of the master’s studies, only the abstracts were read; whereas, in the case of the doctoral studies, the entire works were read, in addition to the abstracts. Thus, the descriptive analysis is developed based on the abstracts and a selective reading of parts of the studies. Subsequently, comments are made about each doctoral study and their findings regarding the relationship between evaluation and management.

Descriptive Analysis

Although the output is mostly in the area of education or teaching (a total of 30 papers in these two areas), other areas have been discussing the relationship between external assessments and school management or administration. In the selection analyzed, there are studies from graduate programs in the areas of Economics (6), Administration (2), Psychology (1), Political Sciences (1), Linguistics (3) and Geography (1). The classification comes from the information recorded on the Work Sheet, in the CAPES.

The main research lines refer to educational assessment (theory and methodology), curriculum, policies, and educational and didactic management, or to specific teaching methodologies. It is also worth noting the increasing interest in lines of research related to other areas (public administration, organizational management, evaluation of public institutions, economic development, economics of education, social economy, evaluation and psychological instrumentation, etc.).

Figure 1 shows that interest in the relationship between external assessment and management has increased considerably in recent years. The first academic studies that sought to associate the two themes were presented in 2003, although it should be noted that discussions on external evaluations began to appear in the academic environment from 1998 (see Bauer, 2012; Bauer; Reis, 2013). The increase in interest in this topic occurred in 2009, with the number of studies presented in 2010 and 2011 (26 papers) exceeding the sum of studies completed in previous years (18 papers).

Source: Author, from the research database.

Chart 1 Time Distribution of Studies 

Table 2 shows that the academic output under discussion comes from 30 different institutions. Most of them (n = 21), produced only one study, with the highest concentration at UnB (University of Brasília) (n = 5). Some institutions, however, seem to cover lines of research in which studies on this topic are more recurrent, such as Unisinos, UFMG, Unesp from Marília and the Federal University of Ceará (UFC), with three studies each.

Table 2 Number of Studies, per Institution 

Institution N
UnB 5
UFC, UFMG, Unesp/Marília, Unisinos 3
FGV, PUC Rio, Unesp Araraquara 2
ENCE, PUCPR, PUCRS, PUCSP, UCB, UCS, UECE, UFAL, UFAM, UFBA, UFGD, UFJF, UFPE, UFRGS, UFSC, Unesp/Ribeirão Preto, Unicamp, Unimarco, Unirio, Unisal, Uniso 1
Total 44

Source: Author, from the research database.

It is worth mentioning that most studies do not discuss, in their abstracts, the research methodology used. Authors and their advisors should pay more attention to this need, given the importance for completeness of this type of text. Among those studies which provide this information, there is a predominance of qualitative research approaches that often deals with the analysis of single cases. Few studies (4.5%) present mixed or quantitative methodology (4.5%) as a way of looking at reality. About 30% of the authors claim to have used qualitative research techniques in their studies. The analysis of the abstracts, in relation to methodological aspects, allows observation of weaknesses in the education of the authors in relation to such matters. Some confusion among the approaches, techniques and theoretical perspectives is observed when the methods that informed the studies that led to the analysis are presented.

Analysis of the abstracts of 34 masters’ studies allowed grouping them according to some thematic foci. Although apparently quite similar in relation to topics of interest, differences are observed among the studies regarding the management level at which the analyses are directed. Vieira (2007), analyzing concepts related to basic education management, summarizes the differences between two levels of management:

[...] educational management refers to a broad spectrum of initiatives developed by different government instances, whether in terms of shared responsibilities in the provision of education, or other actions developed in their specific fields. School management, in turn, as the term suggests, is found in the school plan and is related to tasks that are within its sphere of coverage.

[...] Thus, it is reasonable to say that educational management is located in the macro sphere, while school management is located in the micro sphere. Both are mutually interconnected, since the first is justified from the second (Vieira, 2007, p. 63, our emphasis).

Taking the author’s inputs as reference, it can be observed that some studies focus on the relations between external assessments and educational management (macro or mesopolitics level), restricting the analysis to central level management (municipal and state education departments, boards or education departments). Other studies focus on analyses within schools, seeking to learn how these relations function based on the work of school managers, directors or pedagogic coordinators, as well as the teachers in their daily classroom management. The aim is to understand the features of the management profile or the professional profiles that make a difference (Catunda, 2007; Rigo, 2010; Casseb, 2011; Costa 2011).

Some studies focus on the influences of actions and programs that take place in the context of educational management on school management, principally on the management of pedagogic aspects such as curriculum organization, teachers’ pedagogic practices and assessment practices.

Several studies aim to analyze factors associated with student performance results obtained in several large-scale assessments, based on several variables, constituting a specific analytic framework (Souza, 2000; Ussan, 2000; Catunda, 2007; Mesquita, 2009; Chirineia, 2010; Wiebusch, 2011; Palermo, 2011; Costa, 2006). It should be noted that, often, in these studies, management and its characteristics, at the macro, meso or micro level, are taken as variables for understanding the phenomenon of student performance.

Table 3 shows the distribution of master’s and doctoral studies in the four axes.

Table 3 Distribution of Studies According to Educational Management, School Management or Analysis of Factors Associated with Student Performance 

Study focus N %
School management 16 36.4
Educational management 16 36.4
Discussion of associated factors 8 18.2
School management and Educational management 4 9.1
Total 44 100.0

Source: Author, based on research data.

It should be noted, however, that this organization has didactic purposes, since studies focused on school management sometimes also investigate the use of assessments in educational management at the other levels of implementation of the policy. Nevertheless, there are few studies that seek to link the prospects of system managers with school managers and teachers (n = 4).

There are studies that discuss the principles and assumptions of public management models, analyzing changes in the paradigms which underlie current educational reforms or, even, the proposition of assessment policies and programs aimed at improving the quality of teaching (Melo, 2005; Santos, 2010b; Andrade, 2011). In this group, there are also studies that deal with new forms of management made within the framework of the New Public Management paradigm (Ceneviva, 2006; Formoso Júnior, 2009; Ferrari, 2010; Alcantara, 2010) and that question the naturalization of economic and market assumptions on educational management. This aspect is illustrated by Spinelli (2004), who understands that large-scale assessment is at the service of economic and marketing principles, which may contribute, paradoxically, to the development of new patterns of educational exclusion.

On the other hand, Ceneviva (2006) is interested in investigating the relationship between large-scale assessment and democratic management. The author points out that the potential of assessment mechanisms to become instruments of democratic control depends more on the structures and relationships that are established within the institutions, in which policy or program evaluation is inserted, than on the actual features of such policies.

Moreira (2004) shows the contradictions inherent to the management model underlying external assessment with its logic of regulation and control, contrasting it to the concept of democratic management of the school emphasized in documents like the Brazilian Federal Constitution.

Finally, some studies examine the education evaluation policies that are proposed as a strategy to support the management and improvement of the quality of education. Similarly, the work of Alcantara (2010) can be cited as a reference. Other studies, such as Chirineia (2010), Escobar (2010), Ferrari (2010), Santos (2010a) and Wiebusch (2011), question the role of evaluation in management.

Another group of studies addresses the implications of large-scale assessments in the realm of the management of the system or the school. Horta Neto (2006) examines how the Board of Education of the Federal District uses the information produced by the Assessment System of Basic Education (SAEB) to manage the public network at the elementary school level. Among other findings, the author points out that the importance of the SAEB´s results for managing the system is present in the discourse of the leaders interviewed. However, there is no evidence of management practices based on these results, which is also illustrated by Bauer (2006). In addition, the author realized the difficulty of the leaders interviewed to understand properly the information produced by SAEB, which is certainly implied in the possibilities of its use. Subsequent studies, such as those of Sousa and Oliveira (2007) and Bauer (2011), reinforce this conclusion.

Vergani (2010) studied how the results of the Evaluation System of School Performance in Rio Grande do Sul served as support for planning educational actions aimed at improving student performance and the quality of education offered, assuming that the results obtained by the diagnostic assessment should be used to support the development of the work. Vergani concluded, as did Horta Neto, that the results of the assessment were not being used according to the purpose supported in their study. Other studies carry out this type of analysis, including the management of systems (Battisti, 2011; Oliveira, 2011) or schools (Escobar, 2010; Garcia, 2010; Gewehr, 2010).

Santos (2010a) was interested in the influences of SPAECE on the practices of the school manager, highlighting which ones are positively related to improving student performance.

Table 4 Sources (Master’s and Doctoral Degrees) 

Level Author Title Institution Year
Master’s ALCANTARA, M. S. Bonus policies and quality indicators: control mechanisms in State schools in the state of São Paulo PUCSP 2010
Master’s MAGALHÃES SOB, M. A. Analysis of the evaluation process of basic education of the municipality of Santa Maria do Pará compared with the INEP tests: a supplementary evaluation instrument UECE 2010
Master’s BARRETO, H. P. D. The large scale assessment in Brazil: comparative analysis between SAEB and a private system UCB 2009
Master’s COSTA, L. O. Effects of school management and individual characteristics of the principal as determinants of the performance of elementary school students UFC 2006
Master’s GARCIA, A. L. School management, quality of education and external evaluation: challenges in and out of school UNESP/Marília 2010
Master’s CHIRINEIA, A. M. The Basic Education Development Index (IDEB) and the dimensions associated with the quality of education UNESP/Marília 2010
Master’s SANTANA, C. F. P. A Continuous teacher training policy and its relationship with the topics of performance evaluation UFGD 2011
Master’s GOMES, F. C. Profile of elementary school teachers and their management styles. A study based on the 2001 SAEB PUC Rio 2004
Master’s ESCOBAR, M. M. C. Information and educational evaluation policies: effective instruments to improve educational management? UFPE 2010
Doctoral SOUZA, S. C. Quasi-market mechanisms in the Brazilian public education UNESP/Marília 2010
Master’s FERRARI, R. M. An idea whose time has come: the institutionalization of public policies in education in the context of the Federal Government UnB 2010
Master’s SANTOS, F. D. G. Impacts generated by the Permanent System of Evaluation of Basic Education of the State of Ceará (SPAECE) in improving teaching and learning in high school UFC 2010
Doctoral ZANARDINI, J. B. Ontology and evaluation of basic education in Brazil (1990-2007) UFSC 2008
Master’s GEWEHR, G. G. Evaluation of basic education: policies and practices in the context of municipal public schools in the city of Pato Branco, Paraná PUCPR 2010
Master’s CENEVIVA, R. Democracy, accountability and evaluation: the evaluation of public policies as an instrument of democratic control FGV 2006
Master’s COSTA, M. H. S. P. Evaluation: teachers’ organizations, paths traveled and challenges imposed on everyday school life UNIRIO 2011
Master’s HORTA NT, J. L. External evaluation: using the 2003 SAEB results in the management of the elementary school system in the Federal District UnB 2006
Doctoral FIGUEIREDO, D. M L. Municipal management, quality of teaching and evaluation of school performance: a study of the municipality of Cosmorama UNESP/Araraquara 2008
Doctoral ARAÚJO, A.C. Education evaluation and quality: contradictions and mediations between school policies and practices in the Federal District UnB 2011
Doctoral SQUILASSE, M. C. The decision-making process in the school: public policies in education and community demands and expectations UNESP/Araraquara 2005
Master’s USSAN, J. L. M. 1997 SAEB evaluation-: infrastructure and organizational variables UFRGS 2000
Master’s MESQUITA, S. S. A. Intraschool factors and school performance: what makes the difference? PUC Rio 2009
Master’s COSTA, M. J. A. Post-LDB nº 9394/96 school inspection in Alagoas: an instance of evaluation or an instrument of state control? UFAL 2009
Master’s MELO, M. S. L. School evaluation as a management tool UFAM 2005
Academic Master’s SOUZA, J. L. Influence of teacher performance and school management on student performance in mathematics (Case Study) UFC 2000
Master’s PALERMO, G. A. Factors associated with school performance: an analysis of mathematics proficiency of fifth grade students in elementary school in the Rio de Janeiro municipal network ENCE 2011
Doctoral AUGUSTO, M. H. O. G. The regulation of educational policies in Minas Gerais and the obligation of results: the challenge of school inspection UFMG 2010
Master’s FORMOSO JR, A. Frontline bureaucrats and the pressure for results FGV 2009
Master’s MEDEIROS, M.L. School managers: a study of the characteristics and administrative practices present in the management of public schools with the best relative performance in the state of São Paulo UNESP/Ribeirão Preto 2011
Master’s VERGANI, F. M. External evaluation of school performance: a pedagogic management tool UCS 2010
Master’s WIEBUSCH, E. M External evaluation: a way to pursue quality in education PUCRS 2011
Master’s SANTOS, J. C. The (counter) reform of public education in Minas Gerais: the evaluation program of public basic education/Proeb under analysis UFJF 2010
Master’s SPINELLI, D. R. M Monitoring evaluation and the materialization of educational reforms of neoliberal character: Brazil of the 90s UNISO 2004
Doctoral ANDRADE, R. J. Quality and equity in Brazilian basic education: evidence from 1995-2003 SAEB UFMG 2008
Master’s CASSEB, M. F. S The impact of the unified curriculum on the Board of Education in the Miracatu region UNISAL 2011
Master’s MOREIRA, R. S. M External evaluation as an instrument of management: access and obstacles UNICAMP 2004
Master’s OLIVEIRA, A. P. M Prova Brasil as a regulation policy of the public network of the Federal District UnB 2011
Master’s RIGO, C. A. State schools’ principals and school planning in the greater São Paulo UNIMARCO 2010
Master’s CATUNDA, A. C. Relationship between the school principal and school performance: a study of data from the network of state schools of Bahia UFBA 2007
Master’s LIMA, M. W. Performance requirements and their impact on the identity of school principals: municipality of Contagem, Minas Gerais UFMG 2011
Doctoral LIMA, E. S. The school principal and the evaluations conducted in the school UnB 2011
Doctoral MÖLLMANN, I. Large-scale management and evaluation: an analysis based on the perspective of private schools in Rio Grande do Sul UNISINOS 2010
Master’s ANDRADE, A. C. Large-scale evaluation: perspective of democratic management UNISINOS 2011
Master’s BATTISTI, L. Large-scale evaluation from the perspective of the municipal management UNISINOS 2011

Source: Author, from the research database.

Costa (2011) discusses the appropriation of assessment results by teachers of a school in the suburb of Rio de Janeiro. Santana (2011) also has the teacher as a central concern, but focuses on how the continuous education programs of the municipal system in Dourados are related to the results of Prova Brasil. Also focusing on the discussion of teacher training, Casseb (2011) tries to understand how the school principal conducts in-service training aimed so that the teachers can understand better the results obtained by the school in the evaluations.

There are also studies dealing with the analysis of the profile of the school principal or the features of their management styles, seeking to understand if there are differences in the results correlated with these aspects. The studies of Gomes (2004), Rigo (2010), Medeiros (2011) and Lima (2011a) illustrate this point.

Maria José Alves Costa (2009) understands that school inspection has been an instrument to control teaching throughout the history of Brazilian education. She analyzes, in the context of Alagoas, post-LDB of 1996, what their current functions, dimensions and practices are. She understands that external evaluations introduce new forms of inspection and management of the institutions.

Finally, Barreto (2009), Magalhães Sobrinho (2010) and Andrade (2011) deal with several evaluation policies. They analyze their logic in the realm of educational management, their relationship to other policies present in the municipality and their relationships and clashes with the conceptualization of democratic management, respectively.

Theoretical Frameworks Used

In order to understand the conceptual frameworks that have guided the analyses of the relationships between external assessment and management, it is important to systematize the theoretical sources used by the authors of selected works.

Several of the authors refer to changes in the role of the state that started in the 1980s. These changes are characterized by the weakening of the state of social welfare and the tendency to decentralize social services without, however, losing control of them. In this case, they point out that the assessment of educational systems becomes a prominent point in the proposals of public policies in various countries where they are obligated to rationalize the application of the available resources, to decentralize the management of the educational system and, at the same time, to control it. They assume that, through rationalization and control, management is no longer carried out in the bureaucratic structures, but guided by the results of performance indicators and attainment of predetermined goals set by government decision-makers.

Another type of discussion in the studies analyzed tends to refer to the critique of the so called Neoliberal State, to the influence of the multilateral institutions on the proposition of external assessments proposals and to the rise of the Evaluative State. In this logic, the state absolves itself of the direct management of the system, focusing on management by results, for example. Such studies are generally based on the work by Almerindo Janela Afonso.

Table 5 shows the main works used by the authors of the selected studies. It also shows the number of times they were cited in the 44 studies under discussion.

Table 5 Main Works Used as Theoretical References in the Studies Analyzed 

Title of the work Author Country Type of work N
Educational evaluation: regulation and emancipation. For a sociology of contemporary evaluation policies Almerindo Janela Afonso Portugal Book 9
Education policies and educational evaluation: For a sociological analysis of educational reform in Portugal (1985-1995) Almerindo Janela Afonso Portugal Book 9
Times for educational evaluation: SAEB, its agents, references and trends Alicia Catalano de Bonamino Brazil Book 8
School learning evaluation Cipriano Carlos Luckesi Brazil Book 7
Large-scale evaluation and curricular standards: the scales of proficiency in math and reading in Brazil Nilma S. Fontanive Brazil Book chapter 6
Large-scale evaluation: an innovative proposal Ruben Klein and Nilma S. Fontanive Brazil Article 5
Evaluation - from excellence to the regulation of learnings: between two logics Philippe Perrenoud Switzerland Book 5
Evaluation: building the field and the criticism Luiz Carlos de Freitas Brazil Book 5
Field and ways of evaluation: the evaluation of higher education in Brazil José Dias Sobrinho Brazil Book chapter 5
Possible impacts of evaluation policies on the school curriculum Sandra Maria Zákia Lian Sousa Brazil Article 5

Source: Author, from the research database.

However, in some of the works there is a certain inconsistency between the theoretical base and the kind of analysis proposed. Among studies that propose to analyze the relationships between educational assessment and management, greater interaction with management and public administration issues, or even the role of evaluation in the management of educational policies, would be expected. However, not all the studies relate, more directly, with the authors who discuss educational management. There are some studies that refer mainly to authors of reference on the theme of learning assessment, with little connection to their object of study.

It is understood that studies from the early 1990s may have suffered from the lack of theoretical references that discuss large-scale assessment in Brazil. Therefore, they had to resort to reference works on learning assessmen, more abundant during the 1980s and 1990s, to compensate for the lack of theoretical references. However, in the early 2000s, there is an increase of studies, mainly theses and dissertations, trying to understand the phenomenon of consolidation and expansion of large-scale assessment. The analysis of the references used in the various studies is not enough to confirm that this new production was considered by the authors under discussion, except for a few studies that have become reference (as, for example, Afonso, Bonamino and Luiz Carlos de Freitas).

In addition, based on a detailed reading of the theoretical references reported and actually used in the body of the work (not just cited in the references), it seems that the theoretical discussion does not always encompass the complexity of the phenomenon being studied. An example of this is the set of studies that tend to discuss large-scale, external evaluation within the framework of democratic management, bureaucratic management or the new public management (which are less common), without contextualizing them within the framework of actions and the role(s) that the state has taken on today, supporting the adoption of these management models.

It is also worth mentioning the little dialogue with international authors, except for the Portuguese, especially Afonso (2000), which allows inferring that there is a language barrier that has hampered access to new theoretical contributions. What is not explained by the language barrier, however, is the little use of other authors who have discussed similar topics at the graduate level in recent studies, allowing us to conclude that the dense and accurate literature review, an essential resource to academic production, has been relegated to second place Even in doctoral studies, the predominance of references to articles and book chapters is seen, also with little appropriation of foreign literature and of the national production from graduate programs. While access to scientific literature has expanded considerably, considering the texts that have been the object of this analysis, it seems that such access does not translate into use by authors. This may even be due to the strict deadlines for the production of articles. As a result, one can point to a certain overlap of objects of study and analyses, which do not contribute much to the advancement of knowledge in the field.

A Look at the PhDs Produced in the Period

Table 4 shows nine doctoral works from the specified selection which focus on the relationship between large-scale assessment and educational management.

The first identified study that deals with the influence of external assessment in management is by Squilasse (2005, p. 16). The author sought to “[...] understand the complex school reality and the decision-making mechanisms within the school”, determining the relationship between the center (deliberative and normative bodies of the education system) and the periphery (school unit). To do so, two schools and their actions and reactions to some political devices at the federal and regional levels were analyzed. The LDB guidelines (Brasil, 1996) were used as the national reference base, and the System of School State Performance Assessment of the State of São Paulo (SARESP), the Programa Recuperação de Férias (Vacation Remedial Classes Program) and the Forum A escola dos nossos sonhos (The School of Our Dreams) were used as the state reference. It is one of the few studies that seeks “[...] to investigate how schools, with different management styles and organizational cultures perceive, interpret and implement public policies from governing bodies” (Squilasse, 2005, p. 20), unveiling the power relations and the resistance that are established between schools and central agencies. The results found by the author indicate a distance between the development and the implementation of public policies in education. When comparing two separate school realities, the way that school management bodies understand the guidance from the governing bodies, whether in relation to the guidelines from LDB or in relation to the proposition from SARESP, is seen to be different in the two schools. Therefore, if the school management is democratic, the implemented policies ensure the consolidation of democratic spaces. In contrast, a school management based on a more centralized model induces undemocratic and less participatory practices within the school.

Figueiredo (2008) discussed the role of the school performance assessment, called the Municipal School Performance Assessment System (SAREM), developed by the Municipal Secretariat of Education of Cosmorama (SME) as a way to diagnose the educational reality of this municipality. This was done to analyze whether there had been advances in terms of student learning, based on the assessment. According to the author, the student performance indicators provided by the SAREM subsidized the development of technical-pedagogic intervention proposals by the Municipal Department of Education, enabling the improvement of the education management system and making contributions to the reorganization of the curricular components and of the pedagogical processes in the schools.

Zanardini (2008) examined the ontological assumptions present in the emergence and consolidation process of the external assessment with one of the management pillars of basic education in Brazil, discussing the current role of educational reforms. For the author, the large-scale assessments investigated (SAEB, ENEM5 and PISA) had a liberal-conservative reference which, among other things, valued the measuring of results and not the understanding of the problems within the school, and that strongly affect the results obtained. The assessment would thus serve to maintain a social structure that stands on the exploitation of the work and the maintenance of a social status quo. It was concluded that there were contradictions between the effective assessment model and the proposal defended in the policies and programs supporting the quality of education.

Andrade (2008) set out to analyze the quality and equity of the Brazilian system of basic education based on student performance data, measured by the Evaluation System of Basic Education (SAEB) in the 1995, 1997, 1999, 2001 and 2003 cycles. By analyzing the data, the author wanted to investigate which factors impact student performance, considering that this is a reflection of an interrelation of factors and choosing some of them for analysis: the effect of the school on the student, the administrative dependency of the school and the race/color of the students. By adjusting the hierarchical regression models, the author obtains results that indicate that some schools have a different effect. Therefore, they must have a practice of pedagogic management of their resources that could be shared with other schools, assisting in the improvement of basic education in Brazil. It is worth mentioning that, even if it had been selected by the criteria determined by the selection of the study, the work of Andrade is not, directly, a discussion of educational management. From its considerations of the quality and racial equity in Brazilian schools, the author points to paths that should be the focus of concern of proponents of educational policies and managers, but does not enter into the proposed discussions.

Möllman (2010) tries to understand the relationship between large-scale assessment and the management of basic education from a private school perspective in Rio Grande do Sul, analyzing how they use external assessment results and how these uses affect management. The study included a total of 77 of the 482 private schools that offer basic education. Based on a theoretical framework that distinguishes management and administration, it uses contributions that make it possible to understand the current role of external assessment in educational management. His work includes extensive bibliographical research, mapping and discussing the production of knowledge about basic education evaluation, private school management and large-scale assessment. The author concludes that most private schools studied recognize the importance of external assessments as a tool to help management, both in the planning and the diagnosis of reality.

Augusto (2010) focused on the effects of the changes on school inspection work in effect in the state of Minas Gerais, in a context of modernization of public services influenced by a managerial approach aimed at rationalizing and regulating public services. The author aimed to understand if the introduction of new regulation mechanisms interfered or not in school work, based on the organization of school inspection in the state of Minas Gerais. She started from the assumption that the current policies under the Programa Choque de Gestão (Management Shock Program) would be inconsistent with the very function of the inspection. To support her analysis, she turned to the nature and career of school inspection, analyzed the so called Management Shock policy and focused on the concept of regulation, tracing its trajectory from its origins to the present day. She concluded that two management models coexisted in Minas Gerais: the bureaucratic, which includes the inspection; and the managerial, focused on results, which includes the programs Acordo de Resultados (Results Agreement) and Choque de Gestão (Management Shock), introducing an ambiguity and a contradiction in conducting policies. From Augusto’s work, it can be said that the regulation of SEEMG’s educational policies is represented by a linear and vertical model, based on the intervening measures proposed in the scope of the Management Shock. Inspection, in turn, deals with the role of regulating the regulation policy, i.e., the Acordo de Resultados, in a mix of the two regulatory models.

Souza (2010) sought evidence of the presence of quasi-market mechanisms in Brazilian educational policies, focusing on the National Education Plan, the Education Development Plan, the Basic Education Assessment System (SAEB), the Index Development of Basic education (IDEB), the National Secondary Education Examination (ENEM) and the articles of the Federal Constitution that deals with education as well as the Law of Directives and Bases of National Education. The author is one of the few, among the studies selected, to contextualize the Brazilian educational reform in the context of state reform, introducing the quasi-market concept in education. Supported by several authors, she discusses the replacement of the term school administration by educational management, arguing that the latter introduces business management approaches into educational management. She also points out that the 1988 Federal Constitution would not have assumed the new principles of the new public management, such as efficiency, effectiveness and market, but rather emphasized the principle of democratic management of education that, although not explained in its meaning and forms of consolidation, will also support the proposed Law of Education Guidelines and Bases (LDB), 1996. Thus, the principles more consistent with those of business management would have been implemented by State Reforms, which led to inconsistencies between the desirable ones, stated in the letter of the law, and the ones carried out from the consolidation of educational policies that influence the management of the educational system. The author also points out that the emphasis on the discourse of democratic management would have been part of a strategy of unaccountability of the State, consistent with the parameters of technocratic public management or new public management.

Other studies, besides Souza’s, show inconsistencies between the conception of the current management in the federal legal framework and the conception of educational management expressed in contemporary educational policies and external assessments of educational systems. This is the case of the work of Araújo (2011) and of Lima (2011c).

The former, analyzing two educational programs of the State Department of Education of the Federal District, implemented in the years 2007 and 2010, the Law of Shared Management of Schools and the System of Performance Assessment of Educational Institutions of the Educational System of the Federal District (SIADE), realized that the Lei de Gestão Compartilhada (Law of Shared Management) and SIADE interfered in the organization of school work, inducing a certain standard of quality practiced by schools. Such standard would be highly influenced by a management logic that values efficiency, the achievement of goals, the short term, introducing a phase of results management in the local educational management, in accordance with the dominant characteristics in the management and assessment processes listed by Souza (2010). Thus, Araújo (2011) develops his work pointing out contradictions between what was planned by the Education Department in relation to the two projects, and what was actually done in two public schools in the region which comprised his field study, illustrating the discussion proposed by Souza (2010) in his study.

Lima (2011a) tried to understand the action and influence of the school principal on institutional and large-scale learning assessment practices. In order to do this, the author analyzed the participation of the principal in activities and moments of interaction with teachers, pedagogical coordinators, students and parents, in discussions related to assessment. In the construction of the argument, he discussed the principal’s understanding of the assessment practices that occurred in school and the reflections on these practices. Focusing on the influence of the System of Performance Assessment of Educational Institutions of the Educational System of the Federal District (SIADE) on the work of the principal, the author concludes that the articulation within the school of the three dimensions of the evaluation analyzed does not depend only on its managers, mainly since large-scale assessments involve other actors and interests that interfere with the possibilities of more articulated work within the school. Regarding the institutional evaluation, the study pointed out that it was not yet a well-known practice in the school, which was not prepared to carry it out according to the current proposals.

In summary, the various studies contribute, from several foci and objects of analysis, to the understanding of management conceptions that inform both educational management and school. Whether through the discussion of the institutionalization of an educational management concept based on the principles of the New Public Management, its opposition to the concept of democratic management, or through the use of concepts such as market, state unaccountability and regulation, doctoral works seem to advance in the sense of incorporating theoretical supports that allow moving forward in the understanding of management models based on external assessment.

However, some studies focus on very partial analyses, or express the relations between external assessment and school management in a one-way path, disregarding the normative infidelity of schools, which would lead to the perception, as pointed out by Licínio Lima (2011b, p. 114) that, “[...] even in a centralized administration, the force of normative imposition is not always observed, or translated into power and actions oriented accordingly, whether in the central administration itself or in peripheral schools”. Perhaps because of the influence of the theoretical frameworks used, the works under discussion often do not present an important contribution to elucidate the complex relations that exist between large-scale assessment and management, be it educational or school management, from the increase of the large-scale assessments in the Brazilian context.

Conclusions

This brief description of the topics that have been object of study in the selected works shows an increase in the topics that have been quite controversial in discussions about educational system assessments. Although most studies are qualitative, focused on isolated cases, together they point to trends in the path that assessment policies have taken in several instances, the controversies regarding these policies, how they have influenced the work in schools and the education system itself.

However, studies do not discuss some of the issues that would need to be investigated further, after more than twenty years since SAEB’s implementation and in a context of expansion of the assessment systems in Brazilian states and municipalities. Studies that focus more on the schools and educational systems curricula, considering the intensification of large-scale assessment as well as on the use of the results as the basis for establishing other policies related to the management system (setting bonuses and hiring teachers and managers, distribution of resources to schools, dissemination of the results achieved, development of quality indexes, among others), would provide timely information to create a panel that could describe how these issues are developing in the national territory. There are contradictions in the investigated literature on the influence of assessments on determining the school curriculum, for example. While some studies claim curricular reductionism in relation to what is focused on, in the evidence matrices, others point out that the assessments do not focus on the school curriculum. Such aspects deserve further investigation, mainly on an empirical basis, that take into consideration the contradictions pointed out by the various existing studies.

Finally, to conclude, there are some considerations that concern the interested researcher. After looking into the studies analyzed within the scope of this article, one has the impression that, with rare exceptions, the theoretical production on the topic, in graduate programs, has not provided significant advances in understanding the complex relationships that are established between assessment and educational management.

It seems that the topic would benefit if those interested in expanding the knowledge produced about these relationships interacted more systematically with studies already written on the subject, partially cited in this article, identifying aspects that require additional investigation, after the continuous mapping of the production. A closer look at the existing production, conducted through extensive bibliographical revision, would allow the field to provide its own feedback and new approaches to be found to examine common problems, which seem to have been studied in isolation by researchers who still do not know much about the production on the topic. It seems necessary to broaden the scope of the work, theoretically and methodologically, so that the expansion of interest in the topic results in the production of new knowledge and not just in dispersion.

REFERENCES

AFONSO, Almerindo Janela. Avaliação Educacional: regulação e emancipação. São Paulo: Cortez, 2000. [ Links ]

ALCANTARA, Melina Sant’Anna. Políticas de Bonificação e Indicadores de Qualidade: mecanismos de controle nas escolas estaduais paulistas. 2010. 94 f. Dissertação (Mestrado em Educação: história, política, sociedade) - Pontifícia Universidade Católica de São Paulo, São Paulo, 2010. [ Links ]

ANDRADE, Alenis Cleusa de. Avaliação em Larga Escala na Educação Básica, Prova Brasil, em Perspectiva da Gestão Democrática: um estudo a partir da associação de municípios do vale do Rio dos Sinos e Município de São Leopoldo-RS. 2011. 147 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade do Vale do Rio dos Sinos, São Leopoldo, 2011. [ Links ]

ANDRADE, Renato Júdice de. Qualidade e Equidade na Educação Básica Brasileira: as evidências do Saeb 1995-2003. 2008. 230 f. Tese (Doutorado em Educação) - Programa de Pós-Graduação em Educação: Conhecimento e Inclusão Social, Faculdade de Educação, Universidade Federal de Minas Gerais, Belo Horizonte, 2008. [ Links ]

ARAÚJO, Adilson César de. Avaliação e Qualidade da Educação: contradições e mediações entre políticas e a prática escolar no Distrito Federal. 2011. 269 f. Tese (Doutorado em Educação) - Faculdade de Educação, Universidade de Brasília, 2011. [ Links ]

AUGUSTO, Maria Helena Oliveira Gonçalves. A Regulação das Políticas Educacionais em Minas Gerais e a Obrigação de Resultados: o desafio da inspeção escolar. 2010. 279 f. Tese (Doutorado em Educação) - Programa de Pós-Graduação em Educação, Universidade Federal de Minas Gerais, Belo Horizonte, 2010. [ Links ]

BANDEIRA, Cláudia. A Avaliação Educacional no Plano Nacional de Educação - PNE. De Olho nos Plano, São Paulo, p. 1-3, 16 abr. 2015. Disponível em: <Disponível em: http://www.deolhonosplanos.org.br/wp-content/uploads/2015/04/An%C3%A1lise-do-tema-da-avalia%C3%A7%C3%A3o-educacional-no-PNE-por-Claudia-Bandeira.pdf >. Acesso em: 02 jan. 2019. [ Links ]

BARRETO, Hydnéa Ponciano Domingueti. A Avaliação em Larga Escala no Brasil: análise comparativa entre o Saeb e um sistema privado. 2009. 144 f. Dissertação (Mestrado em Educação) - Universidade Católica de Brasília, Brasília, 2009. [ Links ]

BATTISTI, Luzia. Avaliação em Larga Escala na Perspectiva da Gestão Municipal. 2010. 98 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade do Vale do Rio dos Sinos, São Leopoldo, 2011. [ Links ]

BAUER, Adriana. Usos dos Resultados do SARESP: o papel da avaliação nas políticas de formação docente. 2006. Dissertação (Mestrado em Educação) - Faculdade de Educação, Universidade de São Paulo, São Paulo, 2006. [ Links ]

BAUER, Adriana. Formação Continuada de Professores e Resultados dos Alunos no Saresp: propostas e realizações. Educação e Pesquisa, São Paulo, v. 37, n. 4, p. 809-824, dez. 2011. [ Links ]

BAUER, Adriana. Estudos sobre Sistemas de Avaliação Educacional. Revista @mbienteeducação, São Paulo, v. 5, p. 7-31, 2012. [ Links ]

BAUER, Adriana; REIS, Adriana Teixeira. Balanço da Produção Teórica sobre Avaliação de Sistemas Educacionais no Brasil: 1988-2011. In: REUNIÃO NACIONAL DA ANPED, 36, 2013, Goiânia. Anais...Goiânia: Anped, 2013. P. 1-18. [ Links ]

BAUER, Adriana; SOUSA, Sandra Maria Zákia Lian; HORTA NETO, João; VALLE, Raquel; PIMENTA, Cláudia. Iniciativas de Avaliação do Ensino Fundamental em Municípios Brasileiros: mapeamento e tendências. Revista Brasileira de Educação, Rio de Janeiro, v. 22, p. 1-19, 2017. [ Links ]

BLASIS, Eloísa de. Avaliações em Larga Escala: contribuições para a melhoria da qualidade na educação. Cadernos Cenpec, São Paulo, v. 3, n. 1, p. 251-268, jun. 2013. [ Links ]

BRASIL. Lei nº 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Diário Oficial da União, Brasília, 23 dez. 1996. [ Links ]

BROOKE, Nigel; CUNHA, Maria Amália. A Avaliação Externa como Instrumento da Gestão Educacional nos Estados. Estudos & Pesquisas Educacionais, São Paulo, p. 17-79, 2011. [ Links ]

CASSEB, Maria de Fátima Soares. O Impacto do Currículo Unificado na Diretoria de Ensino da Região de Miracatu. 2011. 92 f. Dissertação (Mestrado em Educação) - Centro Universitário Salesiano de São Paulo, Americana, 2011. [ Links ]

CATUNDA, Arturo Cavalcanti. Relação entre Competência do Diretor Escolar e Desempenho da Escola: um estudo de dados da rede estadual de ensino da Bahia. 2007. 154 f. Dissertação (Mestrado em Administração) - Núcleo de Pós-Graduação em Administração, Escola de Administração, Universidade Federal da Bahia, Salvador, 2007. [ Links ]

CENEVIVA, Ricardo. Democracia, Accountability e Avaliação: a avaliação de políticas públicas como instrumento de controle democrático. 2006. 160 f. Dissertação (Mestrado em Administração Pública e Governo) - Escola de Administração de Empresas de São Paulo, Fundação Getúlio Vargas, São Paulo, 2006. [ Links ]

CHIRINEIA, Andréia Melanda. O Índice de Desenvolvimento da Educação Básica (Ideb) e as Dimensões Associadas à Qualidade da Educação. 2010. 119 f. Dissertação (Mestrado em Educação) - Faculdade de Filosofia e Ciências, Universidade Estadual Paulista “Julio de Mesquita Filho”, Marília, 2010. [ Links ]

COELHO, Maria Inês. Vinte Anos de Avaliação da Educação Básica no Brasil: aprendizagens e desafios. Ensaio: avaliação e políticas públicas educacionais, Rio de Janeiro, v. 16, n. 59, p. 229-258, jun. 2008. [ Links ]

COSTA, Leandro Oliveira. Efeitos da Gestão Escolar e Características Individuais do Diretor Determinantes do Desempenho dos Estudantes do Ensino Fundamental. 2006. 67f. Dissertação (Mestrado em Economia) - Faculdade de Economia, Administração, Atuária e Contabilidade, Universidade Federal do Ceará, Fortaleza, 2006. [ Links ]

COSTA, Maria Helena dos Santos Prazeres. Avaliação: tessituras docentes, caminhos percorridos e desafios que se impõem no cotidiano escolar. 2011. 178 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade Federal do Estado do Rio de Janeiro, Rio de Janeiro, 2011. [ Links ]

COSTA, Maria José Alves. A Inspeção Escolar em Alagoas pós LDB nº 9394/96: uma instância de avaliação ou um instrumento de controle do Estado?. 2009. 100 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação Brasileira do Centro de Educação, Universidade Federal de Alagoas, Maceió, 2009. [ Links ]

ESCOBAR, Maria Mônica Carvalho. Políticas de Informação e de Avaliação Educacional: instrumentos efetivos para a melhoria da gestão pedagógica? 2010. 124 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade Federal de Pernambuco, Recife, 2010. [ Links ]

FERRARI, Rafael Martins. Uma Ideia cujo Tempo chegou: a institucionalização da avaliação de políticas públicas em educação no contexto do Governo Federal. 2010. 23f. Dissertação (Mestrado em Administração) - Programa de Pós-Graduação em Administração da Faculdade de Economia, Administração e Contabilidade, Universidade de Brasília, Brasília, 2010. [ Links ]

FIGUEIREDO, Dione Maribel Lissoni. Gestão Municipal, Qualidade de Ensino e Avaliação do Rendimento Escolar: um estudo do município de Cosmorama. 2008. 367 f. Tese (Doutorado em Educação Escolar) - Pós-Graduação em Educação Escolar, Universidade Estadual Paulista “Julio de Mesquita Filho”, Araraquara, 2008. [ Links ]

FORMOSO JÚNIOR, Aurélio. Burocratas de Linha de Frente e a Pressão por Resultados. 2009. Dissertação (Mestrado em Administração Pública e Governo) - Escola de Administração de Empresas de São Paulo, Fundação Getúlio Vargas, São Paulo, 2009. [ Links ]

FREITAS, Dirce Nei Teixeira. A Avaliação da Educação Básica no Brasil: dimensão normativa, pedagógica e educativa. In: REUNIÃO ANUAL DA ANPED, 28, 2005, Caxambu. Anais... Caxambu, 2005. [ Links ]

FREITAS, Dirce Nei Teixeira. Avaliação da Educação Básica no Brasil: características e pressupostos. BAUER, Adriana; GATTI, Bernadete A.; TAVARES, Marialva R. Vinte e cinco Anos de Avaliação de Sistemas Educacionais no Brasil: origens e pressupostos. Florianópolis: Insular, 2013. P. 70-96. [ Links ]

GARCIA, Ana Lúcia. Gestão da Escola, Qualidade do Ensino e Avaliação Externa: desafios na e da escola. 2010. 97 f. Dissertação (Mestrado em Educação) - Faculdade de Filosofia e Ciências, Universidade Estadual Paulista “Julio de Mesquita Filho”, Marília, 2010. [ Links ]

GEWEHR, Glaer Gianne. Avaliação da Educação Básica: políticas e práticas no contexto de escolas públicas municipais na cidade de Pato Branco-Paraná. 2010. Dissertação (Mestrado em Ciências Humanas) - Pontifícia Universidade Católica do Paraná, Curitiba, 2010. [ Links ]

GOMES, Francisco Carlos. O Perfil dos Professores da Escola Fundamental e seus Estilos de Gestão: um estudo a partir do Saeb 2001. 2004. Dissertação (Mestrado em Educação) - Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, 2004. [ Links ]

HORTA NETO, João Luiz. Avaliação Externa: a utilização dos resultados do Saeb 2003 na gestão do sistema de ensino fundamental no Distrito Federal. 2006. 143 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade de Brasília, Brasília, 2006. [ Links ]

LIMA, Erisevelton Silva. O Diretor e as Avaliações Praticadas na Escola. 2011. 227f. Tese (Doutorado em Educação) - Programa de Pós-Graduação em Educação, Faculdade de Educação, Universidade de Brasília, Brasília, 2011a. [ Links ]

LIMA, Licínio. A Escola como Organização Educativa: uma abordagem sociológica. 4. ed. São Paulo: Cortez Editora, 2011b. [ Links ]

LIMA, Marcos Welington de. As Exigências de Performatividade e seus Impactos na Identidade dos Diretores Escolares: município de Contagem-MG. 2011. 196f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação: Conhecimento e Inclusão Social, Faculdade de Educação, Universidade Federal de Minas Gerais, Belo Horizonte, 2011c. [ Links ]

MAGALHÃES SOBRINHO, Mauro Alves. Análise do Processo de Avaliação do Ensino Básico do Município de Santa Maria do Pará comparado com os Exames do INEP: um instrumento complementar de avaliação. 2010. Dissertação (Mestrado em Políticas Públicas) - Programa de Pós-Graduação em Políticas Públicas, Universidade Estadual do Ceará, Fortaleza, 2010. [ Links ]

MEDEIROS, Mirna de Lima. Gestores Escolares: um estudo das características e práticas administrativas presentes na gestão de escolas públicas com melhor desempenho relativo do estado de São Paulo. 2011. 234 f. Dissertação (Mestrado em Ciências) - Programa de Pós-Graduação em Administração de Organizações, Faculdade de Economia, Administração e Contabilidade de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, 2011. [ Links ]

MELO, Maria Sirleide Lima de. Avaliação Institucional como Instrumento de Gestão. 2005. Dissertação (Mestrado em Educação) - Universidade Federal do Amazonas, Manaus, 2005. [ Links ]

MESQUITA, Silvana Soares de Araujo. Fatores Intraescolares e Desempenho Escolar: o que faz a diferença?. 2009. 126f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Pontifícia Universidade Católica do Rio de Janeiro, Rio de Janeiro, 2009. [ Links ]

MÖLLMANN, Ibanor. Gestão e Avaliação em Larga Escala: uma análise a partir da perspectiva de escolas privadas no Rio Grande do Sul. 2010. 142 f. Tese (Doutorado em Educação) - Programa de Pós-Graduação em Educação, Universidade do Vale do Rio dos Sinos, São Leopoldo, 2010. [ Links ]

MOREIRA, Rozemeiry dos Santos Marques. Avaliação Externa como Instrumento da Gestão: a adesão e os impasses de sua realização. 2004. 122 f. Dissertação (Mestrado em Educação) - Faculdade de Educação, Universidade Estadual de Campinas, Campinas, 2004. [ Links ]

OLIVEIRA, Ana Paula de Matos. A Prova Brasil como Política de Regulação da Rede Pública do Distrito Federal. 2011. 274 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade de Brasília, Brasília, 2011. [ Links ]

PALERMO, Gabrielle A. Fatores Associados ao Desempenho Escolar: uma análise da proficiência em matemática dos alunos do 5º ano do ensino fundamental da rede municipal do Rio de Janeiro. 2011. Dissertação (Mestrado em Estudos Populacionais e Pesquisas Sociais) - Escola Nacional de Ciências Estatísticas, Rio de Janeiro, 2011. [ Links ]

RIGO, Cosmo de Almeida. Diretores de Escolas Estaduais e Planejamento Escolar na Grande São Paulo. 2010. Dissertação (Mestrado em Educação, Administração e Comunicação) - Universidade de São Marcos, São Paulo, 2010. [ Links ]

SANTANA, Cristina Fátima Pires Ávila. A Política de Formação Continuada de Professores e a sua Relação com os Tópicos da Avaliação de Desempenho. 2011. 125 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Faculdade de Educação, Universidade Federal da Grande Dourados, Dourados, 2011. [ Links ]

SANTOS, Francesca Danielle Gurgel dos. Impactos gerados pelo Sistema Permanente de Avaliação da Educação Básica do Estado do Ceará (Spaece) na Melhoria do Ensino e Aprendizagem no Ensino Médio. 2010. 189 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação da Faculdade de Educação, Universidade Federal do Ceará, Fortaleza, 2010a. [ Links ]

SANTOS, Josiane Cristina dos. A (Contra) Reforma da Educação Pública em Minas Gerais: o programa de avaliação da rede pública de educação básica/Proeb em análise. 2010. 172 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Faculdade de Educação, Universidade Federal de Juiz de Fora, Juiz de Fora, 2010b. [ Links ]

SOUSA, Sandra Zákia; OLIVEIRA, Romualdo Portela. Sistemas de Avaliação Educacional no Brasil: características, tendências e uso dos resultados. (Relatório de Pesquisa apresentado à FAPESP). São Paulo: Faculdade de Educação da USP, 2007. [ Links ]

SOUZA, Juceli Lima de. Influência da Performance Docente e da Gestão Escolar no Desempenho dos Alunos em Matemática (Estudo de Caso). 2000. Dissertação (Mestrado em Educação) - Universidade Federal do Ceará, Fortaleza, 2000. [ Links ]

SOUZA, Silvia Cristina de. Mecanismos de Quase-Mercado na Educação Pública Brasileira. 2010. 149 f. Tese (Doutorado em Políticas Públicas e Administração da Educação Brasileira) - Programa de Pós-Graduação em Educação, Faculdade de Filosofia e Ciências, Universidade Estadual Paulista “Julio de Mesquita Filho”, Marília, 2010. [ Links ]

SPINELLI, Daniela Renna Magistrini. A Avaliação de Monitoramento e a Materialização das Reformas Educacionais de Caráter Neoliberal: Brasil dos anos 90. 2004. 192 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade de Sorocaba, Sorocaba, 2004. [ Links ]

SQUILASSE, Mario do Carmo. O Processo de Tomada de Decisão na Escola: as políticas públicas em educação e as demandas e expectativas da comunidade. 2005. Tese (Doutorado em Educação Escolar) - Pós-Graduação em Educação Escolar, Universidade Estadual Paulista “Julio de Mesquita Filho”, Araraquara, 2005. [ Links ]

USSAN, Jorge Lisandro Maia. Avaliação do Saeb - 1997: infraestrutura e variáveis organizacionais. 2000. 80f. Dissertação (Mestrado em Administração) - Programa de Pós-Graduação em Administração, Universidade Federal do Rio Grande do Sul, Porto Alegre, 2000. [ Links ]

VERGANI, Flávia Melice. Avaliação Externa de Rendimento Escolar: um instrumento para a gestão pedagógica. 2010. 124 f. Dissertação (Mestrado em Educação) - Programa de Pós-Graduação em Educação, Universidade de Caxias do Sul, Caxias do Sul, 2010. [ Links ]

VIEIRA, Sofia Lerche. Política(s) e Gestão da Educação Básica: revisitando conceitos simples. Revista Brasileira de Política e Administração da Educação, Porto Alegre, v. 23, n. 1, p. 53-69, jan./abr. 2007. [ Links ]

WERLE, Flávia. Políticas de Avaliação em Larga Escala na Educação Básica: do controle de resultados à intervenção nos processos de operacionalização do ensino. Ensaio: avaliação e políticas públicas de educação, Rio de Janeiro, v. 19, n. 73, p. 769-792, out./dez. 2011. [ Links ]

WIEBUSCH, Eloisa Maria. Avaliação Externa: um caminho para a busca da qualidade da educação. 2011. 152 f. Dissertação (Mestrado em Educação) - Faculdade de Educação, Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, 2011. [ Links ]

ZANARDINI, João Batista. Ontologia e Avaliação da Educação Básica no Brasil (1990-2007). 2008. 208 f. Tese (Doutorado em Educação) - Programa de Pós-Graduação em Educação, Universidade Federal de Santa Catarina, Florianópolis, 2008. [ Links ]

1This article is part of a broader study developed by the author in the Carlos Chagas Foundation, from 2012 to 2014. The author thanks the Foundation for the support to conduct this study.

2The timeframe is due to the beginning of the study year: 2013. The study aimed originally to carry out a survey of the works until 2012. However, the data available at the Capes Database of Theses and Dissertations Capes were up to 2011, which limited the timeframe of the study. It is noteworthy that more recently (2017) the survey was updated, with the same search descriptors used in this study, but the data are still being processed, due to the large volume of existing works.

3The publications that discuss the relations between external assessments and management, used in this paper, were selected from a database that was elaborated for a broader research project and that is dependent on the information provided by the authors, as its object of study, abstracts, titles and keywords.

4A summary table with the titles of works, authors, year of publication and institution is shown in Table 4.

5ENEM: National Secondary Education Examination.

Received: October 05, 2017; Accepted: December 04, 2018

Adriana Bauer is a pedagogue, researcher in the Educational Research Department of the Carlos Chagas Foundation and lecturer in the Department of Methodology of Education and Comparative Education (EDM) at the School of Education of the University of São Paulo. Academic director of the Brazilian Association for Educational Evaluation (ABAVE) from 2017 to 2019. ORCID: http://orcid.org/0000-0002-5942-9181. E-mail: adbauer@fcc.org.br

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License