QUALITY OF BASIC EDUCATION: BEYOND LARGE-SCALE COGNITIVE TESTS

Abstract

This article analyzes the evaluation of the quality of basic education in Brazil, questioning the overemphasis on cognitive test results as the primary indicator of educational excellence. A more comprehensive interpretative model is proposed, grounded in the seven axes of the Quality Matrix of the Sistema de Avaliação da Educação Básica [Basic Education Assessment System] (Saeb), which encompasses multiple dimensions of education. In addition, the limitations of Saeb are outlined, and the adverse effects of large-scale assessments are discussed, highlighting the need for a broader perspective on the dimensions that should constitute educational quality. The article concludes that an evaluation model that goes beyond cognitive tests is essential for a more comprehensive understanding of the quality of education.

EDUCATION QUALITY ASSESSMENT; LARGE-SCALE ASSESSMENT; EDUCATIONAL INDICATORS; PERFORMANCE TESTING

Resumo

Este artigo analisa a avaliação da qualidade da educação básica no Brasil, questionando a ênfase excessiva nos resultados dos testes cognitivos como principal indicador de excelência educacional. Propõe-se um modelo interpretativo mais abrangente, baseado nos sete eixos da Matriz de Qualidade do Sistema de Avaliação da Educação Básica (Saeb), que considera múltiplas dimensões da educação. Além disso, apresentam-se as limitações do Saeb e discutem-se os efeitos adversos das avaliações em larga escala e a necessidade de uma visão mais ampliada sobre os aspectos que devem compor a qualidade educacional. O artigo conclui que um modelo de avaliação que vá além dos testes cognitivos é fundamental para uma compreensão mais abrangente da qualidade da educação.

AVALIAÇÃO DA QUALIDADE DA EDUCAÇÃO; AVALIAÇÃO EM LARGA ESCALA; INDICADORES EDUCACIONAIS; TESTE DE DESEMPENHO

Resumen

Este artículo analiza la evaluación de la calidad de la educación básica en Brasil, cuestionando el énfasis excesivo en los resultados de los test cognitivos como principal indicador de excelencia educativa. Se propone un modelo interpretativo más integral, basado en los siete ejes de la Matriz de Calidad del Sistema de Avaliação da Educação Básica [Sistema de Evaluación de la Educación Básica] (Saeb), que considera múltiples dimensiones de la educación. Además, se presentan las limitaciones del Saeb y se discuten los efectos adversos de las evaluaciones a gran escala y la necesidad de una visión más amplia de los aspectos que deben componer la calidad educativa. El artículo concluye que un modelo de evaluación que vaya más allá de los test cognitivos es esencial para una comprensión más integral de la calidad de la educación.

EVALUACIÓN DE LA CALIDAD DE LA EDUCACIÓN; EVALUACIÓN A GRAN ESCALA; INDICADORES EDUCATIVOS; TEST DE RENDIMIENTO

Résumé

Cet article analyse les évaluations sur la qualité de l’éducation de base au Brésil et remet en question l’importance excessive accordée aux résultats des tests cognitifs comme principal indicateur d’excellence en matière d’éducation. Un modèle interprétatif plwus compréhensif est proposé, structuré autour des sept axes d’analyse de la qualité du Sistema de Avaliação da Educação Básica [Système d’évaluation de l’éducation de base] (Saeb), qui prend en compte les multiples dimensions de l’éducation. Les limites du Saeb et les effets négatifs des évaluations à grande échelle sont aussi discutés, tout comme la nécessité d’une vision plus large concernant les aspects devant intégrer la qualité de l’éducation. L’article conclut qu’un modèle d’évaluation qui dépasse les tests cognitifs est fondamental pour une compréhension plus exhaustive de la qualité de l’éducation.

ÉVALUATION DE LA QUALITÉ; DE L’ÉDUCATION; ÉVALUATION À; GRANDE ÉCHELLE; INDICATEURS ÉDUCATIFS; TEST DE PERFORMANCE

THE EVALUATION OF BASIC EDUCATION QUALITY IN BRAZIL ADHERES TO A DOMINANT international perspective that considers students’ performance in large-scale cognitive tests as the primary indicator of educational excellence. This hegemonic perspective permeates public evaluation policies and is further reinforced by the increasing adoption of self-assessment systems by states and municipalities. However, we argue that this constitutes a reductionist approach, as it overlooks a broader range of factors essential for achieving educational excellence beyond mere student test proficiency. In this article, we propose an interpretative model for evaluating basic education quality, aiming to establish a comprehensive framework that accounts for its multiple dimensions. This approach contrasts with the hegemonic perspective, which prioritizes favorable student performance in large-scale exams as the ultimate - or nearly exclusive - goal of education.

External evaluations are a widespread practice worldwide and play a significant role in shaping global educational policies, with the Programme for International Student Assessment (Pisa), administered by the Organisation for Economic Co-operation and Development (OECD), serving as a prominent example of such initiatives. Held every three years, including in non-OECD countries, Pisa assesses the performance of 15-year-old students in reading, mathematics, and science. Its results influence international education standards, encouraging nations to implement educational reforms aimed at enhancing student performance.

In the Brazilian context, the past 25 years have seen a significant expansion of educational evaluation systems at both federal and state/municipal levels. Oliveira and Clementino (2019, p. 555, own translation) describe these systems as “a set of policies and practices used by the State to measure and hold schools accountable for raising student performance, as well as to stimulate and support improvement when needed”. Various studies (Sousa & Oliveira, 2007; Bauer, 2010; Brooke & Cunha, 2011; Horta, 2013; Sousa, 2013) document this expansion, reflecting the belief that controlling educational processes through standardized tests is essential for promoting improvements in student performance and, consequently, educational quality.

The Índice de Desenvolvimento da Educação Básica [Primary Education Development Index] (Ideb), established in 2007 by Decreto n. 6.094, has become a key instrument for measuring and comparing educational performance across all levels of the federation. This index quantifies or simplifies basic education excellence into a single number, referred to as an “objective indicator for verifying the fulfillment of established goals” (Decreto n. 6.094, 2007, own translation). Its core component is student performance in standardized tests, calculated through an equation that also accounts for student flow (promotion and dropout rates). Cognitive tests are among the instruments of the Sistema de Avaliação da Educação Básica [Basic Education Assessment System] (Saeb), conducted biennially to evaluate the country’s educational quality. These tests are complemented by the contextual questionnaires administered to students, teachers, school principals, and municipal education managers. The Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira [National Institute of Educational Studies and Research Anísio Teixeira] (Inep), affiliated with the Ministério da Educação [Ministry of Education] (MEC), oversees Saeb, fulfilling its institutional mandate to promote studies, research, and evaluations on Brazilian education.

Since the establishment of Ideb, self-assessment systems in Brazilian states and municipalities have expanded significantly. Therefore, we regard Ideb as one of the key drivers behind the expansion of assessment systems developed or acquired by subnational entities, as nearly all of these systems emphasize student performance in cognitive tests. Recent data (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira [Inep], 2024) reveal a significant increase in Brazilian municipalities of varying population sizes developing their own assessments or contracting external companies to conduct them. The proportion rose from 28% in 2014 to 50% in 2021, reflecting a 22-percentage-point increase in just 7 years. Additionally, 11.3% of municipalities were found to use tests to assess preschool children, 41.6% conducted external exams at least semiannually, and 19.3% adopted workbook-based education systems, which are generally associated with teacher training and external assessments.

The prominence of Ideb in shaping the current educational agenda is evident in the Plano Nacional de Educação [National Education Plan] (PNE), enacted through Lei n. 13.005 (2014), where is featured as one of the guiding instruments of public policy in education. In its Goal 7, the PNE mandates the promotion of “quality basic education across all stages and modalities, aiming to enhance school flow and learning outcomes” (Lei n. 13.005, 2014, own translation). It also establishes targets for the early and final years of elementary education as well as high school.

According to Charlot (2021), prevailing conception of “educational quality” emerged during the educational reforms of the 1960s to 1980s, primarily in the United States and Europe, with a focus on improving mathematics and science education. Decades later, the OECD, through initiatives such as Pisa, assumed a central role in promoting and standardizing this concept, rendering it hegemonic and associating educational quality with the capacity of educational systems to prepare students for a more effective participation in the global economy.

This concept is confined to the development of competencies and skills, particularly in mathematics, native language, and sciences, aimed at human capital formation. However, educational quality should be conceived more broadly, encompassing the development of various dimensions essential for life in society. Biesta (2018) contends that the ultimate purpose of education is to support children and young people in being present in the world (and doing so in an adult manner). He differentiates between the language of learning, which lacks substantive content, and the language of education, which must address questions of content, purpose, and relationships. He further warns that student learning is not solely determined by what is said but also by what is done, suggesting that quality education must encompass qualification, socialization, and subjectivation. However, the author emphasizes that decisions, which ought to be made by educators, are instead determined in other spheres, including politics, curriculum developers, textbook authors, and the global assessment system.

Charlot (2021) highlights the significant challenges associated with the term “educational quality”, particularly its ambiguity and the difficulty in defining and measuring this concept. A decade earlier, Gusmão (2013) had systematized the literature in the field to identify the predominant approaches to educational quality, categorizing five key perspectives: the expansion of schooling; performance in large-scale tests; the economic perspective; diversity; and the educational model. Interestingly, this ambiguity and the plurality approaches surrounding “educational quality” closely align with Saeb’s own framework, which emphasizes that “[e]ducational quality is multidimensional, and therefore multiple factors from both within and outside the school context must be considered” (Inep, 2018, p. 8, own translation).

Thus, we argue that “educational quality” is a broad concept and that should encompass the multiple dimensions permeating the educational context, extending far beyond the outcomes of cognitive tests administered to students. In an effort to capture the diverse aspects we deem essential to the Brazilian educational assessment, we propose an interpretative model structured around the axes of the Saeb Quality Matrix, aiming to broaden this concept and enhance its practical implementation. Before proceeding, we highlight several negative aspects of large-scale external assessments.

Limitations of Saeb and adverse effects of large-scale educational assessments

Large-scale external assessments, despite their global adoption as instruments driving the development of public policies in education, have also faced significant criticism. In Brazil, Saeb serves as the primary evaluative mechanism in this domain, with the central objective of “producing information about the quality of basic education, in conjunction with other data, to support government bodies in evaluating, redefining, and establishing programs and public policies in education” (Inep, 2018, p. 6, own translation). However, this evaluation system exhibits a range of limitations that hinder a comprehensive understanding of education.

Since its inception in 1990, Saeb was initially applied using a sample-based approach. From 2005 onwards, the evaluation adopted an almost census-like approach, including only schools with ten or more students enrolled in the evaluated grades. However, the coverage is not fully census-like across the educational system, as it excludes schools that fail to meet this minimum enrollment threshold in the evaluated grades. This exclusion distorts the overall picture by disregarding the realities of smaller schools, often situated in rural or remote communities. These schools embody unique educational contexts, whose realities remain unaccounted for in the evaluation, thereby compromising the representativeness of the collected data.

Another defining feature of Saeb is that its tests are administered exclusively to students in the 5^th and 9^th grades of elementary education and the 3^rd and 4^th grades of high school. Although only four out of the twelve years/grades of basic education are evaluated, this measurement extrapolates the perception of educational quality to the entire school. As a result, substantial portion of the student body in other grades is excluded, leading to fragmented understanding of educational quality across the entire school trajectory.

Saeb primarily emphasizes mathematics and Portuguese tests administered to students in the specified grades, despite efforts to broaden the scope of the assessments. Even with the inclusion of humanities and natural sciences and the extension to the 2^nd year of elementary education, the information made available to society remains largely unchanged. We do not advocate for the exams to be administered across all school grades or to encompass all disciplines, as proposed by MEC during the Bolsonaro government. We emphasize that, despite these acknowledged limitations, the tests pertain to educational quality, yet their results fail to capture the full complexity of the educational process.

We contend that these limitations result in an incomplete and distorted perception of educational quality, impeding a precise and comprehensive evaluation of the educational system. The reliance on a restricted sample, evaluation of a limited number of grades/years, and the narrow focus on a few disciplines fail to adequately capture the diversity and complexity of Brazilian schools. This critique applies exclusively to cognitive tests, disregarding the broader dimensions that, we argue, should be incorporated into an evaluation of educational quality.

A critical perspective addresses the adverse effects of large-scale assessments, particularly on teachers (Grek et al., 2020). The emphasis on these assessments has been observed to foster an educational practice commonly referred to as “teaching to the test”, which involves preparing students specifically to succeed in these tests (Gewirtz, 2002; Menken, 2006). This practice places pressure on educational systems to adapt curricula and teaching methods, prioritizing improved student performance on external tests (Hypolito & Jorge, 2020).

Teacher autonomy is among the most affected elements of this evaluation model centered on cognitive tests. According to Maroy (2013), this accountability framework undermines teacher professionalism by subordinating pedagogical decisions to the pursuit of results in external evaluations. This not only calls teachers’ competence into question but also reflects a lack of trust in their ability to assess and foster comprehensive learning. Ranson (2003) observes that school curricula, once developed within professional communities, are now subject to public scrutiny, thereby limiting educators’ control over content and teaching methods. This dynamic leads to the erosion of teacher autonomy, confining educators to a constrained role dictated by external metrics.

In the Brazilian context, Oliveira and Clementino’s (2020) study revealed that negative impacts on teachers are more pronounced in states that implement high accountability policies. A study conducted in Brazil’s Northeast region found that Paraíba and Pernambuco implement high accountability policies, which include rewards and sanctions for teachers and schools based on student test results. Conversely, in the states of Bahia and Rio Grande do Norte, policies identified are characterized as having low accountability (Oliveira & Clementino, 2020). Teachers in high accountability states experience heightened internal pressure to prepare students for external exams and perceive evaluation results as having a greater impact on school reputation.

Santos (2023), analyzing data from the same study,¹ highlights a disconnect between aspects of external evaluations and objective working conditions. According to the author, the proportion of professionals with permanent contracts reaches 70.6% in low accountability states but decreases to 57.9% in high accountability states. In low accountability states, 56.9% of teachers frequently receive recommendations from school management to align their teaching with the learning standards assessed in tests, compared to 76.4% in high accountability states. These data suggest that in high accountability states, teachers face greater external direction in their pedagogical practices, alongside a lower proportion of professionals with stable employment contracts.

The use of standardized student test results has been contested even within government entities themselves. A recent and controversial case occurred in Minas Gerais, where Lei n. 24.431 (2023) was enacted, establishing criteria for the distribution of part of the Imposto sobre Circulação de Mercadorias e Serviços [Tax on the Circulation of Goods and Services] (ICMS) allocated to Education. Under the new parameters, 50% of the funds are distributed to municipalities based on the performance of elementary education students in state-conducted external assessments, 20% is tied to school flow, 15% to full-time and rural attendance, and the remaining 15% to school management. By excluding the number of enrollments from the criteria for public fund distribution, the law creates significant disparities in the per-student allocation among municipalities, disproportionately impacting more populous ones (Fontes, 2024).

Assessment systems in Brazil predominantly focus on cognitive tests, overlooking other critical aspects of the educational process. This supports Duarte’s (2011, p. 170, own translation) concern, who observes that “the evaluation process, regardless of its scope (federal, state, or municipal), fails to adequately consider the working conditions faced by teachers”. In light of this, the next section proposes an interpretative model aimed at fostering critical reflection on how additional dimensions can be incorporated into the evaluative framework.

Interpretative model of education quality

The strategy we propose to analytically represent an expanded concept of educational quality involves utilizing the structure of the Saeb. Within this framework, we incorporate all its assessment instruments, including tests administered to students and contextual questionnaires completed by municipal education managers, school principals, teachers, and students. This approach aligns with Saeb’s specific objectives by addressing the evaluation of quality, equity, and efficiency in the country’s education system and supports the development, monitoring, and improvement of evidence-based public policies.

The primary reference for the Saeb Quality Matrix is the document titled Sistema de Avaliação da Educação Básica: Documentos de referência versão 1.0, published by Inep in 2018. Notably, this document considers existing legislation, including the 1988 Federal Constitution (Constituição da República Federativa do Brasil, 1988); the Lei de Diretrizes e Bases da Educação Nacional [National Education Guidelines and Framework Law] (LDB) (Lei n. 9.394, 1996); and the Plano Nacional de Educação (Lei n. 13.005, 2014). In this document, educational quality is conceptualized through seven axes or dimensions: 1. School Attendance; 2. Teaching and Learning; 3. Investment; 4. Education Professionals; 5. School Management; 6. Equity; and 7. Citizenship, Human Rights, and Values. Each dimension encompasses major themes, further divided into specific topics.

To construct an interpretative model encompassing all dimensions of quality, it is necessary to define the items that make up each dimension and specify the nature of their interrelationships. Ultimately, this web of interconnections facilitates the formulation of clear and testable hypotheses. Precision in defining these relationships enables the identification of how one variable may influence another, which is particularly valuable given the complexity of educational systems with multiple interrelated variables.

We propose an interpretative model for evaluating educational quality, encompassing the seven dimensions outlined in the Saeb Quality Matrix and reflecting the multidimensionality and complexity of the educational process (Figure 1). Sequentially, the model posits that “Equity” influences the “Investment” axis, which in turn affects “School Attendance”, “Teaching and Learning”, “Education Professionals”, “School Management”, and “Citizenship, Human Rights, and Values”.

Figure 1
Interpretative model for evaluating educational quality

In exploring the model, we emphasize key characteristics of the multiple associations established among the axes or dimensions of educational quality. The first is that axis 6, “Equity”, functions as the independent variable, whereas dimension 3, “Investment”, serves as the dependent variable. Equity, in this context, refers to justice in resource allocation, ensuring that all students and stakeholders, particularly those from disadvantaged groups, receive adequate support. From this perspective, socioeconomically disadvantaged groups should receive greater investment to address pre-existing inequalities and ensure equitable access to quality education for all.

Within the sequence of relationships, axis 3, “Investment”, which depends on axis 6, “Equity”, becomes the independent variable for the other five dimensions. This indicates that investments directly influence the axes “School Attendance”, “Teaching and Learning”, “Education Professionals”, “School Management”, and “Citizenship, Human Rights, and Values”. In other words, investments are essential to improving school infrastructure, valuing education professionals, implementing effective pedagogical practices, promoting efficient management, and fostering education centered on citizenship.

The five dependent axes - “School Attendance”, “Teaching and Learning”, “Education Professionals”, “School Management”, and “Citizenship, Human Rights, and Values” - are interrelated, with their associations rooted in correlation rather than dependence. This suggests that improvements in one axis can positively influence others, without adhering to a specific order or a direct cause-and-effect relationship among them. For instance, effective school management can simultaneously enhance school attendance and teaching and learning while fostering an environment that supports professional development for educators and promotes students’ civic development.

Five quality axes as the ultimate goals of education

The proposed interpretative model acknowledges the complexity of the educational process, moving beyond the narrow perspective focused exclusively on students’ cognitive test results. This framework incorporates diverse dimensions of quality, identifying five as the ultimate outcomes of the educational domain.

The first quality axis, “School Attendance”, emphasizes that schools must accommodate students by providing appropriate facilities and comfort, fostering sociability and exchange of experiences, thereby expanding opportunities for formative activities. As a fundamental premise, schools must be located near students’ homes, be accessible to individuals with disabilities, and provide conditions that enable education professionals to fully engage in the educational process.

In contrast to the traditional perspective, we position student cognitive tests as just one among the various components of the “Teaching and Learning” axis, thereby removing its centrality as the primary indicator of educational quality. This dimension focuses on “generating information that allows us to understand and monitor the teaching provided by school units” (Inep, 2018, p. 16, own translation), extending beyond merely evaluating student scores in external tests. It comprises two themes, curriculum and pedagogical practices, which encompass aspects such as the nature of relationships established in the learning environment, lesson planning, and pedagogical support, among others.

Another dimension pertains to education professionals, encompassing “themes related to performance, professional training, working conditions, and employment of teachers and school principals” (Inep, 2018, p. 30, own translation). In addition to influencing teaching, this dimension captures the conditions and contexts that shape pedagogical practice and educators’ well-being, aspects often overlooked in models that focus exclusively on student performance. Factors such as retention, health, and job satisfaction stem from this dimension and are deemed essential for implementing effective pedagogical practices and fostering a positive work environment.

School management represents another dimension of this framework, encompassing areas such as planning, organization, monitoring, and evaluation of educational activities. This dimension emphasizes strategic management practices that foster the direct involvement of diverse stakeholders in the educational process, addressing factors such as school climate, collaborative work among teachers, and professional development. These aspects highlight the capacity of schools and educational networks to manage resources effectively and cultivate an environment conducive to teaching and learning.

The fifth axis, “Citizenship, Human Rights, and Values”, seeks to broaden the scope of the model by capturing the role of education in fostering a fairer and more equitable society. This axis emphasizes a commitment to inclusion and social justice as foundational principles, acknowledging that educational objectives can only be achieved when the needs of all students, particularly the most vulnerable and those with disabilities, are addressed. This underscores the understanding that educational quality is inextricably linked to its role in fostering civic education and upholding human rights.

The right of education extends beyond learning

The dialectic of accountability policies lies in their aim to improve student performance by promoting practices designed to yield better assessment outcomes. To achieve this, education systems establish standards and goals to encourage schools and teachers to prioritize educational practices considered more effective. These policies also aim to ensure accountability and enhance transparency by publishing results under the premise of involving parents and communities in monitoring school performance.

However, this centrality in student performance is diminished in the proposed model for assessing educational quality, which acknowledges that the right to education extends beyond the acquisition of cognitive skills and entails ensuring a broader spectrum of educational components. From this perspective, indicators are incorporated to reflect the right to a safe and inclusive learning environment, access to a diverse and relevant curriculum, the development of life skills, the promotion of citizenship, equity, and respect for cultural diversity.

By incorporating variables that address student well-being, including school climate and mental health, the model goes beyond traditional measures of student performance. This enables school managers and policymakers to monitor and foster educational environments that effectively accommodate students and support their educational trajectories.

With respect to equity in educational opportunities, the model accounts for the distribution of resources among students from diverse backgrounds (social, identity, racial, geographical), enabling each to become a rights-bearing citizen. This entails ensuring that the right to education is equitable, free from barriers, particularly those of a socioeconomic nature.

The model further acknowledges the significance of education for citizenship, emphasizing the need to evaluate how schools prepare students to become informed and engaged citizens. This is reflected in the inclusion of variables that assess civic engagement, comprehension of human rights, and the cultivation of social values.

By adopting a multidimensional approach, the interpretative model enables education networks to not only uphold the right to learning but also to ensure the holistic realization of education. In doing so, it establishes a vision of educational quality that is inclusive, comprehensive, and aligned with the broader objectives of education as a fundamental human right.

There is a chain of relationships among quality dimensions

The proposed model illustrates a sequential and interactive approach to analyzing educational quality, adhering to the logic of context > inputs > processes > results. In this model, axis 6, “Equity”, is identified as a contextual factor that directly influences axis 3, “Investment”, underscoring that policies and practices aimed at equity should directly impact resource allocation for education. Subsequently, these investments are directed toward fundamental areas, including “School Attendance”, “Teaching and Learning”, “Education Professionals”, “School Management”, and “Citizenship, Human Rights, and Values”.

Analytical models establishing a chain of relationships among variables have already been employed in studies on teacher working conditions. Pereira (2017) examined teacher retention in schools as a dependent variable influenced by four independent variables - students’ economic level, policing, surveillance, and school infrastructure - all of which were interrelated. The study revealed that school infrastructure had the greatest impact on teacher retention. Similarly, Oliveira et al. (2017) demonstrated that classroom conditions and school conditions were interrelated and both contributed to teachers’ professional satisfaction.

Feasibility and strategies for implementing the educational quality assessment model

The implementation of the proposed assessment model is feasible in Brazil, leveraging existing resources and making necessary adjustments. Brazil possesses a substantial volume of data collected by Inep and other public institutions, which can be integrated to support a comprehensive and multidimensional framework of educational quality. Below, we outline strategies that offer a practical pathway for implementing this evaluation model.

Saeb serves as the foundation for implementing this model, as it incorporates a Reference Matrix for educational quality and is designed to evaluate Brazilian education. Grounded in the 1988 Federal Constitution and specific legislation, this Matrix outlines seven dimensions of educational quality, referred to as quality axes. Each axis is structured into themes, which are further subdivided into topics. These topics specify what must be measured, guiding the development of items in Saeb’s four contextual questionnaires, administered to municipal education managers, school managers, teachers, and students, while also identifying data to be collected from other available databases.

In the proposed model, the initial data integrated into the Quality Matrix are those produced by Saeb. Subsequently, additional data already compiled by Inep, including the School Census, Higher Education Census, and the educational indicators, are incorporated. This information enables a more detailed understanding of the topics addressed.

The subsequent stage entails identifying statistical data produced by other ministries or public bodies aligned with the Quality Matrix. The PNE 2014-2024, for instance, stipulates in article 4 that the established goals “must reference the National Household Sample Survey - PNAD, the demographic census, and the most recent national censuses of basic and higher education” (Lei n. 13.005, 2014, own translation). Accordingly, data produced by the Instituto Brasileiro de Geografia e Estatística [Brazilian Institute of Geography and Statistics] (IBGE), as well as by entities such as the Ministério de Desenvolvimento Social [Ministry of Social Development] (MDS) and the Fundo Nacional de Desenvolvimento da Educação [National Education Development Fund] (FNDE), should be incorporated, encompassing socioeconomic and demographic aspects pertinent to educational quality.

If gaps are identified in the Saeb Quality Matrix, revealing the absence of statistical data to represent specific topics, it becomes necessary to plan studies and research to address these deficiencies. Such data can be gathered through sample surveys and need not be updated annually. For instance, smaller schools in rural areas or specific communities can be assessed through studies that account for their unique characteristics, enabling their inclusion in the analysis of educational quality.

The collected information must be integrated into a unified framework, accounting for different units of analysis, such as the national, state, municipal and school levels. This integration enables the development of multidimensional indicators to represent each dimension of the Saeb Quality Matrix, avoiding hierarchies among them and refraining from prioritizing specific data, in contrast to the current approach, which emphasizes student results in standardized tests. These indicators reflect the complexity and diversity of basic education, addressing multiple factors related to the school environment and overall educational quality.

The final product is a system of indicators encompassing all dimensions of the Saeb Quality Matrix, offering a comprehensive and integrated perspective on the Brazilian educational landscape. These organized data is made publicly available, either through data visualization panels designed to enhance comprehension or by providing microdata for research purposes. Transparency and accessibility of data are essential for state, municipal, and school managers to effectively utilize this information in planning and decision-making. Moreover, these data are fundamental for supporting researchers in their studies on basic education, contributing to advancing knowledge on aspects that remain underexplored.

Discussion

Building on the arguments presented, we discuss key topics, articulating our understanding of the current evaluation of basic education quality in Brazil. This analysis adopts a broad perspective, encompassing multiple factors and stakeholders involved. Finally, we will discuss the feasibility of implementing the proposed model in Brazil.

We start by examining the definition of Saeb’s objectives, one of which is “to evaluate the quality, equity, and efficiency of education practiced in the country across various governmental levels” (Inep, 2018, p. 6, own translation). This formulation implies a conceptual distinction between quality, equity, and efficiency in education, treating them as separate dimensions. However, it is crucial to acknowledge that quality is a comprehensive concept that inherently encompasses both equity and efficiency. The inclusion of the “Equity” dimension in the Saeb Quality Matrix reinforces this perspective, emphasizing that educational quality cannot be achieved without addressing equity.

Another objective of Saeb is “to support the elaboration, monitoring, and improvement of evidence-based public policies” (Inep, 2018, p. 6, own translation), underscoring the significance of scientific research in generating empirical data to inform political decisions. This objective highlights the value of academic research, enabling the exploration of relationships between variables, the testing of hypotheses, and the formulation of public policies grounded in empirical evidence. However, we argue that there is a genuine risk of “evidence-based public policies” devolving into “public policies biased by evidence”, not as a result of inadequate methods or insufficient findings to validate the studies, but due to an excessive emphasis on cognitive tests. This dominance may inadvertently narrow the spectrum of evidence, omitting critical aspects such as professional recognition, equity, and inclusion, among others.

We also highlight the commitments outlined in the PNE, a crucial guiding instrument for public educational policies. Goal 7 of the Plan establishes biennial targets for average grades related to student performance in the assessed grades. Conversely, strategy 7.3 proposes the development of a national framework of institutional assessment indicators, encompassing a broader range of factors within the educational context, such as student and teacher profiles, school infrastructure, pedagogical resources, school management, and other dimensions. It further emphasizes the importance of considering the specificities of different educational modalities. However, the implementation of this strategy encounters significant challenges, primarily due to the absence of clear guidelines and concrete targets for these institutional indicators. The absence of a precise definition of measurable indicators and corresponding standards, as seen with student performance, hinders the formulation of public policies aimed at improving various dimensions of education.

Specifically concerning student outcomes in large-scale assessments, we question whether they are adequate even to represent the “Teaching and Learning” axis. When broadening the scope, we contend that factors such as basic education completion, access to higher education, respect for human rights, and others should be incorporated. Each of these should hold equal importance to student performance.

Shifting the discussion to an evaluation framework we consider appropriate, which is comprehensive and does not focus solely on students’ cognitive test scores, we highlight the case of Finland. Although Finland ranks highly on international scales, it does not prioritize actions centered on standardized tests, competition among schools, or privatization. Sahlberg (2015) contends that the opposite is true, emphasizing the combination of factors that create an environment where education is highly valued. According to the author, one key factor in Finnish education is the autonomy granted to teachers, enabling them to apply their knowledge and expertise in the classroom without the pressures associated with standardized exams. As a result, Finland boasts high retention rates in the teaching profession, with many teachers dedicating their entire careers to education.

Finally, we assert that implementing the proposed educational quality assessment model is achievable in Brazil. By leveraging existing data - particularly those provided by Saeb, Inep, and IBGE -, we can develop a multidimensional indicator system that captures the complexity of basic education. This integrated approach facilitates a comprehensive analysis of educational quality, encompassing multiple factors beyond student performance in standardized tests.

Ensuring the accessibility and transparency of this information serves as a valuable tool for managers and researchers. It facilitates planning, decision-making, and the evaluation of various aspects that define the educational system. Thus, by reinforcing the foundation for a multidimensional analysis, we contribute to the continuous improvement of Brazilian education quality, moving beyond large-scale cognitive tests.

Conclusion

The assessment model proposed in this article offers a more comprehensive framework for analyzing the quality of basic education in Brazil, extending beyond the results of large-scale cognitive tests. By integrating multiple dimensions, such as school attendance, teaching and learning, education professionals, school management, equity and citizenship, human rights, and values, the model aims to capture the complexity and diversity of the educational process. This approach enables a more comprehensive and accurate evaluation of the education system in Brazil.

The implementation of this model is feasible, leveraging data already collected through Saeb, as well as other studies and sources, including the School Census, Continuous PNAD, and information from various ministries and public bodies. The primary challenge lies in structuring and utilizing these data in a coherent and integrated manner, offering a comprehensive perspective to support more effective public policies. Adopting this multidimensional approach creates the opportunity to promote improvements in the educational system, ensuring that the quality of basic education is comprehensively understood and evaluated in all its complexity.

• 1
  This is the research As condições da oferta da educação básica pública em quatro estados do Nordeste do Brasil, carried out between September 2021 and March 2022 by the Grupo de Estudos sobre Política Educacional e Trabalho Docente at the Universidade Federal de Minas Gerais (Gestrado/UFMG), with the participation of 1,109 teachers from state and municipal public schools in Bahia, Paraíba, Pernambuco and Rio Grande do Norte.

Acknowledgments

This study was partially funded by the Conselho Nacional de Desenvolvimento Científico e Tecnológico [National Council for Scientific and Technological Development] (CNPq), under the Call 40/2022, project code 421017/2022-8, and the Call 22/2016, project code 440077/2017-6.

• How to cite this article
  Oliveira, D. A., Pereira, E. A., Junior, & Horta, J. L., Neto. (2025). Quality of basic education: Beyond large-scale cognitive tests. Cadernos de Pesquisa, 55, Article e11295. https://doi.org/10.1590/1980531411295_en

Data availability statement

The contents underlying the research text are contained in the manuscript.

References

• Bauer, A. (2010). Usos dos resultados das avaliações de sistemas educacionais: Iniciativas em curso em alguns países da América. Revista Brasileira de Estudos Pedagógicos, 91(228), 315-344. https://doi.org/10.24109/2176-6681.rbep.91i228.576
  » https://doi.org/10.24109/2176-6681.rbep.91i228.576
• Biesta, G. (2018). O dever de resistir: Sobre escolas, professores e sociedade. Educação, 41(1), 21-29. https://doi.org/10.15448/1981-2582.2018.1.29749
  » https://doi.org/10.15448/1981-2582.2018.1.29749
• Brooke, N., & Cunha, M. A. de A. (2011). A avaliação externa como instrumento da gestão educacional nos estados. In Fundação Victor Civita, Estudos & Pesquisas Educacionais (2, pp. 17-79). Fundação Victor Civita. https://fvc.org.br/wp-content/uploads/2018/04/estudos_e_pesquisas_educacionais_vol_2.pdf
  » https://fvc.org.br/wp-content/uploads/2018/04/estudos_e_pesquisas_educacionais_vol_2.pdf
• Charlot, B. (2021). “Qualidade da educação”: O nascimento de um conceito ambíguo. Educar em Revista, 37, Artigo e81286. https://doi.org/10.1590/0104-4060.81286
  » https://doi.org/10.1590/0104-4060.81286
• Constituição da República Federativa do Brasil de 1988 (1988). Brasília, DF. https://www.planalto.gov.br/ccivil_03/constituicao/constituicao.htm
  » https://www.planalto.gov.br/ccivil_03/constituicao/constituicao.htm
• Decreto n. 6.094, de 24 de abril de 2007 (2007). Dispõe sobre a implementação do Plano de Metas Compromisso Todos pela Educação, pela União Federal, em regime de colaboração com Municípios, Distrito Federal e Estados, e a participação das famílias e da comunidade, mediante programas e ações de assistência técnica e financeira, visando a mobilização social pela melhoria da qualidade da educação básica. Brasília, DF. https://www.planalto.gov.br/ccivil_03/_ato2007-2010/2007/decreto/d6094.htm
  » https://www.planalto.gov.br/ccivil_03/_ato2007-2010/2007/decreto/d6094.htm
• Duarte, A. (2011). Políticas educacionais e o trabalho docente na atualidade: Tendências e contradições. In D. A. Oliveira, & A. Duarte (Orgs.), Políticas públicas e educação: Regulação e conhecimento (pp. 161-182). Fino Traço.
• Fontes, L. (2024, 18 janeiro). Com menos recursos, prefeitos questionam nova distribuição do ICMS da Educação: Municípios mais populosos reivindicam maior critério de matrículas de alunos para distribuir 10% da arrecadação com o imposto. O Tempo https://www.otempo.com.br/politica/com-menos-recursos-prefeitos-questionam-nova-distribuicao-do-icms-da-educacao-1.3314406
  » https://www.otempo.com.br/politica/com-menos-recursos-prefeitos-questionam-nova-distribuicao-do-icms-da-educacao-1.3314406
• Gewirtz, S. (2002). The managerial school: Post-welfarism and social justice in education Routledge.
• Grek, S., Maroy, C., & Verger, A. (2020). World yearbook of education 2021: Accountability and datafication in the governance of education Routledge.
• Gusmão, J. B. de. (2013). A construção da noção de qualidade da educação. Ensaio: Avaliação e Políticas Públicas em Educação, 21(79), 299-322. https://doi.org/10.1590/S0104-40362013000200007
  » https://doi.org/10.1590/S0104-40362013000200007
• Horta, J. L., Neto. (2013). As avaliações externas e seus efeitos sobre as políticas educacionais: Uma análise comparada entre a União e os estados de Minas Gerais e São Paulo [Tese de doutorado, Universidade de Brasília]. RIUnB - Repositório Institucional da Universidade de Brasília. http://repositorio.unb.br/handle/10482/14398
  » http://repositorio.unb.br/handle/10482/14398
• Hypolito, Á. M., & Jorge, T. (2020). OCDE, PISA e avaliação em larga escala no Brasil: Algumas implicações. Sisyphus - Journal of Education, 8(1), 10-27. https://doi.org/10.25749/sis.18980
  » https://doi.org/10.25749/sis.18980
• Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep). (2018). Sistema de Avaliação da Educação Básica: Documentos de referência versão 1.0 Inep/Daeb. https://download.inep.gov.br/educacao_basica/saeb/2018/documentos/saeb_documentos_de_referencia_versao_1.0.pdf
  » https://download.inep.gov.br/educacao_basica/saeb/2018/documentos/saeb_documentos_de_referencia_versao_1.0.pdf
• Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep). (2024). Microdados do Sistema de Avaliação da Educação Básica 2023 Inep.
• Lei n. 9.394, de 20 de dezembro de 1996 (1996). Estabelece as diretrizes e bases da educação nacional. Brasília, DF. https://www.planalto.gov.br/ccivil_03/leis/l9394.htm
  » https://www.planalto.gov.br/ccivil_03/leis/l9394.htm
• Lei n. 13.005, de 25 de junho de 2014 (2014). Aprova o Plano Nacional de Educação - PNE e dá outras providências. Brasília, DF. https://www.planalto.gov.br/ccivil_03/_ato2011-2014/2014/lei/l13005.htm
  » https://www.planalto.gov.br/ccivil_03/_ato2011-2014/2014/lei/l13005.htm
• Lei n. 24.431, de 14 de setembro de 2023 (2023). Altera a Lei n. 18.030, de 12 de janeiro de 2009, que dispõe sobre a distribuição da parcela da receita do produto da arrecadação do ICMS pertencente aos municípios. Belo Horizonte, MG. https://www.fazenda.mg.gov.br/empresas/legislacao_tributaria/leis/2023/l24431_2023.html
  » https://www.fazenda.mg.gov.br/empresas/legislacao_tributaria/leis/2023/l24431_2023.html
• Maroy, C. (2013). Estado avaliador, accountability e confiança na instituição escolar. Revista Educação e Políticas em Debate, 2(2), 319-338. https://doi.org/10.14393/REPOD-v2n2a2013-24780
  » https://doi.org/10.14393/REPOD-v2n2a2013-24780
• Menken, K. (2006). Teaching to the test: How No Child Left Behind impacts language policy, curriculum, and instruction for English language learners. Bilingual Research Journal, 30(2), 521-546. https://doi.org/10.1080/15235882.2006.10162888
  » https://doi.org/10.1080/15235882.2006.10162888
• Oliveira, D. A., & Clementino, A. M. (2019). As políticas de responsabilização na educação básica nos estados do Nordeste. In D. A. Oliveira, A. M. C. Duarte, & C. M. L. Rodrigues (Orgs.), A política educacional em contexto de desigualdade: Uma análise das redes públicas de ensino da região Nordeste (pp. 523-562). Mercado das Letras.
• Oliveira, D. A., & Clementino, A. M. (2020). As políticas de avaliação e responsabilização no Brasil: Uma análise da educação básica nos estados da região Nordeste. Revista Iberoamericana de Educación, 83(1), 143-162. https://doi.org/10.35362/rie8313877
  » https://doi.org/10.35362/rie8313877
• Oliveira, D. A., Pereira, E., Jr., Vieira, L. F., Duarte, A. M. C., Silva, A. M. C. J., Duarte, A. W. B., & Jorge, T. A. (2017). Indicadores do trabalho docente na educação básica. Fino Traço.
• Pereira, E. A., Jr. (2017). Condições de trabalho docente nas escolas de educação básica no Brasil: Uma análise quantitativa [Tese de doutorado, Universidade Federal de Minas Gerais]. Repositório Institucional da UFMG. https://repositorio.ufmg.br/handle/1843/BUOS-AQQPSG
  » https://repositorio.ufmg.br/handle/1843/BUOS-AQQPSG
• Ranson, S. (2003). Public accountability in the age of neo-liberal governance. Journal of Education Policy, 18(5), 459-480. http://dx.doi.org/10.1080/0268093032000124848
  » http://dx.doi.org/10.1080/0268093032000124848
• Sahlberg, P. (2015). Finish Lessons 2.0: What can the world learn from educational change in Finland? (2^nd ed.). Teachers College.
• Santos, M. L. K. (2023). As políticas de accountability na região Nordeste em contexto de alta e baixa responsabilização [Dissertação de mestrado]. Universidade Federal de Minas Gerais.
• Sousa, S. Z. (2013). Avaliação externa e em larga escala no âmbito do Estado brasileiro: Interface de experiências estaduais e municipais de avaliação da educação básica com iniciativas do governo federal. In A. Bauer, & B. A. Gatti (Orgs.), Vinte e cinco anos de avaliação de sistemas educacionais no Brasil: Implicações nas redes de ensino, no currículo e na formação de professores (pp. 61-85). Insular.
• Sousa, S. Z., & Oliveira, R. P. (2007). Sistemas de avaliação educacional no Brasil: Características, tendências e uso dos resultados (Relatório de pesquisa). Universidade de São Paulo.

Publication Dates

• Publication in this collection
  26 May 2025
• Date of issue
  2025

History

• Received
  03 July 2024
• Accepted
  03 Dec 2024