ARTICLE - MIXED-METHOD EVALUATION TO PROMOTE THE QUALITY OF EARLY CHILDHOOD EDUCATION: POTENTIALITY AND TENSIONS OF A COLLECTIVE PRODUCTION

ABSTRACT:

The evaluation of the quality of Early Childhood Education constitutes a field of knowledge and practices that mobilize structural aspects into the area considering the competence of the different agents in the evaluation processes and their effects in the effectiveness of public policy. In the framework of an evaluation aligned with the promotion of quality and the recognition of Early Childhood Education as a public policy subordinate to social control, we propose to discuss the potentialities and tensions present in the mixed-method evaluation, a model that combines internal and external procedures and instruments inside and outside of institutions. The discussion starts from a mixed-method evaluation experience carried out in conjunction with the technical team of the Public Ministry of the State of São Paulo (MPSP), in the Special Education Group of Ribeirão Preto. The results indicate that the mixed evaluation was powerful because of its dialogic format: the external evaluation allowed a technical and multiprofessional look aligned with the public quality policy; the internal evaluation facilitated a collective self-reflection on quality but showed different strengths and weaknesses in terms of critical views. By combining these perspectives, possibilities are created for overcoming the limitations of each type of evaluation, and also for reflecting on the dynamics of relationships between individuals and public agencies.

Keywords:
mixed-method evaluation; early childhood education; quality; public policies

RESUMO:

A avaliação da qualidade na e da Educação Infantil constitui-se como um campo de conhecimentos e práticas mobilizadores de aspectos estruturantes para a área, que transitam da competência dos diferentes agentes nos processos avaliativos aos seus efeitos na efetivação da política pública. Nos marcos de uma avaliação alinhada à promoção da qualidade e no reconhecimento da Educação Infantil como uma política pública submetida ao controle social, propõe-se a discussão sobre as potencialidades e tensões presentes na avaliação mista, um modelo de avaliação que combina procedimentos e instrumentos internos e externos às instituições. A discussão parte de uma experiência de avaliação mista realizada junto com a equipe técnica do Núcleo de Assessoria Técnica (NAT) do Ministério Público do Estado de São Paulo (MPSP), no âmbito do Grupo de Atuação Especial de Educação - Núcleo Ribeirão Preto (GEDUC-RP). Os resultados indicam que a avaliação mista foi potente devido ao seu formato dialógico: a avaliação externa possibilitou um olhar técnico e multiprofissional alinhado às proposições da política pública de qualidade; a avaliação interna promoveu uma autorreflexão coletiva da qualidade, mas evidenciou poderes diferenciados e fragilidades no que se refere ao olhar crítico de algumas instituições. Conclui-se que é no cruzamento destes olhares que se criam possibilidades de superação dos limites de cada tipo de avaliação e também convida à reflexão sobre os desafios que se instauram na dinâmica das relações entre as pessoas e órgãos públicos.

Palavras-chave:
avaliação mista; educação infantil; qualidade; políticas públicas

RESUMEN:

La evaluación de la calidad en y de la Educación Infantil constituye un campo de conocimientos y prácticas que movilizan aspectos estructurales para el área, que pasan de la competencia de los diferentes agentes en los procesos de evaluación a sus efectos en la efectividad de las políticas públicas. En el marco de una evaluación alineada con la promoción de la calidad y en el reconocimiento de la Educación Infantil como política pública sujeta a control social, proponemos discutir las potencialidades y tensiones presentes en la evaluación de método mixto, un modelo de evaluación que conjuga procedimientos e instrumentos internos y externos a las instituciones. La discusión es parte de una experiencia de evaluación de método mixto realizada en conjunto con el equipo de Asesoramiento Técnico del Ministerio Público del Estado de São Paulo (MPSP), en el Grupo de Educación Especial de Ribeirão Preto. Los resultados indican que la evaluación mixta fue poderosa por su formato dialógico: la evaluación externa permitió una mirada técnica y multiprofesional alineada con los planteamientos de la política pública de calidad; la evaluación interna promovió una autorreflexión colectiva sobre la calidad, pero mostró diferentes facultades y debilidades en cuanto a la visión crítica de algunas instituciones. Se concluye que es en la intersección de estas perspectivas que se crean posibilidades para superar los límites de cada tipo de evaluación y también invita a reflexionar sobre los desafíos que se presentan en la dinámica de las relaciones entre las personas y los organismos públicos.

Palabras clave:
método mixto de evaluación; educación infantil; calidad; políticas públicas

INTRODUCTION

The topic of evaluating the quality of educational services, in Education in general, emerges in a context of reforms oriented and driven by adjustments by the Brazilian State to international interests. In the 1990s, associated with a discourse on the need for greater control and regulation of services, the debate gained visibility with the adoption of a neoliberal agenda that encouraged privatization, greater market autonomy, and the flexibility of forms of educational provision (BARBOSA, 2008; CÁRIA; OLIVEIRA, 2015; FREITAS, 2005). This discourse made up the proposal for the “modernization” of public management and the institutionalization of evaluation, disseminated in Western democracies, with particular consequences in Latin America (FARIA, 2005).

The impacts of these reforms over the last few decades have been widely cited in academic discussions, particularly their deleterious effects on funding policy, the creation of large-scale assessments, and, consequently, the direction of curricula and content (BONAMINO; SOUSA, 2012; SCHNEIDER; NARD; DURLI, 2018).

In addition to these problems, this process distorted the monitoring practices typical of school management and inherent to the work of teachers in the search for the most appropriate ways of inserting children and adolescents in the appropriation of historically and culturally constructed knowledge. Thus, from the evaluative practice, it ended up blurring its primordial meaning, which, seen as a human activity, is intrinsically linked to other activities constantly submitted to questioning and criticism in the construction of new and better possibilities of performance.

In Brazilian Early Childhood Education, Rosemberg (2010) argues that daycare struggle movements have always associated quality and expansion of vacancies in the quest to overcome the dichotomies of quantity X quality and recognition of local culture X lowering of supply conditions, in addition to assuming a firm position contrary to the “poor to poor” service recommended by multilateral agencies to developing countries (ROSEMBERG, 2007). Despite the importance of these struggles and the advances in the scientific field, the author never tired of warning about the inequalities that cross the offer of daycare and preschool for Brazilian children, constituting a huge obstacle in the construction of a truly democratic Early Childhood Education, that is, that includes coverage, equity, and quality of care.

This historical heritage, of an articulated set of regional, class, race, gender, and place of residence inequalities, calls for actions from different social actors in the elaboration and execution of public policies that aim at the democratization of Early Childhood Education and the overcoming of what Rosemberg calls the “curse of Sisyphus” (ROSEMBERG, 2003); a metaphor that expresses the comings and goings of public policies for daycare and preschool that suffer setbacks depending on the conceptions of Early Childhood Education, childhood, poverty and, mainly, the role of the State. This condition, despite advances, characterizes Early Childhood Education as a chronic public problem, with recurrent demand (RUA, 1997), which perseveres on the public agenda.

In this context, evaluation in and of Early Childhood Education gains space and importance (MORO; SOUZA, 2014; MORO, 2018), which constitutes a field of knowledge and practices that mobilize structuring aspects for the area, which transit from micro to macro. That is, from the competence of the different agents in the evaluation processes to their repercussions in the implementation of public policy and the reduction of inequalities.

Early Childhood Education, marked by historical, political, social, cultural, and institutional constraints, is subject to disputes by different social actors, which extend in time and space and which translate into the adoption of different evaluation models (ALVARENGA; VIANNA, 2021; COUTINHO; MORO, 2017; NEVES; MORO, 2013; ROSEMBERG, 2001, 2013). Throughout its institutionalization, the debate on the relationship between evaluation focused on monitoring children and the interest in the determinants of Early Childhood Education has gained complexity (CAMPOS, 2020).

Until 2016, from a macro point of view, Brazil prioritized the construction of an evaluation policy for Early Childhood Education focused on the analysis of the conditions for providing care, considering that they are the ones who structure the service and, consequently, promote the development of children, quality and equity. This position aligned the evaluation policy of Early Childhood Education with a critical and resistant conception of the importation of quality evaluation models adopted at other educational levels (NEVES; MORO, 2013). These evaluative policies, instead of looking at the educational context in a diagnostic and formative way, are guided by external evaluations that seek to measure the cost-benefit of the policy through the performance of children in tests and scales, which creates margin for exclusion and consolidation of classification and discrimination processes (ALVARENGA; VIANNA, 2021; PIMENTA; SOUSA; FLORES, 2021).

Corsino (2021, p. 6) draws attention to the fact that, in the “field of disputes, the insertion of the debate on the evaluation of/in Early Childhood Education in the LDB (Lei de Diretrizes e Bases) was permeated by tensions between opposing theoretical views and practices” and analyzes that the Law of Guidelines and Bases of National Education 9394/1996 (BRASIL, 1996) contains weaknesses when it mentions only assessment in Early Childhood Education. The development of the area was necessary to formulate the horizon of evaluation of Early Childhood Education, linked to a contextual approach, engaged in broader debates that understand that the models adopted for evaluation, committed to certain worldviews, become instruments of reification, aggravation of inequalities or criticism and overcoming the established reality.

Prominent moments in this construction were the articulations around the National Curriculum Guidelines for Early Childhood Education - DCNEIs (Diretrizes Curriculares Nacionais da Educação Infantil, BRASIL, 2010), the National Education Plan (Plano Nacional de Educação, BRASIL, 2014), and the entire process of institutional discussion at the Ministry of Education that culminated in the elaboration of the Draft Ordinance proposing the creation of the National Assessment of Early Childhood Education - ANEI (Avaliação Nacional da Educação Infantil). These articulations are part of what Rosemberg (2013, p. 48) identified as the thematization of the evaluation of early childhood education as a “social problem” which, in this way, started to compose a delimited field of knowledge and be part of the policy negotiation agenda, public policy, becoming a “disputed territory”.

In this work, it is defended that the evaluation in and of Early Childhood Education must be submitted to a reading key that conceives the service offered to children in the plots of the formulation and implementation of public policy for daycare centers and preschools. Thus, the choice and execution of evaluation models for Early Childhood Education are understood as evaluation policies, but also as part of the so-called public policy cycle, composed of interrelated phases referring to the perception and delimitation of problems, definition, formulation, implementation, and evaluation of the agenda (RAEDER, 2014). As C. Souza (2006) warns us, public policy is a field that is the subject of an investigation by numerous disciplines of knowledge and that mobilizes individual, institutional, ideological and political interests in decision-making for the solution of public problems. Due to such interests and their role in the distribution of social goods, expansion of rights, and confrontation of social inequalities, public policies must be understood as a historical result of the clash between different social groups and between these groups and the State (BOSCHETTI, 2009).

In this way, Tonet (2015) warns us that it is only in understanding the social totality that public policy acquires meaning since in its elaboration and execution, social and class struggles are waged around conceptions of subject and society, of state model and management modes. In these struggles between collective actors, the hegemonic forms of public policies and the possibilities of participation in their elaboration are constructed; if social movements, for example, work to make the problem visible, trying to influence the construction of an agenda and the political agenda around a given social problem, these actors will not always be present in its implementation, dependent on a model more or less committed to participation and citizenship.

With this understanding, the cycle analytical model - and the role that evaluation plays in it -; sees public policy in its deliberative and fundamentally dynamic nature, producing knowledge and practices that may (or may not) improve the effectiveness of the public policy (SOUZA, C., 2006). Each stage of the public policy cycle is complex and involves different proposals, issues, methodologies, and social actors. It is not ridiculous to say that this separation does not always occur in the real world, being useful mainly for analysis purposes (RUA, 1997). Evaluation, seen as a policy (as discussed by FARIA, 2005) and/or as part of the cycle of other public policies, is important for enabling a broad understanding of State action or a program and for guiding decision-making processes, guiding the actors involved about the need to modify or add constitutive elements of the evaluated public policy (RAMOS; SCHABBACH, 2012).

Based on an evaluative experience at kindergartens and preschools in the Ribeirão Preto region, we proposed a discussion on the potentialities and tensions present in a type of evaluation that articulates different actors, commonly called mixed evaluation.

THE EVALUATION EXPERIENCE

The reflections raised here are the result of the implementation of a proposal for the evaluation of daycare centers/preschools carried out within the scope of the actions of the Special Education Action Group (GEDUC- Grupo de Atuação Especial de Educação) of the Public Prosecutor's Office of the State of São Paulo, Ribeirão Preto center.

The Prosecutor's Office appears as a new social actor in the so-called justicialization of public policies (GOULART, 2013a) or judicialization (PALHARES, 2019, p. 11) and, consequently, of the Education agenda. This is mainly due to the change in its institutional mission which, since the 1988 Constitution, has integrated the guarantee and defense of diffuse and collective rights. This change makes the Public Prosecutor's Office a resolutive body, which acts in the control and induction of public policies, that is, as a subject of public policy (GOULART, 2013a). In the case of Education, it has become an important arena for discussing the implementation of educational policies following the precepts of the Federal Constitution and similar legislation, working with other actors (such as families, children, social movements, and education unions) in the surveillance with public bodies responsible for implementing educational policies.

The GEDUC was created at the Prosecutor's Office of the State of São Paulo in 2010, Center of the Capital, and, in 2016, two more Centers were added (Ribeirão Preto and Presidente Prudente), aimed at identifying and acting, administratively and judicially, in the situations of violation of diffuse interests related to Education. The Ribeirão Preto Center covers, in its territorial base, 22 municipalities that correspond to two Regional Boards of Education of Ribeirão Preto. The GEDUC-RP comprises the prosecutor, the official and assistant prosecutor, and the Ribeirão Preto Regional Psychosocial Technical Advisory Center - NAT team (equipe Núcleo de Assessoria Técnica Psicossocial Regional Ribeirão Preto) (two social workers and a psychologist). Pedagogy professional advice is offered by another instance of the Public Prosecutor's Office.

Our relationship with GEDUC took place through the offer of internships to Psychology students, to advise on decisions in projects in the area of Early Childhood Education, Rural Education, family participation in school, situations of classroom overcrowding, and evaluation of school equipment.

The work of the Prosecutor's Office to guarantee a place in daycare and preschool is nationally recognized, although not always accompanied by discussions about the quality of care (FELDMAN; SILVEIRA, 2019). This is why, in one of the projects, it was necessary to build a model of action for this body to defend the integrality of the right to daycare and preschool, which is not limited to access to the vacancy. The project that is the basis for the reflections raised here was established by the prosecutor as a Public Policy Monitoring Procedure (PAA- Procedimento de Acompanhamento de Política Pública) to meet one of the goals of the GEDUC-RP Action Program, on the quality of Early Childhood Education institutions, in compliance with the legal framework of Brazilian Early Childhood Education and the General Action Plan of the Prosecutor's Office of the State of São Paulo.

The model built in the project, after theoretical-methodological, technical, and ethical-political debates, combined external evaluation instruments, carried out by the technical team of the Prosecutor's Office and interns, and the use of internal evaluation instruments. It was developed in two phases. The model was experimented with 11 institutions, from four different municipalities. For the choice, the size of the municipality was considered in the set of municipalities that make up the territorial base of the GEDUC-RP and whether or not it met the demand for places in Early Childhood Education. Thus, the selection was: a municipality that is a regional reference, large and with unmet demand; one of medium size regional and with unmet demand; one of small size with attendance to all the demands. Nine public institutions (4 kindergartens and 5 preschools) and 2 partner institutions (2 kindergartens) were part of the project.

The first phase consisted of an external evaluation by the NAT technical team and interns. The visits took place without prior notice and sought to accompany a complete period (morning or afternoon) of the institution's operation. At first, the team asked to talk to the manager to present the NAT and the GEDUC, as well as the project to which the institution was selected, by sampling, and how it fits into the resolution paradigm of the Prosecutor's Office, of guaranteeing rights collectives through the promotion of projects. The NAT team sought to position itself in a dialogical way, considering the need to engage the institutions in the evaluation process, which did not avoid a certain estrangement from them and, at different levels, a defense of informing the evaluators about the functioning of the institution.

For that moment, a form was created comprising the following dimensions: legal structure; professional staff; physical space; materials; times and routine chart by classes; activities; pedagogical documentation; relationship with families; and food and health promotion. Each evaluator filled out their form and this was used for a later discussion about the institution's impressions and the writing of the Evaluation Reports of each institution, highlighting the positive points and those that demanded attention. A General Assessment report, with recommendations for actions with municipalities, was also prepared.

The second phase consisted of the internal assessment conducted by the institution, after participating in a training meeting with the NAT team and interns, in which they were invited to conduct the self-assessment using a participatory methodology, guided by the application and procedures proposed by Quality Indicators in Early Childhood Education (Indicadores da Qualidade na Educação Infantil, BRASIL, 2009). It would also be necessary to prepare a report with positive aspects and points for improvement.

After the applications, two joint meetings were held: one for the presentation of external evaluations, with notes on general issues to all institutions and singularities, discussed between coordinators, directors, and the team responsible for the evaluation; and a meeting to present the self-assessment carried out by the institutions with the exchange of experiences and discussion on the construction of a quality improvement plan. After the meetings, the institutions also added the vision of the external evaluators (technical team from the Public Prosecutor's Office) to the plan drawn up by the daycare centers and preschools in the internal evaluation. This construction, which combined internal and external views, composed an Institutional Action Plan for the Promotion of Quality delivered by each daycare center and preschool to the GEDUC-RP prosecutor, with a forecast of actions distributed over two years. The technical team of the Public Prosecutor's Office and the interns worked, throughout this period, to monitor the Plan and actions, through visits and new meetings, building a process of monitoring and common learning.

POSSIBILITIES AND LIMITS OF EXTERNAL ASSESSMENT

In the expanded scenario of public policy evaluation, external evaluation is used as a possibility for specialists to collaborate in identifying compliance with the objectives of a project or program.

Sometimes, its use may appear associated with the top-down approach of formulating and implementing public policy. This approach is concerned with structuring elements and with an expanded scalar degree of the intended change, which makes it rely, generally, on procedures and quantitative instruments and parameters constructed by universal references. External evaluation assumes that there are different roles in the evaluated program or institution, in a hierarchical system of competences and establishment of goals and responsibilities. One of his criticisms is that the top-down approach bets on central actors, in addition to implying the separation of public policy formulation and implementation processes (MOTA, 2020; NAJBERG; BARBOSA, 2006).

In Education, this approach appears mainly in large-scale evaluations, whose control of procedures for gathering information and analysis is located and concentrated in actors distant from the implementation of programs and policies, that is, external to the daily reality of institutions. Criticism of this model, as adopted in the country, has been widely documented in the area, which problematizes what public policy this type of evaluation is linked to and questions the at least simplistic and linear view of the functioning of public administration and the process of implementation of the programs (ALMEIDA, 2020; FREITAS, 2005, 2016; NAJBERG; BARBOSA, 2006; SCHNEIDER; NARD; DURLI, 2018). In Early Childhood Education, Cançado and Correa (2020) state that evaluative models inspired by this vision are privileged and criticize the fact that they do not invest in educational agents and the reflection of the work process, prioritizing the focus on children and the skills to be acquired.

However, it should be remembered that, at the municipal scale, the supervision teams, although composed of actors internal to the municipal system, act as evaluators external to daycare centers and preschools. This design plays an important role in the social legitimation of these institutions, their accreditation, quality control, recognition of the pedagogical proposal, and the promotion of improvements in the physical structure and personnel conditions. Thus, in a way, it is a system of balances, especially in the monitoring of the institutions and services of the agreement policy.

This characteristic implies acknowledging that the external evaluation is not harmful and can include participation, reception, and dialogue strategies. However, it has limits, and the uses of its results need to be observed. In this way, its use, its valuation, and its impacts need to be dimensioned and analyzed in the balance between the contributions it offers and the limits of its nature to observe the fundamentals of the public policy that sustains it; the objectives of the evaluating agent; the focus on policy implementation; the scope and extent of the intended change; the relationship between the actors; pretensions and positioning on the different stages of the public policy cycle; the rationale, aspirations and political commitments of its proposal and its actors.

The new roles of the Prosecutor's Office and the technical support to the prosecutor in the evaluation of public policies ended up putting its staff in the place of external evaluators of public programs and policies. This change also inserts pedagogues, psychologists, social workers, and other professionals as new subjects in the cycle of public policy, with differentiated functions in the defense of the right to quality Early Childhood Education. On behalf of this public body, the technical professionals can have access to the development of public policies for Early Childhood Education and, through visits and assessment of different realities, build a kind of thermometer about supply and quality on a broader scale than that of the evaluated institution. In addition, the information collected can generate attributes that contribute to better planning and execution of Early Childhood Education plans, since all external evaluation generates reports that support formal documents (CUNHA, 2018).

What is new for these professionals that are used to act directly in the implementation of policies within institutions, is to position between technical knowledge and the utopia of society and education instituted by the principles of our Federal Constitution of 1988 and the legal framework in area to promote, through the instruments of the Prosecutor's Office, the critique of what exists and “take a step forward in the long march for a democratic Brazilian Early Childhood Education” (ROSEMBERG, 2010, n.p).

In most of the institutions that participated in the PAA developed at GEDUC-RP, it was possible to perceive an inconvenience with the presence of the evaluators upon arrival, transforming it, at times, into a tense event, with an evident defensive posture of the institutions, and need to present a positive image.

Submission to the gaze of the other, by any institution or professional, is not a calm human action and is devoid of mobilizations and discomforts of the most diverse orders. Even if extremely careful, the arrival at the institutions, from the institutional place of the Prosecutor's Office, mobilizes a set of meanings supported by cultural experiences and lived on the action of this body. Identifying the present meanings, removing from the role of superiority that any external evaluation design entails, in addition to those of punishment associated with agents of the justice system, without losing sight of the institutional mandate, are challenges and tensions posed to these actors who carry out the external evaluation. In this tension, the look of specialists from the Public Prosecutor's Office, like another expands the possibilities of incidence in public policy on Early Childhood Education and undeniably carries the risk of concentrating powers and nullifying localities, specificities, and subjects. In this sense, it is essential to listen to those who participate and build the institution, suffer its problems, carry its history and see what can be improved from a perspective of experience (PALHARES, 2019).

POTENTIALITIES AND LIMITS OF INTERNAL ASSESSMENT

Internal evaluation, in general, is associated with the public policy formulation and implementation model called bottom-up.

The bottom-up approach emerges as a criticism of the separation between formulation and implementation of public policy and the watertight and fragmented nature of the cycle of public policies present in the top-down model and recognizes the active and contentious context in which the evaluations are inserted (MOTA, 2020; NAJBERG; BARBOSA, 2006; OLIVEIRA, 2013).

This model defends the existence of a dynamic relationship between these processes and shifts the focus from the central actors to the actors in the policy implementation context, the so-called street-level bureaucrats, terminology proposed by Michael Lipsky (2010), whose developments are discussed by Cavalcanti, Lotta, and Pires (2018). It is the street-level bureaucrats who use different devices in the daily implementation of the policy, shaping practices of appropriation that make political uncertainties and local pressures dialogue, adjust actions to the demands of their beneficiaries, exercise resistance, and act with different powers. Through its actions, politics is always undergoing a process of adaptation to local conditions. This movement imprints a reading of interdependence between the stages and between the actors, in addition to thinking of the constant redefinition and review of the policy as a continuum that is difficult to control since its effectiveness is always situated in the dynamics of relationships. This view is also close to Stephen Ball and Richard Bowe's proposal, which considers the policy in use as part of the policy cycle, that is, the discourses and practices at the local level. Therefore, they are subject to interpretation and can be recreated, transformed, and consolidated (MAINARDES, 2006)

The internal evaluation intends to promote listening among peers and to discuss positive and negative elements in non-intimidating ways, collaborating in contextualized agreements to improve the service (CUNHA, 2018).

In Early Childhood Education, internal evaluation appears linked to the so-called quality promotion models (BRASIL, 2009; PIOTTO et al., 1998), which value processes to the detriment of products or results and which are used to leverage negotiation movements and search for points of common interest and values, without disregarding diversities.

The different conceptions and expectations present in the educational program or institution are equally important and need to be incorporated into the evaluation. In addition to rigor in planning and the instrument used, for Coutinho and Moro (2017), democratic principles must be present to create engagement, generate self-reflection and awareness about the roles of each person in the school and the collective responsibilities in the school process of change and institutional improvement.

These evaluation designs are justified by a vision aligned with democracy and with the construction, even if difficult and laborious, of dialogue channels between the different social and institutional actors. Educational debates are centered, in this perspective, on the defense of the concept of negotiated quality (BONDIOLI, 2004), interested not in strategies for the production of quantitative data, but in processes of construction of collective commitments with the criteria of quality and with its promotion (FILIPE; BERTAGNA, 2017). As Freitas (2005) points out, in the construction of a “pact” around the needs and commitments of the multiple actors of schools and the education system for changes to occur.

In Early Childhood Education, this was the favored model when preparing the Brazilian quality assessment instrument, the Quality Indicators for Early Childhood Education (BRASIL, 2009). The elaboration of the Indicators obeyed procedures that put different groups, institutions, and researchers in the area in conversation, in a dynamic of democratic listening in the elaboration of the evaluation policy, whose continuity also followed these principles, until the ANEI proposal. The methodology foreseen by the Indicators - already developed by the Non-Governmental Organization Ação Educativa for assessing the quality of Elementary Education (Elementary School Quality Indicators) (BRASIL, 2013) - involves the participation of managers, teachers, other daycare/preschool professionals, and family members in building an appreciative picture of the quality of child care. Other initiatives for the elaboration of this type of instrument occurred to adapt it to the specificities of public networks, without losing sight of its participatory principles, as is the case of the Quality Indicators of Early Childhood Education in São Paulo (SÃO PAULO, 2016).

The internal evaluation, linked to this perspective, recognizes that the quality criteria are referenced in the multiple meanings of all subjects and limited to the social, economic, historical, and cultural conditions of the institutions.

In the Public Prosecution's procedure, initially, the proposal for the self-evaluation was received as one more task to be carried out, among many others, being necessary to work on the impressions generated from the proposal and the implications of its accomplishment in complementing the external evaluations.

The 11 kindergartens and preschools built different strategies to mobilize families and school staff. The presentation of the internal evaluation, carried out by the institutions, indicated the degree of self-criticism and maturity for the collective exhibition. Institutions with greater self-criticism, with a staff of professionals more engaged in the evaluation, open to the novelty that the project proposed and less afraid of the Public Prosecutor's Office, presented evaluations that were sometimes more demanding than those of the technical team. However, some institutions, with many points of attention identified by the external team, expressed very positive self-assessments, with difficulty naming aspects to be improved. This behavior had already appeared in an experiment with the application of a self-assessment instrument in a public, contracted, and private institution, in the mid-1990s (PIOTTO et al., 1998).

Power circulation dynamics also emerged in the presentations; sometimes, municipal secretaries accompanied the presentations in an obvious attempt to control the speech of the principals. In others, there was freedom for frank expression of professional difficulties, institutions, and municipal management. These differences also indicate the complexities of power at the time of self-assessments, marked differently by genuine acceptance of the assessments carried out by the actors or by mechanisms of silencing and production of agreement based more on coercion than on consensus.

CROSSING EYES: SIDE-BY-SIDE EVALUATIONS

The mixed evaluation is one of the public policy evaluation models that combine procedures and tools for internal and external evaluation of the programs implementing a given public policy. This model, like the others, is composed of different forms, also considering: who are the responsible agents and/or participants in the evaluation; the moment in which it occurs (ex ante, on going, or ex post the program); the focus on the process or the result (RAMOS; SCHABBACH, 2012).

Several institutions, public and private, responsible for evaluation processes have encouraged the combination of both perspectives through the top-down/bottom-up methodological arrangement as a way of apprehending global aspects (from the perspective of external teams interested in the impact of public policy interventions) and contextual aspects of actions within organizations (which contribute to the role and relationships between executors and beneficiaries) (SILVA; SOUZA, J., 2011).

Also, Osuna et al. (2000) highlight the inadequacy of a single evaluation method and see the purpose of the evaluation as more important. The authors criticize imported methodologies, not created in the sociocultural context of the assessment and which make significant results impossible. They share the view that the evaluation must dialogue with State policies, understand who the subjects of the program are, and assume the territory as a horizon. They also argue that the top-down/bottom-up arrangement promotes a feedback system, in which one boosts and enriches the other, bringing eyes closer together. In addition to the evaluation, one should have the perspective that everyone leaves this process more qualified and that the objectives are met (OSUNA et al., 2000). Mota (2020) situates this approach as a third generation of hybrid theories.

In the reported PAA, the external and internal evaluations analyzed separately, produced satisfactory results for their purposes, also considered in isolation. It would be enough to present to the public prosecutor the aspects that needed improvement in each institution, according to the external team's report. This information already had the power to instruct a set of administrative or judicial actions for measures with the municipal administrations and the expansion of public investments in the conditions of infrastructure and the offer of vacancies in daycare centers and preschools.

The debates and discussions promoted in the internal evaluation also resulted, to a greater or lesser extent, in perceptions of problematic aspects not always aware by the school community, which was made possible by the exercise of looking at themselves with magnifying glasses and the time that the participatory procedure the Indicators promoted.

However, both processes were dynamized precisely by the adoption of a design that placed the different assessments side by side. The choice for the mixed assessment was linked to the vision of a Public Prosecutor's Office (GOULART, 2013b), which bets on the institutional capacity to create spaces for dialogue between the different actors in society to enhance the construction and co-responsibility for the promotion of public policies in all stages of its cycle (ASENSI, 2010; RAEDER, 2014). The mixed assessment sought to be a resource for this type of action and to be an instrument to drive the dynamics of internal assessment. The internal evaluation would have occurred differently and produced different results if its effects and scope had been limited to the scope of the institution and its actors. For some institutions (according to the evaluation of the project with them carried out in one of the meetings), having the Public Prosecutor's Office external evaluation was both a legitimization of their good practices - which left them fulfilled - and also a support for their demands, many times without echo in the municipal management and in the secretariats on whom they depended for the solution of internal problems of the institution - which gave them hope.

Thus, despite the misgivings about the Public Prosecutor's Office, the external evaluation by the GEDUC-RP acted as an important ally in the internal quest to improve the quality of the conditions that structure the care for children in the evaluated daycare centers and preschools. On the other hand, for the team responsible for the external evaluation, there was an increase in the power of their tasks, since they positioned not only in the construction of an opinion that is a means to an end. The mixed design made it possible to overcome the limits of an external assessment carried out in one visit and to expand knowledge about the set of evaluated dimensions, institutions, and relationships. In addition, by leveraging the entire evaluation process, the mixed design also improved its product; the elaboration of the Institutional Plan for the Promotion of Quality resulted from a synthesis effort, enriching it and taking a significant leap between the first (internal) version and the version that dialogued the internal and external evaluations.

Another gain from the crossing of eyes, especially through the promotion of collective meetings between institutions from different municipalities, was the possibility of transcending the singularities of the institutions and apprehending more structural problems, that is, the constituent mediations of the scale of the municipalities. Reports on the particularities that crossed all institutions of the municipalities, as well as all four municipalities, were additionally elaborated on different actions of the GEDUC-RP, in the incidence in the municipal policies of construction of kindergartens and pre-schools; security of physical structures; purchase of toys and materials; convenience policy; training of professionals; construction of the political-pedagogical project; supervisory teams. Some of these indications resulted in new PAAs; however, several were discontinued due to the COVID-19 pandemic. One of the PAAs would be the construction of Conduct Adjustment Terms - an instrument of mutual agreement - for municipalities to build participatory procedures in the preparation of a Permanent Municipal Plan for Monitoring Quality in Early Childhood Education.

The mixed evaluation model, side by side, changed the understandings that placed external and internal evaluation in polarities and generated discomfort and search for alternatives to a model of performance of the Public Prosecutor's Office team. However, it is emphasized that this form of evaluation, which places external and internal evaluators in interaction and conversation, demands interactive sensibilities and skills that are not always easy; and, no less demanding in terms of maturity is the position occupied by the teams from the evaluated day care centers and preschools. The dialogic situation produces conversation zones full of sharing, but also a lot of estrangement and obvious disputes over space and power. There was engagement and commitment but also doubts about the relevance and necessity of the proposal, without apprehending the real meaning of co-responsibility for the evaluation.

As a control body, the Public Prosecutor's Office imposes tensions on the institutions and their actors, since reality is marked by challenges that are felt by those who try to make their work happen daily as executors of the care of young children. Another important consideration is that this way of acting by the Public Prosecutor's Office, due to its novelty since the Federal Constitution of 1988, has not yet been consolidated, and the hegemony of this model is also in dispute, which generates conflicts and different understandings, even among its actors, on ways of acting to guarantee diffuse educational rights. Furthermore, the limitations of professionals external to an institution are also great, since, as they do not share the institutional routine, they always apprehend partial and momentary versions and aspects. This tension, before being overcome, must be recognized so that the approach respects the institutions and their condition as protagonists of their problems.

FINAL CONSIDERATIONS

The experience of evaluating daycare centers and preschools, provoked by an unusual place for their participants to act, encouraged the search for a deeper understanding of the limits and potential of the evaluation proposals most discussed in the literature, close to the formulation and implementation approaches of public policies known as top-down or bottom-up.

In this process, a design was constructed that associated external and internal strategies and instruments with the evaluated institutions. It was the concreteness of the link to the Public Prosecutor's Office and the role it plays in the defense of the Federal Constitution, and consequently, in the dynamics of the cycle of public policies, that pressured the creation of this type of design. Pedagogues, psychologists, and social workers from that place, summoned to assume new roles, not directly involved in the implementation of policies and programs, began to mediate actions that transit between the scale of institutions (implementation and execution of the policy of early childhood education) and the scale of the municipalities, with an impact on the decisions of the prosecutor for the defense of quality early childhood education. In this way, they also brought to the practice of the Public Prosecutor's Office a form of evaluation that is sensitive to localities, loaded with the experience as street-level bureaucrats that their training gives them, capable of mediating different actors in the understanding that policies are also dependent on the environment of the implementation. On the other hand, daycare and preschool professionals carried out the self-assessment, sustaining the relationships and commitments internally to the institution, but also as an important actors in the control of public policies, expanding their functions in the evaluation of Early Childhood Education and being involved in a reading of both the conditions of its offer as well as the effects of its actions.

The constructed design, in a way, valued local agents but did not abandon its focus on acting with agents in decision-making positions, not to dissociate macro and micro contextual aspects. In any case, this design is not necessarily intended to surpass other types, due to a series of issues raised by the experience.

The experience paved the way for the awareness that different places demand instruments and methods that are also different. The social and institutional roles of the actors located in the dynamics of the cycle of formulation and implementation of public policies constrain possibilities of action, even if these places are not exactly fixed and can be exercised creatively. Thus, the instruments and evaluation procedures are organic to their creation contexts and the role of their actors. In the case of the Public Prosecutor's Office, the presence of the technical team as an external evaluator is always crossed by the inspection character that permeates the popular imagination about this body, which can make it difficult to carry out an evaluation that is dialogical with the evaluated service, no matter how much efforts are made to approach and occupation of that place from a resolutive paradigm (GOULART, 2013b).

In addition, the experience also consolidated the notes made in the area about the association between evaluation and the scale of desired changes. If the proposal of a mixed evaluation, side by side, showed a relationship of continuity between changes within the institutions and changes in the scale of municipal politics, it also indicated that this transition does not occur harmoniously; on the contrary, it is constituted by conflicts, both between actors internal to the education system and between them and the external evaluation team. Also, the dialogic perspective of the mixed evaluation, side by side, has challenges that it introduces in the complexity of the relationships between people and public bodies. Power relations and tensions are intrinsic to evaluations will always be, as previously discussed, in a tonality of struggle for models of societies. Every evaluation is always an instrument of power.

These questions thus indicate that, before overcoming and criticizing the types of evaluation, it is their uses and interests that need to be made explicit and problematized. In addition, the joining of two different perspectives on the same educational process needs to be done in a planned way, in a way that generates benefits for all and, mainly, that has the child as its horizon.

External, internal, or mixed evaluations, which contrast with the historically constructed and consolidated conceptions of Early Childhood Education, need to be rejected, as well as those that, regardless of their designs, are not consistent with the principles of democratic participation.

Evidently, from the experience limited to the participating actors, the mixed evaluation was constituted as a possible strategy in the consolidation of action in consonance with the new paradigm of the Public Prosecutor's Office and with the democratic values of the current Brazilian society. The mobilization of different actors, local and involved with management, and the joint construction of action plans commit everyone, collectively, to quality early childhood education and to the promotion of a democratic rule of law. Working with public policies involves different social relationships (GONÇALVES, 2013). For a more democratic posture in the exercise of power, and in its effectiveness, the evaluations must also be constructed jointly, with the participation of the multiple actors involved in this process.

Lessons learned from the experience reinforce that it is the commitment to democratic Early Childhood Education in Brazil and the conception of the State set out in the Federal Constitution of 1988 that should guide the collective construction of public evaluation policies that overcome the historical challenges of the area. Thus, it must be associated with a critical perspective, which problematizes how much public policy has been able to expand rights, provide equity and reduce social inequalities (BOSCHETTI, 2009), in addition to being supported by a conception that contrasts with visions of quality present in neoliberal policies (FREITAS, 2005). Recognition and appreciation of children and family members as subjects of rights, creation of spaces for participation, the establishment of vocalization mechanisms of different social actors, respect for local specificities and dynamics, democratic and dialogical posture, strengthening of the concept of the rule of law are horizons, present in the constitutional principles, which guide the critical look and evaluation of evaluation policies and their different designs. With them in hand and mind, as indicators or criteria for analyzing evaluation policies, it is not difficult to distinguish strategies that seek to reduce inequalities or, on the contrary, encourage and stigmatize children and populations.

Preprint

DOI: https://doi.org/10.1590/SciELOPreprints.3631

Acknowledgments

The authors thank all the institutions that participated in the evaluation and the NAT and GEDUC-RP team for allowing the development of this work together: Alice Vieira de Albuquerque, Pâmela Migliorini Claudino da Silva, Rachel Fernanda Matos dos Santos, Naul Felca, Gabriela Oliveira Lacerda Luciana Leoncini. Thanks also to the retired Prosecutor, Marcelo Pedroso Goulart.

• The translation of this article into English was funded by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES/Brasil.

REFERÊNCIAS

• ALMEIDA, Luana C. Quando o foco passa a ser o resultado na avaliação externa em larga escala: evidências de uma rede. Educação em Revista, v. 36, p. e233713, 2020. <https://doi.org/10.1590/0102-4698233713>
  » https://doi.org/10.1590/0102-4698233713
• ALVARENGA, Carolina F.; VIANNA, Cláudia P. Avaliação, gênero e qualidade na Educação Infantil: conceitos em disputa. Educar em Revista, Curitiba, v. 37, p. e78271, 2021. <http://dx.doi.org/10.1590/0104-4060.78271>
  » https://doi.org/10.1590/0104-4060.78271
• ASENSI, Felipe D. Judicialização ou juridicização? As instituições jurídicas e suas estratégias na saúde. Physis: Revista de Saúde Coletiva [online]., v. 20, n. 1, p. 33-55, 2010. <https://doi.org/10.1590/S0103-73312010000100004>
  » https://doi.org/10.1590/S0103-73312010000100004
• BARBOSA, Ivone G. Das políticas contraditórias de flexibilização e de centralização: reflexões sobre a história e as políticas da educação infantil em Goiás. Intera-Ação, v. 33, n. 2, p. 379-393, 2008. Disponível em: <https://www.revistas.ufg.br/interacao/article/view/5273/4690>. Acesso em 30/07/2021.
  » https://www.revistas.ufg.br/interacao/article/view/5273/4690
• BRASIL. Lei nº 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Brasília, 1996. Disponível em: Disponível em: http://www.planalto.gov.br/ccivil_03/leis/l9394.htm Acesso em 07/10/2022.
  » http://www.planalto.gov.br/ccivil_03/leis/l9394.htm
• BRASIL. Ministério da Educação. Indicadores da Qualidade na Educação Infantil. Brasília: MEC/SEB, 2009. Disponível em: <Disponível em: http://portal.mec.gov.br/dmdocuments/indic_qualit_educ_infantil.pdf >. Acesso em: 20/03/2018.
  » http://portal.mec.gov.br/dmdocuments/indic_qualit_educ_infantil.pdf
• BRASIL. Ministério da Educação. Diretrizes curriculares nacionais para a Educação Infantil. Brasília: MEC/SEB , 2010. Disponível em: <http://portal.mec.gov.br/dmdocuments/diretrizescurriculares_2012.pdf>. Acesso em: 03/04/2018.
• BRASIL. Ministério da Educação. Indicadores da Qualidade na Educação - Ensino Fundamental. Unicef, Pnud, Inep, SEB/MEC (coords.)- São Paulo: Ação Educativa, 2013. Disponível em: <Disponível em: http://portal.mec.gov.br/seb/arquivos/pdf/Consescol/ce_indqua.pdf >. Acesso em: 19/04/2018.
  » http://portal.mec.gov.br/seb/arquivos/pdf/Consescol/ce_indqua.pdf
• BRASIL. Lei nº 13.005, de 25 de junho de 2014. Aprova o Plano Nacional de Educação - PNE e dá outras providências. Brasília, 2014. Disponível em: Disponível em: http://www.planalto.gov.br/ccivil_03/_ato2011-2014/2014/lei/l13005.htm Acesso em: 07/10/2022.
  » http://www.planalto.gov.br/ccivil_03/_ato2011-2014/2014/lei/l13005.htm
• BONAMINO, Alicia; SOUSA, Sandra Z. Três gerações de avaliação da educação básica no Brasil: interfaces com o currículo da/na escola. Educação e Pesquisa, v. 38, n. 2, p. 373-388, 2012. <https://doi.org/10.1590/S1517-97022012005000006>
  » https://doi.org/10.1590/S1517-97022012005000006
• BONDIOLI, Anna. O projeto pedagógico da creche e a sua avaliação: a qualidade negociada. Campinas: Autores Associados, 2004.
• BOSCHETTI, Ivanete. Avaliação de políticas, programas e projetos sociais. In: CFESS; ABEPSS. (Org.). Serviço Social: Direitos Sociais e Competências Profissionais. Brasília: CFESS, 2009. p. 575-592. Disponível em: <Disponível em: http://www.cressrn.org.br/files/arquivos/V6W3K9PDvT66jNs6Ne91.pdf >. Acesso em: 04/02/2021.
  » http://www.cressrn.org.br/files/arquivos/V6W3K9PDvT66jNs6Ne91.pdf
• CAMPOS, Maria M. Avaliação da qualidade na Educação Infantil: impasses e perspectivas no Brasil. Revista Pesquisa e Debate em Educação, v. 10, n. 1, p. 891 - 916, 2020. <https://doi.org/10.34019/2237-9444.2020.v10.32009>
  » https://doi.org/10.34019/2237-9444.2020.v10.32009
• CANÇADO, Natália F. C.; CORREA, B. C. Avaliação, qualidade e Educação Infantil: análise de uma experiência municipal. Estudos em avaliação educacional, v. 32, p. e07719, 2021. <https://doi.org/10.18222/eae.v32.7719>
  » https://doi.org/10.18222/eae.v32.7719
• CÁRIA, Neide P.; OLIVEIRA, Sandra M. S. S. Avaliação em larga escala e a gestão da qualidade da educação. Revista de Ciências Humanas Educação, n. 26, p. 22-40, 2015. Disponível em: <Disponível em: http://revistas.fw.uri.br/index.php/revistadech/article/view/1477/1853 >. Acesso em: 09/04/2021.
  » http://revistas.fw.uri.br/index.php/revistadech/article/view/1477/1853
• CAVALCANTI, Sérgio; LOTTA, Gabriela S.; PIRES, Roberto R. Contribuições dos estudos sobre burocracia de nível de rua. In: PIRES, Roberto; LOTTA, Gabriela; OLIVEIRA, Vanessa E. (Orgs.). Burocracia e Políticas Públicas no Brasil: intersecções analíticas. Brasília: Ipea/Enap, 2018. p. 227-246.
• CORSINO, Patrícia. Apresentação - Movimentos avaliativos na e da Educação Infantil. Educar em Revista, Curitiba, v. 37, p. e83539, 2021. <http://dx.doi.org/10.1590/0104-4060.83539>
  » https://doi.org/10.1590/0104-4060.83539
• COUTINHO, Ângela S.; MORO, Catarina. Educação Infantil no cenário brasileiro pós golpe parlamentar: políticas públicas e avaliação. Revista Zero-a-seis, v. 19, n. 36, p. 349-360, 2017. < https://doi.org/10.5007/1980-4512.2017v19n36p349>
  » https://doi.org/10.5007/1980-4512.2017v19n36p349
• CUNHA, Carla. G. S. Avaliação de políticas públicas e programas governamentais: tendências recentes e experiências no Brasil. Revista Estudos de Planejamento, n. 12, p. 27-57, 2018. Disponível em: <Disponível em: https://revistas.dee.spgg.rs.gov.br/index.php/estudos-planejamento/article/view/4298 >. Acesso em 13/10/2020.
  » https://revistas.dee.spgg.rs.gov.br/index.php/estudos-planejamento/article/view/4298
• FARIA, Carlos A. P. A política da avaliação de políticas públicas. Revista Brasileira de Ciências Sociais, v. 20, n. 59, p. 97-169, 2005. <https://doi.org/10.1590/S0102-69092005000300007>
  » https://doi.org/10.1590/S0102-69092005000300007
• FELDMAN, Marina; SILVEIRA, Adriana A. D. Atuação extrajudicial do Ministério Público e direito à Educação Infantil: um estudo de caso. Educação e pesquisa, v. 45, p. e186597, 2019. <http://dx.doi.org/10.1590/S1678-4634201945186597>
  » https://doi.org/10.1590/S1678-4634201945186597
• FILIPE, Fabiana. A.; BERTAGNA, Regiane. H. A concepção de qualidade educacional impulsionada pelas avaliações externas no estado de São Paulo. Revista Educação em Questão, v. 55, n. 46, p. 188-219, 2017. <10.21680/1981-1802.2017v55n46ID13297>
  » https://doi.org/10.21680/1981-1802.2017v55n46ID13297
• FREITAS, Luiz C. Qualidade negociada: avaliação e contra-regulação na escola pública. Educação e Sociedade, v. 26, n. 92, p. 911-933, 2005. <https://doi.org/10.1590/S0101-73302005000300010>
  » https://doi.org/10.1590/S0101-73302005000300010
• FREITAS, Luiz C. Três teses sobre as reformas empresariais da educação: perdendo a ingenuidade. Caderno Cedes, v. 36, n. 99, p. 137-153, 2016. <https://doi.org/10.1590/CC0101-32622016160502>
  » https://doi.org/10.1590/CC0101-32622016160502
• GONÇALVES, Maria da Graça. M. Psicologia, subjetividade e políticas públicas. São Paulo: Cortez, 2013
• GOULART, Marcelo. P. Ministério Público e Políticas Públicas. In: SABELLA, Walter P.; DAL POZZO, A. A. F.; FILHO, José E. B. (Orgs). Ministério Público: vinte e cinco anos do novo perfil institucional. São Paulo: Malheiros Editores LTDA, 2013a. p. 285-308.
• GOULART, Marcelo P. Elementos para uma teoria geral do Ministério Público. Belo Horizonte: Arraes Editores, 2013b.
• LIPSKY, Michael. Street-level Bureaucracy: dilemmas of the individual in public services. 30ª anniversary expanded Ed. New York: Russel Sage Foundation, 2010.
• MAINARDES, Jefferson. Abordagem do ciclo de políticas: uma contribuição para a análise de políticas educacionais. Educação e Sociedade, Campinas, v. 27, n. 94, p. 47-69, 2006.
• MORO, Catarina. Diferentes olhares para a creche: a avaliação de contexto com o instrumento SPRING em um município da Emilia Romagna. Revista Linhas, v. 19, n. 40, p. 138-160, 2018. <http://dx.doi.org/10.5965/1984723819402018138>
  » https://doi.org/10.5965/1984723819402018138
• MORO, Catarina; SOUZA, Gizele. Produção Acadêmica Brasileira sobre Avaliação em Educação Infantil: primeiras aproximações. Estudos em Avaliação Educacional, v. 25, n. 58, p. 100-125, 2014. <https://doi.org/10.18222/eae255820142743>
  » https://doi.org/10.18222/eae255820142743
• MOTA, Luís F. Estudos de implementação de políticas públicas: uma revisão de literatura. Sociologia, Problemas e Práticas, v. 92, p. 133-150, 2020. Disponível em: <Disponível em: http://journals.openedition.org/spp/7161 >. Acesso em: 14/02/2022.
  » http://journals.openedition.org/spp/7161
• NAJBERG, Estela; BARBOSA, Nelson B. Abordagens Sobre o Processo de Implementação de Políticas Públicas. In: Encontro de Administração Pública e Governança, 2006, São Paulo. Anais. São Paulo: Anpad, 2006. Disponível em: <Disponível em: http://www.anpad.org.br/admin/pdf/ENAPG276.pdf > Acesso em: 26/03/2021.
  » http://www.anpad.org.br/admin/pdf/ENAPG276.pdf
• NEVES, Vanessa F. A.; MORO, Catarina. Avaliação na educação infantil: um debate necessário. Estudos em Avaliação Educacional, v. 24, n. 55, p. 272-303, 2013. <https://doi.org/10.18222/eae245520132727>
  » https://doi.org/10.18222/eae245520132727
• OLIVEIRA, Luciano M. O ministério público brasileiro e a implementação de políticas públicas. Revista de informação legislativa, v. 50, n. 198, p. 223-238, 2013.
• OSUNA, J. L. et al Guía para la evaluación de políticas públicas. Sevilla: Instituto de Desarrollo Regional, Fundación Universitaria, 2000.
• PALHARES, Amanda. M. Assessoria técnica do serviço social no Ministério Público. In: SOUZA, Bianca R. et al (Orgs.). NAT em movimento: práticas do Núcleo de Assessoria Técnica Psicossocial. São Paulo, SP: MPSP/NAT, 2019. p. 8-22.
• PIMENTA, Cláudia. O.; SOUSA, Sandra Z.; FLORES, Maria L. R. Dimensões para análise de propostas de avaliação de políticas de Educação Infantil. Educar em Revista, v. 37, p. e78210, 2021. <http://dx.doi.org/10.1590/0104-4060.78210>
  » https://doi.org/10.1590/0104-4060.78210
• PIOTTO, Débora C. et al Promoção da qualidade e avaliação na educação infantil: uma experiência. Cadernos De Pesquisa, n. 105, p. 52-77, 1998. Disponível em: <Disponível em: http://publicacoes.fcc.org.br/index.php/cp/article/view/703 >. Acesso em: 18/11/2021.
  » http://publicacoes.fcc.org.br/index.php/cp/article/view/703
• ROSEMBERG, Fúlvia. Caminhos cruzados: educação e gênero na produção acadêmica. Educação e Pesquisa, v. 27, n. 1, p. 47-68, 2001. <https://doi.org/10.1590/S1517-97022001000100004>
  » https://doi.org/10.1590/S1517-97022001000100004
• ROSEMBERG, Fúlvia. Sísifo e a educação infantil brasileira. Pro-posições, v. 14, n. 1, p. 177-194, 2003. Disponível em: <Disponível em: https://periodicos.sbu.unicamp.br/ojs/index.php/proposic/article/view/8643915/11382 >. Acesso em: 04/03/21.
  » https://periodicos.sbu.unicamp.br/ojs/index.php/proposic/article/view/8643915/11382
• ROSEMBERG, Fúlvia. Educação infantil pós-FUNDEB: avanços e tensões. In: SEMINÁRIO EDUCAR NA INFÂNCIA: PERSPECTIVAS HISTÓRICO-SOCIAIS, 2007, Curitiba2017. Disponível em: <Disponível em: http://www.diversidadeducainfantil.org.br/PDF/Educa%C3%A7%C3%A3o%20infantil%20p%C3%B3s-FUNDEB%20avan%C3%A7os%20e%20tens%C3%B5es%20 %20F%C3%BAlvia%20Rosemberg.pdf >. Acesso em: 07/08/2020.
  » http://www.diversidadeducainfantil.org.br/PDF/Educa%C3%A7%C3%A3o%20infantil%20p%C3%B3s-FUNDEB%20avan%C3%A7os%20e%20tens%C3%B5es%20 %20F%C3%BAlvia%20Rosemberg.pdf
• ROSEMBERG, Fúlvia. Um passo adiante na longa marcha por uma Educação Infantil brasileira democrática. Palestra proferida no Seminário Internacional de Avaliação da Qualidade da Educação Infantil, 2010. YouTube Disponível em: <Disponível em: http://www.youtube.com/watch?v=HE3PRoc2bJU >. Acesso em: 25/04/2020.
  » http://www.youtube.com/watch?v=HE3PRoc2bJU
• ROSEMBERG, Fúlvia. Políticas de educação infantil e avaliação. Cadernos de Pesquisa, v. 43, n. 148, p. 45-75, 2013. <https://doi.org/10.1590/S0100-15742013000100004>
  » https://doi.org/10.1590/S0100-15742013000100004
• RAEDER, Savio T. O. Ciclo das políticas públicas: uma abordagem integradora dos modelos para análise de políticas públicas. Perspectivas em Políticas Públicas, v. 7, n. 13, p. 121-146, 2014. Disponível em: <Disponível em: https://revista.uemg.br/index.php/revistappp/article/view/856 >. Acesso em 25/05/2021.
  » https://revista.uemg.br/index.php/revistappp/article/view/856
• RAMOS, Maria. P.; SCHABBACH, Leticia M. O estado da arte da avaliação de políticas públicas: conceituação e exemplos de avaliação no Brasil. Revista de Administração Pública, v. 46, n. 5, p. 1271-1294, 2012. <https://doi.org/10.1590/S0034-76122012000500005>
  » https://doi.org/10.1590/S0034-76122012000500005
• RUA, Maria G. Análise de Políticas Públicas: conceitos básicos. Textos elaborados para o Curso de Formação para a carreira de Especialista em Políticas Públicas e Gestão Governamental. Brasília: ENAP/Ministério do Planejamento, 1997.
• SÃO PAULO (Município). Secretaria Municipal de Educação. Diretoria de Orientação Técnica. Indicadores de Qualidade da Educação Infantil Paulistana. São Paulo: SME / DOT, 2016. Disponível em: Disponível em: http://portal.sme.prefeitura.sp.gov.br/Portals/1/Files/35117.pdf Acesso em: 24/10/2020.
  » http://portal.sme.prefeitura.sp.gov.br/Portals/1/Files/35117.pdf
• SCHNEIDER, Marilda P.; NARDI, Elton L.; DURLI, Zenilde. Políticas de avaliação e regulação da qualidade: repercussões na educação básica. Revista e-Curriculum, v. 16, n. 1, p. 109-138, 2018. <http://dx.doi.org/10.23925/1809-3876.2018v16i1p109-138>
  » https://doi.org/10.23925/1809-3876.2018v16i1p109-138
• SILVA, Juliana B.; SOUZA, Tatiana N. Análise da utilização de uma escala para avaliação da qualidade de creches. Estudos em Avaliação Educacional, v. 22, n. 48, p. 137-158, 2011. <https://doi.org/10.18222/eae224820112004>
  » https://doi.org/10.18222/eae224820112004
• SOUZA, Celina. Políticas públicas: uma revisão da literatura. Sociologias, v. 8, n. 16, p. 20-45, 2006. <https://doi.org/10.1590/S1517-45222006000200003>
  » https://doi.org/10.1590/S1517-45222006000200003
• TONET, Ivo. Qual política social para qual emancipação?. SER Social, v. 17, n. 37, p. 279-295, 2015. <https://doi.org/10.26512/ser_social.v17i37.13432>
  » https://doi.org/10.26512/ser_social.v17i37.13432

Publication Dates

• Publication in this collection
  27 Mar 2023
• Date of issue
  2023

History

• Received
  21 Feb 2022
• Accepted
  17 Aug 2022