Effects of changes in a multi-level implementation context: analysis of the Brazilian High School Reform

Abstract

The article seeks to understand how exogenous changes are impacted by implementation contexts established in each place. Based on the analysis of Law 13.415/2017, which reforms High School in Brazil, it verifies how the federal changes proposed by this policy caused alterations in the states and how the local contexts also generated changes in the reform itself. The article is based on a qualitative investigation that monitored the effects of the publication of the Law and the beginning of the process of implementing the reform in the 26 states and the Federal District over two years. The empirical case was analyzed based on the variables of conflict and ambiguity that characterize the contexts of implementation. The article contributes to the literature on the implementation of public policies, reforms in public administration, and education policies.

Keywords:
policy implementation; reforms; high school; changes

Resumo

O artigo busca compreender como mudanças exógenas são afetadas pelos contextos de implementação que encontram em cada local. A partir da análise da Lei 13.415/2017, que reforma o Ensino Médio no Brasil, verifica como as mudanças federais trazidas por esta política provocaram alterações nos estados e qual a variação que os contextos locais geraram na própria reforma. O artigo se baseia em uma pesquisa qualitativa que acompanhou os efeitos da publicação da Lei e o início do processo de implementação da reforma nos 26 estados e Distrito Federal ao longo de dois anos. O caso empírico foi analisado a partir das variáveis de conflito e ambiguidade que caracterizam os contextos de implementação. Estes achados contribuem para a literatura de implementação de políticas públicas, de reformas e mudanças na administração pública e de políticas de educação.

Palavras-chave:
implementação de políticas; reformas; ensino médio; mudanças

Resumen

El artículo busca comprender cómo cambios exógenos son impactados por contextos de implementación establecidos en cada lugar. A partir del análisis de la Ley 13.415/2017, que reforma la Educación Media en Brasil, verifica cómo los cambios federales de esta política provocaron alteraciones en los estados y cómo los contextos locales generaron cambios también a la propia reforma. El artículo se basa en una investigación cualitativa que hizo seguimiento a los efectos de la publicación de la Ley y al inicio del proceso de implementación de la reforma en los 26 estados y el Distrito Federal a lo largo de dos años. El caso empírico fue analizado a partir de las variables de conflicto y ambigüedad que caracterizan los contextos de implementación. El artículo contribuye con la literatura de implementación de políticas públicas, de reformas a la administración pública y de políticas de educación.

Palabras clave:
implementación de políticas; reformas; educación media; cambios

1. INTRODUCTION

Public policies are constantly subject to change, to a greater or lesser extent, brought about by endogenous or exogenous factors (Baumgartner, True & Jones, 1999; Mahoney & Thellen, 2010). Part of the literature on public policies has elaborated on this phenomenon and its effects on the government agenda (Sabatier & Jenkins-Smith, 1993). In general, it analyses how the changes cause alterations in the agenda or in the formation of policies, in a systematic attempt to broadly understand them, comparing different historical moments and factors that may help to explain the changes (Capella, Brasil & Sudano, 2015). What the literature suggests is that political processes have a multi-dimensional nature in which different players, institutions and ideas interact in the shaping of the agenda and the changes (Baumgartner, Jones & Wilkerson, 2011).

Part of the literature is also dedicated to understanding the State’s processes of reform, observing its formation (Osborne & Gaebler, 1998; Pollitt & Bouckaert, 2011), and the effects on the state structures (Rodrigues & Jayo, 2016). However, few advances have been made in the understanding of how the reforms are implemented and the effect of the implementation process on the reforms themselves (Rodrigues & Lotta, 2017).

This article seeks to contribute to filling some of these gaps in the literature, attempting to provide an understanding of the results of the changes in complex realities, made up of many different organizations, in which an exogenous change can lead to different consequences depending upon the context. Another point is an understanding of how the reforms are implemented, what effects the implementation have on them, and to what extent the contexts of the implementation affect them. The object, therefore, is to analyze the effects of the exogenous change in contexts of policy implementation located in the complex and federal institutional environments. The questions that the paper intends to answer are: how are exogenous changes adapted locally in federal contexts? To what extent do these contexts affect the reforms? To what extent do these reforms affect different contexts of implementation? To answer these questions, we will be using, as our analytical model, the understanding of contexts of implementation by means of the relationship between the ‘conflict’ and ‘ambiguity’ variables developed by Matland (1995).

In empirical terms, the reform of Secondary Education in Brazil has been chosen. This paper is based upon a qualitative study (interviews, a case study and survey) which, over the course of two years (2017 to 2018), monitored the start of the implementation process of the reform in the 26 states and the Federal District, as of the publication of Law 13,415 (2017), making comparisons with the context prior to the Law. This monitoring allowed us to analyze how the implementation contexts were transformed as a result of the adopted reform.

2. ACADEMIC CONTRIBUTIONS TO PROCESSES OF IMPLEMENTATION AND CHANGE IN PUBLIC POLICIES

To understand the processes of implementation and policy change in a complex and federal environment, studies on the context of implementation, the implementation of policies in federal contexts and implementation of reforms were used as academic inputs.

2.1 Contexts of Policy Implementation

The literature on policy implementation is concerned with analyzing how the regulations and plans are established as a result of action taken by different players (Hill & Varone, 2016). Part of this literature has sought to understand the characteristics of the contexts of implementation that allow some policies to manage to achieve their results whilst others fail (Hill & Hupe, 2003; Pollitt, 2013). Going beyond a normative perspective, the literature seeks to understand the factors that affect the context of implementation and, therefore, its ability to achieve the planned results (Faria, 2012; Hill & Hupe, 2003; Lotta, Nunes, Cavalcante, Ferreira & Bonat, 2018; Pollitt, 2013). Despite the literature recognizing the importance of the context as a means of explaining the implementation processes, only a few authors have actually made advances in understanding which contextual variables matter and how they affect the execution of the policies (Pollitt, 2013).

Matland (1995) was one of the authors who have been most responsible for guiding the studies of the effects of context on implementation. Inspired by the organizational theory, he proposed an analytical model based upon two central variables: ambiguity and conflict. For this author, every context of public policy is notable for differing degrees of conflict and ambiguity; the interdependence between these two elements leads to contexts that are more or less conducive for implementations that are effective and resolve the issues.

The conflict is related to the inter-dependency between the players and the (in)compatibility of objectives. Every organization and public policy carries inconsistencies that appear as greater or lesser degrees of conflict between the players. The conflict can be characterized on two levels: conflict in relation to the ends (in which players do not agree with the objectives of the policy) and conflict in relation to the means (in which the players do not agree with the ways adopted to achieve the objectives) (Matland, 1995).

The ambiguity concerns the space given by the policy for interpretation and adaptation in relation to both the means and the ends (Matland, 1995). For some of the specialists, the ambiguity is one element that is prejudicial to the policies, since it reduces the ability to control, the imposition (enforcement) of actions and the standardization of results (Pressman & Wildavsky, 1973). Another sector feels that the ambiguity can have positive effects, since it allows learning, experimentation and adaptation (Heclo, 1978). Or in other words, it could be an error in the drafting of the policy, when it shows shortcomings in the specifications, opening up space for unexpected decisions and gaps in democracy (Rothstein, 1998). It could also be intentional, when opportunities for interpretation suggest that those drafting the policies do not wish to assume the risk of pre-determination or do not have the political conditions to approve service standards (Hill & Hupe, 2002).

Matland (1995) highlights that ambiguity and conflict exist in varying degrees and combinations. The association between the two generates differing contexts of implementation, as Box 1demonstrates:

Box 1
Contexts of ambiguity and conflict

In an administrative form of implementation, the political objectives and means of achieving them are clear. These are policies that will tend to have a standard form of execution by players showing little discretion, and with results that will be highly predictable (Matland, 1993; May, 1995). They generally relate to technocratic environments with little politicization.

In the experimental implementation, there is no question of the need for the public policy, but decisions on results and the means of achieving them is open. In these situations, experimentalism is found, allowing the players adaptation and interpretation. The central focus is negotiation and creation. It is generally linked to federal contexts or new policies; the ambiguity is intentional to allow adaptation (Hill & Varone, 2016; Matland, 1995).

The political implementation occurs in situations in which the governing player decides to include an issue in the agenda and implement it, despite the conflict not having been resolved. It creates conditions of enforcement via standards (little ambiguity) (May, 1993). Guaranteeing conformity occurs by means of incentive or coercion, allowing the conflict to be resolved (or not confronted) through standardized and ‘top-down’ forms of implementation. They are normally policies with disruptive subject matters, but which form part of the government agenda (Matland, 1995).

In a symbolic form of implementation, the difficulty in executing the policies is evident, considering that, as well as being highly ambiguous, they are the objects of dispute (Matland, 1995). This context is generally linked to the existence of new values or objectives that require the government to take a stance, but there is no mobilization of resources to put them into practice. The government players assume the subject matter, but do not face up to the conflict, and nor do they reduce the ambiguity to make the policy effective. This can occur because the governing player has no interest in implementing them, or the forces of coalition do not provide the support, or, furthermore, there are no political or material means to be able to do so. The tendency with policies of this type is that they do not lead to effective results (Matland, 1995).

By being one of the few authors to use analytical variables to understand the contexts of implementation, Matland (1995) has been an important benchmark for studies in this area and he will serve as an analytical base for this study.

2.2 Policy Implementation in Federal Contexts

A second important question when thinking of implementation contexts concerns how the policies are adapted and reformulated when they pass through the various levels of players involved in their processes. To analyze the implementation, it is important to understand who decides what alterations to the policies are made in the different levels of decision-making and how the alterations are made (Hill & Hupe, 2003). This idea stems from the presupposition that there is generally a great distance between those forming the policies and those ultimately executing them (Arretche, 2001). This distance leads to contingencies in the implementation due to the decisions taken by a chain of implementers allocated in a wide range of political and institutional contexts (Lotta, 2015). To understand the processes of the implementation of a policy, it is therefore necessary to analyze the levels of decision-making that they undergo.

The literature on the matter proposes two central concepts: ‘multi-level’ implementation and ‘multi-layering’ implementation. The first is a characteristic of almost every policy and concerns the different parties that coexist in the cycle of public policy making but which do not have sufficient autonomy to alter decisions. This is the case, for example, with the organizational hierarchical levels involved in a policy (Hill & Hupe, 2003). The ‘multi-layering’ form of implementation, meanwhile, concerns the levels of different governments that exercise autonomy and are legitimately enabled to take decisions autonomously and with territorial jurisdiction, as occurs in the federal contexts, in which there is a separation of responsibilities between entities with legitimate decision-making rights (Hill & Hupe, 2003). This is an important distinction because it sets out the difference between those who have a legitimate mandate to transform the policy and reform it, and those who do not, and, moreover, who interfere in the day-to-day activities whilst trying to implement what has been formulated (Hill & Hupe, 2003). The more the number of layers and levels, the greater the complexity of implementation, since the policy can be altered in the different spaces and interactions (Lotta, 2019).

Tiers and levels have a direct relationship with conflict and ambiguity: the manipulation of the ambiguity of the policies is the attempt to make the various tiers understand what needs to be done. The reduction of ambiguity has the effect of reducing the discretion of the players, which increases the possibility of the policies ultimately aligning themselves with the original proposal (Lotta, 2019; Matland, 1995). This increase in ambiguity provides space for reinterpretation of the policies, adjusting them to different contexts. But it also consequently creates a variation in the results. Conflict can also be found between both the tiers and the hierarchical levels. In the case of the former, it forms part of the democratic, political and federative dynamic. Between the hierarchical levels, meanwhile, the conflict denotes a management problem, in which the players in the lower levels do not agree with the decisions taken by those operating in the higher levels. The conflicts may be hidden leading to sabotage strategies (Brehm & Gates, 1997) - or explicit, creating insurrection strategies (Mahoney & Thellen, 2010).

To put a policy into practice, it is necessary to map out the degree of conflict and ambiguity that exists as well as the multiple layers and levels of hierarchy that the policy will pass through. Based upon this mapping, it is possible to think about strategies for increasing or decreasing the ambiguity, and for facing up to the conflict (or not), that allow the policy to be executed in the desired manner (Hill & Varone, 2016; Matland, 1995).

To reduce the ambiguity, the answer is to increase the standardization of the planning of public policies, leading to systems of incentive conditioned by conformity, or in other words, by abiding by what was drafted at the central level (May, 1995). When the ambiguity concerns the hierarchical levels (multi-level), the strategy can be established with the regulation of the discretionary actions taken by the players and the creation of systems of control and incentive for operation in the hierarchical approach. The reduction of ambiguity in the federal levels depends on the encouragement of federal cooperation in relation to the drafting of the policies. In this case, Goggin, Bowman, Lester and O´Toole (1990) provide an answer: to guarantee the implementation of public policies in complex federal contexts, the federal messages should be accompanied by credible and viable solutions, emphasizing the problems and the solutions that should be created.

If the issue moves on to the mitigation of conflicts, the strategy for the various levels should be to increase enforcement and compliance, reducing the hierarchical players’ capacity to question (May, 1995). Minimizing ambiguity or increasing spaces for negotiation are two strategies which can combat conflicts. When these concern federal bodies, the strategy should involve negotiation or enforcement. This is what May (1995) calls adhesion via coercive mandates: the local governments are considered to be regulated agents with the responsibility of following regulators established by the more central levels. The national levels decide on standards and procedures so that results can be achieved whilst applying sanctions on those who do not reach them, thereby reducing ambiguity and forcing the policy through when they have federal jurisdiction or when they have sufficient resources to create “voluntary” adhesion with prompted implementation.

The strategies for creating adhesion (reducing conflict and/or ambiguity) pass through the coordination and control instruments, including the creation of regulations, monitoring systems, training strategies, and incentives, etc. These instruments tend to be coercive and mandatory, such as those highlighted by May (1993), or involve cooperation between the players.

The contexts of policy production are extremely complex if only governmental players are considered. However, observing the reality of the current production of public policies in democracies, we can see that governments that produce them alone are rare (Gomide & Pires, 2014). In the institutional environment of public policy production in Brazil, there is a close interaction between state and non-state players. This is the case, for example, of the public policy councils, conferences, ombudsmen and public hearings, amongst other areas that institutionalize public participation in different areas, including education ( Gomide & Pires, 2014; Lotta & Vaz, 2014; Segatto & Abrucio, 2016). In addition to the participating institutions, there exist various different forms of co-production that are under way, such as the contracting of organizations for the provision of services, or partners for the drafting and management of public policies, players that are not State-controlled but which influence the decision-making processes.

In the case of Brazil, another important particularity concerns the size, heterogeneity and inequality of the country. As the literature points out, these characteristics lead to an environment of public policy production in which a balance between the degree of ambiguity and the degree of coordination is necessary. On the one hand, the high level of decentralization (the result of ambiguity and little coordination) can lead to a rise in inequalities (Abrucio, Franzese & Sano, 2010; Arretche, 2012). On the other, the ambiguity allows for the production of innovative experiences that are important for ensuring that the policies are effective at a local level. The last few decades have seen a search for this balance based upon a combination between policies designed to provide incentive and federal inducement (characteristics of an administrative context), but with a certain amount of space for adaptation and creation at local level (characteristics of the experimental context) (Abrucio et al., 2010; Arretche, 2012; Segatto, 2011).

2.3 Implementation of Reforms

Another important issue when thinking about policy implementation is that of institutional and organizational changes. The literature highlights two possible types of change: radical change, when an abrupt shift in direction occurs, and gradual change, when incremental changes take place.

The works of authors such as Baumgartner et al. (1999), and Lindblom (1965) show that the State’s day-to-day operations tend towards ‘incrementalism’, with small, gradual changes over time, due to the difficulty of addressing transformations as much in political terms as in the high level of institutionalization. This stems from the characteristics of the public sector: extremely complex difficulties; players with only partial access to information on problems and solutions; decision-making processes that require a great deal of negotiation (Lindblom, 1965); and decisions from the past that weigh upon the present - called ‘path dependence’ (Pierson, 2000).

As such, the policies are generally characterized by long periods of stability, possibly being interrupted by periods of instability, that lead to significant changes (Baumgartner et al., 1999). This context is named by the authors as ‘punctuated equilibrium’: a tendency towards equilibrium with the punctuation of radical changes that create a new equilibrium. Radical changes are normally rarer and are related to exogenous factors that undermine the institutional status quo - for example, natural disasters, economic crises or wars. However, the change can also come about from alterations in the coalition forces that manage to impose new standards, images and problems on the political agenda, which causes a displacement of the state action, altering the previous equilibrium to create a new balancing point (Baumgartner et al., 1999).

Another segment of the literature seeks to understand the processes of gradual change in the policies (Lindblom, 1965). In one classic text on the issue, Mahoney and Thellen (2010) argue that there exist different types of change in the institutions that vary depending upon the level of intensity and institutional transformation. Similar to the idea set out by Matland (1995), they use the pairing of conflict and ambiguity, but also consider another two variables: the number of veto possibilities (conflict) and the level of discretion of the players (ambiguity). As such, there are institutional contexts with many possibilities for veto and others with fewer possibilities for veto, as well as contexts that allow a high level of discretion for the players, while in others there is a much lower level. The relationship between these elements leads to scenarios that allow for different types of change.

In the specific case of education, the literature has focused on understanding the factors that determine the success and failure of the implementation of educational reforms. By analyzing what has been learned from the literature about the implementation of reforms in the field of education, Bruns and Schneider (2016, p. 4) argue that “[...] educational reform is politically risky”. Taking the political risk as a starting point, the authors argue that reforms in the field of education differ from others in four features: the ubiquity and power of the teachers’ unions; the opacity of the classrooms; the difficulty in monitoring performance; and the fact that the reforms produce results in the long-term. As such, the success of the reforms depends upon elements such as: the sequence of the changes; the decision concerning whether it is to be a legislative or administrative change; the communication strategies adopted; and negotiation with those players holding powers of veto, especially the unions.

The literature on the implementation of educational reforms also highlights a number of factors upon which success is conditional: commitment from the school community, the performance of pilot and/or small-scale experiments, and what has been learned from the failures of previous reforms (Cox, 2006; Holm-Nielsen, Thorn, Brunner & Balán, 2005; Krawczyk & Vieira, 2005; Organization for Economic Cooperation and Development [OECD], 2010). Part of the literature also explains that the success of the implementation of a reform measure in education also depends upon the prioritization the government gives to both education and the reform itself (Hallinger, 2010; Sorensen, 1994; Tan & Gopinathan, 2000; Tan, Koh & Choy, 2016). Finally, the literature shows how the success of education reforms also depends upon the time dedicated to its implementation, meaning that the players involved are convinced of its content and understand how to implement it (Hallinger, 2010; Howie, 2002; Moodley, 2013).

The possibility of creating changes for the implementation of reforms is tied to the capacity to have proposals that are both general and strategic and, at the same time, contextualized and adaptable to the local needs (Matus, 1996). This factor is even more relevant in federal and heterogeneous contexts, as is the case with Brazil. In this case, the federal government can assume the role of promoter and instigator of the proposals for change, but given the particularities of each context and the local autonomy, it is expected that the reform ideas are transformed locally (Segatto, 2011).

Considering, therefore, the heterogeneity expected in the implementation of reforms of this nature and the characteristics inherent to the reforms highlighted by the literature, the conflict and ambiguity pairing becomes an important tool for understanding the capacity of implementation of reforms in education. Based upon the academic references presented here, an analysis will now be provided of the case of recent reforms to Secondary Education in Brazil, observing the different impacts of this process on the contexts of implementation.

3. CONTEXT AND METHODS

3.1 Context

To perform the analyses proposed in this paper, we have chosen the case of the implementation of Secondary Education Reform proposed by the Temer Government in 2016, based on Provisional Measure (PM) 748 (2016), later transformed into Law 13.415 (2017). The implementation of the reform was analyzed in all the states and the Federal District between 2017 and 2018, subsequent to the approval of this federal legal framework.

Education policies in Brazil pass through various different tiers and levels - these resulting from the division of responsibilities guaranteed by the Federal Constitution (1988) that establishes a system of collaboration between the Federal Union, the states and the municipalities in the organization of their teaching systems. It is the Federal Government’s responsibility to coordinate the national policy and articulate the different federal levels (art. 8). The state governments should ensure Primary Education and prioritize the offer of Secondary Education (art. 10). However, the general norms of the policies (including the Law of Basic Directives [LDB], and the Common National Curriculum Base [BNCC]) are defined at national level. The division of responsibilities in education is not set in stone, since there is an overlapping of offers in different state systems and an inequality in terms of distribution of resources (Segatto & Abrucio, 2016).

In addition to being a multi-layering policy, education is also notable for the various levels that are produced in the three federal bodies. Specifically, this means dozens of state education departments, thousands of municipal offices, regional units and schools, and millions of employees.

Or in other words, what exists is a multi-level context that is complex, heterogeneous and historically unequal (Segatto & Abrucio, 2016), in which a high level of discretionary decision-making is accumulated. As such, thinking of processes of change within this reality means developing systems of multi-level adhesion with a great number of players who, in general, do not operate in relation to any sort of hierarchy. As Bruns and Schneider (2016, p. 44) explain, “[...] To be effective, reforms have to reach into even the most distant classrooms, which means they may have to go through multiple levels of administrative hierarchy, including provincial, municipal, and school-level directors”.

Furthermore, the case of Secondary Education reflects the Brazilian institutional environment. As a result of the federalism and the decentralization of social policies, education in Brazil operates in the midst of a complex maze of players located within the three federal levels, in addition to the social players, participating institutions and other powers that interfere in the decision-making process (Segatto & Abrucio, 2016). With this structure in place, it can be argued that, in principle, the implementation of reforms in Brazilian education entail a high degree of negotiation and coalition building, tackling obstacles for enactment and later alterations, that have to be submitted to the same channels of veto and partners (Schneider & Mizala, 2014). Within this context it is important to analyze the implementation of a reform and its effects.

3.2 Collection Method and Analysis

The analyses presented in this article are based upon two research strategies: a study of the literature on the implementation of policies, and a study of the primary data related to the process of implementation of changes in Secondary Education in the various Brazilian states¹. The data was collected in two stages: interviews and field studies in eleven states in 2017; and a survey, interviews and documentary analysis conducted in the 26 states and the Federal District, in 2018.

Eleven states were selected for the interviews and field studies conducted in 2017: Acre, Pará, Piauí, Rio Grande do Norte, Pernambuco, Bahia, Espírito Santo, Minas Gerais, Ceará and Santa Catarina, as well as the Federal District of Brasília. The choice of these states² sought to include states that had already been making moves towards a change in Secondary Education before the 2016 reform. In these states, the collection of data was performed by means of interviews with the experts responsible for Secondary Education at each State Education Department. In the case of the states of Ceará and Santa Catarina, field visits were conducted to perform interviews with other players with important roles in the process of implementation, including representatives from the state boards of education, teachers, school principals and union representatives. These two states were chosen due to their importance in terms of the innovation of educational policies they had made over recent years, which allowed for an understanding of how the reforms were negotiated and implemented in those states where this was done more substantially.

This first stage of data collection allowed us to map out the processes of change adopted through until 2017 in some Brazilian states.

In 2018, other data collection strategies were adopted, with coverage extending to all 26 states and the Federal District: 1) application of a survey, in partnership with the National Board of State Education Departments (Consed); 2) interviews with the experts responsible for Secondary Education in the different states. The aim of survey was to map out and understand the development of the initiatives of change and flexibilization of the curricula in the different states. Data was collected in relation to the changes under way in each state - whether legal changes had already been made, whether they had incorporated the principal measures proposed by Law 13,415(2017), the number of schools impacted, the players involved in the discussions, etc.

The aims of the interviews, held between July and October, 2018, were: to validate and complete the data collected by the survey; to understand the qualitative changes; and to test hypotheses. 40 Interviews were performed with experts responsible for the coordination of Secondary Education at the State Education Departments in the 27 states. As in some states that management of Secondary Education is divided between the Department of Education and the Department of Development (or similar), or between different departments within the Department of Education, in some states more than one interview was performed. They were all transcribed and analyzed in order to create narratives with which to contextualize and improve the understanding of the development of the changes in each state.

The research then moved on to the analysis of the process of change in light of what has been published about policy implementation. Based upon the analytical model developed by Matland (1995), the variables of ambiguity and conflict were observed in each moment in time (before and after - 2016) to understand the contexts of implementation. Once this had been done, analysis was performed to provide answers to the research questions in this article. The aim was not to understand the individual cases, but rather the general movement of the reform, observing the country as a whole. As such, and in agreement with the interviewees, it was decided not to cite specific cases or the names of the experts who took part in the research.

4. ANALYSIS OF THE CONTEXT OF EDUCATION POLICY IN BRAZIL: PRE AND POST-REFORM

4.1 The pre-reform context

By performing an empirical observation of the actions developed by the states in relation to the changes in Secondary Education prior to 2016, two parallel contextual characteristics were identified: 1. the context of experimental implementation, and 2. the context of administrative implementation.

The first was a reflection of the federal characteristics of education, which meant little conflict between the players and a high level of ambiguity concerning the regulations, guaranteeing that each state could manage its own solutions. However, the consequence of the experimental context in a country with high levels of diversity and inequality is the production of heterogeneity in the provision of education policies (Arretche, 2012). Where there exist installed capacities, the states and municipalities manage to produce innovative and inclusive solutions that are suited to the specific location. Where capacities are lacking, the states and municipalities have problems executing the policies and they become ineffective. Therefore, this heterogeneity, on the one hand, produces innovative but also irregular experiences, with results that have both great and little effect, whilst leading to inequality in terms of the quality of education and access to it (Abrucio et al., 2010).

Over the last few decades, in order to address the problems of inequality, the Federal Government has been strengthening its policies of incentive and inducement that seek to encourage standards for the production and results of policies, with the transfer of specific resources to those defined nationally (Abrucio et al., 2010; Arretche, 2012). This is the typical case of intergovernmental cooperation described by May (1993) and one which leads to the construction of a context of administrative implementation (Matland, 1995), with an attempt to achieve adhesion, encouraging standardized actions for the development of local potential.

In both cases, be it in the experimental context, or be it in administrative implementation, the central characteristic of the scenarios of implementation of change in Secondary Education in the states was a low level of conflict, since there existed a degree of stability in the interactions between the different players, which showed a high level of autonomy when making decisions. As such, a gradual process of transformation was created, as was noted by the experts who were interviewed: processes of change in the provision of Secondary Education characterized by experimentation, dialog with the different players, and an effort made to produce new results, which generally, preceded the institutional change.

Some of the state governments were experiencing these processes in the form of tiers, and conversion and reinterpretation at the time of implementation, with deviations due to changes in the environment, without having to deal with a radical change in the laws (Mahoney & Thellen, 2010). As an example, we can cite the implementation of curricular innovations in full-time schools, involving discussions with the players directly involved and with space for adjustments and route corrections. This experience was made possible by the relatively stable environment, by the states’ control over the ways things were done, and by the prior consolidation of an experimental and heterogeneous environment. For example: Espírito Santo, with a reorganization of the curriculum and experiences of the working environment since 2007; Rio Grande do Sul (since 2012) and Rio de Janeiro (since 2011), with a new curriculum; Ceará, with experiences of Full-Time High Schools (EMTIs) and technical and vocational education, as well as a reorganization of the curriculum, since 2008; and Pernambuco, with EMTIs since 2008, and experiences with flexibilization of the curriculum since 2012.

4.2 Reform of Secondary Education: a change of the context of implementation

The analysis of the empirical data allows us to note that, if the institutional environment through until 2016 was notable for its relative stability and low level of conflict between the players, the same cannot be said for the following year. In 2016, President Dilma Roussef was impeached and a president with little social legitimacy moved into office. This new president proposed a provisional measure, which was later approved by Law 13.415 (2017), confirming numerous changes that had been proposed previously, but also introducing new ones which had not been debated to any great extent. There was, however, a loss of the previous balance, creating a critical situation in which one actor changed the status quo and the environment of stability (Baumgartner et al., 1999).

The law in question proposed alterations and regulations for Secondary Education by means of new wordings of the articles of the Law of Basic Directives (LDB). Despite entailing numerous changes from the regulatory perspective, the alterations were not automatically applicable to the local contexts: they needed to be regulated and then implemented by the states. By way of example, some of the most important modifications were: an increase in the working hours; changes to the curricular structure; alteration to the regulations for training teaching staff, changes in the offering of vocational and technical education; and incentives for the offering of full-time education, amongst others.

A large part of the reform was not regulated at the time, leading to a lack of clarity over how the alterations should take place. A new environment of conflicts was therefore created, with ambiguity owing to regulations that still hadn’t been interpreted and without any guidance on their execution, such as, for example, the uncertainty surrounding the approval of the BNCC for Secondary Education and the lack of guidance for implementing training itineraries.

On the one hand, the reform led the context of the policy into enormous conflict, in so far as it was approved and established by a government that was not accepted by part of the population, without taking into account previous gradually developed discussions. On the other hand, since it had not been fully regulated, and since it led to changes that had not been fully agreed upon, the reform created a highly ambiguous context. We therefore have the displacement of a context that varied between experimental implementation and administrative implementation to one of great ambiguity and conflict. This context is referred to as ‘symbolic implementation’, one in which results are not expected, since the conflict hinders the mobilization of the players and the high level of ambiguity paralyzes the ability to act (Matland, 1995). In this case, ambiguities and conflicts were created in relation to both the means and the ends.

In parallel to this, the Federal Government continued operating on the logic of incentives and inductions, or in other words, with administrative implementation, but with measures relating to the reform of Secondary Education - such as the Policy of Promotion of the Implementation of Full-Time High Schools (EMTIs), that was pushed during that period. This policy established the provision of resources by the Ministry of Education (MEC) for the states over a period of ten years per school, whilst conditions were created for the provision of the resources by the MEC.

In other words, at the same time as the general policy was shifted into the zone of symbolic implementation (conflict and ambiguity), specific routes remain part of the logic of administrative implementation via systems of incentive and induction (Matland, 1995; Arretche, 2012).

Considering this new context, the questions that remain are as follows: what is the effect of this displacement for the states? To what extent does the exogenous change, caused by the actions at federal level, produce different results in the different Brazilian states? The analyses that follow take a comparative look at the federal states, in an attempt to understand the different consequences caused by the reform.

5. EXOGENOUS CHANGES IN LOCAL CONTEXTS: EFFECTS OF THE REFORM IN THE STATES

By making a comparative analysis of Secondary Education in the states, a situation- that was notable for its high level of heterogeneity was recognized, both in relation to the degree of the changes being implemented, and in relation to the direction they had taken and the period of their implementation.

By means of an analysis of the empirical data relating to the changes in Secondary Education prior to MP 748 (2016) and Law 13,415 (2017), it was possible to divide the states into two groups, according to the levels of incidence, coverage and institutionalization of the changes implemented. The first group includes those which have had early, basic experience of changes in Secondary Education. The second includes those states that have had experience that could be considered more significant and/or impacting, or in other words, it concerns alterations that were more radical, more institutionalized, or provided greater coverage across the system.

The second group was further divided into two subgroups: states with previous institutionalized experiences that were more specific or connected to certain themes prompted by the federal government; and states that had a more consolidated movement of changes, with inter-related initiatives that were broader and more institutionalized rather than being tied solely to federal initiatives.

By comparing these groups, and taking into account the context prior to 2016 and that of 2018, different degrees of progress were noted that had been created as a consequence of the previous contexts, as summarized in Box 2:

Box 2
Impacts of the reform of secondary education on the States

In the first group, we find states that decided to take advantage of the reform to push forward broader changes, be they more quantitative (for example, establishing more full-time schools than the number the federal government was providing incentives for), or be they more qualitative (investing in new forms of teaching and curricula, or developing the relationship with players across the system). Based upon this group, it is possible to conclude that the federal government’s induction process is capable of enabling important changes in the states’ agendas, but the changes are boosted when there are independent local initiatives to invest in transformations that go beyond this induction.

In the second group, with the arrival of the reform, prior incipient initiatives were impacted by changes that were already expected due to the incentives, without any innovation in the content or reach. In all of them, the biggest impact caused was the creation of EMTIs, which, in general, brought with it a set of initiatives that bolstered flexibilization, comprehensive education, and leadership amongst students, but did not necessarily result in more structural transformations. This group allowed us to realize that, when the federal government proposes alterations that are tied to induction models (administrative implementation); it is capable of including the change in the states’ agendas, since they are interested in embracing it. Differently to the first group, however, it is noticeable that there is less locally developed content present, meaning there are changes that stick more to what the federal government has proposed, with a more linear and more expected trajectory. This scenario indicates that an exogenous change can reposition the internal actions more incrementally when the states adhere to it.

The third and fourth groups differ in the degree of institutionalization and integration between the changes. The third is made up of states which have important prior experience, but which is, in general, connected to other induction actions operated by the Federal Government during previous periods. These transformations, although robust, did not form a single movement, but rather a full set of initiatives with little institutionalization. In these states, the arrival of the reform meant leverage for the changes. In almost all of them, since 2012 (and in some cases still during the 2000s), there were shifts focused on curricular reviews, the creation of vocational technical schools formed into a single class group, complementary work experience and integration with the work market, and the creation of elective subjects, all of which were made possible by Federal Government actions implemented during the previous decade (Programa Ensino Médio Inovador [ProEMI]). In these cases, it is interesting to note how this broader and more consistent process can take advantage of an exogenous change to boost the transformation. This takes place largely because the innovations created were in line with the reform model, since they had followed the movement suggested by the Federal Government via strategies of incentive and induction, making the adoption of the new changes less costly.

Finally, the fourth group also underwent a broad and intense process of change in Secondary Education. However, this group experienced less effects, or effects of paralysis or a reduction in the pace of the changes, upon arrival of the reform. It is interesting to note that, in all of the states in this group, there was a prior tradition of innovation and good results in education, with an important legacy and institutional capacity. Also in these states, the agenda of changes in Secondary Education had already been in place for a long time, fostered not only by federal directives (as in the previous group), but also by the specific trajectory of the education they were pursuing. This concerns States that, for example, had implemented innovations in the creation of EMTIs for more than a decade, approved models for curricula reorganization and the offering of vocational education in a single class group, amongst others. In all of them there was a process of transformation under way, with a clearly defined direction, established objectives, a unique model, and a high level of institutionalization in the changes. For these states, the approval of the Provisional Measure 748 (2016) and Law 13,415 (2017) became a potential obstacle. On the one hand, because it created an environment of indecision, leading to paralysis (for example, the lack of definition for Secondary Education in the BNCC). On the other, because they recognized the high costs involved in the adaptation to the new proposals of those measures already implemented. In short, in these states, the reform had not, through until 2018, provided leverage. On the contrary, it was transformed into a motive for taking a “calm and careful” approach, or in other words, to “wait and see in relation to what will happen, and then adapt the changes that have already been established”.

Another important point in relation to the course taken by the changes concerns the process of standardization, which, in turn, influences the degree of institutionalization. The states from group 4 have invested in large-scale and broad-reaching processes over the past few years. The percentages for the other three groups show variations in the standardization: some established regulations for specific changes, whilst others did nothing since they felt that this was more a moment for experimentation and discussion. Once again, this helps to explain the difficulties involved in the implementation of the changes in group 4.

6. FINAL COMMENTS

The aim of this paper has been to study the effects that an exogenous change can lead to in the context of the implementation of public policies in complex and federal institutional environments. In empirical terms, the start of the process of implementation of the reform of Secondary Education, between 2017 and 2018, was analyzed to observe the extent to which the decisions taken by the Federal Government have altered the contexts of implementation in the states and, at the same time, the extent to which the local contexts have also altered the contexts of the reform. To provide an analytical base, the ‘conflict’ and ‘ambiguity’ variables were employed.

The analysis of the cases provided a view of how the transformation of context took the players into a state of symbolic implementation, and led to paralysis and little capacity for action. At the same time, the incentive and induction initiatives put into operation by the Federal Government, caused situations of administrative implementation. These two general contexts materialized in different ways in the states. This was due to the fact that the exogenous changes found specific contexts in each location, thus leading to different results.

This case also reflects the complexity of the contexts of multi-level and multi-layering implementation, involving various players, with different interests and coordination needs. It also demonstrates how, in these cases, the instruments of coordination become key parts in the effecting of decisions. As the literature that has been published on educational reforms highlights, if there is a lesson to be learned from all the cases, it is that the implementation stage of the reforms is crucial and extremely vulnerable, especially when the players with an interest in the decision-making process are not involved, as has occurred in Brazil (Bruns & Schneider, 2016). The literature also shows how a failure to establish means for debate and the involvement of key-players leads the conflicts to other contexts, making part of the implementation unviable. As the authors point out, “[...] losers in the reform adoption phase retain substantial scope to undermine reforms during implementation”. Despite no analysis of the position of the various players in relation to the reform having been performed, it is possible to understand, solely from the perspective of the players from the state governments, that reforms that have not been negotiated, and in which the conflict and ambiguity have not been resolved, tend to have different effects in each context, depending upon the reaction of the local players.

It is also important to consider that this paper has analyzed a specific period in the progress of the reform (between 2017 and 2018) and that both state and federal governments sought to somehow invest in the implementation strategies, with the aim of reducing conflicts and ambiguities³. However, in relation to the period studied, the results of these actions cannot be observed. As such, one limit of this paper is that it is able to study just a restricted period of time that should be expanded upon in future studies. Other studies may also be able to provide more detailed analyses of these changes of context, elaborating on specific cases, to understand how and why differences in implementation take place in the different states and how the different players behave in these different scenarios.

The findings presented here contribute to both the literature on the implementation of public policies and to that on education, as well as to the literature that addresses reform and change in the public sector. Empirically, it is also intended that contributions should be provided to the understanding of the process of change in Brazilian Secondary Education following the publication of Law 13,415 (2017).

ACKNOWLEDGMENTS

The research upon which this article is based was commissioned by the Instituto Unibanco, which we thank for their technical and financial support. This article would not have been possible without the support and partnership of the National Board of Education Departments (Consed), and especially the experts from the 27 States, who shared the advances and obstacles experienced in the implementation of the changes in Secondary Education policy. We would also like to thank all the researchers from the Núcleo de Estudos da Burocracia (NEB) who helped conducting interviews.

REFERÊNCIAS

• Abrucio, F., Franzese, C., & Sano, H. (2010). Coordenação e cooperação no federalismo brasileiro: avanços e desafios. In A. S. Cunha, L. A. Medeiros & L. Aquino(Org.), Estado, instituições e democracia (v. 1., pp. 177-212). Brasília, DF: IPEA.
• Arretche, M. (2001). Uma contribuição para fazermos avaliações menos ingênuas. In M. C. R. N. Barreira, & M. C. B. Carvalho (Org.), Tendências e perspectivas na avaliação de políticas e programas sociais São Paulo, SP: IEE/PUC, Cenpec, Cedac.
• Arretche, M. (2012). Democracia, federalismo e centralização no Brasil Rio de Janeiro, RJ: Fundação Getulio Vargas/Fiocruz.
• Baumgartner, F., Jones, B. D. & Wilkerson, J. (2011). Comparative Studies of Policy Dynamics. Comparative Political Studies, 44(8), 947-972.
• Baumgartner, F., True, J. L., & Jones, B. D. (1999). Punctuated equilibrium theory: explaining stability and change in American policymaking. In P. Sabatier (Ed.), Theories of the policy process Oxford, UK: Westview Press.
• Brehm, J., & Gates, S. (1997). Working, shirking, and sabotage: Bureaucratic response to a democratic public Ann Arbor, MI: University of Michigan Press.
• Bruns, B., & Schneider, B. (2016). Managing the politics of quality reforms in education: policy lessons from global experience. Background Paper: The Learning Generation, (Working Paper prepared for International Commission on Financing Global Education Opportunity). Nova York, NY. Recuperado de http://report.educationcommission.org/wp-content/uploads/2017/01/Managing-the-Politics-of-Quality-Reforms.pdf
  » http://report.educationcommission.org/wp-content/uploads/2017/01/Managing-the-Politics-of-Quality-Reforms.pdf
• Capella, A., Brasil, F., & Sudano, A. (2015). O estudo da agenda governamental: reflexões metodológicas e indicativos para pesquisas(pp. 1-43). In Anais do 39º Encontro Anual da ANPOCS, Caxambu, MG.
• Constituição Constituição da República Federativa do Brasil: Promulgada em 5 de outubro de 1988 (1988). Brasília, DF. Recuperado de http://www.planalto.gov.br/ccivil_03/constituicao/constituicao.html
  » http://www.planalto.gov.br/ccivil_03/constituicao/constituicao.html
• Cox, C. (2006). Policy formation and implementation in secondary education reform: The case of Chile at the turn of the century Washington, DC: World Bank.
• Faria, C. (2012). Implementação de políticas públicas: teoria e prática Belo Horizonte, MG: PUC Minas.
• Goggin, M, Bowman, A., Lester, P., & O’Toole, L., Jr. (1990). Implementation theory and practice: Toward a third generation Glenview, IL: Scott Foresman/Little, Brown.
• Gomide, A., & Pires, R. (Eds.). (2014). Capacidades estatais e democracia. Arranjos institucionais de políticas públicas Brasília, DF: IPEA.
• Hallinger, P. (2010). Making education reform happen: is there an “Asian” way? School Leadership & Management, 30(5), 401-418.
• Heclo, H. (1978). Issue networks and the executive establishment. In A. King (Ed.), The New American Political System Washington, DC: American Enterprise Institute.
• Hill, M., & Hupe P. (2002). Implementing Public Policy: Governance in theory and practice Londres, UK: Sage Publications.
• Hill, M., & Hupe, P. (2003). The multi-layer problem in implementation research. Public Management Review, 5(4), 471-490.
• Hill, M., & Varone, F. (2016). The public policy process Londres, UK: Routledge.
• Holm-Nielsen, L. B., Thorn, K., Brunner, J. J., & Balán, J. (2005). Regional and international challenges to higher education in Latin America. In H. Wit, I. C. Jaramillo, J. Gacel-Ávila, & J. Knight (Ed.), Higher education in Latin America (pp. 39-69). Washington, DC: World Bank.
• Howie, S. J. (2002). Renewal of secondary education curricula and assessment in South Africa. In World Bank (Ed.),Secondary education in Africa: Strategies for renewal(pp. 41-53). Washington, DC: World Bank .
• Krawczyk, N. R., & Vieira, V. L. (2005). Um estudo sobre o estado da arte da reforma educacional na década de 90 na América Latina: Brasil, Argentina, Chile e México (Relatório de pesquisa). São Paulo, SP: FAPESP.
• Lei Lei n. 13.415, de 16 de fevereiro de 2017 (2017). Altera as Leis n. 9.394, de 20 de dezembro de 1996, que estabelece as diretrizes e bases da educação nacional, e 11.494, de 20 de junho 2007, que regulamenta o Fundo de Manutenção e Desenvolvimento da Educação Básica e de Valorização dos Profissionais da Educação, a Consolidação das Leis do Trabalho - CLT, aprovada pelo Decreto-Lei no 5.452, de 1 de maio de 1943, e o Decreto-Lei no 236, de 28 de fevereiro de 1967; revoga a Lei no 11.161, de 5 de agosto de 2005; e institui a Política de Fomento à Implementação de Escolas de Ensino Médio em Tempo Integral. Brasília, DF. Recuperado de http://www.planalto.gov.br/ccivil_03/_ato2015-2018/2017/lei/l13415.htm
  » http://www.planalto.gov.br/ccivil_03/_ato2015-2018/2017/lei/l13415.htm
• Lindblom, C. (1965). The Science of Muddling Through. Public Administration Review, 19(2), 79-88.
• Lotta, G. (2015). Burocracia e implementação de políticas de saúde Rio de Janeiro, RJ: Fiocruz.
• Lotta, G. (Org.). (2019). Teoria e análises sobre implantação de políticas públicas no Brasil Brasília, DF: ENAP.
• Lotta, G., Nunes, A., Cavalcante, S., Ferreira, D., & Bonat, J. (2018). Por uma agenda brasileira de estudos sobre implementação de políticas públicas. Revista do Serviço Público, 69(4), 779-816.
• Lotta, G., & Vaz, J. (2015). Arranjos institucionais de políticas públicas: aprendizados a partir de casos de arranjos institucionais complexos no Brasil.Revista do Serviço Público, 66(2), 171-194.
• Mahoney, J., & Thellen, K. (2010). Explaining institutional change: Ambiguity, agency and power Cambridge, MA: Cambridge University Press.
• Matland, R. (1995). Synthesizing the implementation literature: The ambiguity-conflict model of policy implementation. Journal of Public Administration Research and Theory, 5(2), 145-174.
• Matus, C. (1996). Estratégias políticas: Chimpanzé, Maquiavel e Gandhi São Paulo, SP: Fundap.
• May, P. (1993). Mandate design and implementation: Enhancing implementation efforts and shaping regulatory styles. Journal of Policy Analysis and Management, 12(4), 634-663.
• May, P. (1995). Can cooperation be mandated? Implementing intergovernmental environmental management in New South Wales and New Zealand. Publius: the Journal of Federalism, 25(1), 89-114.
• Ministério da Educação & Conselho Nacional dos Secretários de Educação. (2018). Guia de Implementação do Novo Ensino Médio Brasília, DF: Autor.
• Moodley, G. (2013). Implementation of the curriculum and assessment policy statements: challenges and implications for teaching and learning (Dissertação de Mestrado). University of South Africa, Pretoria, South Africa.
• Organização para a Cooperação e Desenvolvimento Econômico. (2010). Ontario, Canada: Reform to support high achievement in a diverse context Paris, França: OECD Publishing.
• Osborne, D., & Gaebler, T. (1998). Reinventando o governo: como o espírito empreendedor está transformando o setor público Brasília, DF: MH Comunicação.
• Pierson, P. (2000). Increasing returns, path dependence and the study of politics. American Political Science Review, 94(2), 251-267.
• Pollitt, C. (Ed.). (2013). Context in public policy and management: The missing link? Cheltenham, UK: Edward Elgar Publishing.
• Pollitt, C., & Bouckaert, G. (2011).Public management reform: A comparative analysis - New public management, governance, and the Neo-Weberian State Oxford, UK: Oxford University Press.
• Portaria Portaria nº 649, de 10 de julho de 2018 (2018). Institui o Programa de Apoio ao Novo Ensino Médio e estabelece diretrizes, parâmetros e critérios para participação. Brasília, DF. Recuperado dehttps://www.in.gov.br/materia/-/asset_publisher/Kujrw0TZC2Mb/content/id/29495231/do1-2018-07-11-portaria-n-649-de-10-de-julho-de-2018-29495216
  » https://www.in.gov.br/materia/-/asset_publisher/Kujrw0TZC2Mb/content/id/29495231/do1-2018-07-11-portaria-n-649-de-10-de-julho-de-2018-29495216
• Portaria Portaria nº 1.024, de 4 de outubro de 2018 (2018). Define as diretrizes do apoio financeiro por meio do Programa Dinheiro Direto na Escola às unidades escolares pertencentes às Secretarias participantes do Programa de Apoio ao Novo Ensino Médio, instituído pela Portaria MEC nº 649, de 10 de julho de 2018, e às unidades escolares participantes da avaliação de impacto do Programa de Fomento às Escolas de Ensino Médio em Tempo Integral - EMTI, instituída pela Portaria MEC nº 1.023, de 4 de outubro de 2018. Brasília, DF. Recuperado de https://www.in.gov.br/materia/-/asset_publisher/Kujrw0TZC2Mb/content/id/44100315/do1-2018-10-05-portaria-n-1-024-de-4-de-outubro-de-2018-44100107
  » https://www.in.gov.br/materia/-/asset_publisher/Kujrw0TZC2Mb/content/id/44100315/do1-2018-10-05-portaria-n-1-024-de-4-de-outubro-de-2018-44100107
• Pressman, J., & Wildavsky, A. (1973). Implementation: How great expectations in Washington are dashed in Oakland Berkeley, CA: University of California Press.
• Rodrigues, D., & Jayo, M. (2016). Modernização administrativa em contexto subnacional. Estudo de caso do Detran-SP. Cadernos Gestão Pública e Cidadania, 21(68), 61-83.
• Rodrigues, D., & Lotta, G. (2017). Análise do processo de implementação de reformas em organizações públicas: os casos do Poupatempo e do Detran-SP. Cadernos Gestão Pública e Cidadania, 22(72), 214-237.
• Rothstein, B. (1998). Just institutions matter Cambridge, MA: Cambridge University Press .
• Sabatier, P., & Jenkins-Smith, H. (1993). Policy change and learning: the advocacy coalition approach Boulder, CO: Westview Press.
• Schneider, B., & Mizala, A. (2014). Negotiating education reform: Teacher evaluations and incentives in Chile (1990-2010) Recuperado de https://doi.org/10.1111/gove.12020
  » https://doi.org/10.1111/gove.12020
• Segatto, C. (2011). Como ideias se transformam em reformas: um estudo comparativo das mudanças educacionais orientadas pelo desempenho nos estados brasileiros (Dissertação de Mestrado). Escola de Administração de Empresas de São Paulo/FGV, São Paulo, SP.
• Segatto, C., & Abrucio, F. (2016). A cooperação em uma federação heterogênea: o regime de colaboração na educação em seis estados brasileiros. Revista Brasileira de Educação, 21(65), 411-429. Recuperado de https://doi.org/10.1590/s1413-24782016216522
  » https://doi.org/10.1590/s1413-24782016216522
• Sorensen, C. W. (1994). Success and education in South Korea. Comparative Education Review, 38(1), 10-35.
• Tan, C., Koh, K., & Choy, W. (2016). The education system in Singapore. In S. Juszczyk (Ed.), Asian Education Systems Toruñ, Polônia: Adam Marszalek Publishing House.
• Tan, J., & Gopinathan, S. (2000). Education reform in Singapore: Towards greater creativity and innovation. NIRA Review, 7(3), 5-10.

• 1
  The study was developed with the technical and financial support of the Instituto Unibanco.
• 2
  The states were chosen with the technical support of the Instituto Unibanco and the National Board of State Education Departments (Consed).
• 3
  The New Secondary Education Support Program, established by means of Ordinance numbers 649 (July, 2018) and 1,024 (October 4, 2018), and the New Secondary Education Implementation Guide, are all measures designed to provide guidance for the implementation through the reduction of ambiguities and conflicts.

• [Translated version] Note: All quotes in English translated by this article’s translator.

Publication Dates

• Publication in this collection
  05 May 2021
• Date of issue
  Mar-Apr 2021

History

• Received
  06 May 2019
• Accepted
  29 May 2020