Acessibilidade / Reportar erro

Between External Influence and Autonomy of Schools: Effects of External Evaluation of Schools

Entre a Determinação e a Autonomia Curriculares: Efeitos da Avaliação Externa das Escolas

Entre Determinación y Autonomía Curriculares: Efectos de la Evaluación Externa de los Centros Escolares

Abstract

The article analyzes the relationship between the External Evaluation of Schools (EES) - which takes place in Portugal since 2006 - and the curricular management processes. Its objective is to identify the effects of this evaluation on the curriculum management of schools. Data were collected by 108 questionnaires, two interviews and two focus groups in two Portuguese school groups with two distinct situations: a school group that rose in the rankings from the first to the second evaluation cycle (from 2006-2011/2012 to 2012-2015/2016) and the other one which dropped. The data analysis showed that the EES had a moderate impact on the curricular practices of the school group studied. However, there was a difference: the school group that rose in the rankings recognizes in more aspects and with more clarity the impact of EES on the restructure of its curricular practices.

Keywords:
Education; schools; evaluation; curriculum

Resumo

O artigo analisa a relação entre a Avaliação Externa das Escolas (AEE) - que ocorre em Portugal desde 2006 - e os processos de gestão curricular, com o objetivo de identificar efeitos desta avaliação nos modos de gestão do currículo que ocorrem nas escolas. Os dados foram recolhidos por 108 questionários, duas entrevistas e dois grupos focais, em dois agrupamentos de escolas portuguesas que correspondem a situações distintas: um agrupamento de escolas subiu nas classificações do primeiro para o segundo ciclo avaliativo (de 2006-2011/2012 para 2012-2015/2016) e o outro desceu. A análise dos dados permitiu constatar que a AEE teve um efeito moderado nas práticas curriculares dos agrupamentos de escolas estudados notando-se, no entanto, uma diferença: o agrupamento de escolas que subiu a classificação reconhece em mais aspetos e com mais clareza o efeito da AEE na reorganização das suas práticas curriculares.

Palavras-chave:
Educação; escolas; avaliação; currículo

Resumen: El artículo evalúa la relación entre la Evaluación Externa de los Centros Escolares (EECE) - que tiene lugar en Portugal desde 2006 - y los procesos de gestión curricular. Su objetivo es identificar los efectos de la evaluación externa de los centros escolares en la gestión del currículo que se hace en las escuelas. Los datos fueron recolectados por medio de 108 encuestas, dos entrevistas y dos focus groups en dos grupos de escuelas portuguesas, que corresponden a dos situaciones diferentes: un grupo escolar subió la clasificación del primer ciclo al segundo ciclo de evaluación (de 2006-2011/2012 a 2012-2015/2016) y el otro obtuvo una clasificación menor. El análisis de los datos permitió establecer que la EECE tuvo un efecto moderado sobre las prácticas curriculares de las escuelas estudiadas, sin embargo, se señala una diferencia: el grupo de escuelas que aumentó la calificación reconoce más, y más claramente, el efecto de EECE en la reorganización de sus prácticas curriculares.

Palabras clave:
Educación; escuelas; evaluación; currículo


Curriculum is, admittedly, the center of the school activity, where more comprehensive decisions of national education policies and particular concerns of every teacher regarding the processes of curriculum development and teaching exercise converge. Thus, the search for excellence in learning and improvement of schools cannot pass alongside the ways in which the curriculum is present in every school (Leite, 2012Leite, C. (2012). A articulação curricular como sentido orientador dos projetos curriculares [The curricular articulation as guidance of curricula projects]. Educação Unisinos, 16(1), 87-92. doi:10.4013/edu.2012.161.09
https://doi.org/10.4013/edu.2012.161.09...
). The same reason makes the curriculum in the object for excellence of External Evaluation of Schools (EES), which in Portugal is the responsibility of the General Inspection of Education (IGE) - an agency dependent on the Ministry of Education.

In this game of forces, among the broader dimensions, in this case due to the External Evaluation of Schools, under the responsibility of IGE and the more particular contextualizations of curricular practices of teachers (Mouraz, Fernandes, & Leite, 2014Mouraz, A., Fernandes, P., & Leite, C. (2014). Efeitos da avaliação externa das escolas no desenvolvimento de uma cultura de auto-avaliação [Effects of schools’ external evaluation in developing a self-assessment culture within schools]. Revista Portuguesa de Investigação Educacional, (14), 67-97.), there are also two contrary movements that fall into this game dynamically. On the one hand, there is a centralizing tendency associated with globalization and standardization of procedures (Kuiper, Van den Akker, Letschert, & Hooghoff, 2008Kuiper, W., Van den Akker, J., Letschert, J., & Hooghoff, H. (2008). Curriculum policy and practices in a European comparative perspective: Finding a balance between prescription and professionalism. Enschede, Netherlands: SLO.); on the other, manifestations that recommend a greater decision-making power to the schools curriculum (Kärkkäinen, 2012Kärkkäinen, K. (2012). Bringing about curriculum innovations: Implicit approaches in the OECD area. Paris, France: OECD.; Lopes, 2002Lopes, A. C. (2002). Os parâmetros curriculares nacionais para o ensino médio e a submissão ao mundo produtivo: O caso do conceito de contextualização [The Brazilian curricular parameters for secondary Education and the surrender to the productive world: The case of the concept of contextualization]. Educação e Sociedade, 23(80), 386-400.) and teachers (Mouraz, Leite, & Fernandes, 2013Mouraz, A., Leite, C., & Fernandes, P. (2013). Teachers’ role in curriculum design in Portuguese schools. Teachers and Teaching: Theory and Practice, 19(5), 478-491. doi:10.1080/13540602.2013.827363
https://doi.org/10.1080/13540602.2013.82...
) emerge. Each of these movements has reasonable and supporting arguments that can be understood in the context of the ideas expressed below (Figure 1). The tendency to centralize the curriculum is associated with the globalization of educational phenomena, required by the influence that economy has had in the definition of curricular policies (Lopes & Macedo, 2011Lopes, A. C., & Macedo, E. (2011). Teorias do currículo [Curriculum theories]. São Paulo, SP: Cortez.; Pacheco & Vieira, 2006Pacheco, J. A., & Vieira, A. P. (2006). Europeização do currículo. Para uma análise das políticas educativas e curriculares [Curriculum Europeization. An analysis of the educational and curricular policies]. In A. F. Moreira & J. A. Pacheco (Orgs.), Globalização e educação: Desafios para políticas e práticas (pp. 87-126). Porto, Portugal: Porto Editora.) and its consequent association with the transformation of education into an argument of competitiveness among countries (Seabra, Morgado, & Pacheco, 2012Seabra, F., Morgado, J. C., & Pacheco, J. A. (2012). Policies of accountability in Portugal. International Journal of Curriculum and Instruction Studies, 2(4), 41-51.). The idea that education systems should be guided by purposes that ensure citizens’ participation in economic systems, in order to potentiate the “wealth of nations,” revisits the neoliberal argument that knowledge is power, because powerful people are the ones who have this knowledge. In a sense, this idea is present in the warnings that have been expressed by Young (2011Young, M. (2011). What are schools for? Educação, Sociedade & Culturas, (32), 145-155. Available from https://www.fpce.up.pt/ciie/revistaesc/ESC32/ESC32_Arquivo.pdf
https://www.fpce.up.pt/ciie/revistaesc/E...
) when he calls for the need for the curriculum to focus on the “powerful knowledge,” as it was also shown by Galian and Louzano (2014Galian, C. V. A., & Louzano, P. B. J. (2014). Michael Young e o campo do currículo: Da ênfase no “conhecimento dos poderosos” à defesa do “conhecimento poderoso” [Michael Young and the curriculum field of studies: Emphasis of the knowledge of powerful]. Educação e Pesquisa , 40(4), 1109-1124. doi:10.1590/s1517-97022014400400201
https://doi.org/10.1590/s1517-9702201440...
).

Figure 1
Characteristics of centralizing and decentralizing tendencies.

As it has been politically justified, the scope of schooling systems with the consequent accessibility of education to all citizens is a necessary condition for the productivity of countries and the functioning of the mechanisms for introducing value added (from the association of knowledge with technology) in which Western economies operate. Serving the same purpose but with another set of arguments and in another framework, globalization also happens through the movement of lending policies (Olmedo & Ball, 2015Olmedo, A., & Ball, S. J. (2015). Competition, governance and global education policy. In M. Souto-Otero (Ed.), Evaluating European education policy-making: Privatization, networks and the European Commission (pp. 25-52). London, United Kingdom: Palgrave Macmillan.; Steiner-Khamsi, 2012Steiner-Khamsi, G. (2012). Understanding policy borrowing and lending: Building comparative policy studies. In G. Steiner-Khamsi & F. Waldow (Eds.), World yearbook of education 2012: Policy borrowing and lending in education (pp. 3-17). London, United Kingdom: Routledge.), which, after traveling, become rapidly influential in national decision-making processes. In these situations, the methods of dissemination of the educational results by international rankings often compete, which are used as a legitimation of quality of education systems. At national level, in Portugal, the External Evaluation of Schools (EES), elaborated to improve education, has had as a side effect the comparability among the schools as a result of the ratings assigned to them.

The third key idea that joins the movement of curriculum centralization concerns the concept of powerful knowledge and it governance. Knowledge is understood as powerful, useful to all subjects, allowing them to expand the localized condition of their lives (Young, 2011Young, M. (2011). What are schools for? Educação, Sociedade & Culturas, (32), 145-155. Available from https://www.fpce.up.pt/ciie/revistaesc/ESC32/ESC32_Arquivo.pdf
https://www.fpce.up.pt/ciie/revistaesc/E...
). Somehow this argument relates to the basis of the universality of the subjects and is materialized in the idea that any citizen has the right and the ability to develop the fundamental literacies. The exponent of this argument is evident, whether in the definition of what the fundamental literacies are, whether in modes and structures created for their definition and measurement, of which PISA is an example (Afonso, 2014Afonso, A. (2014). The emergence of accountability in the Portuguese education system. European Journal of Curriculum Studies, 1(2), 125-132. Available from http://pages.ie.uminho.pt/ejcs/index.php/ejcs/article/view/54/24
http://pages.ie.uminho.pt/ejcs/index.php...
). If the political spaces are the places where the agents that promote international comparative assessments are (such as PISA), in the public space (e.g. the space occupied by the media) such international processes of knowledge measurement, as well as the evaluative processes of schools are justified (Abrantes, 2009Abrantes, P. (2009). Os guardiães do templo: A imprensa generalista e a imposição de uma agenda educativa [The guardians of the temple: The press and the educational agenda influences]. Educação, Sociedade & Culturas, (28), 127-144. Available from https://www.fpce.up.pt/ciie/revistaesc/ESC28/28_pedroAbrantes.pdf
https://www.fpce.up.pt/ciie/revistaesc/E...
; Mouraz, Torres, & Nunes, 2015Mouraz, A., Torres, A. C., & Nunes, C. (2015). Currículo em tempos de crise: Os jornais portugueses e a construção social do currículo [Curriculum in times of crisis: The portuguese newspapers and the social construction of curricula]. Investigar em Educação, (3), 111-130. Available from http://pages.ie.uminho.pt/inved/index.php/ie/article/view/91/90
http://pages.ie.uminho.pt/inved/index.ph...
), which constitutes a clear form of knowledge governance (Bonamino & Sousa, 2012Bonamino, A., & Sousa, S. Z. (2012). Três gerações de avaliação da educação básica no Brasil: Interfaces com o currículo da/na escola [Three generations of assessments of basic education in Brazil: Relations between curriculum and school]. Educação e Pesquisa, 38(2), 373-388. doi:10.1590/S1517-97022012005000006
https://doi.org/10.1590/S1517-9702201200...
). The use of arguments of equity and justice (Connell, 1999Connell, R. W. (1999). Escuelas y justicia social [Schools and social justice] (2a ed.). Madrid, España: Morata.; Torres Santomé, 2010Torres Santomé, J. (2010). La justicia curricular: El caballo de Troya de la cultura escolar [The Curricular justice: The Troy horse of the school culture]. Madrid, España: Morata .) to construct this justification, in particular by emphasizing what is due to all students and to each individual, in the treatment that the school affords to them, is also a form of knowledge governance.

The second movement to which we refer recommends a greater curricular decision making power to give schools and teachers, using the arguments aimed at the promotion of equity of the systems and/or their effectiveness. One of those arguments relates to the delegation of competence to the schools and teachers as an exercise to distribute responsibilities and recognize that schools and teachers are key elements in educational processes. This idea is in line with the decentralizing tradition of curriculum organization of countries from the North and center of Europe and is based simultaneously on the recognition of the professionalism of teachers (Kuiper et al., 2008Kuiper, W., Van den Akker, J., Letschert, J., & Hooghoff, H. (2008). Curriculum policy and practices in a European comparative perspective: Finding a balance between prescription and professionalism. Enschede, Netherlands: SLO.; Nóvoa, 1992Nóvoa, A. (1992). Os professores e a sua formação [The teachers and their training]. Lisboa, Portugal: Dom Quixote.) and on the importance of the autonomy of the schools (Barroso, 2006Barroso, J. (2006). A autonomia das escolas: Retórica, instrumento e modo de regulação da ação política [The autonomy of schools: Rethoric and instruments of political action]. In AAVV, A autonomia das escolas (pp. 23-48). Lisboa, Portugal: Fundação Calouste Gulbenkian.; Leite & Fernandes, 2010Leite, C., & Fernandes, P. (2010). Desafios aos professores na construção de mudanças educacionais e curriculares: Que possibilidades e que constrangimentos [Challenges to teachers in the educational and curricular shifting: Possibilities and limits]. Educação (Porto Alegre), 33(3), 198-204. Available from http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/8076/5723
http://revistaseletronicas.pucrs.br/ojs/...
). On the other hand, this argument is often related to the reduction in the central funding of education systems whose central administrations, after seeing the reduction in the funds to be distributed, transfer responsibilities to the institutions and local educational agents. However, in this process of distribution of responsibility, what happens is the delegation of the power to act in accordance with a global principle.

Another key idea that is associa ted with the decentralization is based on the relationship between curriculum development and educational innovation (Kärkkäinen, 2012Kärkkäinen, K. (2012). Bringing about curriculum innovations: Implicit approaches in the OECD area. Paris, France: OECD.). This argument has as an implicit question “what is the best way to organize and develop innovative answers to solve educational and curricular problems?” and has as an explicit answer that “it is at the local level” (Hopkins, Stringfield, Harris, Stoll, & Mackay, 2014Hopkins, D., Stringfield, S., Harris, A., Stoll, L., & Mackay, T. (2014). School and system improvement: A narrative state-of-the-art review. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 25(2), 257-281. doi:10.1080/09243453.2014.885452
https://doi.org/10.1080/09243453.2014.88...
; Leite, 2002Leite, C. (2002). Princípios e implicações da gestão curricular local [Principles and implications of local curriculum management]. Revista Música, Psicologia e Educação, (4), 5-11. Available from http://recipp.ipp.pt/bitstream/10400.22/3122/1/ART_CarlindaLeite_2002.pdf
http://recipp.ipp.pt/bitstream/10400.22/...
).

A third argument, which in some way intersects with the previous ones, focuses its attention on the change of roles and conditions for its assumption and for the recognition that teachers and schools should consider themselves and become full-fledged agents of the educational action. The essential concept here is the “agency,” and the challenge lies in identifying the factors that will allow teachers and schools to be agents (Mouraz et al., 2013Mouraz, A., Leite, C., & Fernandes, P. (2013). Teachers’ role in curriculum design in Portuguese schools. Teachers and Teaching: Theory and Practice, 19(5), 478-491. doi:10.1080/13540602.2013.827363
https://doi.org/10.1080/13540602.2013.82...
; Priestley, Edwards, Priestley, & Miller, 2012Priestley, M., Edwards, R., Priestley, A., & Miller, K. (2012). Teacher agency in curriculum making: Agents of change and spaces for manoeuvre. Curriculum Inquiry, 42(2), 191-214. doi:10.1111/j.1467-873X.2012.00588.x
https://doi.org/10.1111/j.1467-873X.2012...
). The concept of curricular autonomy of schools and teachers has in this argument its main support, and it is associated with the relationship that is considered to exist in the curriculum between theory and practice, of which the curriculum contextualization is an example (Fernandes, Leite, Mouraz, & Figueiredo, 2013Fernandes, P., Leite, C., Mouraz, A., & Figueiredo, C. (2013). Curricular contextualization: Tracking the meanings of a concept. The Asia-Pacific Education Researcher, 22(4), 417-425. doi:10.1007/s40299-012-0041-1
https://doi.org/10.1007/s40299-012-0041-...
). Curricular autonomy is understood as the power and competence of teachers and schools to select the contents and organize the most coherent processes to ensure the identity construction of the citizens. (id) The concept is also a common tendency of discourses that shape the revisions of the national curricula (Sinnema & Aitken, 2014Sinnema, C., & Aitken, G. (2014). Emerging international trends in curriculum. In M. Priestley & G. Biesta (Eds.), Reinventing the curriculum: New trends in curriculum policy and practice (pp. 141-163). London, United Kingom: Bloomsbury.).

It is in the game of these two movements (of curricular centralization and decentralization) that the reflection that we are producing upon the effects of (EES) makes sense, which has been cited as a political measure of curricular management and effectiveness that fits in the larger panorama of policies that tend to the international standardization (Afonso, 2014Afonso, A. (2014). The emergence of accountability in the Portuguese education system. European Journal of Curriculum Studies, 1(2), 125-132. Available from http://pages.ie.uminho.pt/ejcs/index.php/ejcs/article/view/54/24
http://pages.ie.uminho.pt/ejcs/index.php...
).

As we have already indicated, the curriculum that is developed in each school is the object par excellence of the look of external evaluators in the exercise they carry out and that, in Portugal, covers all schools, every four years. Although after 2013, in the EES, the evaluative judgement became completely dependent on the school results obtained by the students and their confrontation with the expected results, the reports produced by the external evaluators do not cease to be liable for a reading about evaluative looks that focus on the conditions of the curricular agency that schools and teachers experience.

It is in this context that our study had the following objectives: to identify contributions from EES in the autonomous exercise of teachers in the curriculum development; to characterize the role of EES in improving the overall process of curricular coherence (planning, methodology, evaluation); to relate the appreciation of teachers’ collaborative work with the EES.

Method

Participants

The methodological procedure followed corresponds to a multi-case study that mobilizes data collected in two groups of Portuguese schools, of basic education, selected because one had gone up and the other went down in ratings assigned by EES, from the first to the second cycle of this evaluation. In a specific way, the principals of the two school groups participated in the investigation, as well as the coordinators of the curricular departments (organized in two focus groups) and 108 teachers of these same groups.

Instruments

Data were obtained through the following procedures: interviews with the principals of the two school groups; Focus groups carried out with the coordinators of the departments; questionnaires applied to teachers of these same groups. In its structure, interviews, focus groups and the questionnaire focused on the three dimensions of research on effects of EES on the curricular autonomy as they emerge from the theoretical foundation of this article. That is, the questions posed intended to know the effects of EES at the level of the curriculum autonomy of the teachers, unfolded in the following dimensions: (i) Collaborative work of teachers; (ii) Involvement and commitment of teachers; and (iii) General process of curriculum coherence. The concept of curricular autonomy, perceived and put into practice by the teachers, is based on the collaborative work they perform, in their involvement and commitment and in the general coherence they perceive in their work (Mouraz et al., 2013Mouraz, A., Leite, C., & Fernandes, P. (2013). Teachers’ role in curriculum design in Portuguese schools. Teachers and Teaching: Theory and Practice, 19(5), 478-491. doi:10.1080/13540602.2013.827363
https://doi.org/10.1080/13540602.2013.82...
).

The questionnaires were organized in the form of a Likert-type scale with five intervals of measurement agreement of the respondents (totally disagree, disagree, undecided, agree, totally agree) and included the items associated with the dimensions mentioned, according to the correspondence expressed in Table 1. The questionnaires were validated by other specialists in evaluation.

Table 1
Aggregation of questionnaire items to the dimensions of the analysis

Explicit questions related to the dimensions referred to as effects of EES were included in the interviews and foci groups, just as they were perceived by the interviewees.

Procedure

Data collection. The interviews with the principals of the two school groups occurred according to the availabilities of those responsible, being always held by two researchers. All interviews were recorded and transcribed. Similarly, the focus groups held with the coordinators of the departments occurred after a call was made by the research team, at an appropriate time to the participants. In this case, the words of the actors were also recorded and transcribed. The questionnaire was filled online, through an invitation sent by e-mail, by 108 teachers of the two groups, which constitutes 33% of the total possible.

Data analysis. The data collected from the qualitative type were treated through content analysis (Krippendorf, 2003Krippendorf, K. H. (2003). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.), using the NVivo10 program, and the responses were gathered according to the dimensions of the research previously presented. For analytical purposes, the phrase was considered a registration unit, being extended the adjacent sentences to the paragraph, in order to translate and sustain an idea. In the encoding process, the principle of non-exclusivity of registration units was followed as belonging to the same subcategory, as defended by Krippendorf (2003)Krippendorf, K. H. (2003). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage., in order to allow the crossing of categories (dimensions of research). The encoding was performed by one of the authors and validated in the system by the other researcher.

The quantitative data obtained by the questionnaire were the object of statistical analysis using the SPSS program, version 21. We made comparative analyses of the measures of central tendency originated from both groups of questionnaires (group 1 and group 2) and assessed the statistical difference between the responses of the two groups of subjects in each of the dimensions in which the items were organized. The differences in statistical significance were considered if p < 0.05.

Ethical Considerations

From the ethical point of view, the following procedures were followed: the participants in the interviews, in the focus groups and in the questionnaire were previously informed that their participation in the survey was voluntary; the schools groups were not identified nor can be identified because many of them have similar characteristics; respondents and the elements that participated in the focus groups and those who answered to the questionnaire were chosen in order to meet the objectives of the study and were guaranteed the anonymity; in the case of focus groups, at the beginning of the session, the participants, all volunteers who were interested in reflecting together upon the relationship between EES and management processes of the curriculum, were informed that they could abandon the session if they so wished; the accounts recorded in the interviews were returned to each respondent in case they wanted to change it; regarding the questionnaire, data that would allow their identification were not collected, which was established before the collection of the responses; participants were informed that they could have access to the global data of the study but not to the individual data.

Results

The analysis of the data collected by the questionnaires, interviews and focus groups allowed obtaining information about the influence and effects of EES in the Groups studied, according to their principals (PG1-Principal of Group 1 and PG2-Principal of Group 2), the department coordinators (FG G1 - Focus Group of Group 1 and FG G2-Focus Group of Group 2) and a group of their teachers. The data on the teachers corresponded to a total of 108 questionnaires, being 74 of group 1 (75% of the teaching population) and 34 of group 2 (22% of the teaching population).

The data are presented following the analysis framework previously referred to, which corresponds to the organization of the questionnaire, or to a subset of questions included in the script of the interviews and focus groups. Thus, the discussion of the dimensions that seem relevant to assess the influence of EES is a graphic presentation in advance of the data obtained from the questionnaire and the enunciation of references encoded in the speech of principals and department coordinators.

Collaborative Work of Teachers

The answers to this set of items of the questionnaire allowed us to know the teachers’ opinions about the impact of External Evaluation in their collaborative work. Figure 2 shows how these responses were distributed by the respondents of both groups. In group 1, which rose in the rankings from the first to the second cycle of EES, there is a clear association of the stimulus with this collaborative work produced by the influence of the external evaluation, and the association does not show wide-dispersion in the opinion of teachers, as the smallest dimension of the chart lines indicates. In group 2, which went down in the rankings from the first to the second cycle of EES, the average value of the answers lies in 3 (= undecided) for the items included in the category and show a greater dispersion of positions among the teachers of that group. There still exists a statistically significant difference as to the influence reported by the teachers of the two groups about the development of the curriculum and the testing in the learning assessment of students.

Figure 2
Relationship between the external evaluation and the work of teachers.

By crossing these data with those obtained by the responses given by the principals and department coordinators, we observed that this influence is clearly assumed in group 1, which even organized the schedules of teachers in order to foster this study, as it can be seen in the following statements by those responsible for this group (which rose in the rankings in EES):

(...) There was a measure that also has to do with the distribution of service, which is the collaborative work. The colleagues in the most structuring disciplinary areas (Mathematics and Portuguese) have 45 or 90 minutes to work collaboratively. (P G1)

(...) that makes us think about how the year went and reflect if there is anything we could do to change, improve, as there is always something to improve. (FG G1)

In Group 2 (which went down in the rankings in EES) the same influence is not generally assumed. If some coordinators consider promoting more collaborative work after EES, it is generally felt that this was an already promoted and widespread practice, before EES, as evidenced in the following statements:

The school has always performed collaborative work.(P G2)

Because of the EES we do more collaborative work focused on the production of assessment instruments, because now we reflect together. (FG G2)

Involvement and Commitment of Teachers

Regarding this dimension of the investigation, we also found a significant difference in the opinion of the teachers of both groups regarding the influence of EES in promoting the commitment and involvement of teachers in educational tasks developed at school and in the classroom. However, this influence seems to be bigger in the development of projects at the school than in the promotion of innovative activities in the classroom, as shown in Figure 3. If there is a positive association in encouraging the commitment and involvement of teachers produced by the exercise of EES in group 1; in group 2, the average value of the responses to the item relating to school projects is 3 (= undecided) and to the item that refers to the influence in the classroom, the value obtained is clearly in disagreement.

Figure 3
Relationship between the external evaluation and the work of teachers.

The crossing of the data obtained by questionnaire with those of the interviews and focus groups shows that the external influence is clearly assumed in the speeches of the interviewees in group 1, which validate the weight of the influence, especially at the level of the number, diversity and rhythm of the school by the effect of EES :

the projects that we have since then were born of this research and we took advantage of it. As the school went into contract of autonomy, all the departments needed to have projects (...). (P G1)

If there were no such projects (...), these projects make us think. (FG G1)

Regarding group 2, the interviewees considered that, as a whole, the group and the teachers develop useful projects both to students and to the school. However, they do not recognize the influence of EES in the decision to perform it. Eventually, they denote some negative effects of EES because of the least good result obtained by the school, which has resulted in some minor involvement. They also explain the lower commitment of teachers as a result of the general climate of the country and the conditions of the profession, rather than a reflection on EES, as the following statements express:

About the involvement and effects at school level, I think they are being a little more dramatic than the previous one because of the results. (FG G2)

I think the moment when the inspection took place was a bad time, (...) nor will it be with what the inspection said, it has to do with what is happening at school, in the profession, in society, it is everywhere. (FG G2)

General process of Curriculum Coherence

The fourth dimension considered joined three aspects that contribute to curricular coherence, namely: i) articulation between cycles and levels of education; II) adequacy of pedagogical practices that occur in the classroom; and iii) adequacy of the curricular practices designed by teachers. These three aspects lead to changes, which were necessary and effective in the management of the curriculum.

The results are generally similar to those reported in previous dimensions, although the participants in the study of group 2 have valued very positively the influence of EES in the development of the articulation between cycles and levels of education organized at school (Figure 4). We found significant differences among the opinions of the teachers of the two groups in the item related to the curricular practices drawn by the teachers and in the item regarding the influence of the EES on the changes operated in the curricular management. It was also these two items that deserved a clear disagreement from the teachers of group 2 regarding the influence of EES in its setting.

Figure 4
Contribution of the external evaluation for teaching and curriculum practices.

The results obtained in the questionnaires are concordant with the statements collected by the principals and coordinators of the Department and help clarify the meaning and means of the influence of EES in group 1 and the more modest influence in group 2.

In group 1, it is evident the influence of EES in the construction process of the curricular coherence and identifies the principal as the vehicle of this influence, in particular by the pressure it exerts on the teachers, as shown the following statements:

In Pedagogy, he (principal) always gives this message, “attention, this is working, this is not, you need to work together.” Even now, because of the relationship between pre and the 1st cycle, (...), he keeps sending messages. (FG G1)

The collaborative work, the work at the level of articulation of cycles is greater, that is, a type of communication that did not exist previously was created. (...) This happened after the evaluation report of the 1st cycle of EES. (P G1)

Also, the EES has determined other planning dynamics and curriculum management, as stated:

The monitoring committee (...) accompanied us in the initiative to improve what can be done. (...) Our Principal is very open to initiatives, new things, to implement something, he wants to do more. (FG G1)

In group 2, the department coordinators report improvements in pedagogical practices and changes in strategies as a way of reacting to the needs of students and the difficulties they encounter. The word innovation does not appear in the interview transcript. The influence that is assumed as a result of the exercise of EES concerns the promotion of curriculum articulation, although it was already a previous practice of the group, especially in the early levels of education, as stated:

The articulation was thought not only on the basis of the external evaluation but according to gaps we observed in different disciplines and the way they are filled. (FG G2)

The communication between the class principals and the guardians here was excellent in relation to other groups, the articulation between preschool and the 1st cycle was also good (...). (FG G2)

In short, in group 2, we recognize that the external look led to improved management practices of the curriculum, as determined by the pressure of the results:

(...) and it was the year we had the worst results, but we have learned and from there we started taking training actions, giving them tests like the others, the ones of previous years, repeat. The same exercises (...) and there has been improvement. (P G2)

An analysis of teachers’ discourses, of the directions of the groups and department coordinators allows us to recognize that the school groups were not indifferent to the EES. However, if in group 1 the reaction of the principal and the department coordinators was to encourage a certain voluntarism to do more and differently, in group 2 the exercise of EES developed a climate of resistance, as it appears in the statements that follow:

EES has stimulated the creative capacity of teachers who want to respond to everything. (FG G1)

EES has interfered with the autonomy of teachers because it forces them to develop activities that they would have done if not pressured. (FG G2)

Discussion

With the study to which this article refers, we intended to meet the purposes of EES at the level of Curriculum autonomy of teachers, unfolded in the following dimensions: (i) Collaborative work of teachers; (ii) Involvement and commitment of teachers; and (iii) General process of curricular coherence. Generally, the data show that in group 1 (group that rose in the rankings) the EES generated more evident effects, at the level of the three dimensions under analysis, than in group 2 (group that went down in the rankings).

For the purposes of EES in the collaborative work of teachers, the data show, in the case of group 1, a clear association between this evaluation of schools and the stimulus of teachers in carrying out collaborative work. In this case, the reorganization of the timetables of teachers represents a strategy of the principal to create conditions for teachers to become involved in improvement actions. This policy may be related to the fact the Principal of group 1 was at the beginning of his term and wanted to see his intervention project implemented, whether by the teachers of his group, whether by EES.

In group 2 (which went down in EES rankings), the relationship between this assessment and the existence of collaborative work practices is not so evident. Collaborative work is perceived as a practice that has been promoted by the principal for several years. It is also recognized that collaborative work practices are the result of a culture of communication established among all educators/teachers of the group, and the adoption of a set of procedures. In summary, this tendency towards a certain standardization of procedures (Kuiper et al., 2008Kuiper, W., Van den Akker, J., Letschert, J., & Hooghoff, H. (2008). Curriculum policy and practices in a European comparative perspective: Finding a balance between prescription and professionalism. Enschede, Netherlands: SLO.), in the case of group 2, can be justified by the fact that the principal is in a situation of continuity term, a condition that has already conferred on him legitimacy to the leadership project, either with the teachers and educators or with EES.

Regarding the involvement and commitment of teachers, the analysis showed differences in relation to the two groups under study. Group 1 has a clearly superior association between EES and the commitment and involvement of teachers in educational activities than group 2. However, this influence seems to be bigger in the development of projects at the school than in the promotion of innovative activities in the classroom, as shown in Figure 3. The effect of the involvement and commitment of the teachers on EES are recognized, in the case of group 1, specifically in the work rhythm of teachers, in the collaborative work they perform and in the volume and diversity of cross-sectional projects to the whole group. The teachers and the principal of group 2 do not recognize the influence of EES on the projects they develop and on the way they engage in the implementation of these projects. Still, they recognize that the fact the group has dropped in the rankings of EES may have had the effect of deterring some teachers to get involved in new projects.

Regarding the fourth dimension, general process of curriculum coherency, in the various aspects included, the results are generally similar to those of the other previous dimensions. The effects of EES are also more valued in group 1 than in group 2. In group 1, the EES is perceived as having generated effect on the improvement of management practices, in the implementation of cross-sectional projects to every school in the group to better respond to students’ difficulties. For their part, the participants in group 2 valued the articulation between cycles and levels of education as the most positive effect of EES.

Despite the distinction of effects of EES, in both groups the external evaluation of the learning policies (national exams) has been a growing focus of attention of schools and teachers, and it is recognized as being influenced by the EES, focusing on what has been achieved at the level of the students’ results. This situation can be associated with a certain tendency to centralize the curriculum and with the influence that the economic dimension has had in the definition of curricular policies (Pacheco & Vieira, 2006Pacheco, J. A., & Vieira, A. P. (2006). Europeização do currículo. Para uma análise das políticas educativas e curriculares [Curriculum Europeization. An analysis of the educational and curricular policies]. In A. F. Moreira & J. A. Pacheco (Orgs.), Globalização e educação: Desafios para políticas e práticas (pp. 87-126). Porto, Portugal: Porto Editora.)

In summary, the relationship between EES and the curricular autonomy of teachers, which was analyzed in the three dimensions that shape the decision process and the curricular practice, is perceived differently by the subjects of group 1 and group 2. In the case of group 1, the data indicate that EES induced a policy of encouragement to “do more and differently,” which is recognized as positive by teachers and as enabler of greater curriculum decision-making power (Kärkkäinen, 2012Kärkkäinen, K. (2012). Bringing about curriculum innovations: Implicit approaches in the OECD area. Paris, France: OECD.) for the teachers. On the contrary, the teachers of group 2 admit that EES interfered with their curricular autonomy, forcing them to perform activities with a logic not always in tune with the educational priorities of the students, in particular those that relate to the necessity of the curriculum to focus on the “powerful knowledge” (Young, 2011Young, M. (2011). What are schools for? Educação, Sociedade & Culturas, (32), 145-155. Available from https://www.fpce.up.pt/ciie/revistaesc/ESC32/ESC32_Arquivo.pdf
https://www.fpce.up.pt/ciie/revistaesc/E...
) and national exams. In this case, the EES seems to have had the effect of reducing the curricular autonomy of teachers.

The data show that the teachers, coordinators and principals of both groups under study “see” EES differently, as well as its effects on school dynamics. These views and positions seem to relate directly to the situation of leaders/principals: beginning of term/continuity of term. In the case of group 1, this condition seems to induce a dynamic leadership that oscillates between the fulfilment of requirements established by EES and the stimulus of teachers for innovation and curricular management processes; in the case of group 2, the leadership dynamics seems to distance themselves from the formalities of EES and value the curricular experiences already carried out by the teachers, stressing the power of their curriculum. In both cases, the ongoing curricular practices seem to relate more to characteristics that can be associated with the decentralist movement of curriculum organization (Kärkkäinen, 2012Kärkkäinen, K. (2012). Bringing about curriculum innovations: Implicit approaches in the OECD area. Paris, France: OECD.) in which the autonomy of teachers (Barroso, 2006Barroso, J. (2006). A autonomia das escolas: Retórica, instrumento e modo de regulação da ação política [The autonomy of schools: Rethoric and instruments of political action]. In AAVV, A autonomia das escolas (pp. 23-48). Lisboa, Portugal: Fundação Calouste Gulbenkian.; Leite & Fernandes, 2010Leite, C., & Fernandes, P. (2010). Desafios aos professores na construção de mudanças educacionais e curriculares: Que possibilidades e que constrangimentos [Challenges to teachers in the educational and curricular shifting: Possibilities and limits]. Educação (Porto Alegre), 33(3), 198-204. Available from http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/8076/5723
http://revistaseletronicas.pucrs.br/ojs/...
) and the creation of conditions for the professional practice (Kuiper et al., 2008Kuiper, W., Van den Akker, J., Letschert, J., & Hooghoff, H. (2008). Curriculum policy and practices in a European comparative perspective: Finding a balance between prescription and professionalism. Enschede, Netherlands: SLO.; Nóvoa, 1992Nóvoa, A. (1992). Os professores e a sua formação [The teachers and their training]. Lisboa, Portugal: Dom Quixote.) are recognized as very important.

References

  • Abrantes, P. (2009). Os guardiães do templo: A imprensa generalista e a imposição de uma agenda educativa [The guardians of the temple: The press and the educational agenda influences]. Educação, Sociedade & Culturas, (28), 127-144. Available from https://www.fpce.up.pt/ciie/revistaesc/ESC28/28_pedroAbrantes.pdf
    » https://www.fpce.up.pt/ciie/revistaesc/ESC28/28_pedroAbrantes.pdf
  • Afonso, A. (2014). The emergence of accountability in the Portuguese education system. European Journal of Curriculum Studies, 1(2), 125-132. Available from http://pages.ie.uminho.pt/ejcs/index.php/ejcs/article/view/54/24
    » http://pages.ie.uminho.pt/ejcs/index.php/ejcs/article/view/54/24
  • Barroso, J. (2006). A autonomia das escolas: Retórica, instrumento e modo de regulação da ação política [The autonomy of schools: Rethoric and instruments of political action]. In AAVV, A autonomia das escolas (pp. 23-48). Lisboa, Portugal: Fundação Calouste Gulbenkian.
  • Bonamino, A., & Sousa, S. Z. (2012). Três gerações de avaliação da educação básica no Brasil: Interfaces com o currículo da/na escola [Three generations of assessments of basic education in Brazil: Relations between curriculum and school]. Educação e Pesquisa, 38(2), 373-388. doi:10.1590/S1517-97022012005000006
    » https://doi.org/10.1590/S1517-97022012005000006
  • Connell, R. W. (1999). Escuelas y justicia social [Schools and social justice] (2a ed.). Madrid, España: Morata.
  • Fernandes, P., Leite, C., Mouraz, A., & Figueiredo, C. (2013). Curricular contextualization: Tracking the meanings of a concept. The Asia-Pacific Education Researcher, 22(4), 417-425. doi:10.1007/s40299-012-0041-1
    » https://doi.org/10.1007/s40299-012-0041-1
  • Galian, C. V. A., & Louzano, P. B. J. (2014). Michael Young e o campo do currículo: Da ênfase no “conhecimento dos poderosos” à defesa do “conhecimento poderoso” [Michael Young and the curriculum field of studies: Emphasis of the knowledge of powerful]. Educação e Pesquisa , 40(4), 1109-1124. doi:10.1590/s1517-97022014400400201
    » https://doi.org/10.1590/s1517-97022014400400201
  • Hopkins, D., Stringfield, S., Harris, A., Stoll, L., & Mackay, T. (2014). School and system improvement: A narrative state-of-the-art review. School Effectiveness and School Improvement: An International Journal of Research, Policy and Practice, 25(2), 257-281. doi:10.1080/09243453.2014.885452
    » https://doi.org/10.1080/09243453.2014.885452
  • Kärkkäinen, K. (2012). Bringing about curriculum innovations: Implicit approaches in the OECD area. Paris, France: OECD.
  • Krippendorf, K. H. (2003). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.
  • Kuiper, W., Van den Akker, J., Letschert, J., & Hooghoff, H. (2008). Curriculum policy and practices in a European comparative perspective: Finding a balance between prescription and professionalism. Enschede, Netherlands: SLO.
  • Leite, C. (2002). Princípios e implicações da gestão curricular local [Principles and implications of local curriculum management]. Revista Música, Psicologia e Educação, (4), 5-11. Available from http://recipp.ipp.pt/bitstream/10400.22/3122/1/ART_CarlindaLeite_2002.pdf
    » http://recipp.ipp.pt/bitstream/10400.22/3122/1/ART_CarlindaLeite_2002.pdf
  • Leite, C. (2012). A articulação curricular como sentido orientador dos projetos curriculares [The curricular articulation as guidance of curricula projects]. Educação Unisinos, 16(1), 87-92. doi:10.4013/edu.2012.161.09
    » https://doi.org/10.4013/edu.2012.161.09
  • Leite, C., & Fernandes, P. (2010). Desafios aos professores na construção de mudanças educacionais e curriculares: Que possibilidades e que constrangimentos [Challenges to teachers in the educational and curricular shifting: Possibilities and limits]. Educação (Porto Alegre), 33(3), 198-204. Available from http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/8076/5723
    » http://revistaseletronicas.pucrs.br/ojs/index.php/faced/article/view/8076/5723
  • Lopes, A. C. (2002). Os parâmetros curriculares nacionais para o ensino médio e a submissão ao mundo produtivo: O caso do conceito de contextualização [The Brazilian curricular parameters for secondary Education and the surrender to the productive world: The case of the concept of contextualization]. Educação e Sociedade, 23(80), 386-400.
  • Lopes, A. C., & Macedo, E. (2011). Teorias do currículo [Curriculum theories]. São Paulo, SP: Cortez.
  • Mouraz, A., Fernandes, P., & Leite, C. (2014). Efeitos da avaliação externa das escolas no desenvolvimento de uma cultura de auto-avaliação [Effects of schools’ external evaluation in developing a self-assessment culture within schools]. Revista Portuguesa de Investigação Educacional, (14), 67-97.
  • Mouraz, A., Leite, C., & Fernandes, P. (2013). Teachers’ role in curriculum design in Portuguese schools. Teachers and Teaching: Theory and Practice, 19(5), 478-491. doi:10.1080/13540602.2013.827363
    » https://doi.org/10.1080/13540602.2013.827363
  • Mouraz, A., Torres, A. C., & Nunes, C. (2015). Currículo em tempos de crise: Os jornais portugueses e a construção social do currículo [Curriculum in times of crisis: The portuguese newspapers and the social construction of curricula]. Investigar em Educação, (3), 111-130. Available from http://pages.ie.uminho.pt/inved/index.php/ie/article/view/91/90
    » http://pages.ie.uminho.pt/inved/index.php/ie/article/view/91/90
  • Nóvoa, A. (1992). Os professores e a sua formação [The teachers and their training]. Lisboa, Portugal: Dom Quixote.
  • Olmedo, A., & Ball, S. J. (2015). Competition, governance and global education policy. In M. Souto-Otero (Ed.), Evaluating European education policy-making: Privatization, networks and the European Commission (pp. 25-52). London, United Kingdom: Palgrave Macmillan.
  • Pacheco, J. A., & Vieira, A. P. (2006). Europeização do currículo. Para uma análise das políticas educativas e curriculares [Curriculum Europeization. An analysis of the educational and curricular policies]. In A. F. Moreira & J. A. Pacheco (Orgs.), Globalização e educação: Desafios para políticas e práticas (pp. 87-126). Porto, Portugal: Porto Editora.
  • Priestley, M., Edwards, R., Priestley, A., & Miller, K. (2012). Teacher agency in curriculum making: Agents of change and spaces for manoeuvre. Curriculum Inquiry, 42(2), 191-214. doi:10.1111/j.1467-873X.2012.00588.x
    » https://doi.org/10.1111/j.1467-873X.2012.00588.x
  • Sinnema, C., & Aitken, G. (2014). Emerging international trends in curriculum. In M. Priestley & G. Biesta (Eds.), Reinventing the curriculum: New trends in curriculum policy and practice (pp. 141-163). London, United Kingom: Bloomsbury.
  • Seabra, F., Morgado, J. C., & Pacheco, J. A. (2012). Policies of accountability in Portugal. International Journal of Curriculum and Instruction Studies, 2(4), 41-51.
  • Steiner-Khamsi, G. (2012). Understanding policy borrowing and lending: Building comparative policy studies. In G. Steiner-Khamsi & F. Waldow (Eds.), World yearbook of education 2012: Policy borrowing and lending in education (pp. 3-17). London, United Kingdom: Routledge.
  • Torres Santomé, J. (2010). La justicia curricular: El caballo de Troya de la cultura escolar [The Curricular justice: The Troy horse of the school culture]. Madrid, España: Morata .
  • Young, M. (2011). What are schools for? Educação, Sociedade & Culturas, (32), 145-155. Available from https://www.fpce.up.pt/ciie/revistaesc/ESC32/ESC32_Arquivo.pdf
    » https://www.fpce.up.pt/ciie/revistaesc/ESC32/ESC32_Arquivo.pdf

Publication Dates

  • Publication in this collection
    12 Sept 2019
  • Date of issue
    2019

History

  • Received
    14 Jan 2017
  • Reviewed
    14 Apr 2018
  • Accepted
    13 June 2018
Universidade de São Paulo, Faculdade de Filosofia Ciências e Letras de Ribeirão Preto, Programa de Pós-Graduação em Psicologia Av.Bandeirantes 3900 - Monte Alegre, 14040-901 Ribeirão Preto - São Paulo - Brasil, Tel.: (55 16) 3315-3829 - Ribeirão Preto - SP - Brazil
E-mail: paideia@usp.br