Conceptions and assessment practices in Rio de Janeiro municipal schools

Concepções e Práticas Avaliativas em Escolas Municipais do Rio de Janeiro

Concepciones y prácticas de evaluación en las escuelas municipales de Río de Janeiro

Abstract

We analyzed the conceptions and assessment practices of elementary school teachers, aiming to investigate the uses of the results of the learning assessments in schools and the results of the Provas Bimestrais (Bimonthly Exams) – instruments of the Assessment System of the Rio de Janeiro Municipal Educational System – for planning and taking pedagogical decisions. We are based on the theories of formative assessment, which indicated assessment as a tool to improve learning and not just to measure it. We conducted interviews and participant observation in two schools, describing and analyzing contradictions between discourse and practices. The teachers’ conceptions approached formative assessment, focusing on defense of diversification of assessment instruments, yet they mainly conduct traditional exams and, frequently, they work in a similar way to the Provas Bimestrais (Bimonthly Exams). The results of both assessments were basically to give superficial feedback to the students and to calculate their grades, distancing the assessment practice from the theoretical conception of formative assessment.

Learning Assessment; External Assessment; Data Use

Resumo

Analisamos concepções e práticas avaliativas de professores do ensino fundamental objetivando investigar os usos dos resultados das avaliações internas de aprendizagem e das Provas Bimestrais - instrumentos do Sistema de Avaliação da Rede Municipal do Rio de Janeiro – para tomada de decisões referentes ao planejamento e a condução das atividades pedagógicas. Baseamo-nos no conceito de avaliação formativa que pressupõe a avaliação como ferramenta de fomento à aprendizagem e não apenas para a sua medição. Realizamos entrevistas e observação participante em duas escolas e mapeamos contradições entre os discursos e as práticas, uma vez que as concepções das professoras se aproximavam da avaliação formativa, com foco na defesa da diversificação de instrumentos avaliativos. Contudo, o uso de provas era majoritário na prática docente e, frequentemente, eram elaboradas de forma semelhante às Provas Bimestrais. Os resultados das avaliações eram direcionados basicamente para dar feedback superficial aos alunos e calcular as suas notas, distanciando a prática avaliativa adotada das concepções teóricas de avaliação formativa.

Avaliação da Aprendizagem; Avaliação Externa; Uso de Dados

Resumen

Analizamos las concepciones y prácticas evaluativas de los profesores de primaria, con el objetivo de investigar los usos de los resultados de las evaluaciones de aprendizaje en las escuelas y los resultados de las Provas Bimestrais (Exámenes Bimestrales) - instrumentos del Sistema de Evaluación del Sistema Educativo Municipal de Rio de Janeiro - para planificar y tomar decisiones pedagógicas. Nos basamos en las teorías de la evaluación formativa, que indicaban la evaluación como una herramienta para mejorar el aprendizaje y no solo para medirlo. Realizamos entrevistas y observación participante en dos escuelas, describiendo y analizando las contradicciones entre el discurso y las prácticas. Las concepciones de los docentes se acercaron a la evaluación formativa, enfocándose en la defensa de la diversificación de los instrumentos de evaluación, pero principalmente realizan exámenes tradicionales y, con frecuencia, trabajan de manera similar a las Provas Bimestrais (Exámenes Bimestrales). Los resultados de ambas evaluaciones fueron básicamente para dar una retroalimentación superficial a los estudiantes y calcular sus calificaciones, alejando la práctica evaluativa de la concepción teórica de la evaluación formativa.

Evaluación del Aprendizaje; Evaluación Externa; Uso de Datos

1 Introduction

The external assessment systems in Brazil have been expanded since the 1990s, as much at the federal level as at the state and municipal levels. Bauer et al. (2015) found that, as of 2005, there has been a significant increase in the number of municipalities, including Rio de Janeiro, that have instituted their own assessments.

The implementation of the external assessment system in Rio’s municipal network began in 2007 with Prova Rio (Rio Exam)¹, instituted by Decree No. 28878 (RIO DE JANEIRO, 2007) and applied annually until 2016. In 2009, the Rio de Janeiro Educational Development Index (IDE-Rio) was created, calculated based on the results of the Rio Exam and the school progression rate. There was also an accountability policy in this network, called the Prêmio Anual de Desempenho² (Annual Performance Award), which was based on the IDE-Rio and the Ideb (Basic Education Development Index). It paid a 14th month salary to professionals in the schools that achieved the goals.

The Rio de Janeiro Municipal Education Secretariat (SME), through Resolution 1010/2009, also established that students were to be assessed by Provas Bimestrais (Bimonthly Exams)³, devised and standardized by teachers of each grade, invited by the Secretariat. The content assessed was directly linked to the Study Booklets for Mathematics, Portuguese and Science, distributed to schools for mandatory use. The proposal was that the Study Material would guarantee the teaching of minimal content in the grades assessed, and that the Bimonthly Exams would measure how much students had learned from this content at the right age.

Although they were conceived outside the schools, these exams were applied and marked by the teachers designated, giving them “hybrid” characteristics of external and internal assessment, with the possibility of being used within a formative assessment perspective. The Bimonthly Exams were not described in the legal text as being directly linked to the SME’s accountability policy, but studies showed that they were used as training tools for the Rio Exam, since the prize offered depended on students’ performance in these exams (ALMEIDA, 2020). Therefore, it is possible to suppose that, in the context of practice, there were articulation and multiple influences among the Study Booklets, the Bimonthly Exams, the Rio Exam and the Annual Performance Award.

The learning assessments devised by the teachers coexisted with external assessments from the federal government, the municipal network, as well as the Bimonthly Exams. In this context, through Resolution No. 1123/2011 (RIO DE JANEIRO, 2011), the SME also established guidelines for assessment of school learning in the network. Schools needed to assign a bimonthly overall score to each student, which should consider “teachers’ assessments, results in the Bimonthly Exams and the formative aspect of student development” (p. 1, Art. 2, 1st paragraph), presupposing the monitoring of student learning throughout the entire process. Thus, we can notice that the resolution provided in its text the formal summative character together with the formative perspective of the assessment, as it allowed combination of internal assessment scores with the Bimonthly Exam scores to compose the students’ overall scores.

The presence of internal, external and “hybrid” assessments in the same educational network raises questions, such as those addressed in this article. We chose to analyze how the teachers conceived and practiced the assessment of student learning against the backdrop of the assessment policies adopted by the municipality. Our argument is that this multi-assessment scenario poses specific pedagogical challenges for teachers and school administrators, and, as we will demonstrate, it produces a context of coexistence between summative and formative assessment proposals, which takes on specific configurations in each classroom⁴.

2 Learning assessment and the concept of formative assessment

Among the modalities of learning assessment – diagnostic, formative and summative – it is the formative that presupposes the monitoring of students’ learning throughout the process, enabling its improvement. For Fernandes (2008), formative assessment regulates learning through the interpretation of results in the classroom context, aiming at immediate local action, taking into account the individual evolution of the student and that of the entire class. The summative assessment, on the other hand, presupposes interpretation of the results based on general criteria, the same for all students, taking into account final classifications. According to the author, formative and summative assessments are often considered dichotomous, although they complement each other. On the other hand, there is a set of hybrid assessment practices that incorporate characteristics of both assessments.

Our understanding of learning assessment is as an instrument that can serve democratization and school and social discrimination based on the form, content, and criteria for correcting the assessment and the use of its results. Therefore, it has a close connection with the school’s social role. However, Luckesi (2011), Fernandes (2014), Sousa (2014) and Villas Boas and Dias (2015) emphasized that the Brazilian experiences of assessment have been marked by a selective, exclusionist classification perspective, where the naturalized culture of exam realization directs the social role of the school to certify a pass or fail, requiring students to memorize content. In this context, the assessment investigates knowledge in a bureaucratic, superficial way, aiming at a final result and not effective, procedural learning.

Some authors believe that the use of assessment results can contribute to improving learning by monitoring student progress through replanning of pedagogical activities and Teaching methods, which also contributes to improvement in teaching (BLACK; WILLIAM, 2001; DEPRESBITERES; TAVARES, 2009; FERNANDES, 2008; GREENSTEIN, 2010; LUCKESI, 2011; SOUSA, 2014). In this sense, we understand that internal, external and hybrid assessments can contribute to the organization and development of pedagogical work, as long as they are understood in terms of their potential and limits. Together, they can configure diagnostic, formative or summative models, the configuration of which depends on the interpretations and actions of the school administrators and teachers.

3 The SME-RJ Bimonthly Exams

The Bimonthly Exams consisted of multiple-choice questions with an answer card. The answers were typed by professionals from each school and processed in a computerized system. Thus, information on student and class learning was generated, such as: the number of right answers and errors per student and the level of hit/miss for each question, enabling the monitoring of learning, identification of difficulties, as well as the administrative and pedagogical uses of these results.

As the Bimonthly Exams contain characteristics of learning assessment and external assessment, we found that they were close to the hybrid assessment model proposed by Fernandes (2008). A hybrid assessment instrument can take on the role of both summative assessment (of learning) and formative assessment (of learning), according to the use teachers and schools make of its results.

The use of Bimonthly Exam results in a formative perspective, together with the assessments prepared by teachers, can contribute to learning. From this perspective, the diversification of assessment instruments, the analysis of their results, the practice of feedback to students, and teacher replanning, when necessary, are fundamental.

4 Data Use in Formative Assessment

The results of the assessments constitute information on the activities performed by the students, such as exams, standardized or not, exercises and individual or group work, generating a set of data, which, when systematized, helps to break away from the classification perspective of assessment and thinking about curricular, inclusion and learning improvement strategies (FERNANDES, 2014; MARSH, 2012; MERTLER, 2014; SOUSA, 2014).

The data can be used from a formative perspective, when it favors learning and teaching, and is summative if used only for recording, classification and dissemination (SANTOS, 2016). Thus, “the type of use interferes with the social role played by the school as a socializer of systematized knowledge, and responsible for student learning” (ALMEIDA, 2020, p. 101).

The review of national and international surveys carried out by Cerdeira et al. (2017) demonstrated an absence of culture using assessment data and educational indicators for pedagogical planning, and difficulty in accessing data related to external assessments. For the authors, there are communication problems among different school administrative levels and the teaching staff, denoting a lack of objective guidance on the pedagogical possibilities of such data, as it was found that they were misused, applying gaming strategies to superficially improve performance in exams: manipulation of results, exam practice, curriculum limitation and inflexibility. According to the authors, in order to minimize these problems, in the USA, there are initiatives for training administrators and teachers to use data (data literacy). However, currently, in Brazil, such initiatives are incipient.

In dialogue with the literature on the use of educational data, we intended to investigate whether the results from the Bimonthly Exams, as well as from the assessments prepared by the teachers, were analyzed and used in an articulated manner and from the perspective of formative assessment.

In our analysis, we use the Greenstein cycle (GREENSTIEN, 2010) as a guiding element. In this proposal it is understood that teaching and assessment are continuous improvement processes in which feedback is an essential element that provides self-knowledge and motivation to students (Figure 1).

Figure 1
Formative Assessment Cycle

Source: Greenstein (2010, p. 4)

The cycle begins with definition of the objectives, followed by teaching guided by these. The application of assessment tools and analysis of their results serves both to verify the achievement of objectives and for pedagogical replanning. In this sense, both the results of the internal assessments and the Bimonthly Exams can be analyzed by the teachers (data collection and analytical steps, respectively). However, right at the beginning of the field work, we realized that, although theoretically possible and desirable, in the schools under study, we did not find this type of action. We observed a gap between teachers’ sense of belonging to theory and their actions in practice. In their theoretical affiliations, some interviewees corroborated the Greenstein cycle. Nevertheless, in terms of practical actions, all of them distanced themselves from it, even those that, in theory, legitimized it.

5 Research Methodology

We chose to research 4th grade (age range 9 - 10) Elementary School classes, as there was no application of external assessments at national or municipal level in 2018. The possible influence on teacher assessment practices was presumed to be mainly due to the Bimonthly Exams. We chose a Regional Education Coordination department (CRE) in the north of the city with a diverse socioeconomic level. We conducted interviewed four teachers in in two schools, and made observations⁵ of them and the administrators in order to map their conceptions about their own assessment practices, as well as the SME-RJ assessments. It is important to clarify that the inferences made here are based exclusively on the data built in the course of the fieldwork, and they refer solely to the two schools analyzed. However, we understood that our analyses allowed, albeit at a hypothetical level, reflection on the dynamics existing between external and internal assessments in the Rio de Janeiro municipal network. Such relationships – between cases and research universes – have already been exhaustively discussed by Fonseca (1999). It is not a matter of arguing about the whole based on one case, but of thinking about it.

The two schools had the same ICG (Administrative Complexity Index)⁶, similarities in the IDE-Rio and differences in the Ideb. The interviews carried out revealed diversity in behavior, and opposition to guiding the teaching work. The School A teachers⁷ reported work with predominantly summative characteristics, while those from School B reported work with formative ones.

The observation of the teachers’ activities in this pair of schools intended to expand knowledge in relation to pedagogical practices, especially on assessments, by revealing how the actions reported in the interviews took place in day-to-day practice. The observation was not intended to explain the indices, but to try to answer the research questions in two environments that contrasted in their way of working, thus allowing comparison of the elements present.

6 Results and discussion

The tensions between formative and summative dimensions in the teachers’ discourses and practices are presented in the light of this research’s theoretical perspective, divided into four categories.

6.1 Assessment concepts and Teaching practices

In the interviews, the teachers reported their conceptions of assessment, which they considered to be the most appropriate way to assess, approaching formative assessment according to our theoretical framework. The four teachers defended diversification of assessment instruments and the use of observation to regulate student learning, thus aiding them throughout the process. According to Depresbiteris and Tavares (2009), observation is the basis of formative assessment, and, together with the diversity of instruments, allows teachers to diagnose students’ situations.

[...] the assessment is carried out on a daily basis, right? [...] You can already see if the child has learned or not, if they have understood, if they have any doubts, a problem [...] I think that observation sometimes works very well (Teacher Vitória – School B).

We can assume that such conceptions are influenced by their initial and continued formations. However, our analysis points out that the practices differ from the statements presented. We observed that the teachers adopted different practices based on their prior conceptions and the existence of the Bimonthly Exams, which proved to be central in their work. The teachers were in favor of the Bimonthly Exams, although they had reservations, such as limitations in the verification of students’ learning and the possibility of being right by luck (guessing) because they are multiple-choice questions. They stated that other types of assessment instruments “make the child think”.

Regarding the assessment of the content taught, we observed that Schools A and B worked differently. School A carried out fortnightly simulations prepared by the teachers, similar to the external assessments and Bimonthly Exams, for the subjects, mathematics and Portuguese. School B, on the other hand, did not guide the teachers as to how to assess the students. This difference between schools was fundamental in choosing them for observation.

Teacher Marta explained that the adoption of fortnightly simulations by School A was a very good project for the students because they often had problems with the external assessment cards: “Sometimes, they knew the correct answers to the questions, but marked the wrong ones, and our own simulation corroborated this. So, this helped the students a lot”. She complemented arguing that the difficulty level of the simulation would be more suitable for the 4th grade than the Bimonthly Exams, considered to be “easier”. Teacher Graça, on the other hand, reported adding “surprise exams”, as well as the bimonthly and simulated exams, without specifying whether there was any similarity with the others. For this school’s administrator, who was also observed during the research, the simulations kept the students studying and helped to raise the Ideb, reminding us of the training students for the exams (MORAIS, 2012).

In School B, when pointing out the limitations of the Bimonthly Exams, teacher Vitória informed that she varied the assessment instruments in order to allow a more reliable result.

In contrast, teacher Alexandra, from the same school, said that, until 2017, she had based her assessment on observation and Bimonthly Exams. However, she intended to use other types of assessment in 2018 as a way of verifying content not covered in the SME assessment, but worked on by her. She highlighted the children’s learning difficulties, relating them to the social environment, as well as the concern to complement the contents of the Study Booklets.

In previous years, I didn’t make [my own assessment] [...] This week I was going to make my own assessment based on the specific subjects I had taught to help those who had got a bad grade. It was because some students really have a lot of difficulty [...]. For example, the Portuguese study booklet only has text interpretation. It has no spelling, no grammar (Teacher Alexandra – School B).

Both the issue of learning difficulties due to the socioeconomic level of the students, and the need to complement the content of the Study Booklets, are recurrent points made by the teachers interviewed. However, when teaching “extra” content, such as grammar, for example, teachers Alexandra, Marta and Graça also assessed them through exams and simulations with multiple-choice questions. This practice denotes a lack of diversification of instruments and approximation with the model of Bimonthly Exams, highlighting the contradiction between the conceptions and teaching practices.

I think 4 years ago the school returned to this [simulated] project, which is very good for the students [...]. Also, because in the Brazil Exam or the Rio Exam, they got a lot wrong. Sometimes, they knew the correct answers to the questions, but marked them wrongly, and our simulation had the same feedback (Teacher Marta).

As they are mandatory, but also linked to the Bimonthly Exams, the teachers predominantly used the Study Booklets to teach the class content. However, they had autonomy to use textbooks and other teaching materials, in addition to the inclusion of mathematics, Portuguese, history and geography, that is, content not covered in the Study Booklets.

We are not obliged to work with the book [...]. The Study Booklets is so small it doesn’t cover. For example, Portuguese is just text interpretation (Teacher Graça – School A).

Here, the teachers’ freedom to plan is highlighted. We noticed the inclusion of Portuguese language content beyond what was demanded in the Study Booklets.

Such an insertion was due to the criticism that this study material focused mainly on text interpretation, due to the fact that external assessments only assess this dimension of the language, causing curriculum inflexibility (CERDEIRA et al., 2017; FREITAS, 2011). In this aspect, the teachers seem to be guided by their formative conceptions of teaching and assessment (LUCKESI, 2011).

At the same time that they expanded the content during classes, they coached them in the periods preceding the Bimonthly Exams, making the relationship between the Study Booklets, the classes and the Bimonthly Exams evident when the teachers revised the content. The term “revision” was used by the four teachers for the reinforcement activity with the students before the exams. To determine what content needed to be revised, the teachers checked the Bimonthly Exam questions as soon as they arrived at the schools.

At School B, the two teachers based their revision on the Bimonthly Exams after their arrival at the school. Teacher Vitória’s justification was that some content taught in the two-month period was not subjected to examination, there being a need to “check the content to be assessed in order to set up the revision.” As for Teacher Alexandra’s revision, it was the same as the exam that arrived from SME-RJ.

The exam arrives. I take the questions, modify them, the numbering [...] A revision. [...] You have to tell them what is going to come up. Not giving the answer, but, by showing them [...] the marks improved considerably. [...] When I started doing these revisions, I didn’t do the same, but I realized that the time was right. There were still those who made mistakes. I said: [...] I’ll do the same thing to see (Teacher Alexandra - School B).

In the Figures 2 and 3, we can see the Bimonthly Exam questions that had not yet been applied and the revision exercises proposed by the teachers.

Figure 2
Revision by Teacher Alexandra (School B)

Source: Survey data (2018)

Figure 3
Teacher Vitória’s Revision (School B)

By stating that giving the identical exam content to the students was the way she had found to reach the goal, teacher Alexandra transformed the moment of content revision into exam training, to the detriment of learning. During the interview, the teacher had already stated that she did not prepare her own assessments, using only those from the Secretariat, and that she thought the training for external exams was valid.

Again, we noticed a contradiction between discourses/conceptions and practices/actions, moving away from the formative perspective of assessment and approaching the summative concept (VILLAS BOAS; DIAS, 2015). In this case, there was investment in students’ memorization as a way of encouraging correct answers to the questions and the elevation of marks in the Bimonthly Exams, which, in turn, could train the students for external assessments that boost indices and generate prizes. These exercises were carried out on the eve of the exams, or, at most, two days before.

In School A, teacher Marta also waited for the Bimonthly Exams to arrive, but she formulated dissertative questions to revise content. However, the question content was also identical to that of the Exams. On the other hand, teacher Graça did not rely so much on the exams, but on the Study Booklets to carry out the revision.

Although they worked in a different way, we could see that the Bimonthly Exams influenced the way of conducting the revision, either directly by checking the exam before the revision, or indirectly when the revision was based on the Study Booklets.

In summary, for the teachers, the Bimonthly Exams were not sufficient to assess the students’ learning, as they were only made up of multiple-choice questions with an “easy” difficulty level for the 4th year. It is important to point out that they were not criticizing the content itself, but the questions created to assess it. At the same time, they did not have very different practice. Of the four, only teacher Vitória (School B) varied the way she assessed her students, one approaching the formative perspective of assessment. In the others’ practice, the perspective of such diversification, both in relation to the assessment instrument used and the periodicity, rarely occurred, distancing themselves from their own conceptions and from the theoretical assumptions of formative assessment. Despite the concern with the students’ social inclusion, most teachers prepared traditional exams, similar to the bimonthlies (multiple-choice), in order to train them and improve their marks.

6.2 Using the results of internal learning assessments

Although formative assessment is presented in educational literature as an ideal type of assessment for improving student learning, and, consequently, for the role played by the school, the presence of a classifying, certifying and naturalized perspective through exams still permeates daily life in Brazil (FERNANDES, 2014; LUCKESI, 2011; SOUSA, 2014; VILLAS BOAS; DIAS, 2015). This dichotomy between theory and practice highlighted by the authors could be verified during the interviews and in the observation period in the two schools.

In the interviews, the teachers reported using the results of the assessments, both from the Bimonthly Exams and from the internal assessments, to compose the students’ average and to give feedback through the collective marking of the exams. They stated that they returned to the contents of the questions with the highest percentage of error, although they highlighted that this practice was limited by the mandatory use of Study Booklets and the limited time available.

I correct along with them. They see where they went wrong, where they got it right (Teacher Alexandra – School B).

I go back to the contents because, in fact, you end up measuring once again what they grasped and what they didn’t (Teacher Graça – School A).

However, we observed that the teachers followed the sequence proposed by the Study Booklets, adjusted to the weekly planning of subjects made by themselves, leaving little space to return to content taught during class time. The return to the content took place mainly through the correction of answers to exercise questions, exams or simulations, collectively on the blackboard, explaining the content and answering students’ questions. This was the way reported and implemented by the teachers to give feedback to students, emphasizing the questions most got wrong so as to reinforce the most difficult content.

Of the four teachers, only teacher Vitória used to call her students to the blackboard so that they could solve the questions. For those who had some difficulty, she helped with the solution. She reported that moments like this supported her assessment and adjustments to the students’ overall performance score.

Reports on the identification of students’ knowledge gaps, and returning to content with different teaching strategies, date back to the formative assessment modality. This presupposes new planning based on the information collected and a change in teaching strategies by the teacher (DATNOW; HUBBARD, 2015; GREENSTEIN, 2010). However, during the observations, the results of the internal assessments, including those from simulations in School A, were not systematically analyzed by the teachers and did not provide for replanning of activities. The contents were quickly and superficially revised with the students, ensuring continuation of the planning.

6.3 Use of the Bimonthly Exam results

Typing the Bimonthly Exam answers into the computerized system enabled generation of reports on students’ mistakes and successes per question, subject and class. We argue that such information could be used in a pedagogical manner, since these exams provide not only collective results for the Education system, but also collective and individual student results for teachers, which gives the Bimonthly Exams a hybrid character, given the possibility of being worked on from the formative assessment perspective.

Although the teachers revealed that they knew the reports and statistics were provided by the system, according to their statements during the observations, it was only possible to verify the feedback of these data in the case of teacher Vitória (School B), although she did not analyze the information in the reports. The use of results for content revision did not differ among them.

CRE’s returns take time, but when we type in the system the percentage appears on the side. We can see it, but then CRE sends it. […] We make a revision of what was not achieved (Teacher Marta – School A).

Now, even a chart is coming, displaying the questions on which they make more or less mistakes. It comes through the system (Teacher Vitória – School B).

Despite marking the Bimonthly Exams and obtaining results after their application, the use of these results was also incipient in terms of performance feedback to students, and in the intervention and regulation of learning during the process (FERNANDES, 2008, 2014).

Individual results could be used to create mechanisms to help students, while the results per class, derived from the consolidated statistics of the answers, would support improvement of the pedagogical work (DATNOW; HUBBARD, 2015; GREENSTEIN, 2010). It is our understanding that these practices alone do not guarantee student learning and their inclusion, but they have the potential to favor it, above all for those facing difficulties, and so most need the school’s pedagogical work.

Comparing the practice of the two schools with the Greenstein cycle (GREENSTEIN, 2010), we observed that there was no school administration role in guiding the use of results, mainly with regard to data analysis and replanning of teaching actions, evidence already pointed out by Cerdeira et al. (2017). As the teaching objectives were already outlined by the SME-RJ and were conducted through Study Booklets, the data analysis stages for replanning would lead to a teaching protagonism towards learning.

6.4 Attribution of the overall score and student recovery

The obligatory nature of remedial studies, preferably parallel to the academic period, is provided for in Article 24 of the Law of Guidelines and Bases of National Education –, as well as in Article 3 of Resolution No. 1123 of the SME-RJ (RIO DE JANEIRO, 2011), which presents the way students conceptualize their learning, called the overall score. Although the resolution presents a formative perspective on student assessment and achievement, it also provides the criteria for their classification, indicating that the classification culture remains by means of scores.

When following the composition of the students’ overall score, we observed that the teachers established the score according to their personal perceptions about the students, their performance and their socioeconomic status. Besides the social aspects, they also considered commitment, participation and discipline, introducing elements into the scoring that were alien to the cognitive aspects of content learning, measured by assessment. We believe that it is possible to assess aspects related to student behavior and attendance, provided that objective, transparent criteria are established for the scoring, in order to compose a comprehensive picture of the student.

We were able to verify that each teacher proceeded differently when attributing the students’ overall score, according to the statements by teachers Graça and Alexandra below. We infer that there is a lack of clarity in the guidelines proposed by Resolution No. 1123, given that the first paragraph defines that teacher assessments, results in Bimonthly Exams and the formative aspect of student development will be considered, while the third paragraph indicates that the Bimonthly Exam marks and the overall score must be taken into account, but it does not define how the relationship among all of these can be operationalized.

I don’t use the mathematical average. I use the average in my mind. I look at everything: behavior because it influences learning. Whether or not he/she does homework, whether he/she is a committed student [...] How am I going to know if he/she is really committed? With the surprise exam mark!? (Teacher Grace – School A).

[...] sometimes, in the exam, they don’t do very well, but they do their homework. I see they know the stuff. So [...] if they get a below average score, I do justice by increasing the mark myself (Teacher Alexandra – School B).

Based on reports of lack of knowledge of Resolution No. 1123 (RIO DE JANEIRO, 2011), and the different procedure adopted by teachers from the same school when composing the overall score, we can assume that there was a lack of guidance from higher administrative levels for the running of schools, and, in turn, such a lack on the part of these for the teachers. Although non-direction can be interpreted as non-fixing the composition of the score by the SME-RJ, respecting its singularities, the coherence of procedures in the same school unit was also not verified, denoting lack of guidance.

The four teachers had a point in common, namely the practice of adjusting the scores after the release of the Bimonthly Exams in the system. As they had already reported during the interviews, the adjustments considered the following aspects:

• Teacher Vitória and Marta - cognitive aspects;

• Teacher Grace - cognitive and behavioral aspects, adjusting the overall score higher or lower, depending on the student; and

• Teacher Alexandra - cognitive and behavioral aspects, adjusting the overall score only to benefit the student, i.e. increasing the mark.

As an example of these adjustments, in the Table 1, we can see, the alteration of two averages performed by teacher Marta at the third bimonthly meeting to assess the students performance (COC). These arithmetic means were not calculated mathematically. Rather, they were entered based on the teacher’s own subjective criteria.

Table 1
Example of the Composition of Teacher Marta’s (School A) Overall Score 3rd COC - Marks and Averages

Portuguese Writing	Portuguese Reading	Mathematics	Sciences	Average (given by Teacher. Marta)	Average - Arithmetic
9.5	8.0	8.7	9.0	7.8	8.8
4.0	8.0	7.3	6.0	5.0	6.3

Source: Research data (2018)

The overall score defined which students would be in parallel recovery. In Schools A and B, the four teachers had students who needed these classes. In School A, it officially took place two days a week after the regular class schedule indicated by the teacher and authorized by the parents/guardians.

Until the second semester, teacher Marta had performed the reinforcement during class, since she had two students she classified as R (regular, “weak”), who had grade I (insufficient). On the other hand, teacher Graça informed that she tried to do a different job with some students – she had three students with grade I, but they “didn’t want to learn” and didn’t show up. Later, she reported she had given up teaching them.

In School B, the reinforcement was performed in the counter shift by two literacy teachers, with the student being indicated by the teacher in charge. Teacher Alexandra informed that one of her students showed significant progress in mathematics and was making improvements in writing as a result of the work done by the school and the family commitment. Despite having R and I students, teacher Vitória indicated only one I student for follow-up with reinforcement classes. Vitória accompanied the other three students during their classes, although we did not observe any differentiated work with them.

At the end of the school year, all of Alexandra, Vitória (School B) and Marta’s (School A) students passed with grade R at least. Of the three students attributed grade I by teacher Graça (School A), one failed. According to these teachers, passing the students followed throughout the school year considered their entire learning path, their performance in the Bimonthly Exams and in the recovery activities, accompanied by the observations of the teachers.

However, we could observe assessment practices in both schools, in part contradictory to the conceptions of formative assessment, guided by the predominant use of exams and exercises, often resembling the Bimonthly Exams and external assessments, little use of the results of internal assessments and Bimonthly Exams, in addition to a subjective construction of the students’ overall scores. Thus, we observed that only one teacher, Vitória, from School B, came closer to the formative concept of assessment.

7 Final Considerations

Despite criticizing the multiple-choice model of the Bimonthly Exams, the teachers partially reproduced this model in their own assessments. In addition, even identifying content limitations in the Bimonthly Exams, they directed the reinforcement activities toward the specific topics that would be examined. We emphasize that, although there was the proposition that the Study Booklets ensured teaching of minimal content and that the Bimonthly Exams assess its compliance, we cannot say that this fact occurred in the classes observed, as the analysis of the contents of the Study Booklets against the matrix curriculum was not the objective of this research.

We observed that, although teachers could work in accordance with the precepts of formative assessment, they did not use the results of learning assessments or Bimonthly Exams in a systematic way for school (re)planning due to socialization in a particular work model, the reduced access to the results of the network assessments, and, mainly, the limited culture of using data for planning. Thus, we evidenced a certain distance between the assessment practices presented and the formative Greenstein (2010) cycle assessment. In this context, the role of teachers in defining goals is minimized since they are defined by the SME-RJ for the entire network.

In summary, this investigation revealed contradictions between the teachers’ conceptions/discourses and their assessment practices. The statements revealed concepts aligned with the perspective of formative assessment, while the practice was outlined as more summative and focused on training for the Bimonthly Exams and external assessments. Assessment results were used sparingly to give students feedback on their learning, and enable them to calculate their grades and overall score.

Considering that the approximation of teaching action from the formative or summative perspective of assessment interferes with the social role played by the school, it seems necessary for the dissemination of information to be improved, and that educational policies should include continuing Education of teachers and administrators for formative assessment, and use its results, as well as those of external assessments, to achieve the objective of consolidating an assessment culture that can both contribute to Teaching work and rethink educational policy.

References

• ALMEIDA, A. B. Avaliação da aprendizagem em contexto de avaliações externas: um estudo sobre as Provas Bimestrais. Tese (Doutorado) – Universidade Federal do Rio de Janeiro, Faculdade de Educação, 2020.
• BAUER, A., et al. Avaliação em larga escala em municípios brasileiros: o que dizem os números? Estudos em Avaliação Educacional, São Paulo, v. 26, n. 62, p. 326-352, maio/ago. 2015. https://doi.org/10.18222/eae266203207
  » https://doi.org/10.18222/eae266203207
• BLACK, P.; WILLIAM, D. Inside the black box: raising standards through classroom assessment. King’s College London School of Education, 2001 [cited 2018 May 21]. Available at https://weaeducation.typepad.co.uk/files/blackbox-1.pdf
  » https://weaeducation.typepad.co.uk/files/blackbox-1.pdf
• BRASIL. Lei nº 9.394, de 20 de dezembro de 1996. Estabelece as diretrizes e bases da educação nacional. Diário Oficial da União, Brasília, DF, 23 dez. 1996.
• CERDEIRA, D. G. S., et al. Conhecimento e uso de indicadores educacionais no Município do Rio de Janeiro. Estudos em Avaliação Educacional, São Paulo, v. 28, n. 69, p. 926-968, set./dez. 2017. https://doi.org/10.18222/eae.v0ix.4104
  » https://doi.org/10.18222/eae.v0ix.4104
• DATNOW, A.; HUBBARD, L. Teachers’ use of assessment data to inform instruction: lessons from the past and prospects for the future. Teachers College Record, New York, v. 117, n. 4, Apr. 2015. https://doi.org/10.1177/016146811511700408
  » https://doi.org/10.1177/016146811511700408
• DEPRESBITERIS, L.; TAVARES, M. R. Diversificar é preciso...: instrumentos e técnicas de avaliação de aprendizagem. São Paulo: Senac São Paulo, 2009.
• FERNANDES, D. Para uma teoria das avaliações no domínio das aprendizagens. Estudos em Avaliação Educacional, São Paulo, v. 19, n. 41, p. 347-369, set./dez. 2008.
• FERNANDES, C. O. Avaliação das aprendizagens: sua relação com o papel social da escola. São Paulo, Cortez, 2014.
• FONSECA, C. Quando cada caso NÃO é um caso: pesquisa etnográfica e educação. Revista Brasileira de Educação, Rio de Janeiro, n. 10, p. 58-78, jan./abr. 1999.
• FREITAS, L.C. Responsabilização, meritocracia e privatização: conseguiremos escapar ao neotecnicismo? In: Seminário de Educação Brasileira, 3. Simpósio PNE – Diretrizes para Avaliação e Regulação da Educação Nacional. Campinas: CEDES, 2011.
• GREENSTEIN, L. A study guide for what teachers really need to know about formative assessment. ASCD, 2010 [cited 2017 June 25]. Available from: http://www.ascd.org/publications/books/ 110017/chapters/The-Fundamentals-of-Formative-Assessment.aspx
  » http://www.ascd.org/publications/books/ 110017/chapters/The-Fundamentals-of-Formative-Assessment.aspx
• INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA – INEP. Nota técnica nº 040, de 17 de dezembro de 2014. Nível de complexidade da gestão escolar. Brasília, DF, 2014. Disponível em: https://nota_tecnica_indicador_escola_complexidade_gestao_adjustment_2015 Acesso em: 08 de maio de 2018.
  » https://nota_tecnica_indicador_escola_complexidade_gestao_adjustment_2015
• LUCKESI, C. C. Avaliação da aprendizagem escolar: estudos e proposições. 22.ed. Sao Paulo: Cortez, 2011
• MARSH, J. A. Interventions promoting educators’ use of data: research insights and gaps. Teachers College Record, New York, v. 114, n. 11, p. 1-48, Nov. 2012. Htttps://doi.org/10.1177/016146811211401106
  » Htttps://doi.org/10.1177/016146811211401106
• MERTLER, C. A. The data-driven classroom: how do I use student data to improve my instruction? Alexandria: ASCD, 2014.
• MORAIS, A. G. Políticas de avaliação da alfabetização: discutindo a Provinha Brasil. Revista Brasileira de Educação, Rio de Janeiro, v. 17 n. 51, p. 551-572, set.-dez. 2012. https://doi.org/10.1590/S1413-24782012000300004
  » https://doi.org/10.1590/S1413-24782012000300004
• RIO DE JANEIRO (Cidade). Decreto nº 28.878 de 14 de dezembro 2007. estabelece diretrizes para a avaliação escolar na rede pública do sistema municipal de ensino da cidade do rio de janeiro e dá outras providências. Diário Oficial do Rio, Rio de Janeiro, 15 dez. 2007
• RIO DE JANEIRO (Cidade). Decreto nº 33.399, de 16 de fevereiro de 2011. Dispõe sobre os critérios de premiação a ser concedida aos servidores na forma que menciona. Diário Oficial do Rio, Rio de Janeiro, 17 fev. 2011.
• RIO DE JANEIRO (Cidade). Resolução nº 1.123, de 24 de janeiro de 2011. Publicado no Diário Oficial do Rio de Janeiro em 25/01/2011.
• SANTOS, L. A articulação entre a avaliação somativa e a formativa, na prática pedagógica: uma impossibilidade ou um desafio? Ensaio: Avaliação e Políticas Públicas em Educação, Rio de Janeiro, v. 24, n. 92, p. 637-669, jul./set. 2016. https://doi.org/10.1590/S0104-40362016000300006
  » https://doi.org/10.1590/S0104-40362016000300006
• SOUSA, S. Z. Avaliação na escola básica: controvérsias e vicissitudes de significados. In: FERNANDES, C. O. (org.). Avaliação das aprendizagens: sua relação com papel social da escola. São Paulo: Cortez, 2014. p. 93-111.
• VILLAS BOAS, B. M. F.; DIAS, E. T. G. Provinha Brasil e avaliação formativa: um diálogo possível? Educar em Revista, Curitiba, n. spe 1, p. 35-53, 2015. http://dx.doi.org/10.1590/0104-4060.41421
  » http://dx.doi.org/10.1590/0104-4060.41421

• 1
  External assessment of elementary education comprising Portuguese and mathematics exams.
• 2
  Decree No. 33399/2011 (RIO DE JANEIRO, 2011)
• 3
  In 2019, the Provas Bimestrais (Bimonthly Exams) became Semestral Exams and the study material became semestral too.
• 4
  Research carried out in 2018, within the scope of the Research Laboratory, with funding from the National Conselho Nacional de Desenvolvimento Científico e Tecnológico – CNPq.
• 5
  The observations lasted about four months.
• 6
  The school management complexity indicator summarizes in a single measure the information on size, operating shifts, level of complexity of the stages and number of stages offered. (Source: INEP, 2014).
• 7
  The names shown are fictitious.

Publication Dates

• Publication in this collection
  14 Mar 2022
• Date of issue
  Oct-Dec 2022

History

• Received
  13 Sept 2021
• Accepted
  17 Feb 2022