SciELO - Scientific Electronic Library Online

 
vol.45Basic teacher training and emotional competences: analysis of curricular content in Brazilian universitiesIs Spinozism a Form of Libertarian Education? author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Educação e Pesquisa

Print version ISSN 1517-9702On-line version ISSN 1678-4634

Educ. Pesqui. vol.45  São Paulo  2019  Epub Apr 18, 2019

https://doi.org/10.1590/s1678-4634201945187778 

SECTION: ARTICLES

Reading skills in the literacy stage (grades 1-3): an analysis on equality of knowledge

1Pontifícia Universidade Católica do Rio de Janeiro – PUC-Rio, Rio de Janeiro, Rio de Janeirio, Brasil. Contatos: naira.muylaert@hotmail.com; alicia@puc-rio.br.

2Universidade Federal de Juiz de Fora – UFJF; Juiz de Fora, Minas Gerais, Brasil. Contato: hilda.micarello@uab.ufjf.br.


Abstract

Considering the concept of equality of knowledge, the present study analyzes the variation in the percentage of correct answers given by 7,781 third-grade students (primary education) in the state of Espírito Santo, Brazil, on items considered difficult in the PAEBES-Alfa cognitive test administered in the 2014. The variation in the rate of correct answers was analyzed for two groups examined: high and low-performance classes. Of the set of skills assessed, five were analyzed for their degree of complexity, considering the linguistic aspects involved in their development. Our results indicate that the correct-answer rate variation between the classes is relatively small, thus suggesting that the acquisition of knowledge is relatively equitable. As for the most complex skills, low correct-answer rates were found for both groups of students. This reveals that most of the students in both high and low-performance classes did not consolidate their learning of more complex skills in the literacy stage (grades 1-3) of Brazilian primary education.

Key words: Reading skills assessment; Equality of knowledge; Educational inequalities

Resumo

À luz do conceito da igualdade de conhecimentos, este estudo analisa a variação do percentual de acertos de 7.781 estudantes do 3º ano do ensino fundamental do estado do Espírito Santo quanto a itens considerados difíceis do teste cognitivo do PAEBES-Alfa, aplicado no ano de 2014. A variação do percentual de acerto aos itens foi analisada em relação a dois grupos que foram avaliados: turmas de alto desempenho e turmas de baixo desempenho. Do conjunto de habilidades avaliadas, cinco foram analisadas quanto ao seu grau de complexidade, considerando os aspectos linguísticos envolvidos no desenvolvimento dessas habilidades. Os resultados deste estudo apontam que a variação do percentual de acerto entre as turmas é relativamente baixa, indicando que a aquisição de conhecimentos se dá de forma mais ou menos equitativa. No que concerne às habilidades mais complexas, observou-se um baixo percentual de acerto nos dois grupos de alunos. Tal resultado revela que a maioria dos alunos, tanto das turmas de baixo quanto das de alto desempenho, não consolidou a aprendizagem de habilidades mais complexas do ciclo de alfabetização.

Palavras-Chave: Avaliação de habilidades de leitura; Igualdade de Conhecimento; Desigualdades Educacionais

Introduction

In Brazil, universal access to primary education and the increasing amount of data on student performance drove the emergence of new studies on the relationship between social and educational inequalities. These studies focus on school knowledge and point to school as a key institution in widening or reducing inequalities between different social groups (SOARES, MAROTTA, 2008; BERNARDO, 2008; BARBOSA, 2011).

Despite the access to basic education has expanded, it remains far from being fully guaranteed at different education levels, therefore, its offer continues to be persistently unequal according to income, race/color and gender differences among students.

With the implementation and consolidation of large-scale education assessment, studies on inequality began to address the issue based on the results of tests that assess student proficiency levels, thus leading analyzes to also cover academic achievement inequality, as well as the school factors that can help explain it. These studies have added new dimensions to the problematization of both the education system and the school as entity levels that can promote social justice (WALTENBERG, 2002; SOARES, MAROTTA, 2008; VALLE, 2010).

Crahay (2002, 2013, 2014) and Dubet (2004, 2004a) resume the discussion on the relationship between school and social justice in the light of the concepts of equal opportunity, equal treatment and equality of knowledge. According to these authors, while the first two concepts are part of classic research on education inequality, the concept of equality of knowledge has been incorporated into analyzes more recently.

As they discuss the fair school, Crahay (2002, 2013, 2014) and Dubet (2004, 2004a) reflect on the limits of the concepts of equal opportunity and equal treatment. According to Dubet (2004), equal opportunity materializes when access to basic education becomes universal, i.e., when children of both the working class and the bourgeoisie began to have access to school education. However, equal access to school, albeit necessary, did not mean that school became more just.

Once universal access to school was ensured, school success and failure came to be justified by student merit, without considering the quality of the education that school provided. Therefore, just as students of all social classes began to enter school, the problem of unequal access was replaced by that of unequal school success (DUBET, 2004; OLIVEIRA, 2007).

According to Dubet (2004, p. 541), the school “did not become fairer because it reduced the difference of positive outcomes between social categories, but rather because it allowed all students to enter the same competition.” Therefore, if the concept of equal opportunity is comprised in universal access to school, the implementation of this concept would not suffice to ensure school justice, because equal opportunity would not be able to encompass equal results (DUBET, 2004a).

On the other hand, according to the concept of equal treatment, schools in the same system must provide the same teaching quality. This implies homogenizing teaching conditions and making the offer of pedagogical resources equivalent for all schools. It therefore implies that educational authorities must organize and manage in an equal manner the quality of opportunities offered to students, making education contents, methods and conditions uniform, homogeneous, equal. In other words, equal treatment would translate into providing the same education for students of different social backgrounds.

From the perspective of equal treatment, once all students are granted the same teaching quality, each student’s success or failure would be legitimized by her/his own merit and by the effort she/he puts into her/his school activities. Thus unequal results would be acceptable and justified by merit.

Both equal opportunity and equal treatment contain premises related to the pursuit of a fair school. According to Crahay (2002, 2013), the limits of this equality, however, become visible in the impossibility to ensure the acquisition of knowledge common to all students. The notions of equal opportunity and equal treatment, while pointing to relevant issues, would therefore be insufficient to guide the materialization of a fair school. The principle of equality of knowledge is presented by Crahay (2002, 2013) as a goal for public policies so as to ensure that Brazilian schools can actually provide equal opportunities for all.

According to Crahay (2002, 2013), equality of knowledge is connected to the definition of corrective or compensatory justice, and it affirms that school should ensure that all students learn the desired knowledge. To that end, according to him, the school should act in order to avoid increasing the socioeconomic inequalities found between students by implementing a pedagogical project organized according to objectives to be achieved by all students. To Crahay (2002, 2013), this means that teaching and learning processes should be established according to basic knowledge – i.e., knowledge that the author defines as fundamental and to be acquired by everybody, regardless of each student’s social class, race/color or gender. Learning basic knowledge is a right of all individuals and it should be realized through effective pedagogical practices (CRAHAY, 2002, 2013).

The present study aims to investigate the notion of equality of knowledge proposed by Crahay, based on proficiency data3 from the Basic Education Assessment Program of the State of Espírito Santo (PAEBES-Alfa) for the third grade of primary education – the school year in which the literacy stage ends. In other words, the study proposes to analyze whether the reading skills measured by PAEBES-Alfa are being acquired by all students – an idea advocated by the concept of equality of knowledge – in the school year that corresponds to the end of the literacy stage in Brazil.

To that end, we analyze the variation in the percentage of correct answers on difficult items of the reading test administered in 2014 for classes with high and low average performances. A total of 7,781 students of 1,168 classes in 349 schools were assessed. Our selection of these data considered some criteria: the education system, the number of classes in each school and the longitudinal design of PAEBES-Alfa. Thus, we selected students from public schools (in both municipal and state education systems), who were enrolled in schools with two or more classes and had proficiency data for the four PAEBES-Alfa assessment rounds, as follows: first-grade entry assessment, in 2012; first-grade exit assessment, in 2012; second-grade exit assessment, in 2013; and third-grade exit assessment, in 2014.

The correct-answer rate variation was also pedagogically analyzed in items that assessed five specific skills, namely: D15 - Finding explicit information in longer texts or in texts presenting data; D17 - Inferring information from texts; D18 - Identifying the subject of texts; D13 - Identifying various genres; and D14 - Recognizing the purpose of various genres. These reading skills, among others, are described in the PAEBES-Alfa Reference Model4 – a document that presents the competences and descriptors that cognitive tests should assess.

According to the literature, although some reading skills are more elementary than others, they all combine to form the stages of textual processing, and their acquisition is fundamental to consolidate the reading dimension of literacy.

Finding explicit information refers to an elementary dimension of the reading process, and it mobilizes knowledge about the elements found on the text’s surface (KOCH, 2016). As for the ability to infer information from a text, it concerns the semantic dimension of the reading competence and encourages building new representations of this nature, i.e., building new meanings from information that is explicit in the text. Identifying the subject of the text, for example, is an inferential process as the reader must consider the different information contained in the text to synthesize its contents. In turn, the ability to identify various genres and recognize their purposes refers to the social uses of reading in various communicative situations, since different genres are related with particular social practices.

The main goal of the present article is to examine how the learning of these skills is distributed among the students assessed by PAEBES-Alpha by the end of the literacy stage.

Apart from this introduction, the present work is organized in three sections. The next section describes the study’s methodological procedures. Then, the main pedagogical analyzes of the statistical results are discussed, and the final considerations summarize the findings of the study.

Methodology

One possible way of examining the knowledge acquired by students and how that knowledge is distributed is to analyze both the percentage of correct answers on items of tests of proficiency levels and the variation of that percentage. In this study, we start from three assumptions: i) an item with a high correct-answer rate indicates that the skill assessed has been acquired by and is consolidated in the majority of students; ii) an item with a low correct-answer rate indicates that few students have acquired the skill assessed; and iii) if the variation of this rate between classes is high within a school, then students’ acquisition of the skill is unequal, and if the variation is small, then students’ learning of the skill is more homogeneous or egalitarian.

In analyzing correct-answer rates on items of an educational assessment, a problem that arises is the instability of means, since these depend directly on students’ performance. Two situations are possible here. One in which a given item has a mean correct-answer rate of nearly 100%, thus revealing a state of learning consolidation by all students. And another, in which the mean correct-answer rate for an item is 0 or close to 0, indicating that the skill measured may have been inadequately addressed by the teacher, or even not been addressed at all in the classroom.

In PAEBES-Alfa, we treated this instability using b parameter, which, in Item Response Theory5 (IRT), indicates the item’s degree of difficulty. The same skill can be measured by an easy and a difficult item, and the percentage of correct answers to each of these will reveal the degree of complexity that this skill can present, as well as its development by the students assessed.

In all, the 2014 PAEBES-Alfa administered 80 reading items. These items assess the 20 skills presented in the Reference Model descriptors. Each item has a particular b parameter, which indicates the item’s degree of complexity. Thus, some items assess the same skill, but have different b parameters, indicating that several items can assess the same skill at different degrees of difficulty.

Considering the limits of this article, we propose to analyze the percentage of correct answers to items considered difficult, i.e., the ones whose IRT b parameter is considered high, since they represent the assessed skills at their most complex and indicate whether such skills were satisfactorily acquired.

In order to select the difficult items, we divided the item difficulty scale6 (b parameter) in four parts and classified the items into 3 difficulty levels: high (4th quartile), medium (2nd and 3rd quartiles) and low (1st quartile). The items considered difficult were the ones in quartile 4, i.e., those with a b parameter higher than 715.1. Table 1 shows the quartiles of b parameter measure.

Table 1 Division of B Parameter in Quartiles 

Degree of difficulty Quartile B Parameter Scale
Low Quartile 1 From 293.89 to 540.09
Medium Quartile 2 540.10 to 624.21
Quartile 3 624.22 to 715.08
High Quartile 4 Above 715. 09

Source: Prepared by the authors based on data from PAEBES-Alfa.

Before analyzing the correct answer rates, two statistical procedures were performed. The first was correcting for accidentally correct answers,7 and the second was estimating the rates of achievement controlled by the students’ social and demographic characteristics more associated with school performance, such as student socioeconomic status, color/race, and gender (CARVALHO, 2003; SOARES, ALVES, 2013; XAVIER, ALVES, 2015). Using these variables, we built a linear hierarchical model with three levels: student, class and school. The correct-answer rate estimation for each item was corrected by accidentally correct answers and controlled by the social variables (socioeconomic index (ISE), color/race and sex) at the student level, while at the class and school levels, the model is unconditional (i.e., without explanatory variables).

Chart 1 shows which variables had statistical significance in the social control of the correct-answer rate estimation. The items in bold are the ones considered difficult according to b parameter, which were analyzed in this work.

Chart 1 Variables that formed the regression models for the control of correct-answer rates by social characteristics 

Variables with statistical significance Reading Items of Third Grade (Primary Education)
Socioeconomic Index (ISE) P1; P4; P6; P9; P10; P20; P22; P23; P30; P32; P34; P38; P3; P46; P50; P55; P57; P59; P64; P65; P66; P68; P70; P72; P76; P78; P79
ISE, Race, Sex P2; P12; P25; P26; P41; P43; P48; P67; P71; P73
ISE, Sex P3; P5; P14; P17; P27; P28; P29; P31; P37; P45; P47; P49; P56; P69; P77; P80
ISE, Race, Sex P7; P8; P21; P24; P40; P44; P52; P62
Race P16
Sex P11; P15; P18; P33; P35; P58; P75
Race, Sex P13; P19
No significance P36; P32; P51; P53; P54; P60; P61; P63; P74

Source: Prepared by the authors based on data from PAEBES-Alfa.

Below we present the variation in the rate of correct answers to the items considered difficult (those in quartile 4) for the 3rd grade of primary education and discuss its pedagogical meaning.

Results

In order to check the equality of knowledge between the classes, their mean performance was divided in tertiles – upper, middle and lower. The lower tertile refers to low-proficiency classes, the middle tertile to medium-proficiency classes, and the upper tertile to high-proficiency classes. In this study, the data for high and low mean proficiency classes were analyzed in order to find whether the complex skills were acquired in a homogeneous (egalitarian) or heterogeneous (unequal) way between both groups of classes in the same schools.

The analysis consisted of finding the percentage of correct answers on difficult items and its variation according to the classes’ level of performance, and observing the skills assessed by those items. On the one hand, we consider that a high correct-answer rate for an item shows that the assessed skill is mastered by nearly all students, and that this result evidences the effectiveness of the school’s work in ensuring the learning of the skill. On the other hand, we consider that an item with a low correct-answer rate indicates that few students have acquired the skill assessed and, therefore, the school likely did not address, or addressed improperly or insufficiently that skill.

In terms of equality of knowledge – a concept that guides this study –, items with a high correct-answer rate variation between the classes show that the skill assessed was addressed differently between the groups of students. The same occurs with the analysis of items with a high correct-answer rate variation, but in which the mean rate is low for both high and low-performance groups.

Analyzing the differences in the rate of correct answers to items that evaluate certain reading skills between these two groups of classes can help understanding which skills were the object of greater or less equality in the distribution of that knowledge between the classes assessed in each school .

Although the PAEBES-Alfa Reference Model for Portuguese expresses only a limited portion of the curriculum in terms of what can be measured through the standardized tests typically used in large-scale assessments, it presents important elements related to the skills and competences to be developed in the literacy stage of primary education. This feature allows analyzing the percentage of correct answers to the items and, therefore, identifying the emphasis given to different and relevant skills in the teaching and learning process involved in the literacy stage.

Table 2 shows the items considered difficult for the 3rd grade of primary education, the rate of correct answers to these items for both high and low-performance classes, the variation of this rate, and the skill assessed by the item.

Chart 2 – Correct-answer rates and their variation for difficult items by level of performance – 3rd grade of primary education 

Item Performance Level Correct-Answer Rate Rate Difference between Both Groups Skill
64 Low 11.69 14.39 Identifying the number of syllables in a word (syllable awareness).
High 26.09
72 Low 47.45 14.18 Establishing thematic continuity relationships based on retrieving elements from the text’s reference chain.
High 61.63
51 Low 38.34 12.65 Identifying a text’s subject.
High 50.99
52 Low 45.62 12.25 Recognizing the purpose of diverse genres.
High 57.87
79 Low 33.82 12.10 Inferring information implicit in texts.
High 45.92
62 Low 56.42 11.21 Identifying the effect of meaning arising from the use of graphical resources, punctuation, lexical selection and repetition.
High 67.63
70 Low 44.63 9.28 Identifying humor effects in diverse texts.
High 53.91
65 Low 9.07 9.17 Identifying linguistic marks that evidence the enunciator in direct or indirect speech.
High 18.23
32 Low 66.19 8.71 Identifying diverse genres.
High 74.90
48 Low 52.18 8.62 Identifying the effect of meaning arising from the use of graphical resources, punctuation, lexical selection and repetition.
High 60.80
29 Low 69.69 8.53 Identifying the effect of meaning arising from the use of graphical resources, punctuation, lexical selection and repetition.
High 78.21
63 Low 23.41 8.30 Recognizing the purpose of diverse genres.
High 31.71
25 Low 72.81 7.83 Identifying a text’s subject.
High 80.63
45 Low 63.42 7.32 Establishing logical, discursive relationships existing in the text.
High 70.74
57 Low 56.42 6.88 Finding explicit information in longer texts or in texts presenting data.
High 63.30
71 Low 81.24 6.23 Identifying diverse genres.
High 87.48
78 Low 48.95 5.81 Identifying humor effects in diverse texts.
High 54.76
30 Low 59.49 5.60 Finding explicit information in longer texts or in texts presenting data.
High 65.08
60 Low 54.90 5.50 Inferring information implicit in texts.
High 60.40
66 Low 76.00 5.37 Finding explicit information in longer texts or in texts presenting data
High 81.37
47 Low 83.42 4.87 Establishing logical, discursive relationships existing in the text.
High 88.28
77 Low 76.31 3.74 Recognizing the purpose of diverse genres.
High 80.05
56 Low 68.48 3.40 Establishing logical, discursive relationships existing in the text.
High 71.88
36 Low 38.46 2.94 Finding explicit information in longer texts or in texts presenting data.
High 41.40

Source: Prepared by the authors based on data from PAEBES-Alfa.

Given the breadth of the Reference Model and the limits of this article, our analysis considers five descriptors: two of them describe skills that, from a linguistic perspective, may be considered less complex; the other three descriptors refer to more complex skills, for reasons we will explore later.

Firstly, we will reflect on the nature of these reading skills, the different difficulty levels at which they can be assessed, and their role in educating a proficient reader. Then, we will analyze the different correct-answer levels observed in both groups of classes and the items that evaluate these skills.

Skills Related to Explicit Information in the Text and in Making Inferences

The first skill to be analyzed is the one described in the Reference Model under code D15 – Finding explicit information in longer texts or in texts presenting data. Finding explicit information refers to a rather elementary dimension of the reading process: identifying elements which are on the text’s surface, i.e., information whose identification does not depend on making more complex inferences. This is, therefore, a rather elementary and localized dimension of text processing, since reading with comprehension requires a series of inferences based on which the reader produces meanings for what she/he reads.

Based on Heinemann and Viehweger (1991), Koch (2016) says that three types of knowledge operate in text processing: linguistic, encyclopedic and interactional knowledge. Linguistic knowledge is responsible:

[...] for the organization of linguistic material on the text’s surface, for the use of cohesive means provided by the language for referral or text sequencing, for the lexical selection appropriate to the subject and/or to the cognitive models mobilized. (KOCH, 2016, p. 32).

Encyclopedic knowledge refers to the reader’s experience of the world, which is mobilized in making predictions about the content of the text, as well as in filling any gaps (things left unsaid by the text, which the reader knows thanks to previous experience with the genre, subject, etc.). Interactional knowledge concerns recognizing the communicative situation in which the text materializes, i.e., its purpose in a given social situation.

The ability to find explicit information in texts mobilizes chiefly linguistic knowledge (KOCH, 2016), since in order to identify some particular information explicitly marked in the text, it is necessary, primarily, to decode the writing so as to find the requested information. It is, therefore, a rather specific and elementary skill in the broader set of skills required in processing a text. The levels of complexity this skill may involve will depend on factors such as the length of the text, the vocabulary used, the amount of information it presents and the position of the particular information being sought within the text. Information appearing early on and with some degree of emphasis is normally easier to find than when it is situated in the middle of the text as this requires further reading, thus posing a major challenge for readers still in the process of acquiring literacy, for whom the decoding of writing takes up a good part of the reading efforts.

With regard to students in the process of acquiring literacy, developing the ability to find information in a text marks a transition between writing decoding and reading comprehension. This is because finding some specific information requested in the item stem requires the student to decipher the writing in order to select some specific information from what the text provides, thus requiring some understanding, though at an elementary level, since it is restricted to elements on the surface of what is being read.

In the test administered to the 3rd grade of primary education, four items corresponded to this descriptor. Table 3 shows the percentage of correct answers to these items for low and high-performance classes, as well as the differences between them.

Chart 3 – Correct-answer rates and their variation for items related to descriptor D15 

Skill Item Performance Level Correct-answer rate (%) Correct-answer rate difference between both groups
Finding explicit information in longer texts or in texts presenting data. 36 Low 38.46 2.94
High 41.4
66 Low 76 5.37
High 81.37
30 Low 59.49 5.6
High 65.08
57 Low 56.42 6.88
High 63.3

Source: Prepared by the authors based on data from PAEBES-Alfa.

Table 3 allows concluding that the ability to find explicit information in texts seems to have been developed by the majority of the students assessed. With regard to item 66, the correct-answer rate for both high and low-performing classes is above 75%, which indicates that most of the students in both classes have developed this skill. Correct-answer rates are also high for items 57 and 30 for both high and low-performance classes, as these rates are above 60%.

The item with lowest correct-answer rates – i.e., item 36 – had poor rates both for high and low-performance groups. In fact, this is the item with the smallest performance difference between the classes – i.e., 2.94 percent points. With regard to the most difficult items, the difference between high and low-performing classes is small, up to 6.88 percent points for the item with the greatest performance variation between the classes. These data indicate that the learning of this skill is distributed with relative equity across the classes considered in this study.

We will now focus on understanding the performance differences between high and low-performing classes regarding a reading skill that can be considered more complex: making inferences from the reading of various texts. This skill is described in the Reference Model under code D17 – Inferring information from texts.

Making inferences is an important part of text processing which concerns the semantic dimension of that processing. In Koch’s (2014, p.143) words:

[...] inferences can be seen as cognitive processes through which the listener or reader starts from text information that is explicitly conveyed, considering also the context, to build new semantic representations.

Therefore, making inferences is a fundamental part of the production of meanings by the reader for the text she/he reads, and more than a reading skill, it is a set of various skills that may contain various levels of complexity, depending on both the amount of text information available and the context in which the inferences are required. With regard to large-scale assessments using multiple-choice items, both the text’s complexity and the context are given by the problem situation constructed by the item.

Generally speaking, these assessments have approached text complexity through factors such as length, vocabulary and the subjects addressed. However, that is a topic requiring studies that might contribute to greater clarity on how these factors combine to make a text more or less complex for a given audience.

Because the images of the items used in the test were not available for the present study, we cannot analyze these factors (text length, vocabulary and subject), therefore, we will not address this aspect. Our interest here is to demonstrate that making inferences, which the Reference Model treats as a specific skill assessed by the PAEBES-Alpha’s reading section, can be considered a significantly more complex skill than finding explicit information in texts.

While finding explicit information in a text requires the reader to use mainly his/her linguistic knowledge, making inferences requires him/her to mobilize, in different proportions, his/her linguistic, encyclopedic and interactional knowledge. With regard to linguistic knowledge, which we discussed earlier as we analyzed the ability to find information in texts, it is the knowledge that allows understanding how the text is organized, i.e., its structure, since it encompasses knowledge of both grammar and lexicon (KOCH, 2016).

Encyclopedic knowledge, or knowledge of the world, builds on the reader’s previous experiences, which allow forming hypotheses and filling gaps he/she may come across when reading a text. Finally, interactional knowledge, which encompasses other types of knowledge,8 is knowledge about the interactional contexts in which texts are produced, whether oral or written. In making inferences, these different systems of knowledge are mobilized by the reader.

According to Marcuschi, to understand a text is to infer. According to the author:

Comprehension is influenced by textual, pragmatic and cognitive conditions, as well as interests and factors such as the reader’s knowledge, genre and the form of textualization. Therefore, text comprehension is a complex question that involves not only linguistic, but also anthropological, psychological and factual phenomena. Inferences deal with the relationships between these types of knowledge and many other aspects. (MARCUSCHI, 2008, p. 94).

One may therefore conclude that making inferences is a complex, multi-dimensional process in which several knowledge systems are at play to help the reader produce new information beyond what is already explicit in the text.

In the PAEBES-Alfa test administered to 3rd grade students, two items were related to descriptor D17. Chart 4 shows the correct-answer rate for these items and the difference between low and high-performance classes.

Chart 4 Correct-answer rates and their variation for items related to descriptor D17 

Skill Item Performance Level Correct-answer Rate Correct-answer rate difference between both groups
Inferring information implicit in texts 79 Low 33.82 12.1
High 45.92
60 Low 54.9 5.5
High 60.4

Source: Prepared by the authors based on data from PAEBES-Alfa.

As we can see, the correct-answer rates for the items that assess information inference are significantly lower than those shown in table 3 for the items that assess the ability to find explicit information in the text, both for thigh and low-performing groups. This finding is consistent with the fact that the ability to infer is more complex than that of finding explicit information, as discussed earlier.

Regarding the easiest item – item 60 –, this rate is 60.4% for the high-performance classes, and regarding the most difficult one – item 79 –, it is only 45.92% for the same classes. In turn, for low-performance classes, this rate is 54.9% for item 60, and only 33,82% for item 79, the most difficult. It is also noteworthy that, regarding this item, the performance difference between high and low-performing classes is 12.1 percent points, which demonstrates that the knowledge assessed by this item is unequally distributed between the high and low-performing classes, with an advantage for the former. This finding corroborates the hypothesis that when it comes to more complex reading skills, there is a greater inequality in the distribution of knowledge between both groups of classes.

We can also see that, regarding information inference (a more complex reading skill), the smaller the correct-answer rate on the item for both classes, the greater the difference, in percent points, between high and low-performance classes. Therefore, the more complex the skill, the greater the difference between the classes, and the smaller the equality in the distribution of that knowledge among students.

The uneven distribution of knowledge when it comes to more complex reading skills can also be seen in the items related to another Reference Model descriptor: D18 – Identifying the subject of texts.

Identifying the subject matter of a text is also an inferential process as it requires the reader to consider the diverse information provided by the text in order to reach that particular information which summarizes its content. In this process, all three knowledge systems previously mentioned – linguistic, encyclopedic and interactional knowledge – are mobilized by the reader to conclude on the subject treated in the text.

Identifying the text’s subject can present different levels of complexity depending on factors related to the text itself – its length, vocabulary, thematic approach, whether the subject is indicated in the title, among others –, as well as factors related to the reader’s previous experience – his/her familiarity with the genre and theme, his/her knowledge of the vocabulary, among others. It is possible, therefore, to assess this skill using items with different levels of complexity.

In the PAEBES-Alfa assessment for the 3rd grade, two items assessed the ability described in D18. Table 5 shows the results for high and low-performance classes in these items and the comparison between them.

Chart 5 Correct-answer rates and their variation for items related to descriptor D18 

Skill Item Performance Level Correct-answer rate Correct-answer rate difference between both groups
Identifying the subject of texts. 51 Low 38.34 12.65
High 50.99
25 Low 72.81 7.86
High 80.63

Source: Prepared by the authors based on data from PAEBES-Alfa.

As shown in Table 5, items assessing the ability to identify the text subject have different levels of difficulty. Item 51 may be considered the most difficult, since in low-performing classes, only 38.34% of the students answered it correctly, and in high-performing classes, only 50.99% did so. On the other hand, item 25 was easy, since correct-answer rate is above 70% for both classes. It is also noteworthy that regarding item 25, the easiest one, the correct-answer rate difference between the high and low-performing classes was 7.86 percent points, while in item 51, the most difficult, that rate was 12.65 percent points. One can conclude that, regarding the easiest item, there is less inequality in the distribution of knowledge between the classes, and regarding the most difficult one, there is a greater inequality in the distribution of that knowledge.

Two other PAEBES-Alpha Reference Model descriptors were also analyzed, namely, D13 – Identifying various genres, and D14 – Recognizing the purpose of various genres.

Both skills relate to the dimension of social uses of reading in different communicative situations. Regarding these skills, the linguistic, encyclopedic and sociointeractional knowledge systems are also mobilized, although sociointeractional knowledge may be said to be determinant, since these descriptors require recognizing that different genres relate to particular social functions.

Regarding the identification of a particular genre, the student is required to have not only the linguistic knowledge necessary to decipher the text, but also knowledge of the genre from previous experience with it whether in school or elsewhere, and interactional knowledge, which allows recognizing the text as representative of a particular genre. Regarding students in the 3rd grade of primary education, one must consider that both their world experience and their linguistic experience are still limited by the fact that they are in the initial stages of the process of systematizing their use of Portuguese. Therefore, the repertoire of genres whose knowledge would be reasonable to assess at this stage is also restricted.

Regarding the ability to recognize the purpose of various genres, in addition to the knowledge above, the reader should also have some knowledge of the social practices in which texts circulate, as well as the nature and purpose of these practices so they can identify the communicative purposes that a particular gender is intended for. Therefore, it may be considered a more complex skill compared to the one described in D13.

Table 6 shows the correct-answer rates for the high and low-performance classes on the items assessing descriptors D13 and D14 of the PAEBES-Alfa Reference Model.

Chart 6 Correct-answer rates and their variation for items related to descriptors D13 and D14 

Skill Item Performance Level Correct-answer Rate Correct-answer rate difference between both groups
Identifying various genres. 32 Low 66.19 8.71
High 74.9
71 Low 81.24 6.23
High 87.48
Recognizing the purposes of various genres. 52 Low 45.62 12.25
High 57.87
73 Low 23.41 8.3
High 31.71
67 Low 76.31 3.74
High 80.05

Source: Prepared by the authors based on data from PAEBES-Alfa.

Regarding the two items assessing genre identification (items 32 and 71), correct-answer rates are above 65% for both low and high-performing classes. The performance difference between them is 6.23 percent points in the easiest item – with a correct-answer rate above 80% for both classes – and 8.71 percent points in the most difficult item – correct-answer rates between 66% and 74.9% for low and high-performing classes, respectively.

From these data, some important conclusions can be drawn. The ability to identify the genre of a text seems to present a low degree of difficulty for the great majority of students in item 71, and a greater degree of difficulty in item 32. Regarding the easiest item (item 71), the performance difference between low and high-performing classes is smaller in percent points – i.e., 6.23 points. Regarding the most difficult item (item 32), the performance difference between low and high-performing classes is higher than in the previous case – i.e., 8.71 percent points. Therefore, regarding the most difficult item, which represents a more complex level of the ability to identify a text’s genre, there is a greater inequality between the knowledge appropriated by students in low-performing classes compared to those in high-performing classes.

This also occurs with the ability to recognize the purposes of various genres. Compared to the items assessing the skill described in D13 (identifying various genres), those assessing D14 (recognizing the purposes of various genres) had smaller correct-answer rates, except for item 67, in which both groups had rates above 75%. In general, the ability to recognize the purpose of texts seems to be more complex than identifying the genre of a text, a hypothesis corroborated by both the assessment data and the linguistic nature of the descriptor. It is also noteworthy that, regarding the easiest item, the difference between low and high-performing classes is the smallest in percent points (3.4 points) compared to the other items.

Final Considerations

Considering the concept of equality of knowledge (CRAHAY, 2002, 2013), the present study sought to analyze and pedagogically interpret the variation in the percentage of correct answers on difficult items of the cognitive test administered to students of the 3rd grade of primary education in the state of Espírito Santo in 2014 by means of the PAEBES-Alfa assessment test.

The study’s main finding shows that the most elementary skills related with the learning of reading were acquired by practically all students, but only a few appropriated the most complex skills. Some items presented significant correct-answer rates for high and low-performance groups, indicating that, possibly, the assessed ability is consolidated in both groups. However, other items had a much smaller correct-answer rate for both groups of classes, thus indicating that some skills, at their most complex level, were not fully learned by most students, regardless of the class they were enrolled in.

One finding that stood out was the small correct-answer rate on an item assessing the ability to find explicit information in longer texts or in texts presenting data. In text processing, this ability refers to recognizing elements situated on the surface of the text, therefore it is considered a simple skill for this education stage. However, one of the items presented a small correct-answer rate even for high-performance classes. Although the correct-answer rate between both groups of students is small – thus indicating homogeneous results –, this finding indicates that the ability to find explicit information in a text at a greater level of complexity has not been consolidated by the majority of the students.

On the other hand, regarding the ability to identify different genres – which is considered a more complex text processing activity as it mobilizes elements related to interactional knowledge, in addition to linguistic and encyclopedic knowledge –, most of the students answered correctly (more than 60%), both in high and low-performance classes. This indicates that schools in the education system analyzed may be fulfilling, if still somewhat timidly, their role of promoting the access of students with different levels of literacy to various textual genres, thus increasing these students’ literacy levels.

These findings raise a few questions: from the perspective of cognitive development, can students master more complex reading skills (e.g., inference) without having first consolidated more elementary skills (e.g., finding information)? Does the non-consolidation of more elementary skills necessarily compromise the acquisition of more complex ones? Do the items used in large-scale reading assessments actually assessed the skills proposed?

Regarding literacy, these issues are important to elucidate how the school has been fulfilling its role as a promoter of equal access to knowledge that is central to the exercise of citizenship. The ability to competently interact with texts of different genres has impacts on students’ academic trajectory and, therefore, on their chances to become integrated in a society that is centered on reading and writing practices.

Other questions arising from the present study refer to the relevance of research that, building on the analysis not only of item parameters but on their structure – the supports they use, the construction of the problem situations whose resolution indicates the development of particular skills by the students – can contribute to understanding the degrees of complexity that the same reading skill can present throughout students’ schooling process. Such studies may help improve assessment instruments – items and tests – so as to make them better adjusted to the population assessed and to the assessment’s very objectives, which can contribute to greater test reliability and adjustment to the constructs assessed, thus ensuring the validity of the results obtained.

These questions can and should help inform public policy making not only in the field of educational assessment, but also, and particularly, in the field of curriculum, in addition to encouraging new studies and research designs.

REFERENCES

BARBOSA, Maria Lígia de Oliveira. Desigualdade e desempenho: uma introdução à sociologia brasileira. Belo Horizonte: Fino Traço, 2011. [ Links ]

BERNARDO, Elisangela da Silva. Composição social e cognitiva de turmas e desempenho em leitura e matemática: como evoluem as desigualdades educacionais? 2008. 202 fls. Tese (Doutorado em Educação) – Pontifícia Univeridade Católica do Rio de Janeiro, Rio de Janeiro, 2008. [ Links ]

CARVALHO, Marília Pinto de. Sucesso e fracasso escolar: uma questão de gênero. Educação e Pesquisa, São Paulo, v. 29, n. 1, p.185-193, jun. 2003. [ Links ]

COSTA RIBEIRO, Carlos Antonio. Desigualdade de oportunidades no Brasil. Belo Horizonte. Argvmentvm, 2009. [ Links ]

CRAHAY, Marcel. L´école peut-elle être juste et efficace? De l´égalité des chances à l´égalité des acquis. Belgique: DeBpeck, 2013. [ Links ]

CRAHAY, Marcel. Poderá a escola ser justa e eficaz? Da igualdade de oportunidades à igualdade de conhecimentos. Tradução Vasco Farinha. Lisboa: Instituto Piaget, 2002. [ Links ]

CRAHAY, Marcel; BAYE, Ariane. Existem escolas justas e eficazes? Esboço de resposta baseado no PISA 2009. Cadernos de Pesquisa, São Paulo, v. 43, n. 150, p. 858-883, 2014. [ Links ]

DUBET, François. O que é uma escola justa? Cadernos de Pesquisa, São Paulo, v. 34, n. 123, p. 539-555, 2004. [ Links ]

DUBET, François. L´école des chances: Qu´est-ce qu´une école juste? Paris, 2004a. [ Links ]

HEINEMANN,Wolfgang; VIEHWEGER, Dieter. Textlinguistik: eine Einführung. Tübingen: Niemeyer, 1991. [ Links ]

KOCH, Ingedore Grunfeld Villaça. As tramas do texto. São Paulo: Contexto, 2014. [ Links ]

KOCH, Ingedore Grunfeld Villaça. O texto e a construção dos sentidos. São Paulo: Contexto, 2016. [ Links ]

MARCUSCHI, Luiz Antonio. Produção textual, análise de gêneros e compreensão. São Paulo: Parábola, 2008. [ Links ]

OLIVEIRA, Romualdo Portela. Da universalização do ensino fundamental ao desafio da qualidade: uma análise histórica. Educação e Sociedade, Campinas, v. 28, n. 100, p. 661-690, out. 2007. [ Links ]

SOARES, José Francisco; ALVES, Maria Teresa Gonzaga. Contexto escolar e indicadores educacionais: condições desiguais para a efetivação de uma política de avaliação educacional. Educação e Pesquisa, São Paulo, v. 39, n. 1, p. 177-194, jan./mar. 2013. [ Links ]

SOARES, José Francisco; MAROTTA, Luana. Desigualdade no sistema de ensino fundamental brasileiro. In: VELOSO, Fernando et al. (Ed.). Educação básica no Brasil. Rio de Janeiro: Elsevier, 2008. p. 73-91. [ Links ]

VALLE, Ione Ribeiro. Justiça na escola: das desigualdades justas à igualdade sem adjetivos!. In: VALLE, Ione Ribeiro; SILVA, Vera Lúcia Gaspar; DAROS, Maria das Dores (Org.). Educação escolar e justiça social. v. 1. Florianópolis: Centro de Ciências da Educação/UFSC, 2010. p. 19-48. [ Links ]

WALTENBERG, Fábio Domingues. Análise economica de sistemas educativos: uma resenha crítica da literatura e uma avaliação empírica da iniquidade do sistema educativo brasileiro. 2002. 159 fls. Dissertação (Mestrado em Economia) – Universidade de São Paulo, São Paulo, 2002. [ Links ]

XAVIER, Flavia Pereira; ALVES, Maria Teresa Gonzaga. A composição social importa para os efeitos das escolas no ensino fundamental? Estudos em Avaliação Educacional, São Paulo, v. 26, n. 61, p. 216-243, jan./abr. 2015. [ Links ]

3We decided to use data from an Assessment Program focused on literacy as there is a consensus among experts that the learning related to this school stage is considered as basic knowledge – the type of knowledge defined by Crahay (2002; 2013) as universal and fundamental for the development of the individual.

5Item Response Theory is a psychometric assessment method that does not just count the total number of correct answers in the cognitive test. One of its assumptions says that the item is the basic unit of analysis, therefore, the performance can be explained both by the skill assessed and by the item’s characteristics. B parameter is one of the characteristics that form the item. Moreover, according to this theory, one can, based on the items, build a proficiency scale that allows determining at what point students are situated on it according to the level of proficiency they achieved in the test.

6The PAEBES-Alpha proficiency scale has a mean of 500 and a standard deviation of 100. This means, roughly speaking, that this scale ranges from 0 to 1000 points.

7In a large-scale assessment, performance means are estimated from data for students’ answers to the items of the test, these variables having a binary coding in which the null value corresponds to error, and one corresponds to a correct answer. In the reading test, all items were multiple-choice, with four response options. With this configuration, the likelihood to answer correctly (25%) is significant, even if the respondent lacks the knowledge for doing so. In order to mitigate this problem in the interpretation of correct-answer rate means, we used a procedure to correct for accidentally correct answers, expressed by the formula: Corrected_rate = Real_rate - (100 – Real_rate)/(N - 1).

8According to Koch (2016), sociointeractional knowledge encompasses the illocutionary, communicational, metacommunicative and superstructural types of knowledge.

Received: November 29, 2017; Revised: April 24, 2018; Accepted: June 13, 2018

Naira Muylaert is a postdoctoral student in the area of public education policies in the Graduate Program in Education at the Pontifícia Universidade Católica Rio de Janeiro (PUC Rio). She is a member of the Laboratory for the Assessment of Education - (LAEd).

Alicia Bonamino is an associate professor at the Department of Education at the Pontifícia Univeridade Católica do Rio de Janeiro (PUC Rio). She coordinates the Laboratory for the Assessment of Education - (LAEd).

Hilda Micarello é Holds PhD in education and is a professor at the Faculdade de Educação, at the Universidade Federal de Juiz de Fora (UFJF); she is also a researcher at the Center for Evaluation and Public Policy for Education (CAEd), at the Universidade Federal de Juiz de Fora. She serves at the PPGP/FACED/UFJF and PPGP/CAEd/ UFJF.

Creative Commons License  This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.