READING COMPREHENSION OF STUDENTS OF PRIMARY SCHOOL II: A STUDY ON PERFORMANCE USING DIFFERENT TYPES OF TASKS AND INFERENTIAL CATEGORIES

ABSTRACT

This research had the goal to examine the reading comprehension of primary school students according to their level of education and the proposed tasks. The research subjects were 62 students from the 6th to the 8th year of Primary School II, from a public school in the city of Porto Alegre (RS / Brazil). Data collection was performed using Questionnaire, True or False and Multiple Choice. The questions had a correspondence with the contents, inferential categories, and Cloze. Each subject answered only one task. This distribution was carried out randomly in each class of students. The collected data were organized and treated statistically, thus enabling the following results: a significant difference in reading comprehension (p = 0.007) in regard to the level of education, indicating progression of scores as the schooling process goes on; a significant difference (p = 0.003) regarding the type of task, in which Multiple Choice task presented the highest average score (4.43), the True or False task showed the lowest average score (2.60) and the Quiz tasks and Cloze indicated intermediate averages (3.29 and 3.28, respectively); and a development on the ability to make inferences (relating to the inferential category) was seen as the educational level increases.

reading comprehension; inferential category of questions; primary school

RESUMO

Essa pesquisa teve como objetivo examinar a compreensão leitora de alunos do Ensino Fundamental II em suas relações com o nível de escolaridade e as tarefas propostas. Os sujeitos da pesquisa foram 62 alunos do 6º ao 8º ano do Ensino Fundamental II, de uma escola pública da cidade de Porto Alegre (RS/Brasil). A coleta de dados foi realizada utilizando Questionário, Verdadeiro ou Falso e Múltipla Escolha, com correspondência das questões entre os conteúdos e as categorias inferenciais, e Cloze. Cada sujeito respondeu a uma tarefa apenas, sendo essa distribuição realizada aleatoriamente em cada turma de alunos. Os dados coletados foram organizados e tratados estatisticamente, possibilitando a obtenção dos seguintes resultados: com relação ao nível de escolaridade, diferença significativa no desempenho em compreensão leitora (p=0,007), indicando progressão dos escores à medida do avanço da escolaridade; com relação ao tipo de tarefa, diferença significativa (p=0,003), com a tarefa de Múltipla Escolha apresentando a maior média (4,43), a tarefa de Verdadeiro ou Falso a menor média (2,60) e as tarefas de Questionário e Cloze médias intermediárias (3,29 e 3,28, respectivamente); com relação à categoria inferencial, evolução da capacidade de realizar inferências com o aumento do nível de escolaridade.

compreensão leitora; categoria inferencial das questões; ensino fundamental

Introduction

Official tests at national and state level have been showing the unsatisfactory conditions of primary school students with regard to Portuguese Language and Mathematics, thus leading to social and academic concerns. According to the results of SAEB 2017 concerning Portuguese and Mathematics in the initial years, 5.94 is the standardized average score of public schools. It is similar to previous years, just a little higher than 2015 rate (5.74). In 2017, in the final years, the Portuguese language score was 253.74, slightly surpassing the result of 2015 (247.33). The 2017 standardized average score for Portuguese and Mathematics (5.09) showed a very small evolution compared to 2015 (4.97). In the initial years of public schools, IDEB 2017 was 5.5 and, in the final years, 4.4. In 2015, in the same context, the result was 5.3 in the initial years, and 4.2 in the final years, indicating a very slight growth. There are understandable concerns about the situation, considering both such standards and the way they impair the overall knowledge acquisition in other fields, and as they also impact the evolution of schooling and social inclusion.

This data and information indicate that all school series present potential needs. This study chooses, however, only the 6th, 7th and 8th years of Primary School II, to continue the already started research and to be able to attain more complex understanding in following years.

In this context, this article studies reading comprehension at school from the perspective of Psycholinguistics, with special attention to the inferential perspective of each question (GIASSON, 2000; GRAESSER; SINGER; TRABASSO, 1994; VIDAL-ABARCA; RICO, 2003).

The objective was to examine the reading comprehension of students from Brazilian public primary schools, according to schooling level, task and the inferential category of the question. Given the theme and the objective, the methodology was characterized by the use of four types of tasks - Questionnaire, Multiple Choice and True or False - with correspondence of questions to both content and inferential characteristics -, and Cloze, enabling data collection, analyzes and results, according to the established objective.

These results contribute to shed light on such problems and to more reflections that may lead to possible solutions. Therefore, in this article, the authors first expose the theoretical foundations about reading, focusing on reading comprehension, inference and evaluation of reading comprehension. Next, they present the research, in its definition, organization and realization. Subsequently, they provide the results and conclusions, favoring the reader’s understanding of the processes developed and the products obtained.

The fundamentals

In this topic, the fundamentals of the research are presented with regard to reading comprehension (research axis), inference and reading comprehension (inferential levels of the questions) and evaluation of reading comprehension (types of tasks used for data collection).

Reading comprehension

Reading means comprehension (COLOMER; CAMPS, 2002), for which it is necessary to perform cognitive processes (SCLIAR-CABRAL, 2008, 2009; LEFFA, 1996). Comprehension and processing are then recognized in an interconnected way within a cognitive paradigm (COSTA; PEREIRA, 2009). Thus, comprehension means performing an interactive process, mainly based on two cognitive procedures - bottom-up and top-down.

The first, bottom-up, is carried out in an ascending manner, that is, from the parts to the whole, from the smallest units to the largest. This occurs when the user takes the information they need mostly from the text itself to get to their comprehension. It is a procedure in which the clues that the author leaves in the text are the foundation of a composition process, since the parts gradually form the whole (SCLIAR-CABRAL, 2008).

The second process, the top-down (GOODMAN, 1991; SMITH, 1999), is seen as a downward movement, from the whole to the parts, meaning that it moves from the largest to the smallest units, from the macrostructure to the microstructure. It is a procedure in which the user’s previous knowledge forms the foundation of a decomposition process. The previously available information allows the user to create connections with the linguistic clues.

Both movements (upward and downward) exist and are used interactively - between brain and text and between prior knowledge and linguistic clues, passing through all linguistic levels. In this interactive dimension, the formulation of hypotheses and their verification rely on linguistic units at their various levels, being used in a double movement. Studies on this topic contribute to shed light on such processes. They highlight the formulation of hypotheses as a predictive procedure that involves linguistic and non-linguistic, which is a mechanism that happens in all its dimensions. Such studies try to understand these occurrences and their variables, especially DeLong, Troyer, Kutas (2014), about pre-processing in sentence comprehension, and Balass, Nelson, Perfetti (2010), about word processing.

In this way, the upward and downward movements are neither exclusive nor serial, but converging to the comprehension situation itself, involving the text in question - gender, type (ADAM, 2008; BAZERMAN, 2009), the reading objective and the reader - previous knowledge, cognitive style (KATO, 2007). Comprehension success relies, then, on the interactive combination of both ascending and descending processes, considering the constituent elements of the comprehension situation, in which variables are interrelated and influence the subject’s definitions.

One of the most accepted interactive models in the literature, the Kintsch’s Construction-Integration Model (1998), considers comprehension to be an inferential process by nature. This model consists of two dimensions that are related to each other: the text base, a mental representation constructed from information featured in the text; and the situational model, a mental representation based on the reader’s prior knowledge in which the gaps in the text are filled through the establishment of inferences. It is an interactive model: as they read, the reader integrates the elements of the text base in order to build the mental representation of the text, linking information at both intratextual and extratextual level (BARETTA; PEREIRA, 2018).

Inference and reading comprehension

According to Marcuschi (2008), inference is the activity that we perform when we gather some known information to arrive at other new information, which can be based on explicit and implicit textual information, as well as on information provided by the reader.

Spinillo (2013, p.179, our translation) considers that:

[…] textual comprehension is an inferential process par excellence. Inference happens because not everything is explicit in the text, and the reader has to establish connections between different segments and to use his world knowledge to fill in the gaps and to build the appropriate and coherent mental representation of the text.

Therefore, reading is, as stated by Koch and Elias (2011), a highly complex interactive activity to produce meaning, which is based on the relationship between the stored knowledge in the reader’s memory and the information brought in the text. For the authors, one cannot speak of the meaning of the text, but of a meaning for the text, which will only be constructed in the interaction with the reader.

Thus, it is essential that the reader reconstructs the path taken by the author, recognizing the linguistic clues they left in the text. That is why Kato (2007, p.72, our translation) states that the text is “a set of footprints to be used to recap the author’s strategies and to reach their proposed goals through them”. This means that, although it is not possible to speak of a single meaning for a text, one may not consider that all forms of understanding are possible, as the understanding can only be effective when the reader’s inferences find support in the materiality of the text. Other conceptions of reading highly value or depreciate the linguistic elements of the text. Here, they are considered part of a triangular relationship between author-text-reader, in which the reader uses the data provided by the author, selecting the relevant information so as to get to a meaning that the text authorizes.

Marcuschi (2008) illustrates this relationship using the metaphor created by Dascal (1981), which associates the text with an onion. According to the author, the inner layers represent the objective information, textual elements that are typically informational and that are not subject to different understandings, such as names, places etc. The intermediate layer, in turn, is where inferences happen, that is, the different, yet valid, readings. The outermost layer is the most prone to misunderstanding, as it consists in the domain of our beliefs and values. It is in this domain that meanings not authorized by the text emerge.

This process of construction of meaning during reading happens, according to Poersch (1991), at levels established by criteria of textual comprehensiveness and depth of understanding. The comprehensiveness criterion is related, according to the author, with the triple linguistic articulation: lexical, phrasal and textual. Lexical comprehension refers to the recognition of the meaning of words; phrasal comprehension, on the other hand, consists in recognizing the meaning that words take in the sentence, because, as Poersch (1991, p.130, our translation) states, “the meaning of a sentence does not correspond to the sum of the meaning of the words”. Textual comprehension, in turn, involves the recognition of the overall meaning of the text.

The second criterion (depth) corresponds to the understanding of the explicit meaning, that is, of what is actually written in the text; the implicit meaning, which is not written, and is still part of the text; and the meta-implicit meaning, which comes from data out of the text, related to the communicational situation.

The construction of these senses is accomplished through processes that demand different cognitive demands. The first, the explicit content, originates from a decoding activity. Implicit content requires more complex mental processes, such as inference and assumption. The meta-implicit, in turn, is built from the prior knowledge of each individual.

The tasks proposed in this study analyze the reading comprehension from the depth criterion, verifying the comprehension of the explicit and implicit meaning of the text, considering the inferential levels.

Assessment of reading comprehension

Studies on reading comprehension have pointed out several factors that can influence the performance on textual comprehension tasks. One of these factors is the type of task used in the evaluation (PEREIRA, 2009; CADIME et al., 2017; SPINILLO; ALMEIDA, 2015). The most common are Multiple Choice, True or False and Questionnaire. Cloze’s task is also seen as important.

According to Tinkelman (1967), Multiple Choice limits the student to the alternatives presented, with little or no opportunity to become disoriented with qualification or exception problems. Vianna (1982) also highlights the fact that, among the existing varieties, this type of test is less prone to deficiencies common to the others and can be more easily adjusted to different purposes. Tinkelman (1967) points out that it can efficiently measure fundamental knowledge, as well as intellectual skills, as the chance factor is considerably reduced, especially when it presents four or five alternatives.

Vianna (1982) pays attention to the complexity of the elaboration of objective tasks and presents inputs to qualify its construction. It is important: to develop the item from relevant ideas; to select ideas that enable the elaboration of items with discriminative power; to elaborate items that measure important objectives; to present the item as clearly as possible; to avoid including non-functional elements in the item; to avoid building items based on overly specific elements; to adapt the item’s difficulty to the group level; to avoid the inclusion of elements that might suggest the answer; and to eliminate the use of stereotyped structure.

In the True or False type of task, it is up to the subject to indicate whether they consider each of the statements presented to be true or false. This basic form, says Gronlund (1974), can present variations in which the examinee must answer yes or no, agree or disagree, right or wrong, fact or opinion, among others. It is a simple and direct task that connects to the subject’s knowledge, as they must decide on the truth or falsity of a proposition. True or False tasks present particular ease and are time saving, also providing a variety of results, such as the understanding of new data; the analysis of known material (presented in a different way); the statement or definition of theories, principles and laws; the critical evaluation of different interpretations of facts or phenomena, distinguishing the false from the true and facts from opinions. When judging whether an affirmative is right or wrong, the subject directly shows their mastery over the knowledge related to it.

According to Gronlund (1974), the True or False task is one of the most difficult to be elaborated, as the statements must be unquestionable - either true or false and, at the same time, it must enable evaluation of important aspects of knowledge. In addition, as it presents only two possibilities of choice, even the non-proficient examiner has a 50% chance of making the correct choice without knowing the answer. The author also points out another problem regarding this type of task: the choice made by the examinee does not provide evidence about the knowledge that they really have (or the lack of it). In case the subject marks a true statement as false, the task does not allow the identification of the wrong concept that led to the chosen answer. In case the subject marks a statement that is effectively false as false, there is not enough evidence to know if he knows the truth.

The Questionnaire, according to Vianna (1982), constitutes a type of task called free or dissertation answer. It is characterized by the fact that the examinee presents his own answer instead of selecting it from several offered alternatives. The nature of such an answer varies according to its type. Simpler questions require only that a specific piece of information presented in the text be recalled, while more complex questions may require more elaborate answers, analysis and connecting elements.

This type of task is recommended when one wants to verify complex objectives, such as, for example, the ability to interpret principles, make inferences, interpret data, critically analyze an idea, and connect elements.

Vianna (1982) understands that the Questionnaire may only be an efficient measure when some aspects are observed, such as item formulation, limitation of the answer, item adequacy, time availability and item correction. Regarding item formulation, the author states it must be sufficiently specific, thus favoring the understanding of how to proceed, yet sufficiently general, thus allowing the subject to structure their response. The limitation of the answer must also be considered beforehand by the task’s creator, since questions that allow broad answers present problems that hinder their success. The author then advises that limited response items should be used because the response parameter is better defined, leading to simple correction and objective assessment. The third aspect he points out is the item adequacy to the time available to answer. It is important to provide more time than what was estimated, so that the subjects, regardless of their writing speed, may have time to consider all questions.

Cloze’s, in turn, is a highly recognized task and consists of “randomly or systematically eliminating the words of a text so that the student tries to replace them, while reading, supported by the context of the remaining words” (SÖHNGEN, 2002, p.65, our translation). Since its presentation by Taylor (1953), the technique has undergone modifications as its elaboration and methodology advanced. The author argues that, in some cases, the Cloze procedure can be used to measure different types of variables in the communication process and in the reading comprehension.

Despite various definitions, the most popular characterizes Cloze task as a procedure that uses texts with a minimum of 250 to 300 words, eliminating every fifth word of the text and preserving the first and last paragraphs intact. Thus, the length of the chosen text determines the process of word erasure and guides the counting of intervals. Due to its configuration, Cloze is widely used in research involving reading comprehension and the readability of texts.

These four types of tasks - Multiple School, True or False, Questionnaire and Cloze - were used in the research on display, and in the next topic, we have the presentation of the structure and deployment, based on the theoretical foundations already exposed.

Research structure

As already mentioned, the research analyzes the reading comprehension performance of primary school students in relation to reading comprehension in its inferential nature, considering categories of inference, types of tasks and schooling level.

The research sample consisted of 62 participants, distributed among three school years: 26 from the 6th year, 20 from the 7th year and 16 from the 8th year. The participants were of both sexes, aged between 11 and 15 years - within the regular school-age -, from a public school in the city of Porto Alegre, Rio Grande do Sul.

In each school year, the participants were randomly distributed according to the type of task: 16 participants - Multiple Choice (MC), 15 participants - True or False (TF), 17 participants - Questionnaire (Q) and 14 participants - Cloze (CL). The participants were thus organized to allow the association of reading comprehension scores with both the schooling level and the type of task, also considering the inferential characteristic of each question.

Then, four reading comprehension tasks were used with these four different formats, all with the same supporting text: the short story “False Test”, by Stanislaw Ponte Preta. The text selection considered content, language and length adequacy to the participating students and the demand that they use inference to achieve comprehension. Some structures were adapted to make the content more appropriate to the linguistic universe of the target audience of the research (BARETTA; PEREIRA, 2018).

Next, we have a description of each task:

1. Multiple Choice Task: six questions, each one presents a question related to the text, offering five possible alternatives as answers, with only one correct answer.

2. True or False task: six questions, based on the information present in questions of the other tasks, with statements related to the text, and the participant must tick T (true) or F (false) and explain when false.

3. Questionnaire task: six open questions corresponding to the content focused on the other tasks.

4. Cloze’s task: 36 gaps that must be filled in, with an interval of five words between each one, using the same text as the other tasks.

In this research, the concept of comprehension refers to an essentially inferential activity (KINTSCH, 1998). Therefore, the questions in the tasks were elaborated based on studies that described different types of inferences. Four categories were then established, from studies by Graesser, Singer and Trabasso (1994), Giasson (2000) and Vidal-Abarca and Rico (2003). The first relates to non-inferential information and the others to inferential information in various forms. The four categories are described and exemplified below:

a. Category 1: the information is explicit and can be directly found in the text without using this strategy.

Question: How should the dog be to please the owner of the house?

Expected response: The dog should be obedient and minimally polite.

Explanation: To answer the question, the reader must find information clearly expressed in the text.

b. Category 2: the inference comes from intratextual relations, as it is found between propositions in the text.

Question: What made the man win the war against the dog?

Expected response: The man won the war because the dog started to pee where it should not.

Explanation: To answer the question, the reader must rely on implicit information. In the text, there is no explicit bond between the fact that the dog begins to pee where it shouldn’t and the fact that the owner wins the war against the dog. The reader needs to add “why” between the two sentences in order to answer the item.

c. Category 3: the inference comes from extratextual information based on the reader’s previous knowledge, and it is up to them to associate it with the propositions of the text.

Question: Whose pee was in the woman’s dress?

Expected response: The pee was from the dog’s owner.

Explanation: To answer the question, the reader must activate his previous knowledge about the circumstances in which a person feels remorse. This way, they might relate the reason for the remorse to the authorship of the action in question.

d. Category 4: inference depends on the macro-structural sense of the text and contributes to establish global coherence.

Item: Why is the title of the text False Proof?

Expected response: The title of the text is False Proof because the owner created a false proof (the pee) against the dog to send it away.

Explanation: To answer the question and explain the meaning of the title, the reader must systematize and relate the various pieces of information inside the text.

The six questions that make up the reading comprehension tasks are distributed as described in Table 1.

Table 1
– Questions by category

Task question	Categories
	1	2	3	4
(Q1) How did the family get the dog?	X
(Q2) How should the dog be to please the owner of the house?	X
(Q3) Why did the man win the war against the dog?		X
(Q4) Why did the man give the animal to a friend who loves dogs?		X
(Q5) Whose pee was in the woman’s dress?			X
(Q6) Why is the title of the text False Proof?				X

Source: Authors’ elaboration.

As for the task evaluation, each correct answer added a point. In the TF task, the false answers were considered correct only when properly justified.

Previously instructed undergraduate and graduate students carried out the data collection at the school, during the regular shift, in Portuguese language lessons of each class, in about 40 minutes each (one class period). Each student in the participating classes answered only one of the tasks, chosen randomly, since the same text was used for the different types of instruments. Table 2 below shows the distribution of students by type of task and by school year.

Table 2
– Distribution of students by task and by school year

Task type	School year
	6th year	7th year	8th year
MC	7	5	4
Q	7	5	5
CL	6	5	3
TF	6	5	4
Total	26	20	16

Source: Authors’ elaboration.
• MC: Multiple choice Q: Questionnaire CL: Cloze TF: True or false

One of the objectives that guided this research was to compare the reading comprehension performance of students from different levels of education. Table 4 shows the average scores for each of the Year variables regardless of the Task variable.

Table 4
– Average score of correct answers per school year

Year	Averages	Standard error	95% confidence interval
			Inferior limit	Upper limit
6th	3,0	1,41	2,42	3,57
7th	3,25	1,44	2,57	3,92
8th	4,31	1,49	3,51	5,10

Source: Authors’ elaboration.

The following topic presents the results obtained based on the collected data.

Results and discussion

As shown, 62 different subjects were submitted to the assessment instruments, providing a single dependent quantitative variable (correct answers), controlled or segmented by two independent qualitative factors or variables (task and school year). The statistical treatment of the data was performed based on the analysis of variance (ANOVA) of double classification (Two-way ANOVA). In this way, the results originate from these variables, according to the objectives outlined and research design.

Each variable (year and task) presents a main effect. An interaction effect or combined effect (in this case year x task) is also assumed by the technique and therefore was tested. The test is performed using the F statistics and the significance (values equal to or less than 0.05 = 5%) of F shows whether that factor or interaction is significant for the result (dependent variable or correct answers). In the researched group, the two variables proved to be significant, indicating that both the type of task and the level of education (year), when analyzed individually, influence the reading comprehension performance. However, this does not occur when analyzing the combined data (Year x Task), since the interaction between these variables was not significant, as shown in Table 3.

Table 3
– Tests of the main effects and the interaction effect

	GL	Sum of Squares.	Medium Squares	F	Significance
Task	3	27,17	9,05	5,16	0,003
Year	2	18,75	9,37	5,34	0,007
Task x Year	6	7,42	1,23	0,70	0,646
Residue	50	87,74	1,75

Source: Authors’ elaboration.

As in Table 4, the overall performance of 6th year students was 3.0, 7th year students was 3.25 and 8th year students was 4.31. The standard error which defines how much we can be wrong when stating the correctness of each level average score. The interval is expressed in the last two columns and considers the possible variations taken the standard error. Thus, 6th year students had a minimum score of 2.42 and a maximum of 3.57. The 7th year students had a minimum score of 2.57 and a maximum of 3.92. The 8th year students, in turn, had a minimum score of 3.51 and a maximum of 5.10.

The table shows the evolution of participants taking school years in account and points to a possible evolution for other groups. Although these results are inserted in a specific situation of one school, they may be considered positively, since the difficulties regarding learning how to read are big as evidenced by official data. However, it is important to remember that the teacher should assess, in educational school practice, whether this difference between educational levels in fact means a substantial evolution in reading comprehension.

The research was also guided by an objective to analyze the participants’ reading comprehension performance in relation to the type of task performed. Table 5 below shows the average score per task in the school years investigated.

Table 5
– Average score of correct answers per task

Task	N	Average
MC	16	4,43
Q	17	3,29
CL	14	3,28
TF	15	2,60

Source: Authors’ elaboration.

It is possible to observe, based on the results shown in the table, that the Multiple Choice task had the highest average score - 4.43. The True or False task, in turn, had the lowest average score (2.60) among the four types of tasks investigated. The Questionnaire and Cloze tasks had intermediate average scores and were very similar to each other - 3.29 and 3.28, respectively.

The Multiple Choice task had the highest average score. This may be explained by the fact that the student benefits from the presentation of alternative answers, contributing to the process of reflection and evaluation on what would be more plausible regarding the proposed question (PEREIRA, 2008, 2018). Thus, the results from these groups show that this type of task favors the participant’s perception of the perspective posed by the question, also helping them to compare the information from each possible answer to the information inside the text and to identify which would be the most appropriate answer.

The Questionnaire task presented the second highest average score, showing that the possibility for the students to organize their own perspective is also indicated to assess the reading comprehension of students at these levels. However, this is an intermediate average score, since it is closer to the average of Cloze’s tasks than to the average of the Multiple Choice task.

The fact that the True or False task had the lowest average score can be explained by the characteristics of this task. The participant needs to present an explanation for the statements considered false, as there is an equal possibility of success and error. For participants at these school levels, it is possible that this task may not be as familiar and easy to understand.

Such results can guide teachers in the planning of their classes and assessments. For students of the analyzed school years, it seems that the tasks with alternatives and with open items are more effective when compared to those that require choosing true or false statements, thus requiring the student to justify. Apparently, for these students, this points to a challenge factor, as does the task that uses gaps.

The research also sought to verify the performance of the participants in relation to the required level of inference. These data are presented in table 6 below.

Table 6
– Number and percentage of correct answers by category - non-inferential / inferential

Years	CATEGORIES
	Category 1	Category 2	Category 3	Category 4
6th year	36 (90%)	18 (45%)	3 (15%)	7 (35%)
7th year	26 (86,6%)	13 (43,3%)	4 (26,6%)	4 (26,6%)
8th year	23 (88,4%)	16 (61,5%)	7 (53,8%)	9 (69,2%)

Source: Authors’ elaboration.

We can see that, in all groups, the number and percentage of correct answers tend to decrease as the inferential category of the question progresses, meaning that less correct answers increase. The only exception is when comparing categories 3 and 4 in the 6th and 8th years.

When we compare school years, we can see that the correct answers were more frequent among students of the 8th year than with those of 7th and 6th years. However, the correct answers were not more frequent among 7th year students compared to 6th year students, except for category 3. Despite this, in general, these data indicate that the level of education is associated with the construction of inferences, since 8th year students answered more correctly than the students from lower school levels.

There is a significantly higher performance in category 1 (Q1 and Q2 of Table 1) in relation to the other categories when we compare data related to each inferential category. This difference is even more evident in the comparison between the percentage of correct answers in category 1 for each group and the percentage of correct answers in all the other categories together. In the 6th year, for example, 90% of students gave correct answers in category 1, while the percentage of correct answers in categories 2, 3 and 4 reached 31.6%. In the 7th year, in categories 2, 3 and 4, we have a score of 32.1% of correct answers, whilst in category 1 alone we have 86.6% of correct answers. The 8th year group, in turn, had 88.4% correct answers in category 1 and 61.5% in the other categories. These results are due to the fact that the first two questions, category 1, require that the reader search for information that was clearly expressed, while the other questions require some type of inference. It is noted, therefore, that students have more difficulties when the questions require complex inferential procedures.

The type of inference required in category 2 (Q3 and Q4 in table 1) is based on causal relationships between propositions in the text, while category 3 (Q5 in table 1) requires inference that comes from information that is added based on the relationship between the previous knowledge of the reader and the linguistic material, that is, the text. The data in table 6 suggest that inferences based on intratextual information are more easily made than those based on extratextual information. This is observed by the percentage of correct answers in category 2 compared to category 3. Despite this, it is noted that this difference decreases when the evolution of the school year is compared.

Finally, category 4 (Q6 in table 1) requires inference that contributes to establishing the overall coherence of the text. The percentage of correct answers in this category was higher in the 6th and 8th years when compared to the performance of the same groups in category 3; differently, the 7th year percentage of correct answers was the same in both levels.

We have a research carried out with students of the 4th and 5th years of Primary School by NUCCLIN (PPGL - PUCRS, Catholic University from Rio Grande do Sul), a group in which the authors of this article also participate. In such, data suggested that students from these school groups have difficulty in making inferences that require the analysis of macro-structural aspects of the text. They are able to understand local information, but unable to relate it globally. The results showed that the performance of the participants in both groups was considerably lower when compared to the other levels. The percentage of correct answers did not exceed 30%. In a previous study carried out with the 6th and 7th years of primary school (BARETTA; PEREIRA, 2018), we have that students make global inferences more easily at such school level, suggesting that this skill evolves as the school years progress. The authors’ conclusions, therefore, corroborate the results presented here.

The equivalence of the types of tasks on these studies is confirmed using Tukey multiple comparison test. The test assesses the extent to which task types offer the same results when researching reading comprehension. Table 7 presents below the results of this analysis.

Table 7
– Multiple comparisons - Test Factor

Statistical Test	Tasks	center	Inferior limit	Upper limit	p-value
Tukey	MC-CL	1,154	-0,204	2,508	0,123
	Q-CL	0,008	-1,329	1,346	0,999
	TF-CL	-0,685	-2,063	0,691	0,556
	Q-MC	-1,143	-2,434	0,147	0,100
	TF-MC	-1,837	-3,169	-0,505	0,003

Source: Authors’ elaboration.

Data in table 7 show that a significant difference was found only between TF and MC (p = 0.003, that is, p <0.05). Therefore, the tasks are not equivalent and each of them may influence the result differently. The data presented in table 5 refers to the average score of correct answers per task. There is a great difference between Multiple Choice, the highest average score of correct answers, at the top of the table, and True or False, the lowest average score of correct answers, at the bottom.

There was no significant difference for the other task groups (p> 0.05). This suggests a certain equivalence between the other tasks, thus using one or another does not influence the results when assessing reading comprehension.

After presenting the collected data and the results obtained in the research, we now have the conclusions of the study below.

Conclusions

This study researched the reading comprehension of students from the 6th, 7th and 8th years of primary school in a Brazilian public school. The focus was the relationships between school year, the type of task and their inferential categories. This is the intermediate level of Primary Education, that is, a transition between the Early Years of Primary School and High School. Therefore, to understand how the reading comprehension evolves in this stage may contribute to teacher planning, also helping to mitigate the evident general reading difficulties of Brazilian students.

According to the results of SAEB 2017, previously presented, we have made a small progress when compared to 2015: in the final years of Primary Education, in 2017, the score of Portuguese Language was 253.74, while in 2015 it was 247.33. These results suggest that schools have managed to make little if any progress, with difficulties regarding teaching and learning. Relating these official data to the results from this research, it is clear that the level of education does in fact influence the resulting reading comprehension scores, as well as the type of task and their inferential categories.

Therefore, to work with inferential categories in the classroom should assist in the development of reading comprehension. In this sense, this research presents potentialities on exploring textual information in relation to such categories. In addition, the type of task also led participants to achieve different performances, suggesting that working with different types of tasks in the classroom is also a way to develop reading comprehension, given that each activity explores information in different ways and requires different cognitive paths for the students to accomplish them.

To achieve the objectives of the study, the authors were supported theoretically by the conception of reading as a cognitive process (COLOMER; CAMPS, 2002), which involves upward and downward movements that occur interactively (SCLIAR-CABRAL, 2008; GOODMAN, 1991; SMITH, 2003). In view of one of the main objectives of the study - to investigate the influence that inferential categories of each question have on reading comprehension -, the conception of comprehension as an activity of an inferential nature also supports this research (KINTSCH, 1998; MARCUSCHI, 2008; GIASSON, 2000; SPINILLO; ALMEIDA, 2015), being an activity that involves explicit and implicit elements (POERSCH, 1991). As for the assessment of reading comprehension, specific studies supported the parameters (GRONLUND, 1974; TINKELMAN, 1967).

The theoretical data presented here were ground for the elaboration of reading comprehension tasks - addressing inferential categories of each question and generating the empirical data collected from students in the 6th, 7th and 8th years of Primary School that participated in this research. Thus, the results of the study are the theoretical data themselves, as well as the tasks and the information on the students’ reading conditions.

With regard to the overall performance of each group, when comparing school years, we have a higher frequency of correct answers as the level of education increases. The average score of correct answers for each group was, respectively, 3.0; 3.25 and 4.31; showing evolution from the 6th to the 7th year and from the 7th to the 8th. Possibly, these results are related to the properties of the cognitive processing of these subjects, indicating that, as desired, educational progression itself might contribute to more complex cognitive procedures.

Considering the performance of the participants in relation to the type of task performed, the highest average score was Multiple Choice, Questionnaire and Cloze had the intermediate averages and True or False had the lowest. The results allow us to conclude that the True or False offers difficulties to the participants of this school group, mainly because the participant must justify their answer when the false alternative was chosen, which was a precondition for their answer to be considered correct. The Multiple Choice average scores indicate, on the contrary, that students in this school range are more familiar with such activity, perhaps due to its frequent use in everyday life. In this case, the student’s cognitive procedures flow better due to the presentation of alternative answers that help their thinking.

When comparing the inferential categories, results suggest that school year is related to the ability to make inferences, since 8th year students had a higher number of correct answers than students from other school years. This did not occur, however, when comparing 7th and 6th year scores, except for category 3. In addition, generally, the percentage of correct answers decreased as the inferential category of each question progressed at all school levels. This was different only when comparing category 3 and category 4 in the 6th and 8th years, which suggests their greater difficulty in questions that require extratextual information (outside of the text) when compared to questions that help establishing the overall meaning of the text.

Tukey multiple comparison test was used to assess equivalence between types of tasks. The results showed that the Multiple Choice and True or False tasks are not equivalent, since there was significant difference between their results. Such a result highlights the difference between the requirements of the two types of task: while the Multiple Choice task offers alternatives that can help cognitive processing towards a plausible answer to the question, as already noted, the True or False task is more complex, for it requires that the student identify whether a statement is true or false and justify when false as well.

Comparisons between other types of tasks did not point out significant differences, indicating equivalences between them. This means that there is no significant influence on reading comprehension scores regardless of the chosen task.

Some peculiars can be seen regarding scores on each inferential category, especially in answers by school year. Among them, it is worth mentioning these performances: 6th year performance was higher than the others in category 1; the 8th year presented the highest scores among all three school years in categories 2, 3 and 4; the 7th year had a higher score than that of the 6th year in category 3. It is also noted that the 8th year results on category 1 were inferior to those of the 6th year, although in a small proportion; in the 7th year, in turn, results were lower in categories 1, 2 and 4. Despite these peculiarities, in the overall result, as already said, we can see a progressive evolution in the results, considering the school year variable.

Other important educational aspects may contribute to understand these peculiarities. Nevertheless, the objective of the research was achieved: to verify reading comprehension using questions created based on inferential categories.

One of these elements might relate to how work is organized by each teacher and in each school and school year. Even though there are official guidelines for the teaching practice, there is also some freedom for institutions and teachers to plan and execute as they choose to, considering their knowledge and characteristics of each school year.

We also have particularities on the part of the students that might affect these results: they sometimes need to change schools due to family circumstances, which can impair both teaching and learning processes. In these circumstances, some students may not have developed the knowledge required for inferential work. In addition, there is also no guarantee that the teacher will be able to help in this matter.

From the study carried out, we can see the educational importance of the ability to make inferences for reading comprehension, since it is the intratextual, extratextual and global relationships that will determine what will be understood, involving linguistic materiality, the previous knowledge of the reader, their cognitive style and the entire contextual situation at the moment of reading. Thus, it is noted that students have a greater difficulty in processing implicit information, as they require links to those stored in memory. This poses an obstacle to the processing of linguistic units and to attain textual comprehension.

To contribute to the pedagogical planning of teaching reading in this school group, it is important to consider well those didactic situations that focus on the development of the student’s ability to analyze different categories of inference. They certainly must be taught to work with different types of information, and to understand them, whether they are intratextual, extratextual or global, since these are all essential for the effective realization of the comprehension process. In addition, the data referring to the different types of tasks can be used to select the most adequate for the development of reading comprehension in such school years, as well as to show the tasks that need to be further developed given the difficulty these students presented with certain formats and requirements.

Acknowledgments

The authors would like to thank the contributions of doctoral students Gabrielle Perotto (CAPES scholarship) and Giselle Liana Fetter (CAPES scholarship) and of the scientific initiation scholarships Agata Marmitt (FAPERGS), Dhaiele Santana Schmidt (BPA / PUCRS) and Diovana da Silveira Baldez (FAPERGS).

REFERÊNCIAS

• ADAM, J.-M. A Linguística: introdução à análise textual dos discursos. São Paulo: Cortez, 2008.
• BARETTA, D.; PEREIRA, V.W. Compreensão literal e inferencial em alunos do Ensino Fundamental. Signo, Santa Cruz do Sul, v.43, n.77, p.53-61, 2018. Disponível em: https://online.unisc.br/seer/index.php/signo/article/view/11533/pdf Acesso em: 03 mar. 2020.
  » https://online.unisc.br/seer/index.php/signo/article/view/11533/pdf
• BALASS, M.; NELSON, J. R.; PERFETTI, C. A. Word learning: An ERP investigation of word experience effects on recognition and word processing. Contemporary educational psychology, San Diego, v.35, n.2, p.126-140, 2010. Disponível em: https://doi.org/10.1016/j.cedpsych.2010.04.001 Acesso em: 01 abr. 2019.
  » https://doi.org/10.1016/j.cedpsych.2010.04.001
• BAZERMAN, C. Handbook of research on writing: history, society, school, individual, text. Abingdon: Routledge, 2009.
• CADIME, I. et al. Compreensão de textos: diferenças em função da modalidade de apresentação da tarefa, tipo de texto e tipo de pergunta. Análise Psicológica, Lisboa, v.35, n.3, p.351-366, 2017. Disponível em: https://doi.org/10.14417/ap.1234 Acesso em: 01 abr. 2019.
  » https://doi.org/10.14417/ap.1234
• COLOMER, T.; CAMPS, A. Ensinar a ler e ensinar a compreender Porto Alegre: Artes Médicas, 2002.
• COSTA, J. C.; PEREIRA, V. W. Linguagem e cognição. Porto Alegre: EDIPUCRS, 2009.
• DASCAL, M. Strategies of understanding. In: PARRET, H.; BOUVERESSE, J. (ed.). Meaning and understanding Berlin; New York: W. De Gruyter, 1981. p.327-352.
• DELONG, K. A.; TROYER, M.; KUTAS, M. Pre-processing in sentence comprehension: Sensitivity to likely upcoming meaning and structure. Language and linguistics compass, Chichester, v.8, n.12, p 631-645, 2014. Disponível em: https://doi.org/10.1111/lnc3.12093 Acesso em: 01 abr. 2019.
  » https://doi.org/10.1111/lnc3.12093
• GIASSON, J. A compreensão na leitura Lisboa: ASA, 2000.
• GOODMAN, K. S. Unidade na leitura – um modelo psicolinguístico transacional. Letras de Hoje, Porto Alegre, n.86, p.9-43, 1991.
• GRAESSER, A. C.; SINGER, M.; TRABASSO, T. Constructing inferences during narrative text comprehension. Psycological Review, Princeton, v.101, n.3, p.371-395, 1994. Disponível em: https://doi.apa.org/doiLanding?doi=10.1037%2F0033-295X.101.3.371 Acesso em: 01 abr. 2019.
  » https://doi.apa.org/doiLanding?doi=10.1037%2F0033-295X.101.3.371
• GRONLUND, N. E. A elaboração de testes de aproveitamento escolar. São Paulo: EPU, 1974.
• KATO, M. A. O aprendizado da leitura. São Paulo: Martins Fontes, 2007.
• KINTSCH, W. Comprehension: a paradigm for cognition. Cambridge: Cambridge University Press, 1998.
• KOCH, I. V.; ELIAS, V. M. Ler e compreender os sentidos do texto. São Paulo: Contexto, 2011.
• LEFFA, V. J. Aspectos da leitura: uma perspectiva psicolinguística. Porto Alegre: Sagra-Luzzatto, 1996.
• MARCUSCHI, L. A. Produção textual, análise de gêneros e compreensão São Paulo: Parábola, 2008.
• PEREIRA, V. W. Um estudo comparativo sobre o desempenho em compreensão leitora de alunos da educação básica 2018. Relatório de pesquisa apresentado a FAPERGS, Porto Alegre, 2018.
• PEREIRA, V. W. Predição leitora e inferência. In: COSTA, J. C (org.). Inferências linguísticas nas interfaces Porto Alegre: EDIPUCRS, 2009. p.10-22.
• PEREIRA, V.W. Compreensão leitora em alunos do Ensino Médio. ReVEL, [s. l.], v.6, n.11, p.1-15, 2008.
• POERSCH, J. M. Por um nível metaplícito na construção do sentido textual. Letras de Hoje, Porto Alegre, v.26, n.4, p.127-143, 1991.
• SCLIAR-CABRAL, L. Processamento da leitura: recentes avanços das neurociências. In: PEREIRA, V. W.; COSTA, J. C. (org.). Linguagem e cognição: relações interdisciplinares. Porto Alegre: EDIPUCRS, 2009. p.49-60.
• SCLIAR-CABRAL, L. Processamento bottom-up na leitura. Revista de Estudos Linguísticos Veredas: Psicolinguística, Juiz de Fora, v.12, n.2, p.24-33, 2008. Disponível em: https://www.ufjf.br/revistaveredas/files/2009/12/artigo02.pdf Acesso em: 01 abr. 2019.
  » https://www.ufjf.br/revistaveredas/files/2009/12/artigo02.pdf
• SMITH, F. Compreendendo a leitura: uma análise psicolinguística da leitura e do aprender a ler. Porto Alegre: Artes Médicas, 2003.
• SMITH, F. Leitura significativa. Tradução de Beatriz Affonso Neves. Porto Alegre: Artmed, 1999.
• SÖHNGEN, C. O procedimento “cloze”. Letras de Hoje, Porto Alegre, v.37, p.65-74, 2002. Disponível em: http://revistaseletronicas.pucrs.br/ojs/index.php/fale/article/view/14174/9411 Acesso em: 15 maio 2018.
  » http://revistaseletronicas.pucrs.br/ojs/index.php/fale/article/view/14174/9411
• SPINILLO, A. G. A dimensão social, linguística e cognitiva da compreensão de textos: considerações teóricas e aplicadas. In: MOTA, M. P. da; SPINILLO, A. G. (org.). Compreensão de textos. São Paulo: Casa do Psicólogo, 2013. p.171-198.
• SPINILLO, A. G.; ALMEIDA, D. D. Compreendendo textos narrativo e argumentativo: há diferenças? Arquivos Brasileiros de Psicologia, Rio de Janeiro, v.66, n.3, p.115-132, 2015. Disponível em: https://www.redalyc.org/pdf/2290/229035339010.pdf Acesso em: 01 maio 2019.
  » https://www.redalyc.org/pdf/2290/229035339010.pdf
• TAYLOR, W. “Cloze procedure”: a new tool for measuring reability. Journalism Quarterly, Urbana, v.30, 1953. Disponível em: https://www.gwern.net/docs/psychology/writing/1953-taylor.pdf Acesso em: 19 maio 2019.
  » https://www.gwern.net/docs/psychology/writing/1953-taylor.pdf
• TINKELMAN, S. N. Melhorando os testes na sala de aula: um manual de procedimento na elaboração de testes para os professores. Rio de Janeiro: USAID, 1967.
• VIANNA, H. M. Testes em Educação São Paulo: IBRASA, 1982
• VIDAL-ABARCA, E.; RICO, G. M. Por que os textos são tão difíceis de compreender? as inferências são a resposta. In: TEBEROSKY, A. (org.). Compreensão da leitura: a língua como procedimento. Porto Alegre: Artmed, 2003. p.139-153.

Publication Dates

• Publication in this collection
  28 Feb 2022
• Date of issue
  2022

History

• Received
  10 Apr 2020
• Accepted
  1 Apr 2021