CLOZE TEST: EVIDENCE OF VALIDITY BY RESPONSE PROCESS

ABSTRACT

The objective of this research was to derive evidence of validity through the response process, through the analysis of the types of errors made by students, in a Cloze test and to analyze the difficulty of their items through morph syntactic analysis. For this research participated 593 elementary school students from de sixth to ninth grades from public schools. Two groups of protocols were formed, one with the children with the highest averages and the other with the lowest averages. The results showed that children with higher averages made more syntactic errors and those with lower averages made semantic errors. The assessment of homogeneity in the distribution of types of errors allowed us to find evidence of validity by the response process. The total numbers of grammatical classes found through morph syntactic analysis gave the text a balance regarding the difficulty of recovering words, indicating its use to measure reading comprehension.

Keywords:
reading; Cloze test; validity

RESUMO

O objetivo desta pesquisa foi derivar evidências de validade por processo de resposta, por meio da análise dos tipos de erros cometidos pelos alunos, em um teste de Cloze e analisar a dificuldade de seus itens por meio de análise morfossintática. Participaram 593 estudantes do ensino fundamental II, de escolas públicas. Foram formados dois grupos de protocolos, um com os das crianças com médias mais altas e outro com as médias mais baixas. Os resultados mostraram que as crianças com médias mais altas cometeram mais erros sintáticos e as com médias mais baixas os erros semânticos. A avaliação da homogeneidade na distribuição dos tipos de erros permitiu encontrar a evidência de validade por processo de resposta. Os números totais das classes gramaticais encontradas por meio da análise morfossintática conferiram ao texto um equilíbrio quanto à dificuldade de recuperação das palavras, indicando seu uso para medir a compreensão de leitura.

Palavras-chave:
leitura; teste de Cloze; validade

RESUMEN

El objetivo de esta investigación fue buscar evidencias de validad por proceso de respuesta, por intermedio del análisis de los tipos de errores cometidos por los alumnos, en un test de Cloze y analizar la dificultad de sus ítems por intermedio de análisis morfosintáctica. Participaron 593 estudiantes de la enseñanza básica II, de escuelas públicas. Se formaron dos grupos de protocolos, uno con los de los niños con promedios más altos y otro con los promedios más bajos. Los resultados apuntaron que los niños con promedios más altos cometieron más errores sintácticos y los con promedios más bajos los errores semánticos. La evaluación de la homogeneidad en la distribución de los tipos de errores permitió encontrar la evidencia de validad por proceso de respuesta. Los números totales de las clases gramaticales encontradas por intermedio del análisis morfosintáctico proporcionaron al texto un equilibrio con respeto a la dificultad de recuperación de las palabras, indicando su uso para medir la comprensión de lectura.

Palabras clave:
lectura; test de cloze; validad

INTRODUCTION

The “Cloze procedure” was introduced, by Taylor (1953), as a method of measuring the readability or diffi,ira, 2009).

The Cloze correction consists of verifying the correct filling of the gaps, assigning a point for the correct answers and zero for the errors (Boruchovitch, Santos, & Oliveira, 2007; Oliveira, Boruchovitch, & Santos, 2009). The difficulties of the items are related to intrinsic factors, which depend on structural elements and facilitators of the text and extrinsic factors, which include the student’s knowledge and strategies used by him to carry out the task. In this sense, Abraham and Chapelle (1992) consider that the successes in Cloze, as well as in other instruments, do not depend only on the subject’s ability, but on the difficulty of the text items, as already mentioned.

Scholars agree that this technique is efficient to assess reading comprehension, since it uses the semantic and syntactic aspects of the text (Bedento & Moreira, 1990; Bensoussam, 1990; Santos, Suehiro, & Oliveira, 2004; Santos, Primi, taxa, & Vendramini, 2002; Santos, Sisto, & Noronha, 2010). It is important to highlight that the reader’s prior knowledge stands out as a determining component, as it makes sure that only proficient readers can fill in the gaps in a coherent way. Therefore, the test is presented in a structured, simple way and with the possibility of being used from elementary school to higher education to assess reading comprehension (Cunha & Santos, 2014; Joly, Bonassi, Dias, Piovesan, & Silva, 2014; Joly & Piovesan, 2012; Klein & Lamprecht, 2012; Monteiro & Santos, 2013; Mota et al., 2012; Mota & Santos, 2014).

The current situation refers to the virtually inexistence of formal instruments for assessing comprehension that present psychometric quality indicators and are marketed by publishers. Thus, almost all the evaluation procedures found are dispersed in theses, dissertations, or scientific articles (Spinillo & Mahon, 2007), which are the only source of resources for professionals working with evaluation.

It is worth noting that several studies have been published in order to analyze the type of processing required and the skill assessed by the Cloze test, which basically depend on the method adopted in creating the gaps. There are a number of studies carried out for this purpose, whose Cloze texts were more complex in content and were applied to samples of university students. Among them, one can mention that of Santos et al. (2002) who found that certain types of gaps favor the use of context and another type of gap that allows the use of prior knowledge (lexical knowledge), as suggested by Storey (1997).

With pre-university students, who responded to a text prepared in Cloze, Oliveira, Cantalice and Freitas (2009) performed a morphosyntactic analysis for the classification of the items. The authors also considered the difficulty of each item to be completed, dividing them into three levels, namely, easy (29 items), medium difficulty (12 items) and difficult (5 items). In the results, they observed that the students got more prepositions right, followed by nouns and articles. The most difficult item was the adjective phrase.

In another study with university students in the Literature and Pedagogy courses, Rosa (2014) investigated the items in two versions of the Cloze test. The data revealed that in both tests the students got items considered to be of low difficulty. Few were able to get items considered difficult. The average number of correct answers in both courses was less than expected for university students, especially those who will be future readers’ trainers, as they will be teachers of elementary and high school.

Among the studies carried out with elementary school students, such as the sample of the present study, Cunha and Santos (2009) analyzed the errors made in a Cloze test, based on considerations on evidence of validity by the response process that, according to Prieto and Delgado (1999), is the study of the cognitive processes involved in the resolution of the items, including predominantly the analysis of response protocols. The presented study had 266 students between 8 and 13 years old, from third and fourth grades (current fourth and fifth years) from public and private schools in São Paulo that participated. The protocols of students with the best and worst scores were grouped and classified according to a scale of the types of errors committed (Garcia, 1998), namely, blank, phonological, lexical, syntactic and semantic. The results showed that students with higher averages made more lexical errors and those with lower averages more semantic errors. The authors concluded that evidence of validity was found through the response process, which allowed the assessment of homogeneity in the distribution of types of errors, according to what was theoretically expected.

Using Item Response Theory (IRT), Oliveira, Santos, Boruchovitch, and Rueda (2012) investigated not only the adjustment of a Cloze Test to the Rasch model, but also assessed the difficulty in responding to the item (DIF) due to the gender of students. A total of 573 students from the second cycle of elementary education from state public schools in the states of Sao Paulo and Minas Gerais participated in the research, most of them female, with an average age of 12.8 years. In the analysis of the instrument, consisting of the text “Coisas da Natureza”, the same used in the present study, the authors showed a good fit to the Rasch model, identifying that the gaps were filled in according to the expected standard, considering people’s ability and difficulty. As for the DIF, only three items were more favorable for one of the genders, which allowed the authors to verify that there was a balance in the answers given by the boys and girls.

A similar study with students in the first cycle of elementary school was carried out by Carvalho, Monteiro, Alcará and Santos (2013) to verify the parameters of the items and the subjects, through the IRT, in a Cloze test, which included quantitative and qualitative analyzes of the item map. They also investigated the presence of item differential functioning (DIF) in relation to sex. The authors confirmed the unidimensional aspect of the instrument; they found a mean theta higher than the average difficulty of the items and the presence of DIF was observed in some items according to the years of teaching, although in general the test was considered adequate.

The analysis of the research items presented showed that the different Cloze tests have been shown to be adequate measures to provide reliable data on students’ reading comprehension. Given the importance of Cloze, which has been widely used to assess reading comprehension, the present study aims to analyze the types of errors made by children, as well as the difficulty of their items through morphosyntactic analysis. It is important to remember that other item analysis studies have been carried out in Brazil with several of the Cloze tests available. The objective here intended is to analyze the errors made by the students, as well as a morphological analysis of the test items.

METHOD

Participants

A total of 593 elementary school students participated in the research, that is, from the 6th to the 9th grade, 136 (22.9%) from the 6th grade, 158 (26.6%) from the 7th grade, 155 (26.1%) from 8th grade and 144 (24.3%) of the 9th grade. A total of 295 (49.7%) were male and 298 (50.3%) female. The ages ranged from 11 to 15 years. The students came from three public schools in the countryside of the State of São Paulo, 196 (33.1%) of school 1, 69 (11.6%) of school 2 and 328 (55.3%) of 3.

Instrument

Cloze Test “Coisas da Natureza”¹ by Santos (2005)

It is a text prepared according to Cloze technique, in its traditional version; thus, the 5th words have been omitted. In the default site was placed a trace of proportional size to the word deleted. It contains approximately 250 words and 40 gaps, and the respondent should fill them in with the words he deems most appropriate to make sense of the text. The score is given considering zero for errors or blanks and one point for each correct answer. The correction is made literally, that is, the word must be written exactly as it was removed from the text.

The text has been specially developed to evaluate reading comprehension in elementary school students with students from 5th to 8th grade (current 6 to 9 years). It has evidence of criterion validity, since it separated students by school stage, forming a subgroup for each school year, with an increase in the average of the scores compatible with the progress in schooling. Results of further study by Oliveira, Boruchovitch and Santos (2007), extended its validity evidence.

Procedure

After approval by the Research Ethics Committee of Sao Francisco University, for proper authorization, data were collected. Each text was answered in approximately 20 minutes.

Data analysis

Through qualitative analysis, errors were categorized into five types: 1 - no answer; 2 - phonological error (orthographic and accent errors); 3 - lexical error (use of synonym); 4 - syntactic error and 5 - semantic error. Then, descriptive statistics tests were used to calculate the percentage of each type of error and the Chi-square test to verify the homogeneity of the distribution of these errors. The level of significance was set at p <0.05.

RESULTS

The average of all students’ hits (n = 593) was 19,85 with a standard deviation of 5.03. There were no subjects who reached the maximum points allowed by the instrument, with the minimum of points being 1 and the maximum 37.

The protocols were separated relating to Cloze tests of students who had the best and worst scores for the construction of a range of types of errors. In view of the large number of participants, it was decided to use the 30% protocols with the highest and lowest scores. Thus, the number of 24 protocols in the category “high averages” was obtained. Then, 24 student protocols were drawn in the “low average” category. The mistakes made by the students were categorized. Table 1 presents descriptive statistics of the types of errors committed in the Cloze by students who had high averages.

Table 1
Descriptive statistics of the types of errors in Cloze of students who had high averages.

Type of Error - High average students
Right word1	No answer	Phonological	Lexical	Syntactic	Semantic	Total
passarinho	0	3	7	0	0	10
e	0	0	0	0	0	0
parada	1	0	1	17	1	20
até	1	0	0	8	0	9
na	0	0	0	0	0	0
menos	0	0	0	0	0	0
mesmo	1	0	2	9	0	12
na	0	0	0	1	0	1
de	0	0	0	0	0	0
se	0	0	0	0	0	0
resolveu	0	0	0	24	0	24
do	0	0	0	0	0	0
pedaço	0	1	5	5	1	12
no	0	0	0	2	0	2
sempre	0	0	0	10	0	10
e	0	0	0	0	0	0
água	0	4	0	0	0	4
o	0	0	0	0	0	0
e	0	0	0	0	0	0
ele	0	1	0	8	0	9
e	0	0	0	0	0	0
a	0	0	0	1	0	1
aves	1	1	0	2	0	4
tentou	0	0	1	5	0	6
mas	0	1	0	4	0	5
embora	0	0	0	0	0	0
o	0	0	0	6	0	6
deu	0	1	0	4	0	5
tantas	1	0	10	12	1	24
não	0	0	1	0	0	1
contou	1	0	0	6	5	12
colega	1	0	9	7	6	23
de	0	0	0	0	0	0

• 1 passarinho (little bird); e (and); parada (stop); até (until); na (at); menos (less); mesmo (the same); de (of); se (if); resolveu (solved);do (of); pedaço (piece); no(at); sempre (always); água (water); o(the); ele (he); aves (birds); tentou (tried); mas(but); embora (although); deu (gave); tantas (so many); não (no); contou (told/said); colega (colleague/classmate); de (of); que (that); dos (of); fugirem (run away); assim (thus); sobrevivência (survival).

For students with high averages, syntactic errors were the most committed (n = 178), significantly higher than lexical errors (n = 43), which were the second most frequent type of error. Semantic errors were the least committed (n = 15). The words that most provided syntactic errors were: “resolveu” (n = 24), “dos” (n = 20) and “parada” (n = 17). Lexical errors were made in greater numbers with the words “tantas” (n = 10) and “colega” (n = 9). It is worth noting that the word “tantas” had a high frequency of errors in more than one category: syntactic and semantic and it did not only lead to lexical error. The word that led to the greatest number of phonological errors was “sobrevivência” (n = 7), mainly due to the absence of accentuation. Many words did not get any error from the students. It was also found that they left virtually no gaps (n = 9).

Continuing the analyzes, the Chi-square test was performed to assess the homogeneity in the distribution of the types of errors made by these students. These analyzes are shown in Table 2.

Table 2
Chi-square test to assess homogeneity in the distribution of the types of errors made in Cloze by students with high averages.

Types of error	F	Fo (%)	fe	(f-fe)2/fe
No answer	9	3,41	52,8	36,33
Phonological	19	7,19	52,8	21,63
Lexical	43	16,28	52,8	1,81
Syntactic	178	67,43	52,8	296,87
Semantic	15	5,69	52,8	27,06
Total	264	100	264	383,70

Table 2 shows that the degree of freedom for the test was 4, the χ2 index was 383.70. The index achieved was statistically significant (p <0.001), as the results did not show homogeneity in the distribution. The descriptive statistics of the types of errors made in Cloze are presented below, by students with low averages, in Table 3.

Table 3
Descriptive statistics of types of errors in Cloze of students with low averages.

Type of Errors - Low average students
Right word	No answer	Phonological	Lexical	Syntactic	Semantic	Total
passarinho	2	3	25	6	12	48
e	2	3	0	1	0	6
parada	4	0	0	5	14	23
até	3	0	0	15	5	23
na	3	0	0	15	0	18
menos	4	0	0	1	13	18
mesmo	7	0	0	3	14	24
na	1	0	0	3	4	8
de	4	1	0	4	11	20
se	4	1	0	12	3	20
resolveu	2	0	4	10	8	24
do	3	0	0	13	0	16
pedaço	3	0	3	5	12	23
no	2	0	0	11	5	18
sempre	5	0	0	9	10	24
e	2	3	0	0	2	7
água	1	9	0	0	0	10
o	1	0	1	4	3	9
e	1	1	0	4	4	10
ele	1	0	0	10	12	23
e	2	3	0	0	3	8
a	1	0	0	0	4	5
aves	4	1	0	6	8	19
tentou	3	0	0	10	9	22
mas	5	0	0	12	4	21
embora	2	6	0	1	7	16
o	4	0	0	10	8	22
deu	5	0	0	5	12	22
tantas	4	0	0	5	15	24
não	4	0	0	6	9	19
contou	7	0	0	2	15	24
colega	5	0	1	1	17	24
de	5	0	0	1	11	17
que	5	1	0	9	5	20
dos	4	0	0	11	9	24
fugirem	7	1	0	2	14	24
assim	4	0	0	6	12	22
sobrevivência	5	1	1	5	12	24
Total	131	34	35	223	306	729

The increasing order of the total errors made coincided with the criteria adopted for classifying the errors of students with low averages. The words that most provided phonological errors were “água” (n = 9) and “embora” (n = 6). As for lexical errors, the students made more mistakes in “passarinho” (n = 25), which appeared twice in the text. The syntactic errors were more committed in the words “até” (n = 15), “na” (n = 15) and “do” (n = 13). The words “colega” (n = 17), “tantas” and “contou” (n = 15) and “parada”, “mesmo” and “fugiram” (n = 14) were the ones that most provided semantic errors.

Finally, it was found that there were errors for all words. After this analysis, the Chi-square test was performed to assess the homogeneity in the distribution of the types of errors by category, of the students with the lowest averages, which is shown in Table 4.

Table 4
Chi-square test to assess homogeneity in the distribution of the types of errors made in Cloze by students with low averages.

Types of Errors	F	fo(%)	fe	(F-fe)2/fe
No answer	131	17,97	145,8	1,50
Phonological	34	4,67	145,8	85,73
Lexical	35	4,80	145,8	84,20
Syntactic	223	30,59	145,8	40,88
Semantic	306	41,97	145,8	176,02
Total	729	100	729	388,33

The χ2 index was 388.33, and the degree of freedom was 4. The significance index was p <0.001, which allowed us to infer that the distribution was not fair. Still according to the objectives of the study, a morphosyntactic analysis of the words deleted from the text

It can be seen, in Table 5, that the text “Coisas da Natureza” that the part of speech with more words suppressed is nouns (n = 8). As for the class of articles, it is found that there are only two.

Table 5
Morphosyntactic analysis of text items in Cloze.

Part of Speech	Items	Total
Verbs	11, 26, 30, 33, 38	5
Adverbs	4, 6, 17, 28, 32, 39	6
Nouns	1, 12, 14, 16, 19, 25, 34, 40	8
Pronoums	7, 22, 29, 31	4
Prepositions	5, 8, 9, 13, 15, 35, 37	7
Articles	20, 24	2
Conjunctions	2, 10, 18, 21, 23, 27, 36	7

DISCUSSION

Initially, it is worth remembering that the Cloze technique is the most used method to assess reading comprehension in Brazil (Cunha, 2009). It is considered that it is instigating to carry out an investigation like this, which proposes to access the lexical competence involved in the response process underlying the evaluation by the instrument. In addition, one more text is made available for future research with Elementary School students and scientific ways of analyzing the effectiveness of a test for reading comprehension.

Thus, this study sought to derive evidence of validity the response process through the analysis of errors made by Elementary School students in a text structured in the standards of the Cloze test, and to analyze the difficulty of its items through the analysis morphosyntactic of the words eliminated from the text. The results of the Chi-square tests showed that there was no homogeneity in the distribution of errors made by students, with a statistically significant index, both for students with high means obtained by calculating the test scores, and for those with low means. These results confer validity through the test response process, as the fact that there is no homogeneity in the distribution indicates increasing difficulties, according to the established and different parameters for the groups of students.

For students with high averages, syntactic errors were the most committed (n = 178) and semantics were the least committed (n = 15). This last result was expected, since these students should have more prior knowledge and strategies to find the words that would make sense of the text (Bensoussam, 1990).

The words that most provided syntactic errors were: “resolveu” (n = 24), “dos” (n = 20) and “parada” (n = 17), respectively verb, preposition and adjective. The verbs and adjectives are considered words that are difficult to recover, as they are classified as content words. As for the preposition, although it is considered of an easier degree, in this text, its placement may have impaired the recovery by the respondents, confirming that the creation of the gaps can impair understanding (Rosa, 2014; Santos et al., 2002).

Lexical errors were made in greater numbers with the words “tantas” (n = 10) and “colega” (n = 9), because synonyms like “muitas” and “amigo” were used, respectively, indicating that there was no prejudice to the understanding of the text. The word that led to the greatest number of phonological errors was “sobrevivência” (n = 7), mainly due to lack graphical accentuation. This data has not been observed in any of the studies researched in the literature, however, it can be hypothesized that many students are unaware the writing standard. It is often possible to observe in written productions that students write with deficits ranging from inadequate use of letters, as well as showing that they do not know how to use accentuation and or punctuation.

The criteria adopted for classifying the errors of students with low averages coincided with the increasing order of the total errors made, meeting the results of the study by Cunha and Santos (2009). Thus, the total number of semantic errors was 306, 20.4 times greater than the number of errors of students with higher averages. This result was as expected because it demonstrates the difficulty of these students in recovering the words of content through long-term memory (Abraham & Chapelle, 1992).

The words that most produced the phonological errors were “água”, a noun, but more for lack of a graphic accent and “embora”, an adverb, considered a difficult word to recover (Santos et al., 2002). As for lexical errors, the students made more mistakes with the word “passarinho”, because it appeared twice in the text.

This research also corroborated the results of studies previously carried out by Abraham and Chapelle (1992), in which adjectives, verbs and adverbs were among the most difficult gaps to be completed because they were classified as content words, related to the cohesive function of the speech. With regard to cognitive processing, content words require more semantic processing, with the recovery of words from long-term memory, as opposed to syntactic processing, which requires the recovery of words from the syntactic structure of sentences, in which the omitted word is inserted. Prepositions and articles were the easiest gaps to fill, as they form units with the words immediately adjacent. The nouns presented medium difficulty, but there was a great dispersion of the difficulty indexes.

The words “colega”, “tantas” and “contou” (n = 15) and “parada”, “mesmo” and “fugiram” (n = 14) were the ones that most provided semantic errors. However, verbs and adverbs are actually more difficult to retrieve grammatical categories and nouns can have diffusion of difficulty indexes (Santos et al., 2002). It is noteworthy that in the sample of protocols of students with low averages, mistakes were made in all words, in contrast to the sample of protocols of students with high averages, in which there were several words without any error.

In total, the text “Coisas da Natureza” contains, from the category of the most difficult words to recover, verbs (n = 5), adverbs (n = 6) and adjectives (n = 1), totaling 12. The words with medium level of difficulty, there are nouns (n = 8) and pronouns (n = 4), with a total of 12. The prepositions (n = 7), articles (n = 2) and conjunctions (n = 7), considered easier to total, 16. Based on the studies recovered (Cunha & Santos, 2009; Santos et. al., 2002), these total numbers of grammatical classes give the text a balance regarding the difficulty of recovering words. In this sense, it is worth mentioning the need that, in future investigations, the text studied here may be submitted to the Item Response Theory - Rasch Model to search for possible variations in the response to items due to the students’ gender, as they did Oliveira et al. (2012).

It is worth reflecting that the present research presented some limitations, among which we can mention the fact that only participants from a Brazilian State were included, which indicates the need for similar analyzes to be carried out with sample protocols with greater representation of the country as one all. The comparison of regional differences would bring the possibility of studying existing linguistic variations. It also seems necessary to investigate possible differences in the successes of students from private education and those who study in public education. New studies, with the objective of seeking answers to the limitations indicated here, should be proposed, configuring a future research agenda about the topic.

Reading comprehension skills are essential not only for the acquisition of formal school knowledge, but also for the formation of a critical and reflective awareness. A fluent reader is able to assign meaning to the content read, giving it new meanings, open to understanding and the transformation of reality itself. The subject’s worldview is often shaped by the way he interprets what he reads. From this perspective, the evaluation of reading comprehension is a topic that has long aroused the interest of researchers in the area for bringing up the discussion of the difficulties inherent to its acquisition and its expansion. The school, therefore, would have the role of ensuring that the reading comprehension process was fully developed over the years in which the individual is submitted to formal education.

Considering Cloze as a low-cost measure, easy to elaborate, apply and correct, it is important that studies which attest to the validity of its interpretations on reading comprehension are the focus of many investigations. That said, its use in the school environment seems quite plausible, as it allows for diagnosis, mapping and subsequent intervention, aiming to minimize the difficulties inherent in reading comprehension, as well as improving skills involved in this process, which are attention, creativity, verbal fluency, metalinguistic awareness, among others.

Based on their results, it would be possible to think of interventional programs that could enable teachers, especially those in basic education, to perceive themselves as readers, as it will be through them that new readers will be trained. In this sense, the role of the university stands out, whose political-pedagogical projects should predominantly be aimed at improving the education of Pedagogy and Literature teachers, so that they can achieve what the Basic Education Curriculum Guidelines - Portuguese Language (Brazil, 1997), who predict that the elementary school student should leave this stage of schooling having reached a level of understanding of literal and inferential reading.

Along the same lines, it is worth mentioning that, due to the lack of reading ability, the student often only decodes the text, but does not understand the content read. Thus, reading would not work as a means of promoting the development of questioning / critical thinking about reality. Unfortunately, what has been observed is that students have been victims of an exclusionary educational policy that trains readers, with only a sufficient reading level to deal more immediately with the reality in which they live. In contrast, what should be achieved is the training of readers, whose fluency, creativity and criticality of the content would allow them to actively participate in the socio-cultural reality. The lack of comprehension skills makes the citizen hostage to the information that is passed on to them by others.

As macro-structural changes require great financial investment from governmental spheres and take time to be implemented, if they are, the results of the present research show the possibility and the need for investment in intervention programs, as pointed out, which may be the result of decisions at the management level each school unit. The feasibility of decisions of this carat depends on the inclusion in the pedagogical projects of each school, of alternatives that make it possible in a serious and committed way to systematically evaluate the students’ reading comprehension skills, with wide involvement of the teachers so that, they based on the results of the evaluations, the ways of coping were defined with the structuring of programs in which students from different grades could participate effectively.

Evaluate well, with instruments with validity evidence that support the interpretations provided by them can be an important tool to face the difficulties that public education goes through. The assumption of this assumption can help us to improve our education with decisions made from reliable situational diagnoses.

REFERÊNCIAS

• Abraham, R. G.; Chapelle. C. A. (1992). The Meaning of Cloze Test Scores: An Item Difficulty Perspective. The Modern Language Journal, 76(4), 468-479.
• Bedento, J. M.; Moreira, L. C. (1990). Desempenho em leitura entre estudantes de enfermagem: um estudo com a técnica de Cloze. In Anais do2ºSimpósio Brasileiro de Comunicação em Enfermagem(pp. 701-714). Ribeirão Preto, SP.
• Bensoussan, M. (1990). Redundancy and cohesion Cloze. Journal of Research in Reading, 13(1), 18-37.
• Bormuth, J. R. (1968). Cloze test readability: Criterion reference scores. Journal of Educational Measurement, 5, 189-196.
• Boruchovitch, E.; Santos, A. A. A.; Oliveira, K. L. (2007). Análise da fidedignidade entre dois tipos de pontuação do Teste de Cloze. Psicologia em Pesquisa, 1(1), 41-51.
• Brasil. Ministério da Educação; Secretaria de Educação Fundamental(1997). Parâmetros Curriculares Nacionais: Língua Portuguesa Brasília: MEC/SEEF.
• Carvalho, L. F.; Monteiro, R. M.; Alcará, A. R.; Santos, A. A. A. (2013). Aplicação da TRI em uma medida de avaliação da compreensão de leitura. Psicologia: Reflexão e Crítica, 26(1), 47-57.
• Cunha, N. B. (2009). Pesquisas com o teste de Cloze no Brasil. In Santos, A. A. A.; Boruchovitch, E.; Oliveira, K. L. (Eds.), Cloze: um instrumento de diagnóstico e intervenção(pp. 79-117). São Paulo: Casa do Psicólogo.
• Cunha, N. B.; Santos, A. A. A. (2009). Validade por processo de resposta no teste de Cloze. Fractal, 21(3), 549-562.
• Cunha, N. B.; Santos, A. A. A. (2014). Estudo de validade do questionário de avaliação de consciência metatextual. Psicologia: Teoria e Prática, 16(1), 141-154.
• García, J. N. (1998). Manual de dificuldades de aprendizagem: linguagem, leitura, escrita e matemática(J. H. Rodrigues, Trad.). Porto Alegre: Artmed.
• Joly, M. C. R. A.; Bonassi, J.; Dias, A. S.; Piovezan, N. M.; Silva, D. V. (2014). Avaliação da compreensão de leitura pelo Sistema Orientado de Cloze (SOC). Fractal, 26(1), 223-242.
• Joly, M. C. R. A.; Piovezan, N. M. (2012). Avaliação do programa informatizado de leitura estratégica para estudantes do ensino fundamental. Paidéia, 22(51), 83-90.
• Klein, A. I.; Lamprecht, R. R. (2012). A compreensão em leitura e a consciência fonológica em crianças com transtorno de déficit de atenção e hiperatividade. Signo, 37(63), 25-54.
• Monteiro, R. M.; Santos, A. A. A. (2013). Recursos familiares e desempenho de crianças em compreensão de leitura. Psico, 44(2), 273-279.
• Mota, M. M. P. E.; Santos, A. A. A. (2014). O Cloze como instrumento de avaliação de leitura nas séries iniciais. Psicologia Escolar e Educacional, 18(1), 135-142.
• Mota, M. M. P. E.; Vieira, M. T.; Bastos, R. R.; Dias, J; Paiva, N.; Mansur-Lisboa, S.; Andrade-Silva, R. (2012). Leitura contextual e processamento metalinguístico no português do Brasil: um estudo longitudinal. Psicologia: Reflexão e Crítica, 25(1), 114-120.
• Oliveira, K. L.; Boruchovitch, E.; Santos, A. A. A. (2007). Compreensão de leitura em alunos de sétima e oitava séries do ensino fundamental. Psicologia Escolar e Educacional, 11(1), 41-49.
• Oliveira K. L.; Boruchovitch, E.; Santos, A. A. A. (2009). A técnica de Cloze na avaliação da compreensão em leitura. In Santos, A. A. A.; Boruchovitch, E.; Oliveira, K. L. (Eds.), Cloze: um instrumento de diagnóstico e intervenção (pp. 47- 70). São Paulo: Casa do Psicólogo.
• Oliveira, K. L.; Cantalice, L. M.; Freitas, F. A. (2009). Compreensão em leitura no ensino médio: análise de acertos por item. In Santos, A. A. A.; Boruchovitch, E; Oliveira, K. L. (Eds.), Cloze: um instrumento de diagnóstico e intervenção (pp. 165-185). São Paulo: Casa do Psicólogo .
• Oliveira, K. L.; Santos, A. A. A.; Boruchovitch, E.; Rueda, F. J. M. (2012). Compreensão da Leitura: Análise do funcionamento diferencial dos itens de um teste de Cloze. Psicologia: Reflexão e Crítica, 25(2), 221-229.
• Prieto, G.; Delgado, A. R. (1999). Medición cognitiva de las aptitudes. In Olea, J.; Ponsoda, V.; Prieto, G. (Eds.), Tests informatizados: Fundamentos y aplicaciones (pp. 207-226). Madrid: Pirámide.
• Rankin, E. F.; Culhane, J. W. (1969). Comparable Cloze and multiple-choice comprehension test scores. Journal of Reading, 13, 193-198.
• Rosa, M. T. (2014). Compreensão leitora em formadores de leitores: um estudo com alunos e professores dos Cursos de Letras e Pedagogia (Dissertação de Mestrado em Educação). Universidade Estadual de Londrina, Londrina.
• Santos, A. A. A. (2005). O Cloze na avaliação da compreensão em leitura com alunos de 5ª a 8ª série Manuscrito não publicado.
• Santos, A. A. A.; Boruchovitch, E.; Oliveira, K. L. (Eds.). (2009). Cloze: um instrumento de diagnóstico e intervenção São Paulo: Casa do Psicólogo .
• Santos, A. A. A.; Primi, R.; Taxa, F.; Vendramini, C. M. M. (2002). O teste de Cloze na avaliação da compreensão em leitura. Psicologia: Reflexão e Crítica, 15(3), 549-560.
• Santos, A. A. A.; Sisto, F. F.; Noronha, A. P. P. (2010). TONI 3: Forma A e teste de cloze: evidências de validade. Psicologia: Teoria e Pesquisa, 26(3) 399-405.
• Santos, A. A. A.; Suehiro, A. C. B.; Oliveira, K. L. (2004). Habilidades em compreensão em leitura: um estudo com alunos de psicologia. Estudos de psicologia, 43(2), 29-41.
• Spinillo, A. G.; Mahon, E. R. (2007). Compreensão de texto em crianças: comparações entre diferentes classes de inferência a partir de uma metodologia on-line. Psicologia: Reflexão e Crítica, 20(3), 463-471.
• Storey, P. (1997). Examining the test-taking process: A cognitive perspective on the discourse cloze test. Language Testing, 14(2), 214-231.
• Taylor, W. L. (1953). “Cloze procedure”: A new tool for measuring read abilility. Journalism Quarterly, 30(4), 415-433.
• Taylor, W. L. (1956). Recent developments in the use of “Cloze Procedure”. Journalism Quarterly, 33(1), 42-48.
• Weaver, W.; Kingston, A. (1963). A factor analysis of the Cloze procedure and other measures of reading and language ability. Journal of Communication, 13, 252-261.

• 1
  Things of nature.

• This paper was translated from Portuguese by Ana Maria Pereira Dionísio.

Publication Dates

• Publication in this collection
  03 Aug 2020
• Date of issue
  2020

History

• Received
  14 Feb 2018
• Accepted
  03 June 2019