Acessibilidade / Reportar erro

Adult reading assessment tools: a psychometric study

ABSTRACT

Purpose:

to investigate the psychometric properties of a performance test for assessment of word recognition and a self-report checklist for reading difficulties/ dyslexia indicators in adults.

Methods:

a total of 54 subjects aged 18-57 years (M = 24.16, SD = 7.34) who had complete high school or were enrolled in college were investigated. The following instruments were used: the Word Reading Competency Test for Adults (WRCT-2), and the self-report questionnaire Adult Dyslexia Checklist (ADC).

Results:

there were no differences in the participants' performance in terms of educational level or gender. Response time of WRCT-2 items was shorter for the Correct Word items than for the incorrect ones (Inversion, Phonological Exchange, Misspelling, and Pseudo-homophone). Of the 54 participants, 18.5% reported greater difficulties on the ADC. Analysis of the extreme groups showed that the participants with higher scores or difficulties on the ADC had poorer performance on the Correct Word items of the WRCT-2. Factor analysis resulted in a one-factor solution for the WRCT-2 items. Spearman-Brown and Cronbach's alpha values greater than 0.70 indicated good reliability of the instruments used. Low to moderate correlation between the instruments provided evidence of validity to both of them.

Conclusion:

this study presents psychometric data from two adult reading assessment instruments. Both instruments had satisfactory validity and reliability. Given the lack of national standardized instruments for adult reading assessment, the present study contributes to the development of future instrumentation for this field.

Keywords:
Reading; Evaluation; Validity of Tests; Adults

RESUMO

Objetivo:

investigar as propriedades psicométricas de um teste de desempenho para avaliação de reconhecimento de palavras e de um checklist de autorrelato de dificuldades de leitura/indicadores de dislexia, em uma amostra de adultos.

Métodos:

foram avaliados 54 sujeitos, idades entre 18 e 57 anos (M=24,16; DP = 7,34), com Ensino Médio completo ou cursando a graduação. As avaliações foram realizadas utilizando o Teste Computadorizado de Competência de Leitura de Palavras para Adultos (TCLP-2) e o questionário de autorrelato Adult Dyslexia Checklist (ADC).

Resultados:

não foram observadas diferenças de desempenho em função da escolaridade e do gênero. O tempo de resposta foi menor no julgamento dos itens Corretos do TCLP-2 em relação aos itens incorretos (inversão, troca fonológica, erro ortográfico e pseudopalavra homófona). 18,5% dos participantes relataram dificuldades mais severas no ADC. Análise de grupos extremos mostrou que participantes com maiores pontuações/dificuldades no ADC tiveram pior desempenho nos itens Corretos do TCLP-2. Análise fatorial retornou solução com fator único para tipos de itens do TCLP-2. Dados de precisão se mostraram adequados para ambos os instrumentos, com valores de Spearman-Brown e alfa de Cronbach maiores que 0,70. Relações de baixa a moderadas foram observadas entre os dois instrumentos, provendo evidências de validade a ambos.

Conclusão:

o estudo apresentou dados psicométricos de dois instrumentos para avaliação de leitura em adultos. Ambos mostraram índices satisfatórios de precisão e evidências de validade por relação com outras variáveis. Frente à carência de instrumentos padronizados para avaliação de leitura em adultos no contexto nacional, o estudo estende sua contribuição à futura instrumentalização desta área.

Descritores:
Leitura; Avaliação; Validade dos Testes; Adultos

Introduction

Reading skills have been widely studied in the literature due to their impact on virtually all life stages. Failure to acquire adequate reading skills may cause learners to have a disadvantage compared to others, not only in terms of academic performance, but also in terms of social and occupational performance11. Porcacchia SS, Barone LMC. Building readership: a reading workshop experience. Estud Psicol. 2011; 28(3):395-402.. Given the importance of reading in today's society, in the national context, several studies have emphasized the need to develop and make available assessment tools and interventions to prevent and remediate reading and writing difficulties. Some studies with significant contributions to this field have been conducted by different research groups 22. Araujo MR, Minervino CA da SM. Avaliação cognitiva: leitura, escrita e habilidades relacionadas. Psicol em Estud. 2008;13(4):859-65.

3. Dias NM, Leon CBR, Pazeto TCB, Martins GL, Prust AP, Seabra AG. Avaliação da leitura no Brasil: Revisão da literatura no recorte 2009-2013. Psicol Teor e prática. 2016;18(1):113-28.

4. Joly MCRA, Bonassi J, Dias AS, Piovezan NM, Silva DV da. Avaliação da compreensão de leitura pelo Sistema Orientado de Cloze (SOC). Fractal Rev Psicol. 2014;26(1):223-42.

5. Machado AC, Capellini SA. Aplicação do modelo de tutoria em tarefas de leitura e escrita para crianças com dislexia do desenvolvimento. Rev Ibero-Americana de Estudos em Educação. 2014;9(1):37-48.

6. Mota MMPE da, Santos AAA dos. O Cloze como instrumento de avaliação de leitura nas séries iniciais. Psicol Esc e Educ. 2014;18(1):135-42.

7. Salles JF de, Piccolo LDR, Zamo RDS, Toazza R. Normas de desempenho em tarefa de leitura de palavras/pseudopalavras isoladas (LPI) para crianças de 1o ano a 7o ano. Estud e Pesqui em Psicol. 2013;13(2):397-419.

8. Silva C da, Capellini SA. Eficácia do programa de remediação fonológica e leitura no distúrbio de aprendizagem. Pró-Fono R Atual. Cientí. 2010;22(2):131-8.

9. Jardini RSR, Ruiz LSR, Ramalho W, de Paula AV. Protocolo lince de investigação neurolinguística (PLIN): instrumento lúdico para conhecer habilidades de leitura e escrita. Rev. Psicopedag. 2015;32(97):49-60.

10. Oliveira DG de, Lukasova K, Macedo EC de. Avaliação de um programa computadorizado para intervenção fônica na dislexia do desenvolvimento. Psico-USF. 2010;15(3):277-86.

11. Corso HV, Piccolo LR, Miná CS, Salles JF. Normas de Desempenho em Compreensão de Leitura Textual para Crianças de 1o Ano a 6a Série. Psico. 2015;46(1):68-78.
-1212. Dias NM, Seabra AG. Instrumentos de avaliação de componentes da leitura: investigação de seus parâmetros psicométricos. Aval. Psicol. 2014;13(2):235-45..

However, a considerable gap has been identified. A recent literature review of articles published in national journals between 2009 and 2013 has identified more than 50 reading assessment instruments in the selected articles. Among the instruments identified, 12 were identified as instruments for the use of professionals, and they are commercially available or are published in books or journal articles. An important conclusion from that review was the lack of instruments to assess adolescents and, mainly, adults; the only resources identified for evaluation of this population were instruments based on the Cloze33. Dias NM, Leon CBR, Pazeto TCB, Martins GL, Prust AP, Seabra AG. Avaliação da leitura no Brasil: Revisão da literatura no recorte 2009-2013. Psicol Teor e prática. 2016;18(1):113-28. procedure.

The Cloze technique has been used as a device for assessment of reading comprehension. It consists of a reading passage from which some words are removed and replaced with blank underlined spaces. The reader's task is to read the passage and fill in the blanks with the words that they think best fits the passage1313. Suehiro ACB. Produção Científica sobre o Teste de Cloze. Psicol Esc e Educ. 2013;17(2):223-32.. The instruments based on the Cloze technique are not yet available for professional use, but some versions have been investigated and some data on their psychometric properties are available in theses and dissertations1414. Lima TH. Compreensão de leitura em alunos do ensino fundamental II: o teste de Cloze como alternativa de avaliação [Tese]. Itatiba (SP): Universidade São Francisco; 2015..

It is important to mention that reading comprehension is a complex process, and there are many models that attempt to explain it. Thus, it can be said that there are several skills such as word recognition, linguistic comprehension (including vocabulary), and reading fluency (which includes appropriate rate of speed and prosody), which contribute to reading competencies (for a review, see Seabra, Dias & Montiel1515. Seabra AG, Dias NM, Montiel JM. Estudo fatorial dos componentes da leitura: velocidade, compreensão e reconhecimento de palavras. Psico-USF. 2012;17:273-83. and Seabra & Dias1616. Seabra AG, Dias NM. Reconhecimento de palavras e compreensão de leitura: Dissociação e habilidades linguístico-mnemônicas preditoras. Neuropsicol Latinoamerican. 2012;4(1):43-56.). Therefore, reading comprehension evaluation would not be sufficiently informative to identify the components, whether deficient or poorly developed, which would underlie individual difficulties.

Accordingly, tools have been developed aiming to contribute to the assessment of reading components for adults. This is important because there is evidence indicating that reading difficulties in adolescence tend to persist into adulthood1717. Maughan B, Messer J, Collishaw S, Pickles A, Snowling M, Yule W, Rutter M. Persistence of literacy problems: spelling in adolescence and at mid-life. J Child Psychol Psychiatry. 2009;50(8):893-901.. Thus, based on an instrument originally developed for children reading assessment, the Bateria de Avaliação de Leitura e Escrita Computadorizada - BALE (Computerized Reading and Writing Assessment Battery)1818. Macedo EC, Capovilla FC, Diana C, Covre P. Desenvolvimento de instrumentos computadorizados de avaliação de funções cognitivas na WWW: O possível e o necessário. In: Macedo EC, Gonçalves MJ, Capovilla FC, Sennyey AL, organizadores. Tecnologia em (Re)habilitação Cognitiva: Um novo olhar para a avaliação e intervenção. São Paulo: Sociedade Brasileira de Neuropsicologia, Edunisc; 2002. p. 21-32., a version for adult assessment was developed, called Bateria de Avaliação de Leitura e Escrita para Adultos - BALE-2 (Adult Reading and Writing Assessment Battery); it is composed of three tests: Writing Test with word dictation (WTWD-2); Word Reading Competency Test (WRCT-2); and Phonological Awareness Test-2 (PAT-2 )1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.,2020. Pinto IS, Macedo EC. Busca por Evidências de Validade de uma Bateria Informatizada de Leitura e Escrita para Adultos. In: Anais da VII Jornada de Iniciação Científica. São Paulo: Universidade Presbiteriana Mackenzie; 2011..

Some studies have investigated this instrument, especially the Word Reading Competency Test (WRCT-2), and reasonable evidence of validity to this instrument has been provided1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.,2020. Pinto IS, Macedo EC. Busca por Evidências de Validade de uma Bateria Informatizada de Leitura e Escrita para Adultos. In: Anais da VII Jornada de Iniciação Científica. São Paulo: Universidade Presbiteriana Mackenzie; 2011.. The WRCT-2 assesses, specifically, word recognition allowing the differential assessment of the phonological route (in which the reader has to identify and decode graphemes into phonemes) and the lexical route (in which the reader recognizes the item written accessing orthographic lexical representations)2121. Ellis AW, Young AW. Human Cognitive Neuropsychology. Hove, UK: Erlbaum; 1988.. These routes are developed throughout the school years and are used depending on the type of item that the reader comes across. Pseudowords, for example, are read via the phonological route, whereas irregular words can only be properly read via the lexical route2222. Dias NM, Montiel JM, Seabra AG. Development and interactions among academic performance, word recognition, listening, and reading comprehension. Psicol Reflexão e Crítica. 2015;28(2):404-15.. In a study involving 100 college students (60% female), it was found that the average score of good readers on the WRCT-2 is 64 points. There was a low, but statistically significant, positive correlation between performance on the WRCT-2 and performance on phonological awareness and writing using dictation2020. Pinto IS, Macedo EC. Busca por Evidências de Validade de uma Bateria Informatizada de Leitura e Escrita para Adultos. In: Anais da VII Jornada de Iniciação Científica. São Paulo: Universidade Presbiteriana Mackenzie; 2011.. Another study compared the performance on the WRCT-2 of 28 adults with dyslexia with that of 26 adults without learning disabilities. The results showed that the dyslexic adults had significantly lower performance than the control group in terms of WRCT-2 total score and also in terms of all specific items of this instrument, which are: Correct Words, Misspelling, Inversion, Pseudo-homophone (pseudwords that are phonetically identical to a word, such as TROFÉL, the correct spelling in Portuguese is "troféu"), and Phonological Exchange. The response time of adults with dyslexia was significant longer than that of the control group1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.. These studies show that the WRCT-2 has evidence of validity based on its correlation with other variables, considering the convergence with similar constructs, such as writing and phonological awareness, and the relationship with an external criterion, dyslexia diagnosis. In the future, WRCT-2 may be included in the evaluation process (along with other instruments for the assessment of linguistic and reading comprehension, as well as fluency of adults with reading difficulties and contribute to the identification of specific problems and the diagnosis of reading disorders/dyslexia in this population.

In addition to performance tests, international studies have reported the use of scales or self-report checklists2323. Lindgrén SA, Laine M. The adaptation of an adult group screening test for dyslexia into Finland-Swedish: Normative data for university students and the effects of language background on test performance. Scand J Psychol. 2007;48(5):419-32.

24. Jones LO, Asbjornsen A, Manger T, Eikeland O. An Examination of the Relationship between Self-Reported and Measured Reading and Spelling Skills among Incarcerated Adults in Norway. J Correct Educ. 2011;62(1):26-50.

25. MacFarlane A, Al-Wabil A, Marshall CR, Albrair A, Jones SA, Zaphiris P. The effect of dyslexia on information retrieval: A pilot study. J Doc. 2013;66(3):307-26.

26. Pothos EM, Kirk J. Investigating learning deficits associated with dyslexia. Dyslexia. 2004;10(1):61-76.

27. Snowling M, Dawes P, Nash H, Hulme C. Validity of a protocol for adult self-report of dyslexia and related difficulties. Dyslexia. 2012;18(1):1-15.
-2828. Stoet G, Markey H, López B. Dyslexia and attentional shifting. Neurosci Lett. 2007;427(1):61-5. that are used to assess a wider range of reading problems and related skills and perform a quick screening assessment of suspected dyslexia. Thus, participants with a certain score or number of difficulty indicators can undergo a more detailed assessment. The Adult Dyslexia Checklist (ADC) is a self-report scale developed for this purpose. It is available in its original version (English) on the British Dyslexia Association (BDA) website and is frequently used in international studies to, for example, rule out the presence of dyslexia in control participants2525. MacFarlane A, Al-Wabil A, Marshall CR, Albrair A, Jones SA, Zaphiris P. The effect of dyslexia on information retrieval: A pilot study. J Doc. 2013;66(3):307-26.,2626. Pothos EM, Kirk J. Investigating learning deficits associated with dyslexia. Dyslexia. 2004;10(1):61-76.,2828. Stoet G, Markey H, López B. Dyslexia and attentional shifting. Neurosci Lett. 2007;427(1):61-5.. The ADC is composed of 20 questions about common characteristics of dyslexia. For each question, test takers are asked to identify whether they have that specific difficulty. Obviously, this instrument score will not provide enough information for a diagnostic of this disorder; scores indicative of severe difficulties require a more comprehensive assessment by a multidisciplinary team2929. Vinegrad MA. A revised adult dyslexia checklist. Educare. 1994;48:21-3.. Another study suggests that even in a non-clinical sample (without known reading disabilities), more specifically an incarcerated population in Norway, the ADC scores were correlated with performance on standardized reading and writing tests2424. Jones LO, Asbjornsen A, Manger T, Eikeland O. An Examination of the Relationship between Self-Reported and Measured Reading and Spelling Skills among Incarcerated Adults in Norway. J Correct Educ. 2011;62(1):26-50..

In the original study of this instrument, Vinegrad2929. Vinegrad MA. A revised adult dyslexia checklist. Educare. 1994;48:21-3. assessed 679 adults, students and non-students aged 18-68 years. The sample contained 32 known dyslexic individuals. The results showed that 60% of respondents gave four or fewer 'yes' responses and 90% gave eight or fewer 'yes' responses. A total of 9 or more 'yes' responses on the questionnaire was considered highly indicative of difficulty. In the national context, in the study carried out by Dias3030. Dias NM. Estudo das características psicométricas do Adult Dyslexia Checklist [Monografia]. Itatiba (SP): Universidade São Francisco; 2007. with college students, the ADC was translated and adapted to Brazilian Portuguese. The authors investigated the psychometric properties of the instrument and found a satisfactory internal consistency index and a weak relationship between the ADC and the students' academic grades. However, to the authors' knowledge, there are no checklists available or studies on their psychometric characteristics published in Brazil.

It is known that national studies commonly address school-age children, whereas in other countries, studies on adult reading assessment or reading disorders have been published since the 1980s1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.. Aiming at contributing to the future development of instrumentation for professional use, the aim of the present study is to investigate the psychometric properties of a performance test for assessment of word recognition, the WRCT-2, and a self-report checklist for reading difficulties/dyslexia indicators, the ADC, in a sample of adults. This study also aims to investigate validity evidence of these instruments based on correlation with other variables (relationship between performances on both tests), as well as their reliability.

Methods

This study was approved by the Research Ethics Committee of the Centro Universitário FIEO (FIEO University Center), under CAAE number 38966714.6.0000.5435.

Participants

A total of 54 subjects (61% female) aged between 18 and 57 years (M = 24.16, SD = 7.34) were assessed. Of these 54 participants, 20% had completed high school and 80% were enrolled in undergraduate courses in a private higher education institution. All participants were recruited from the university (the participants who had graduated high school were companions of university students or were visiting the university). The participants had no history of known neurological and/or psychiatric disorders or physical and/or sensory disabilities that could affect the instruments' scores.

Instruments

Reading Competency Test for Adults (WRCT-2)

The WRCT-2 1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.,2020. Pinto IS, Macedo EC. Busca por Evidências de Validade de uma Bateria Informatizada de Leitura e Escrita para Adultos. In: Anais da VII Jornada de Iniciação Científica. São Paulo: Universidade Presbiteriana Mackenzie; 2011. assesses word recognition ability. It consists of 80 items, and each item includes a spoken and a written word, which are either matching (congruent) or non-matching (incongruent) pairs. Firstly, the test taker listens to the spoken word and the written word is displayed on the computer screen for 1 second. Then the test taker has to determine whether the spoken-written word pair matches or not. Matching pairs are those in which the spoken-written words are the same; there are four types of non-matching pairs: Misspelling (Mi), in which not following a spelling rule leads to changing the sound pattern of the word being read (example of words in Portuguese, Mi: /carinho /- for the spoken word /carrinho /); Phonological Exchange (PE), when a sound in the word is replaced (PE: /envestiado /- for the spoken word /enfestiado /, which is the correct word in Portuguese); Pseudo-homophone (PH), despite being a pseudoword, the sound pattern is identical to that of the real word (PH: /Gibóia /- for the spoken word /jiboia /which is the correct spelling in Portuguese); and Inversion (I), in which a letter is in the wrong order or misplaced in the word (I: /texturizavle /- for the spoken word /texturizável /, which is the correct spelling in Portuguese). WRCT-2 is a computer-based test to be administered individually and which lasts 10 minutes. Morão1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.) pointed out that this test is appropriate for reading assessment of both dyslexic adults and adults without reading difficulties. The items are scored 1 point for each correct answer or 0 for incorrect ones.

Adult Dyslexia Checklist (ADC)

The ADC2929. Vinegrad MA. A revised adult dyslexia checklist. Educare. 1994;48:21-3. is a questionnaire consisting of 20 items related to common characteristics of dyslexia, including reading problems and difficulties in related skills, such as memory or phonological awareness, for example: 'Does it take a long time (or more than it should) to read a page of a book?' or 'When you say a long word, do you sometimes find it difficult to pronounce it or to remember all the sounds in the correct order?'. The questions are dichotomous; the respondent answers 'Yes' or 'No' to each question. It has been used to screen for possible reading problems, and the result can indicate the need for a more comprehensive assessment. Therefore, it may be useful in the assessment of adolescents and adults with suspected reading disabilities.

This instrument is answered by the respondents, who can be assisted if their reading difficulties make it difficult to answer the questions. The administration time is approximately 5 minutes. National studies on this instrument3030. Dias NM. Estudo das características psicométricas do Adult Dyslexia Checklist [Monografia]. Itatiba (SP): Universidade São Francisco; 2007.,3131. Dias NM, Gurgel LG, Bueno JMH, Seabra AG, Reppold CT. Estudo preliminar do Adult Dyslexia Checklist: Desempenho de universitários de dois estados brasileiros. In: Anais do III Congresso Brasileiro de Psicologia: Ciência e Profissão. São Paulo: FENPB; 2010. have reported satisfactory internal consistency and low but significant correlations with academic performance in college students. The percentage of respondents that gave eight or fewer 'yes' responses in these national studies was 94.8%3030. Dias NM. Estudo das características psicométricas do Adult Dyslexia Checklist [Monografia]. Itatiba (SP): Universidade São Francisco; 2007. and 93.6%3131. Dias NM, Gurgel LG, Bueno JMH, Seabra AG, Reppold CT. Estudo preliminar do Adult Dyslexia Checklist: Desempenho de universitários de dois estados brasileiros. In: Anais do III Congresso Brasileiro de Psicologia: Ciência e Profissão. São Paulo: FENPB; 2010., corroborating the findings of Vinegrad2929. Vinegrad MA. A revised adult dyslexia checklist. Educare. 1994;48:21-3. (90%). It is worth highlighting that 9 or more 'yes' responses can be considered indicative of severe difficulties thus requiring a more comprehensive assessment2929. Vinegrad MA. A revised adult dyslexia checklist. Educare. 1994;48:21-3..

Procedure

Participants were recruited from a higher education institution and were invited to participate in the study. The instruments were administered in a suitable room at the university, according to the availability of the participants. On the day of the evaluation, after signing the consent form, the participants responded to the ADC questions and took the WRCT-2. The instruments were administered individually in a single session lasting approximately 20 minutes. Participants who scored 9 or more 'yes' responses in the ADC were referred to the University Psychopedagogy Clinic, (where the study was carried out) for further evaluation since this score may be indicative of reading/writing disabilities.

Data Analysis

Descriptive statistics was conducted for the performances on the WRCT-2 and ADC. In order to justify the subsequent analyses considering the sample as a whole (high school graduates x college students) and to investigate the effect of gender, the Mann-Whitney U test was performed according to education level and gender. To compare the participants' performance on the items of WRCT-2, raw scores were converted to z-scores, and an intra-subject analysis (for both score and response time) was carried out. As for the ADC, a simple frequency analysis was carried out to determine the percentage of respondents that gave 9 or more 'yes' responses. The Mann-Whitney U test was then performed comparing the extreme groups (group that scored 9 or more in the ADC x group that scored 8 in the ADC), considering the WRCT-2 performances as dependent variables (nonparametric statistics was used in this analysis and in the analysis of the effect of gender and education level because of the unequal sample sizes of the groups). Factor analysis was carried out for the different types of items of the WRCT-2 (however, it could not be carried out for the ADC because the sample was not adequate for factor analysis, KMO= 0.571). The reliability of both instruments was estimated measuring the internal consistency and homogeneity of the items using Cronbach's alpha and the Spearman-Brown split-half reliability estimates. To verify evidence of validity based on correlation with other variables, the Pearson correlation coefficient was used to describe the relationship between the scores and response time (milliseconds) in the WRCT-2 (considering total score and the score of each type of item) and ADC scores.

Results

Descriptive statistics for WRCT-2 scores and response time and ADC scores are shown in Table 1. The Mann-Whitney U test revealed no significant differences between the groups regarding education level (WRCT-2 total score with U= 192, p= 0.338; WRCT-2-response time with U= 203, p= 0.472; ADC, with U= 198, p= 0.410). There were no significant differences between the performances on the WRCT-2 regarding gender (WRCT-2 total score with U= 263.5, p= 0.658; WRCT-2-response time with U= 279, p= 0.902), but a marginal trend was observed for the ADC (U= 195, p= 0.063), with a tendency of men to report greater difficulties.

Table 1:
Scores of the Adult Dyslexia Checklist and the Word Reading Competency Test for Adults

Two intra-subject analyses were carried to investigate the performance of the adult participants across the different types of items in the WRCT-2. One was based on the scores obtained and the other on the mean response time per item type. The results are shown in Table 2. With regard to the scores, the analysis revealed no significant differences in performance on the different types of items although descriptive statistics showed higher means for some types of items (easier items, e.g. Inversion; or more difficult items, e.g. Pseudo-homophones). On the other hand, significant differences were observed between the types of items in terms of response time. Pairwise comparison with Bonferroni correction showed that the response time was shorter for items with Correct Words than for the other types of items.

Table 2:
Descriptive and inferential statistics obtained after Within-Subjects ANOVA for performance (z-score and response time) on the Word Reading Competency Test for Adults

With respect to the ADC, a simple frequency analysis was carried out to determine the number of respondents that gave 9 or more 'yes' responses. It was found that 81.5% of the respondents gave up to 8 affirmative responses, and 18.5% gave 9 or more (among which only 20% were from the subsample of high school graduates, and thus this result does not reflect bias in the inclusion of participants with lower level of education). After identifying participants with score 9 or higher in the ADC, the analysis of the extreme groups was carried out comparing the performance on the WRCT-2 of participants with score 9 or higher (G9+) and those with score ≤ 8 points (G0-8) on the ADC. Descriptive and inferential statistics are presented in Table 3. Although the descriptive statistics show that the G9+ had, in general, lower scores on the WRCT-2 (except for Phonological Exchange), there was a significant effect of this group only for the item type Correct Word. There was no difference between the groups for response time.

Table 3:
Descriptive and inferential statistics obtained after analysis of the extreme groups in the Adult Dyslexia Checklist (Mann-Whitney) comparing the performances on the different types of items of the Word Reading Competency Test for Adults

Factor analysis of the different types of items of WRCT-2 (KMO= 0.747; Bartlett Sphericity Test <0.001), considering the scores only, resulted in a one-factor solution (Eigenvalue = 2.221; variance explained = 44.422%). All types of items of this instrument were grouped with factor loadings between 0.60 and 0.70.

Subsequently, the reliability of the instruments was measured based on internal consistency and the split-half method. For the WRCT-2 (N= 80 items), Cronbach's α= 0.77 and Spearman-Brown coefficient= 0.73. For the ADC (N = 20 items), Cronbach's α = 0.71 and Spearman-Brown coefficient= 0.82. In general, the indexes obtained can be considered satisfactory.

Finally, in order to verify evidence of validity based on correlation with other variables, Pearson's correlation was carried out to determine the relationship between the instruments. The results showed negative and significant low correlation between total performance on the WRCT-2 and the scores on the ADC (r = - 0.34, p = 0.013). Negative and significant correlations were also found between some types of items of the WRCT-2 and the ADC scores. There were moderate correlations between the ADC scores and the items type Correct Words and low correlations with the items type Misspelling. There were no significant correlations between the ADC scores and response time in the WRCT-2. Table 4 summarizes the results obtained.

Table 4:
Correlations between performance and total response time in the items of the Word Reading Competency Test for Adults and Adult Dyslexia Checklist scores

Discussion

The present study aimed to investigate the psychometric characteristics of a reading test (WRCT-2) and a self-report checklist that evaluates dyslexia indicators (ADC) in a sample of Brazilian adults. Thus, in view of the lack of adult reading assessment instruments33. Dias NM, Leon CBR, Pazeto TCB, Martins GL, Prust AP, Seabra AG. Avaliação da leitura no Brasil: Revisão da literatura no recorte 2009-2013. Psicol Teor e prática. 2016;18(1):113-28.,1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011., this study discussed two instruments that allow the assessment of this population. The first instrument, WRCT-2, assesses isolated word recognition ability and the second, ADC, can be considered as a screening tool and allows the screening of common characteristics of dyslexia.

Since there were no significant differences in performance on the WRCT-2 and ADC regarding education level (high school x undergraduate education), the sample was considered homogeneous and underwent subsequent analysis. The total mean score obtained in the WRCT-2 was similar to that found by Pinto and Macedo2020. Pinto IS, Macedo EC. Busca por Evidências de Validade de uma Bateria Informatizada de Leitura e Escrita para Adultos. In: Anais da VII Jornada de Iniciação Científica. São Paulo: Universidade Presbiteriana Mackenzie; 2011., who investigated college students only. It is important to remember that the present study also investigated high school graduates, and their performance was similar to that of the college students investigated, indicating that the WRCT-2 can also be used to assess reading in this population group.

As for gender, no significant differences were found between men and women in the performance on the WRCT-2, corroborating the results found by Morão1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.. However, a non-significant trend was observed in the ADC regarding gender, indicating that men tended to report greater difficulties. There were no differences in performance regarding the type of the item of the WRCT-2, i.e., there was no difference in terms of item difficulty in this instrument. Considering that a non-clinical sample (there were no individuals with known diagnosis of dyslexia in the sample) composed of high school graduates and college students was evaluated, it can be said that the results found were expected.

However, the results showed that the response time of the items type Correct Words were shorter than that of the incorrect words or pseudowords. This may be explained due to the type of processing involved in reading these types of item. For example, correct words can be read employing the logographic processing strategy (if they are very frequent), via the phonological route (if they are regular), or accessing the lexicon (lexical route). On the other hand, Inversions, Misspellings, and Phonological Exchanges can be read only via phonological or lexical routes. Peseudo-homophones can be read using the lexical processing only2121. Ellis AW, Young AW. Human Cognitive Neuropsychology. Hove, UK: Erlbaum; 1988.,2222. Dias NM, Montiel JM, Seabra AG. Development and interactions among academic performance, word recognition, listening, and reading comprehension. Psicol Reflexão e Crítica. 2015;28(2):404-15.. A possible explanation is that some type of conflict would result in reading words with errors or pseudowords (e.g. incongruent reliance of pathways when reading via phonological and lexical routes), requiring more time to respond to this type item, which would not occur (or it would occur to a lesser extent) in the case of Correct Words. Therefore, Correct Words were recognized more quickly by the participants.

With respect to the ADC, it was observed that 81.5% of the participants gave up to 8 affirmative responses. In the original study on this instrument, Vinegrad2929. Vinegrad MA. A revised adult dyslexia checklist. Educare. 1994;48:21-3.) found 90%, and previous studies carried out in Brazil with college students found 94.8% and 93.6%3030. Dias NM. Estudo das características psicométricas do Adult Dyslexia Checklist [Monografia]. Itatiba (SP): Universidade São Francisco; 2007.,3131. Dias NM, Gurgel LG, Bueno JMH, Seabra AG, Reppold CT. Estudo preliminar do Adult Dyslexia Checklist: Desempenho de universitários de dois estados brasileiros. In: Anais do III Congresso Brasileiro de Psicologia: Ciência e Profissão. São Paulo: FENPB; 2010.. Thus, in the present study there were more subjects with 9 or more affirmative responses. As previously mentioned, this fact was not associated with the inclusion of high school graduates not attending college in the sample; therefore, other variables need to be considered to explain these results. Since the sample studied here and those of previous studies include a population from very different regions, it is possible that socioeconomic variables may have had some influence. Future research should investigate the socioeconomic impact of education level (public school x private school) on the difficulties in the ADC.

Analysis of the extreme groups showed that the ADC is indeed able to identify individuals with greater reading difficulty, even in a non-clinical sample (participants without a known reading disability). It was found that the performance of the participants with score 9 or higher in the ADC was lower than that of the other participants on the reading of the Correct Words in the WRCT-2. In fact, descriptive statistics suggested that the group with the greatest difficulties on the ADC showed lower performance in all types of items of the WRCT-2; this trend, however, may not have been statistically significant due to the small number of subjects in this group and in the sample.

Factor analysis resulted in a one-factor solution for the different types of WRCT-2 items. Another study with a sample of elementary school children (using a version of WRCT specific for this age group) identified different factors that separate the items that can be read logographically or via the phonological route from those requiring the lexical route1515. Seabra AG, Dias NM, Montiel JM. Estudo fatorial dos componentes da leitura: velocidade, compreensão e reconhecimento de palavras. Psico-USF. 2012;17:273-83., which did not occur in the present study. This result may be related to aspects of development since that study by Seabra et al.1515. Seabra AG, Dias NM, Montiel JM. Estudo fatorial dos componentes da leitura: velocidade, compreensão e reconhecimento de palavras. Psico-USF. 2012;17:273-83. was carried out with children in elementary school, a period in which word recognition routes are being achieved and consolidated2222. Dias NM, Montiel JM, Seabra AG. Development and interactions among academic performance, word recognition, listening, and reading comprehension. Psicol Reflexão e Crítica. 2015;28(2):404-15.. On the other hand, the present study involved adults, and therefore it is expected that all routes to reading be fully consolidated, which can make it difficult to dissociate them in terms of performance.

This study also investigated the reliability of the WRCT-2 and ADC using the Cronbach's alpha coefficient and split-half method (Spearman-Brown). The results showed satisfactory internal consistency and item homogeneity. These measurement procedures are essential because the content of the items can be a source of measurement errors3232. American Educational Research Association, Association AP, National Council on Measurement in Education. The standards for educational and psychological testing. Washington: American Educational Research Association; 1999.,3333. Urbina S. Fundamentos da Testagem Psicológica. Porto Alegre: Artmed; 2007.. Finally, evidence of validity was investigated based on the correlation between the participant performances on the instruments. Negative significant correlations were found between reporting difficulties in the ADC and overall performance and performance on the items Correct Word and Misspelling of the WRCT-2. These relationships indicate that the higher the number of correct answers in these items in the WRCT-2, the fewer the reports of difficulties in the ADC. It is worth mentioning that these low to moderate correlations were present even in a non-clinical sample, which is consistent with the findings of a previous study2424. Jones LO, Asbjornsen A, Manger T, Eikeland O. An Examination of the Relationship between Self-Reported and Measured Reading and Spelling Skills among Incarcerated Adults in Norway. J Correct Educ. 2011;62(1):26-50..

These results provide evidence of validity to these instruments based on correlation with other variables, corroborating, in the case of WRCT-2, the results of previous studies1919. Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.,2020. Pinto IS, Macedo EC. Busca por Evidências de Validade de uma Bateria Informatizada de Leitura e Escrita para Adultos. In: Anais da VII Jornada de Iniciação Científica. São Paulo: Universidade Presbiteriana Mackenzie; 2011.. As for the ADC, although it is widely used internationally 2424. Jones LO, Asbjornsen A, Manger T, Eikeland O. An Examination of the Relationship between Self-Reported and Measured Reading and Spelling Skills among Incarcerated Adults in Norway. J Correct Educ. 2011;62(1):26-50.

25. MacFarlane A, Al-Wabil A, Marshall CR, Albrair A, Jones SA, Zaphiris P. The effect of dyslexia on information retrieval: A pilot study. J Doc. 2013;66(3):307-26.
-2626. Pothos EM, Kirk J. Investigating learning deficits associated with dyslexia. Dyslexia. 2004;10(1):61-76.,2828. Stoet G, Markey H, López B. Dyslexia and attentional shifting. Neurosci Lett. 2007;427(1):61-5. and made available by a major international organization (BDA- British Dyslexia Association), to date, the present study is the first one to investigate the psychometric properties of this instrument in Brazil (previous studies, including the translation of ADC into Portuguese, have been published in conference proceedings or undergraduate senior projects).

Among its limitations, it is worth highlighting that the present study did not investigate a clinical sample, which should be included in further studies in order to investigate evidence of validity of both instruments regarding an external criteria (diagnosis of dyslexia), as well as to verify the sensitivity and specificity of the instruments and analyze the types of errors and the most common difficulties among individuals with dyslexia. To date, few tools to evaluate adult reading have been developed and studied. Accordingly, although providing preliminary evidence, this study can bridge a gap in this field by presenting, along with initiatives of other research groups1313. Suehiro ACB. Produção Científica sobre o Teste de Cloze. Psicol Esc e Educ. 2013;17(2):223-32., new possibilities for psychoeducational assessment of adult reading.

Final Considerations

The present study investigated the WRCT-2, a performance test for assessment of word recognition ability, and the ADC, a self-report questionnaire for the screening of common dyslexia characteristics. These instruments were developed for adults in a field characterized by a lack of standardized instruments in the national context. Accordingly, this study showed that both instruments have satisfactory reliability. Moreover, evidence of validity based on correlation with other variables was provided to these two instruments. New studies investigating the evidence of validity of these instruments have been carried out in order to make them available for professionals in this field.

Referências

  • 1
    Porcacchia SS, Barone LMC. Building readership: a reading workshop experience. Estud Psicol. 2011; 28(3):395-402.
  • 2
    Araujo MR, Minervino CA da SM. Avaliação cognitiva: leitura, escrita e habilidades relacionadas. Psicol em Estud. 2008;13(4):859-65.
  • 3
    Dias NM, Leon CBR, Pazeto TCB, Martins GL, Prust AP, Seabra AG. Avaliação da leitura no Brasil: Revisão da literatura no recorte 2009-2013. Psicol Teor e prática. 2016;18(1):113-28.
  • 4
    Joly MCRA, Bonassi J, Dias AS, Piovezan NM, Silva DV da. Avaliação da compreensão de leitura pelo Sistema Orientado de Cloze (SOC). Fractal Rev Psicol. 2014;26(1):223-42.
  • 5
    Machado AC, Capellini SA. Aplicação do modelo de tutoria em tarefas de leitura e escrita para crianças com dislexia do desenvolvimento. Rev Ibero-Americana de Estudos em Educação. 2014;9(1):37-48.
  • 6
    Mota MMPE da, Santos AAA dos. O Cloze como instrumento de avaliação de leitura nas séries iniciais. Psicol Esc e Educ. 2014;18(1):135-42.
  • 7
    Salles JF de, Piccolo LDR, Zamo RDS, Toazza R. Normas de desempenho em tarefa de leitura de palavras/pseudopalavras isoladas (LPI) para crianças de 1o ano a 7o ano. Estud e Pesqui em Psicol. 2013;13(2):397-419.
  • 8
    Silva C da, Capellini SA. Eficácia do programa de remediação fonológica e leitura no distúrbio de aprendizagem. Pró-Fono R Atual. Cientí. 2010;22(2):131-8.
  • 9
    Jardini RSR, Ruiz LSR, Ramalho W, de Paula AV. Protocolo lince de investigação neurolinguística (PLIN): instrumento lúdico para conhecer habilidades de leitura e escrita. Rev. Psicopedag. 2015;32(97):49-60.
  • 10
    Oliveira DG de, Lukasova K, Macedo EC de. Avaliação de um programa computadorizado para intervenção fônica na dislexia do desenvolvimento. Psico-USF. 2010;15(3):277-86.
  • 11
    Corso HV, Piccolo LR, Miná CS, Salles JF. Normas de Desempenho em Compreensão de Leitura Textual para Crianças de 1o Ano a 6a Série. Psico. 2015;46(1):68-78.
  • 12
    Dias NM, Seabra AG. Instrumentos de avaliação de componentes da leitura: investigação de seus parâmetros psicométricos. Aval. Psicol. 2014;13(2):235-45.
  • 13
    Suehiro ACB. Produção Científica sobre o Teste de Cloze. Psicol Esc e Educ. 2013;17(2):223-32.
  • 14
    Lima TH. Compreensão de leitura em alunos do ensino fundamental II: o teste de Cloze como alternativa de avaliação [Tese]. Itatiba (SP): Universidade São Francisco; 2015.
  • 15
    Seabra AG, Dias NM, Montiel JM. Estudo fatorial dos componentes da leitura: velocidade, compreensão e reconhecimento de palavras. Psico-USF. 2012;17:273-83.
  • 16
    Seabra AG, Dias NM. Reconhecimento de palavras e compreensão de leitura: Dissociação e habilidades linguístico-mnemônicas preditoras. Neuropsicol Latinoamerican. 2012;4(1):43-56.
  • 17
    Maughan B, Messer J, Collishaw S, Pickles A, Snowling M, Yule W, Rutter M. Persistence of literacy problems: spelling in adolescence and at mid-life. J Child Psychol Psychiatry. 2009;50(8):893-901.
  • 18
    Macedo EC, Capovilla FC, Diana C, Covre P. Desenvolvimento de instrumentos computadorizados de avaliação de funções cognitivas na WWW: O possível e o necessário. In: Macedo EC, Gonçalves MJ, Capovilla FC, Sennyey AL, organizadores. Tecnologia em (Re)habilitação Cognitiva: Um novo olhar para a avaliação e intervenção. São Paulo: Sociedade Brasileira de Neuropsicologia, Edunisc; 2002. p. 21-32.
  • 19
    Morão C. Perfil de Desenvolvimento no TCLP-2 em Adultos com e sem Dislexia do Desenvolvimento. São Paulo; 2011.
  • 20
    Pinto IS, Macedo EC. Busca por Evidências de Validade de uma Bateria Informatizada de Leitura e Escrita para Adultos. In: Anais da VII Jornada de Iniciação Científica. São Paulo: Universidade Presbiteriana Mackenzie; 2011.
  • 21
    Ellis AW, Young AW. Human Cognitive Neuropsychology. Hove, UK: Erlbaum; 1988.
  • 22
    Dias NM, Montiel JM, Seabra AG. Development and interactions among academic performance, word recognition, listening, and reading comprehension. Psicol Reflexão e Crítica. 2015;28(2):404-15.
  • 23
    Lindgrén SA, Laine M. The adaptation of an adult group screening test for dyslexia into Finland-Swedish: Normative data for university students and the effects of language background on test performance. Scand J Psychol. 2007;48(5):419-32.
  • 24
    Jones LO, Asbjornsen A, Manger T, Eikeland O. An Examination of the Relationship between Self-Reported and Measured Reading and Spelling Skills among Incarcerated Adults in Norway. J Correct Educ. 2011;62(1):26-50.
  • 25
    MacFarlane A, Al-Wabil A, Marshall CR, Albrair A, Jones SA, Zaphiris P. The effect of dyslexia on information retrieval: A pilot study. J Doc. 2013;66(3):307-26.
  • 26
    Pothos EM, Kirk J. Investigating learning deficits associated with dyslexia. Dyslexia. 2004;10(1):61-76.
  • 27
    Snowling M, Dawes P, Nash H, Hulme C. Validity of a protocol for adult self-report of dyslexia and related difficulties. Dyslexia. 2012;18(1):1-15.
  • 28
    Stoet G, Markey H, López B. Dyslexia and attentional shifting. Neurosci Lett. 2007;427(1):61-5.
  • 29
    Vinegrad MA. A revised adult dyslexia checklist. Educare. 1994;48:21-3.
  • 30
    Dias NM. Estudo das características psicométricas do Adult Dyslexia Checklist [Monografia]. Itatiba (SP): Universidade São Francisco; 2007.
  • 31
    Dias NM, Gurgel LG, Bueno JMH, Seabra AG, Reppold CT. Estudo preliminar do Adult Dyslexia Checklist: Desempenho de universitários de dois estados brasileiros. In: Anais do III Congresso Brasileiro de Psicologia: Ciência e Profissão. São Paulo: FENPB; 2010.
  • 32
    American Educational Research Association, Association AP, National Council on Measurement in Education. The standards for educational and psychological testing. Washington: American Educational Research Association; 1999.
  • 33
    Urbina S. Fundamentos da Testagem Psicológica. Porto Alegre: Artmed; 2007.
  • Source: CAPES; CNPq

Publication Dates

  • Publication in this collection
    Sep-Oct 2016

History

  • Received
    30 Dec 2015
  • Accepted
    14 July 2016
ABRAMO Associação Brasileira de Motricidade Orofacial Rua Uruguaiana, 516, Cep 13026-001 Campinas SP Brasil, Tel.: +55 19 3254-0342 - São Paulo - SP - Brazil
E-mail: revistacefac@cefac.br