Acessibilidade / Reportar erro

Scale of Perception and Analysis of Reading Fluency - SOLAR: usability and consistency

ABSTRACT

Objective:

to analyze the consistency of responses of evaluators and to verify the usability of the perception and analysis of SOLAR (Science of Language and Reading)

Methods:

a non-experimental descriptive cross-sectional study. Elementary school teachers and undergraduate students of speech therapy and pedagogy participated as the judges. The judges used the SOLAR Scale as an instrument to analyze the reading fluency of twenty audios recorded by elementary school students. Usability comparisons were performed using the Kruskal Wallis test and two-by-two comparisons using the Mann Whitney test. To analyze the consistency of the responses between the judges, the calculation of the interclass correlation coefficient was used.

Results:

the analysis of the reliability of the answers showed an excellent intraclass correlation coefficient for all the skills of the Scale. As for the usability of the Scale, more than 80% of the participants positively evaluated the assessment. This was verified through the usability questionnaires which confirmed that it was easy to use, with clear guidelines and that users felt comfortable and confident using it.

Conclusion:

SOLAR obtained good results in reliability and consistency with excellent agreement between the evaluators. These results indicated satisfactory reliability of the SOLAR items and favorable qualitative ratings from users.

Keywords:
Reading; Education; Language Development; Learning; Evaluation Study

RESUMO

Objetivo:

analisar a consistência de respostas entre avaliadores e verificar a usabilidade da Escala de Percepção e Análise da Fluência Leitora - SOLAR.

Métodos:

estudo descritivo não experimental de corte transversal. Participaram como juízes professores do Ensino Fundamental e acadêmicos de Fonoaudiologia e Pedagogia. Os juízes utilizaram como instrumento a Escala SOLAR para a análise de fluência leitora de áudios de 20 estudantes do Ensino Fundamental. A comparação da usabilidade foi realizada por meio do teste Kruskal Wallis e comparação dois a dois pelo teste Mann Whitney. Para análise da consistência de respostas entre juízes foi utilizado o cálculo do coeficiente de correlação interclasse. O nível de significância utilizado foi de 5%.

Resultados:

a análise da fidedignidade das respostas demonstrou coeficiente de correlação intraclasse excelente para todas as habilidades da escala. Quanto à usabilidade do instrumento, mais de 80% dos participantes o avaliaram positivamente. Verificou-se, por meio dos questionários de usabilidade, que é uma escala de fácil uso, com orientações claras, os seus usuários se sentem confortáveis e confiantes em utilizá-la.

Conclusão:

a SOLAR obteve bons indicadores de fidedignidade e consistência, com excelente concordância entre os avaliadores. Tais resultados indicam confiabilidade satisfatória dos itens e avaliações qualitativas favoráveis dos usuários.

Descritores:
Leitura; Educação; Desenvolvimento da Linguagem; Aprendizagem; Estudo de Avaliação

Introduction

Reading is a complex mental activity, which requires the use of different cognitive processes to achieve the interpretation of a written text. Such processes start with the understanding that lines are letters and that these letters are represented by sounds11. Alves LM, Barcelos J, Scoralick R, Martin Celeste LC. Monitoramento da Fluência leitora. In: Coares AM, Capovilla FC, Simão JROR, Neves LM, editors. Caminhos da aprendizagem e inclusão: Entretecendo múltiplos saberes. Belo Horizonte: Artesã, 2021. p.57-70..

There are two decoding routes in reading processing. The phonological route uses grapheme-phoneme conversion, translating letters or their groupings into phonemes. The lexical route triggers reading by representation of familiar words which are in the reader's mental lexicon22. Coltheart M, Rastle K, Perry C, Langdon R. Projection mapping as a means of creative visual expression and its future prospects. Psychol Rev. 2010;108(I):450-6.

3. Morton J. Facilitation in Word recognition experiments causing change in the Logogen Model. In: Press P, editor. Processing os visible language, v 1. New York; 1979. p.259-68.
-44. Morton J. Interaction of information in word recognition. Psychol Rev. 1969;76(2):165-78. and the visual representation of a familiar word can link to the meaning of the word through the semantic system and with that, the word can be articulated. The decoding route directly influences the reader's fluency, considering that in the lexical route, access to the lexicon is faster, whereas in the phonological route, the reader will take more time to decode the word, and consequently reading speed will be reduced which directly impacts reading fluency55. Cunha VLO, Martins MA, Capellini SA. Relação entre fluência e compreensão leitora em escolares com dificuldades de aprendizagem. Psicol Teor e Pesqui. [journal on the internet] 2017 [accessed 2021 oct 10]; 33:1-8. Available at: https://www.scielo.br/j/ptp/a/4G3GsrWzMSk7SSQ7FtrSC5h/?lang=pt
https://www.scielo.br/j/ptp/a/4G3GsrWzMS...
,66. Salles JF de, Parente MA de MP. Processos cognitivos na leitura de palavras em crianças: relações com compreensão e tempo de leitura. Psicol Reflexão e Crítica. [journal on the internet] 2002 [accessed 2021 oct 10]; 15(2):321-31. Available at: https://www.scielo.br/j/prc/a/3FcxqfWxLMVq3c83Gs3q8mJ/abstract/?lang=pt
https://www.scielo.br/j/prc/a/3FcxqfWxLM...
.

To achieve the ability to read fluently and understand what is written it is necessary to established reading processing routes and all the mental activities there are envolved. Reading fluency is the ability to read accurately, with a speed similar to that of conversation, in order to give expressiveness to what is read77. Alves LM, Celeste LC. Escala de percepção de fluência leitora. Rev Formação Docente. [journal on the internet] 2019 [accessed 2021 oct 10]; 11(2):195-204. Available at: https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
https://www.metodista.br/revistas/revist...
,88. Alves LM, Santos LF, Carvalho IM, Ribeiro GL, Freire LS, Martins-Reis VO et al. Reading speed in elementary school and junior high. CoDAS [journal on the internet]. 2021 [accessed 2021 oct 10]; 33(5):1-7. Available at: https://www.scielo.br/j/codas/a/PcCR78M7pjNWHKhtGs4Lt8f/?format=pdf⟨=en
https://www.scielo.br/j/codas/a/PcCR78M7...
. In this process, the word sequences are separated into tonal groups, with breathing pauses appropriately allocated, so as not to compromise the sound of the text and divide it into meaningful units.

According to the National Reading Panel (2000)99. NRP: National Reading Panel. Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instructions. Tecnical Report. USA: National Institute of Child Health & Human Development, NRP; 2000., the elements that make up reading fluency are accuracy, automatic processing (speed) and expressiveness. Accuracy refers to the ability to read correctly and accurately88. Alves LM, Santos LF, Carvalho IM, Ribeiro GL, Freire LS, Martins-Reis VO et al. Reading speed in elementary school and junior high. CoDAS [journal on the internet]. 2021 [accessed 2021 oct 10]; 33(5):1-7. Available at: https://www.scielo.br/j/codas/a/PcCR78M7pjNWHKhtGs4Lt8f/?format=pdf⟨=en
https://www.scielo.br/j/codas/a/PcCR78M7...
,1010. Hudson RF, Lane HB, Pullen PC. Reading fluency assessment and instruction: What, why, and how? Read Teach. 2005;58(8):702-14.. From the moment the reader manages to consolidate his reading, it develops quickly and without substantial effort. The reader will be able to allocate their cognitive resources to the ability of fluency and to make use of prosodic resources, printing melodic intonation, use of pauses and emphases in appropriate places, in addition to creating expressive sentences, which will help in understanding the meaning of the text1111. González-Trujillo MC, Calet N, Defior S, Gutiérrez-Palma N. Escala de fluidez lectora en español: midiendo los componentes de la fluidez. Estud Psicol. [journal on the internet]. 2014 [accessed 2021 oct 10]; 35(1):104-36. Available at: http://dx.doi.org/10.1080/02109395.2014.893651
http://dx.doi.org/10.1080/02109395.2014....
.

After the publication of the National Reading Panel report99. NRP: National Reading Panel. Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instructions. Tecnical Report. USA: National Institute of Child Health & Human Development, NRP; 2000., different authors and researchers in the field of education have studied the importance of reading fluency to the literacy process and its relationship with academic success. Students who have difficulties in maintaining reading fluency with adequate speed and expression of the different modalities of a text, we can see that for the process of reading to be carried out properly the cognitive skills responsible for the attentional mechanisms are generally task-oriented, decoding the words. For this reason, current studies show that the selective attention needed to understand the text will consequently be lower, which will influence the interpretation of the message read1212. Martins MA, Capellini SA. Relation between oral reading fluency and reading comprehension. CoDAS [journal on the internet]. 2019 [accessed 2021 oct 10]; 31(1):1-8. Available at: https://www.scielo.br/j/codas/a/ghNL5wkkLdLL9f5ZMbDbTkb/?lang=en
https://www.scielo.br/j/codas/a/ghNL5wkk...

13. O'Connor RE. Reading fluency and students with reading disabilities: how fast is fast enough to promote reading comprehension? J Learn Disabil. 2018;51(2):124-36.

14. Puliezi S, Maluf MR. A fluência e sua importância para a compreensão da leitura. Psico-USF. [journal on the internet] 2014 [accessed 2021 sep 10];19(3):467-75. Available at: https://www.scielo.br/j/pusf/a/jdXqdwBLBR hVswxj4Yj3dYs/?lang=pt&format=pdf
https://www.scielo.br/j/pusf/a/jdXqdwBLB...

15. Rasinski TV. Readers who struggle: why many struggle and a modest proposal for improving their reading. Read Teach [journal on the internet]. 2017 [accessed 2021 sep 10]; 70(5):519-24. Available at: https://timrasinski.com/presentations/rasinski-2017-rt-readers_who_struggle.pdf
https://timrasinski.com/presentations/ra...
-1616. Andrade AJL, Celeste LC, Alves LM. Characterization of reading fluency in elementary school students. Audiol - Commun Res [journal on the internet]. 2019 [accessed 2021 oct 10]; 24:1-8. Available at: https://www.scielo.br/j/acr/a/DtXN6jJxZptKd8CMvk5FL5s/?lang=en
https://www.scielo.br/j/acr/a/DtXN6jJxZp...
.

This scenario and the impact of reading fluency during the academic years gives one the ability to realize the huge importance of evaluating and monitoring students throughout their school years. Health and education professionals trained in the use of strategies that aim to help children in the development of this fluency during reading, commonly use different ways to assess each predictive skill, such as accuracy, speed and expressiveness. Reading speed, i.e., the number of words the reader can read in a given time, can be objectively determined by analyzing the number of words read in a one-minute interval (WPM). Likewise, when analyzing the number of correct words read in this same interval (WCPM), an objective measure of accuracy is obtained1717. Kawano CE, Kida ASB, Carvalho CAF, Ávila CRB. Fluency parameters and types of errors in the reading of students with signs of reading and writing difficulties. Rev Soc Bras Fonoaudiol [journal on the internet]. 2011 [accessed 2021 oct 10]; 16(1):9-18. Available at: https://www.scielo.br/j/rsbf/a/Dv5xytVH5NtYBtwsxwwjCPD/abstract/?lang=en
https://www.scielo.br/j/rsbf/a/Dv5xytVH5...
. The expressiveness measures concerns the way the reader interacts with the text, demonstrating emotions and attitudes when reading77. Alves LM, Celeste LC. Escala de percepção de fluência leitora. Rev Formação Docente. [journal on the internet] 2019 [accessed 2021 oct 10]; 11(2):195-204. Available at: https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
https://www.metodista.br/revistas/revist...
. Studies attempt to understand this, based on the objective analysis of phonetic parameters, such as voice intensity, melodic variation and the temporal organization of speech1818. Celeste LC, Pereira ES, Pereira NRR, Alves LM. Prosodic parameters of reading in 2nd to 5th grade students. CoDAS [journal on the internet]. 2018 [accessed 2021 oct 10]; 30(1):1-4. Available at: https://www.scielo.br/j/codas/a/hDGpC8xRCCrmQwHfJn5tqnr/?lang=en
https://www.scielo.br/j/codas/a/hDGpC8xR...
.

Objective and subjective assessment strategies have emerged as the scientific community has advanced studies in the areas of reading fluency. As an alternative that is easily accessible and applicable by teachers, parents and different professionals concerned with students reading, subjective scales are presented with the aim of evaluating through recorded or instantaneous reading, in a single instrument and without the need for specific training, the predictive skills of fluent reading77. Alves LM, Celeste LC. Escala de percepção de fluência leitora. Rev Formação Docente. [journal on the internet] 2019 [accessed 2021 oct 10]; 11(2):195-204. Available at: https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
https://www.metodista.br/revistas/revist...
.

Currently, among the instruments described in various studies, three of them stand out in the assessment of elementary school students. The National Assessment of Education Progress - NAEP was created in 2002 by the US Department of Education and allows for the assessment of a student's reading ability in four distinct reading levels, requiring the evaluator to judge the reading and classify it accordingly with the corresponding level1919. Daane MC, Campbell JR, Grigg WS, Goodman MJ, Oranje A. Fourth-grade students reading aloud: NAEP 2002 special study of oral Reading (NCES 2006-469). 2005. U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. Washington, DC: Government Printing Office.. The Multi-Dimensional Fluency Scale - MDFS, developed by Zutell and Rasisnski in 1991, attempts to assess children and adolescents based on four areas of reading fluency, allowing the sum of the scores of a Likert scale that will guarantee a classification of reading development between restricted and complete2020. Smith GS, Paige DD. A study of reliability across multiple raters when using the NAEP and MDFS rubrics to measure oral reading fluency. Read Psychol. [journal on the internet]. 2019 [accessed 2021 oct 10]; 40(1):34-69. Available at: https://doi.org/10.1080/02702711.2018.1555361
https://doi.org/10.1080/02702711.2018.15...
. In 2015, Alves, Lalain, Ghio, & Celeste2121. Alves LM, Lalain M, Ghio A, Celeste LC. Escala Multidimensional de Fluência em Leitura: avaliação perceptiva da leitura em escolares com e sem dislexia do desenvolvimento. In: Editora W, editor. Dislexia: novos temas, novas perspectivas. Rio de Janeiro; 2015. p.151-64. translated the MDFS into Brazilian Portuguese and French In order to assess the instrument's internal validity in other languages, the authors presented recordings of readings from 10 dyslexic students and 10 normal readers to 10 French judges and 10 Brazilian judges. The result was that the study showed that the evaluators of both groups had similar scores, in addition to evaluating the dyslexic students with a low score, and normal readers with a high score. Consequently the purpose of the study purpose was achieved.

The scales were proposed and developed based on English language readers. Therefore considering the specific characteristics of Brazilian Portuguese, an instrument was developed by Alves & Celeste77. Alves LM, Celeste LC. Escala de percepção de fluência leitora. Rev Formação Docente. [journal on the internet] 2019 [accessed 2021 oct 10]; 11(2):195-204. Available at: https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
https://www.metodista.br/revistas/revist...
. The reading perception fluency scale allows for the assessment of students' reading fluency through five domains, namely: fluidity, pauses, speed, expressiveness and intonation. The student's reading can then be judged using a Likert scale with a score from one to five, which will allow the classification of reading competency as undeveloped or fully developed77. Alves LM, Celeste LC. Escala de percepção de fluência leitora. Rev Formação Docente. [journal on the internet] 2019 [accessed 2021 oct 10]; 11(2):195-204. Available at: https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
https://www.metodista.br/revistas/revist...
.

Alves and associates developed a previous study with the scale and identified low agreement among judges in some of the evaluated parameters, suggesting a possible difficulty in differentiating between them11. Alves LM, Barcelos J, Scoralick R, Martin Celeste LC. Monitoramento da Fluência leitora. In: Coares AM, Capovilla FC, Simão JROR, Neves LM, editors. Caminhos da aprendizagem e inclusão: Entretecendo múltiplos saberes. Belo Horizonte: Artesã, 2021. p.57-70.. Four elementary school teachers, four parents of elementary school students and four speech therapy undergraduate students took part in the study. The judges were instructed to analyze 10 audios with readings by students with different classifications of reading fluency. Participants did not receive prior training on how to perform the analyses. From these results, a reduced format of the scale was developed with the objective of increasing the agreement of the answers among the judges in the evaluation through the tool. In this new model called Scale of Perception and Analysis of Reading Fluency - SOLAR, the authors grouped the skills fluidity and pauses, in addition to Intonation and Expressiveness2222. Alves LM, Celeste LC. SOLAR: Escala de Percepção e Análise da Fluência Leitora. Kognos, Belo Horizonte; 2021..

Considering the importance of monitoring and evaluating the reading fluency of students during their school years and the role played by subjective scales with the expansion of access for parents, students and professionals, this study aimed to analyze the consistency of responses between raters and verify the usability of the SOLAR Scale.

Methods

This is a non-experimental descriptive cross-sectional study approved by the Research Ethics Committee of the Federal University of Minas Gerais - UFMG, Brazil, opinion no 4.453.235. All participants signed a participant's information and consent form (FICF).

Elementary School teachers, speech therapy undergraduate students and pedagogy were invited to participate in the study, acting as judges of the audios to be evaluated. The recruitment of participants was carried out by advertising in colleges running courses of interest, as well as on social networks and in elementary schools. Data collection was carried out between November 2020 and March 2021.

Participants of both sexes who had signed the consent form participated in the training on how to use the scale and performed the assessments through Google online form. The groups studied were as follows, a teachers group composed of teachers from the 2nd to 9th grades of elementary schools of both public and private schools in Belo Horizonte and region and from Brasília, Federal District, with experience varying from between 1 and 30 years of experience. A group of speech language pathology students composed of students enrolled from the fifth period of the undergraduate course in speech language pathology at the federal university of Minas Gerais. A group of pedagogy students comprising of students enrolled from the fifth period of the undergraduate course in pedagogy in Belo Horizonte and region and from Curitiba - PA.

Participants who did not have internet access or who were not available to carry out the training given before filling out the online form, or did not fully comply with the evaluations of the audios, were excluded from the research.

The stages of the study were as follows selection and randomization of the audios, organization of the SOLAR scale as an online form, a training course for evaluating reading samples from the scale, and evaluation by judges of the reading samples and the usability of the tool.

To achieve the objectives of the study the participants used the SOLAR scale as an assessment tool (Figure 1)2222. Alves LM, Celeste LC. SOLAR: Escala de Percepção e Análise da Fluência Leitora. Kognos, Belo Horizonte; 2021..

Figure 1:
Reduced version of the perception of reading fluency Scale in printed format

Since this study was carried out during the COVID-19 pandemic, social distancing made it impossible to have meetings in person which would have utilized the printed scale to analyze the selected audios, therefore it was necessary to convert the scale to a digital format, transforming it into a Google form, which made it possible to expand the study group to participants in different regions of the country. With this new format, we tried to maintain the original design of the scale, using all the explanations of competences and assessment parameters, in addition to the colored scale for each existing level.

As an object of the reading fluency analysis, audio files from a database of 20 Elementary School students were used as a sample with 10 proficient readers and 10 students with a diagnosis of developmental dyslexia2323. Alves LM, Reis C, Pinheiro Â. Prosody and reading in dyslexic children. Dyslexia. 2015;21(1):35-49..

The following reading parameters were analyzed, according to the scale, fluidity, speed and expressiveness. The judges were responsible for listening to the students' recordings and to use the scale to classify each parameter of the evaluated audio from 1 to 5.

The files contained recordings of students reading a simple text2424. Scliar-Cabral L. Guia prático de alfabetização. Contexto, editor. São Paulo; 2003. aloud in a natural manner and the recordings were saved as .wav files without identifying the students. To organize the audios for analysis, a randomized sample was selected using Excel software.

Before starting the assessments, all the judges took part in online training on how to use the SOLAR Scale. The course consisted of a 40-minute lesson in which the researchers introduced the judges to the concept of reading fluency, as well as its components and also a presentation of the scale together with its objectives. In addition to the above the three parameters were evaluated and how they could be classified were explained. To show this audios were played that exemplified each one of them and their different forms of classification.

Microsoft Teams and Google Meet apps were used to carry out the course and the participants could answer questions related to the assessment of reading fluency and about using the scale itself. All questions asked by the participants were discussed and answered.

After completing the course the judges completed the reading fluency assessment step, answering the scale as presented in Google Forms. At this stage each participant received two links, one corresponding to the scale in a digital version, and the other with access to audios stored in a Google Drive folder. Each judge listened to audio readings lasting approximately 60 seconds. Each audio was numbered using a numerical sequence from 1 to 20 and which was to be evaluated in the form section numbered with the equivalent numeral.

After evaluating all the readings, the judges answered the System Usability Scale - SUS2525. Martins AI, Rosa AF, Queirós A, Silva A, Rocha NP. European Portuguese validation of the System Usability Scale (SUS). Procedia Comput Sci [journal on the internet]. 2015 [accessed 2021 oct 10]; 67(Dsai):293-300. Available at: http://dx.doi.org/10.1016/j.procs.2015.09.273
http://dx.doi.org/10.1016/j.procs.2015.0...
in order to assess their way of accessing the content of the tool used. The scale consisted of 10 confirmation statements with scores on a Likert scale ranging from 1+ (strongly disagree) to 5+ (strongly agree). In addition the participants answered nine yes and no questions about the System Acceptability Scale. The questionnaire aimed to evaluate the tool used in the study.

All the data was entered into an Excel spreadsheet created by the researchers and analyzed by running it against SPSS, a statistical software package.

To analyze usability by group of participants, a continuous variable was created that took into account the sum of the scores of each item presented, in which the values 4 and 5, neutral 3 and negative 0 and 1 were allocated.

To compare the usability results between the three groups, the Kruskal Wallis test was used and two-by-two comparisons were made using the Mann-Whitney test.

To assess the consistency between raters in order to answer the study's guiding question "Is there consistency in the use of the Scale between raters?" the intraclass correlation coefficient (ICC) was calculated. In this analysis, the proposals of Lii Koo2626. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med [journal on the internet]. 2016 [accessed 2021 oct 2]; 15(2):155-63. Available at: http://dx.doi.org/10.1016/j.jcm.2016.02.012
http://dx.doi.org/10.1016/j.jcm.2016.02....
were followed, as follows: CCI less than 0.5 are poor, values between 0.5 and 0.5 are moderate, values between 0.75 and 0.9 are good, and values greater than 0 .9 are excellent.

Results

The study included 56 judges, aged between 20 and 56 years of age who were residents of Belo Horizonte and surrounding region, Curitiba and Brasília. Participants were divided into three different groups, 21 teachers from the 2nd to 9th grades of Elementary School (37.5%), 25 speech therapy undergraduate students (44.6%) and 10 students from an undergraduate course in Pedagogy (17.9%).

To analyze the reliability and consistency between the raters, the intraclass correlation coefficient (ICC) was calculated for each skill assessed by the instrument, which reached an excellent degree26 for all measures of the scale (Table 1).

Table 1:
Intraclass correlation coefficient for the items of the perception of reading fluency scale

All the participants reported that they had enjoyed using the SOLAR Scale and felt motivated to continue its use when necessary, and that they would recommend the scale. Of the 56 participating judges, 98.2% consider the scale to be useful and that its guidelines were considered to be clear and easy to understand (Table 2).

Table 2:
Description of the applicability and usability of the reading fluency perception scale for teachers and speech therapy undergraduate students and pedagogy (n=56)

Overall, more than 80% of the participants positively evaluated the usability of the scale. Regarding the aspect of confidence in using the scale only 7.2% of the participants (3 judges from the teachers category and 1 judge from the category of undergraduate students in Pedagogy) did not feel confident using the instrument. In addition, 5.4% of the judges (including 2 from the teachers category and 1 judge from the undergraduate students in speech language pathology and audiology) consider that a steep learning curve was required to be able to deal with the scale.

To analyze the usability by groups of participants, a continuous variable was created that took into account the sum of the scores for each item presented in the previous table. The Kruskal Wallis test showed a statistically significant difference between the three groups (X2=30.217; p<0.001). When comparing two by two using the Mann-Whitney test, it was found that the group of speech language pathology students had the highest score, followed by the group of pedagogy students and the group of teachers, with the lowest.

Table 3:
Comparison of continuous variables of applicability and usability (25) of the reading fluency perception scale between the delimited groups (n=56)

As for training to use the Scale, all participants reported that the guidelines received were useful for their day to day lives.

Discussion

Reading fluency assessment is widely recommended by educational institutions in countries such as Australia, the United States and the United Kingdom. For the subjective assessment of this ability, studies have shown the efficiency of using assessment and monitoring scales11. Alves LM, Barcelos J, Scoralick R, Martin Celeste LC. Monitoramento da Fluência leitora. In: Coares AM, Capovilla FC, Simão JROR, Neves LM, editors. Caminhos da aprendizagem e inclusão: Entretecendo múltiplos saberes. Belo Horizonte: Artesã, 2021. p.57-70.,77. Alves LM, Celeste LC. Escala de percepção de fluência leitora. Rev Formação Docente. [journal on the internet] 2019 [accessed 2021 oct 10]; 11(2):195-204. Available at: https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
https://www.metodista.br/revistas/revist...
,1919. Daane MC, Campbell JR, Grigg WS, Goodman MJ, Oranje A. Fourth-grade students reading aloud: NAEP 2002 special study of oral Reading (NCES 2006-469). 2005. U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. Washington, DC: Government Printing Office.

20. Smith GS, Paige DD. A study of reliability across multiple raters when using the NAEP and MDFS rubrics to measure oral reading fluency. Read Psychol. [journal on the internet]. 2019 [accessed 2021 oct 10]; 40(1):34-69. Available at: https://doi.org/10.1080/02702711.2018.1555361
https://doi.org/10.1080/02702711.2018.15...
-2121. Alves LM, Lalain M, Ghio A, Celeste LC. Escala Multidimensional de Fluência em Leitura: avaliação perceptiva da leitura em escolares com e sem dislexia do desenvolvimento. In: Editora W, editor. Dislexia: novos temas, novas perspectivas. Rio de Janeiro; 2015. p.151-64..

The results in this study reached through the descriptive analysis of the usability of the reduced scale showed that the group of speech language pathology students had a higher score of the sum of the values of the system usability scale - SUS when compared to the group of pedagogy students. This fact may be related to the greater workload of topics related to reading fluency in the curriculum of the undergraduate courses, with speech language pathology students being more familiar with the study of reading fluency in their educational practice. The teachers' score was lower compared to the other two groups, and it is possible to assume that the teachers direct their observations more towards the content read (comprehension) and aspects related to the decoding and word recognition, consequently they may pay less attention to the assessment and monitoring strategies of reading fluency during their academic training, which means that these instruments are not commonly used in their professional practice, as reported by other authors2727. Dall' Aqua MF, Takiuchi N, Zorzi JL. Efetividade de um treinamento de professores de uma escola de educação especial usando os princípios dos métodos Hanen e V.O.E.: veja, ouça e espere. Rev. CEFAC [journal on the internet]. 2008 [accessed 2021 oct 1]; 10(4):433-42. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516-18462008000400003⟨=pt%0Ahttp://www.scielo.br/pdf/rcefac/v10n4/v10n4a03.pdf
http://www.scielo.br/scielo.php?script=s...
. Furthermore it can also be assumed that professionals may not have recognized fluency as a primordial content2828. Castejón LA, González-Pumariega S, Cuetos A. Teaching reading fluency in the clasroom: from research to practice. Ocnos Rev Estud sobre Lect. [journal on the internet] 2019 [accessed 2021 oct 1]; 18(2):17-30. Available at: https://revista.uclm.es/index.php/ocnos/article/view/ocnos_2019.18.2.1961/pdf
https://revista.uclm.es/index.php/ocnos/...
or they may have considered that the analysis of reading fluency is an area of activity for the speech therapist. These findings reinforce the importance of educational speech language pathology, especially when monitoring fluency in the school environment.

The result of the consistency of responses between the evaluators showed the judges that the grouping of the parameters pause and speed and intonation and expressiveness of the reduced scale were essential, considering that in a research carried out before this one11. Alves LM, Barcelos J, Scoralick R, Martin Celeste LC. Monitoramento da Fluência leitora. In: Coares AM, Capovilla FC, Simão JROR, Neves LM, editors. Caminhos da aprendizagem e inclusão: Entretecendo múltiplos saberes. Belo Horizonte: Artesã, 2021. p.57-70., in which the judges used the full scale for evaluation and did not receive any prior training on the use of the application, the agreement result obtained from the speed parameter was moderate agreement and the expressiveness was unsatisfactory, thus also completing the need for training for the effective use of the scale. The comparison between the studies carried out makes the authors consider the availability of the instrument together with previously documented material that aims to train those who aim to use the scale to assess or monitor the reading ability of students.

It is evident that among the groups of judges, the speech language pathology students had a greater knowledge about reading fluency, considering the entire approach to the topic during their training, which allows the correlation of some domains used for the assessment through the used scale. The reduction of the domains present in the scale of this study allowed the judges of both groups to assess the reading speed parameters, together with the assessment of pauses, allowing for a single and direct assessment to identify fast or slow reading, as well as the recognition of inappropriate pauses. The same happened with the intonation and expressiveness parameters, which allowed, in a single domain, the evaluation of the melodic variation of reading (intonation), and also the student's interaction with the text and its expressiveness. In this scenario, the study established its objective by obtaining excellent results in the correlation between the judges' evaluations, reinforcing the premise of the reliability of the use of the scale as a safe tool for the assessment of reading fluency in elementary school students.

The excellence of the result of the intraclass correlation coefficient (ICC) points to the effectiveness of the perception and analysis of the reading fluency scale (SOLAR), which can be widely used to assess and monitor the reading fluency of elementary school students. The Scale is freely accessible and can be used by educators and health professionals alike involved in the learning process and even by parents or the student themselves.

It is important to highlight the fact that for the efficient use of the scale students must be offered a text appropriate to their age and educational level, because if not and the text is beyond or below the student's capacity the results obtained may present results that are not consistent with the actual performance of the student. The scale can be used both in a clinical setting, in a school environment or at home when used by parents. Reading should always be in context, that is, full texts, so that it is possible to monitor all the reading parameters (fluidity, speed and expressiveness) and in individual observation11. Alves LM, Barcelos J, Scoralick R, Martin Celeste LC. Monitoramento da Fluência leitora. In: Coares AM, Capovilla FC, Simão JROR, Neves LM, editors. Caminhos da aprendizagem e inclusão: Entretecendo múltiplos saberes. Belo Horizonte: Artesã, 2021. p.57-70..

Resolution No. 309/2005 of the Federal Council of Speech-Language Pathology (CFFa) points out that the role of the educational speech language pathologist is to develop training and advice, through clarifications, lectures, guidance, and case studies among others2929. Conselho Federal de Fonoaudiologia. Resolução CFFa no 309. 2005.. Therefore, this study clearly shows the importance of the practice of educational speech language pathology, considering that the speech language pathologist can offer educational professionals the theoretical support on reading fluency, contributing to the change of pedagogical conduct when necessary, favoring the direction of the teacher's gaze and contributing for the development of the student3030. Luzardo R, Nemr K. Instrumentalização fonoaudiológica para professores da educação infantil. Rev. CEFAC [journal on the internet]. 2006 [accessed 2021 sep 10]; 8(3):289-300. Available at: https://www.redalyc.org/pdf/1693/169320536006.pdf
https://www.redalyc.org/pdf/1693/1693205...
,3131. Seno MP. Contribuições da fonoaudiologia educacional para a formação continuada de professores. Brazilian J Dev [journal on the internet]. 2020 [accessed 2021 oct 2]; 6(9):69507-21. Available at: https://www.brazilianjournals.com/index.php/BRJD/article/view/16792
https://www.brazilianjournals.com/index....
.

In addition the liaison between the speech therapists and educator appears as a way to ensure a more global and specialized assistance and caring for students, with the development of pedagogical practices that take into account the entire construction of the skills necessary for fluent reading, guaranteeing the improvement of students3131. Seno MP. Contribuições da fonoaudiologia educacional para a formação continuada de professores. Brazilian J Dev [journal on the internet]. 2020 [accessed 2021 oct 2]; 6(9):69507-21. Available at: https://www.brazilianjournals.com/index.php/BRJD/article/view/16792
https://www.brazilianjournals.com/index....
.

The excellent response from the judges and the usability of the instrument highlights that the present study brought a contribution to the scientific and academic world and presented an effectiveness in the use of a studied and developed tool for Brazilian Portuguese. The study highlighted that the SOLAR Scale is very accessible with little need to expand theoretical knowledge on the subject, in addition to what is offered in training. In addition, it can be included in the professionals' routine, which will allow both monitoring and detection of reading difficulties, favoring early interventions and monitoring of the child throughout the school years.

Some limitations of the study need to be considered, such as the restricted number of participants and it would be beneficial to expand the study to other health care professionals working in education, such as psychologists, in addition to parents and students themselves, which could include usage of the tool in the home environment and in tutoring situations. In addition, there is important new research that confirms the isolated effect of training, in which one group receives guidelines for using the scale and the other uses it without prior training.

Conclusion

SOLAR achieved good indicators of reliability and consistency, with excellent agreement between the evaluators. In addition, it was highlighted throughout the usability questionnaires that this is an easy-to-use scale, with clear and easy-to-understand guidelines, its users feeling comfortable with and confident to use it. Such results showed not only satisfactory reliability of the SOLAR items, but also favorable qualitative evaluations from users.

Acknowledgment

To the Pro-Reitoria de Pesquisa (PRPq) of the Universidade Federal de Minas Gerais for the support for the publication of the study and to CAPES for providing scientific material for the research.

REFERENCES

  • 1
    Alves LM, Barcelos J, Scoralick R, Martin Celeste LC. Monitoramento da Fluência leitora. In: Coares AM, Capovilla FC, Simão JROR, Neves LM, editors. Caminhos da aprendizagem e inclusão: Entretecendo múltiplos saberes. Belo Horizonte: Artesã, 2021. p.57-70.
  • 2
    Coltheart M, Rastle K, Perry C, Langdon R. Projection mapping as a means of creative visual expression and its future prospects. Psychol Rev. 2010;108(I):450-6.
  • 3
    Morton J. Facilitation in Word recognition experiments causing change in the Logogen Model. In: Press P, editor. Processing os visible language, v 1. New York; 1979. p.259-68.
  • 4
    Morton J. Interaction of information in word recognition. Psychol Rev. 1969;76(2):165-78.
  • 5
    Cunha VLO, Martins MA, Capellini SA. Relação entre fluência e compreensão leitora em escolares com dificuldades de aprendizagem. Psicol Teor e Pesqui. [journal on the internet] 2017 [accessed 2021 oct 10]; 33:1-8. Available at: https://www.scielo.br/j/ptp/a/4G3GsrWzMSk7SSQ7FtrSC5h/?lang=pt
    » https://www.scielo.br/j/ptp/a/4G3GsrWzMSk7SSQ7FtrSC5h/?lang=pt
  • 6
    Salles JF de, Parente MA de MP. Processos cognitivos na leitura de palavras em crianças: relações com compreensão e tempo de leitura. Psicol Reflexão e Crítica. [journal on the internet] 2002 [accessed 2021 oct 10]; 15(2):321-31. Available at: https://www.scielo.br/j/prc/a/3FcxqfWxLMVq3c83Gs3q8mJ/abstract/?lang=pt
    » https://www.scielo.br/j/prc/a/3FcxqfWxLMVq3c83Gs3q8mJ/abstract/?lang=pt
  • 7
    Alves LM, Celeste LC. Escala de percepção de fluência leitora. Rev Formação Docente. [journal on the internet] 2019 [accessed 2021 oct 10]; 11(2):195-204. Available at: https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
    » https://www.metodista.br/revistas/revistas-izabela/index.php/fdc/article/view/2091/1089
  • 8
    Alves LM, Santos LF, Carvalho IM, Ribeiro GL, Freire LS, Martins-Reis VO et al. Reading speed in elementary school and junior high. CoDAS [journal on the internet]. 2021 [accessed 2021 oct 10]; 33(5):1-7. Available at: https://www.scielo.br/j/codas/a/PcCR78M7pjNWHKhtGs4Lt8f/?format=pdf⟨=en
    » https://www.scielo.br/j/codas/a/PcCR78M7pjNWHKhtGs4Lt8f/?format=pdf⟨=en
  • 9
    NRP: National Reading Panel. Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instructions. Tecnical Report. USA: National Institute of Child Health & Human Development, NRP; 2000.
  • 10
    Hudson RF, Lane HB, Pullen PC. Reading fluency assessment and instruction: What, why, and how? Read Teach. 2005;58(8):702-14.
  • 11
    González-Trujillo MC, Calet N, Defior S, Gutiérrez-Palma N. Escala de fluidez lectora en español: midiendo los componentes de la fluidez. Estud Psicol. [journal on the internet]. 2014 [accessed 2021 oct 10]; 35(1):104-36. Available at: http://dx.doi.org/10.1080/02109395.2014.893651
    » http://dx.doi.org/10.1080/02109395.2014.893651
  • 12
    Martins MA, Capellini SA. Relation between oral reading fluency and reading comprehension. CoDAS [journal on the internet]. 2019 [accessed 2021 oct 10]; 31(1):1-8. Available at: https://www.scielo.br/j/codas/a/ghNL5wkkLdLL9f5ZMbDbTkb/?lang=en
    » https://www.scielo.br/j/codas/a/ghNL5wkkLdLL9f5ZMbDbTkb/?lang=en
  • 13
    O'Connor RE. Reading fluency and students with reading disabilities: how fast is fast enough to promote reading comprehension? J Learn Disabil. 2018;51(2):124-36.
  • 14
    Puliezi S, Maluf MR. A fluência e sua importância para a compreensão da leitura. Psico-USF. [journal on the internet] 2014 [accessed 2021 sep 10];19(3):467-75. Available at: https://www.scielo.br/j/pusf/a/jdXqdwBLBR hVswxj4Yj3dYs/?lang=pt&format=pdf
    » https://www.scielo.br/j/pusf/a/jdXqdwBLBR hVswxj4Yj3dYs/?lang=pt&format=pdf
  • 15
    Rasinski TV. Readers who struggle: why many struggle and a modest proposal for improving their reading. Read Teach [journal on the internet]. 2017 [accessed 2021 sep 10]; 70(5):519-24. Available at: https://timrasinski.com/presentations/rasinski-2017-rt-readers_who_struggle.pdf
    » https://timrasinski.com/presentations/rasinski-2017-rt-readers_who_struggle.pdf
  • 16
    Andrade AJL, Celeste LC, Alves LM. Characterization of reading fluency in elementary school students. Audiol - Commun Res [journal on the internet]. 2019 [accessed 2021 oct 10]; 24:1-8. Available at: https://www.scielo.br/j/acr/a/DtXN6jJxZptKd8CMvk5FL5s/?lang=en
    » https://www.scielo.br/j/acr/a/DtXN6jJxZptKd8CMvk5FL5s/?lang=en
  • 17
    Kawano CE, Kida ASB, Carvalho CAF, Ávila CRB. Fluency parameters and types of errors in the reading of students with signs of reading and writing difficulties. Rev Soc Bras Fonoaudiol [journal on the internet]. 2011 [accessed 2021 oct 10]; 16(1):9-18. Available at: https://www.scielo.br/j/rsbf/a/Dv5xytVH5NtYBtwsxwwjCPD/abstract/?lang=en
    » https://www.scielo.br/j/rsbf/a/Dv5xytVH5NtYBtwsxwwjCPD/abstract/?lang=en
  • 18
    Celeste LC, Pereira ES, Pereira NRR, Alves LM. Prosodic parameters of reading in 2nd to 5th grade students. CoDAS [journal on the internet]. 2018 [accessed 2021 oct 10]; 30(1):1-4. Available at: https://www.scielo.br/j/codas/a/hDGpC8xRCCrmQwHfJn5tqnr/?lang=en
    » https://www.scielo.br/j/codas/a/hDGpC8xRCCrmQwHfJn5tqnr/?lang=en
  • 19
    Daane MC, Campbell JR, Grigg WS, Goodman MJ, Oranje A. Fourth-grade students reading aloud: NAEP 2002 special study of oral Reading (NCES 2006-469). 2005. U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics. Washington, DC: Government Printing Office.
  • 20
    Smith GS, Paige DD. A study of reliability across multiple raters when using the NAEP and MDFS rubrics to measure oral reading fluency. Read Psychol. [journal on the internet]. 2019 [accessed 2021 oct 10]; 40(1):34-69. Available at: https://doi.org/10.1080/02702711.2018.1555361
    » https://doi.org/10.1080/02702711.2018.1555361
  • 21
    Alves LM, Lalain M, Ghio A, Celeste LC. Escala Multidimensional de Fluência em Leitura: avaliação perceptiva da leitura em escolares com e sem dislexia do desenvolvimento. In: Editora W, editor. Dislexia: novos temas, novas perspectivas. Rio de Janeiro; 2015. p.151-64.
  • 22
    Alves LM, Celeste LC. SOLAR: Escala de Percepção e Análise da Fluência Leitora. Kognos, Belo Horizonte; 2021.
  • 23
    Alves LM, Reis C, Pinheiro Â. Prosody and reading in dyslexic children. Dyslexia. 2015;21(1):35-49.
  • 24
    Scliar-Cabral L. Guia prático de alfabetização. Contexto, editor. São Paulo; 2003.
  • 25
    Martins AI, Rosa AF, Queirós A, Silva A, Rocha NP. European Portuguese validation of the System Usability Scale (SUS). Procedia Comput Sci [journal on the internet]. 2015 [accessed 2021 oct 10]; 67(Dsai):293-300. Available at: http://dx.doi.org/10.1016/j.procs.2015.09.273
    » http://dx.doi.org/10.1016/j.procs.2015.09.273
  • 26
    Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med [journal on the internet]. 2016 [accessed 2021 oct 2]; 15(2):155-63. Available at: http://dx.doi.org/10.1016/j.jcm.2016.02.012
    » http://dx.doi.org/10.1016/j.jcm.2016.02.012
  • 27
    Dall' Aqua MF, Takiuchi N, Zorzi JL. Efetividade de um treinamento de professores de uma escola de educação especial usando os princípios dos métodos Hanen e V.O.E.: veja, ouça e espere. Rev. CEFAC [journal on the internet]. 2008 [accessed 2021 oct 1]; 10(4):433-42. Available at: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516-18462008000400003⟨=pt%0Ahttp://www.scielo.br/pdf/rcefac/v10n4/v10n4a03.pdf
    » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1516-18462008000400003⟨=pt%0Ahttp://www.scielo.br/pdf/rcefac/v10n4/v10n4a03.pdf
  • 28
    Castejón LA, González-Pumariega S, Cuetos A. Teaching reading fluency in the clasroom: from research to practice. Ocnos Rev Estud sobre Lect. [journal on the internet] 2019 [accessed 2021 oct 1]; 18(2):17-30. Available at: https://revista.uclm.es/index.php/ocnos/article/view/ocnos_2019.18.2.1961/pdf
    » https://revista.uclm.es/index.php/ocnos/article/view/ocnos_2019.18.2.1961/pdf
  • 29
    Conselho Federal de Fonoaudiologia. Resolução CFFa no 309. 2005.
  • 30
    Luzardo R, Nemr K. Instrumentalização fonoaudiológica para professores da educação infantil. Rev. CEFAC [journal on the internet]. 2006 [accessed 2021 sep 10]; 8(3):289-300. Available at: https://www.redalyc.org/pdf/1693/169320536006.pdf
    » https://www.redalyc.org/pdf/1693/169320536006.pdf
  • 31
    Seno MP. Contribuições da fonoaudiologia educacional para a formação continuada de professores. Brazilian J Dev [journal on the internet]. 2020 [accessed 2021 oct 2]; 6(9):69507-21. Available at: https://www.brazilianjournals.com/index.php/BRJD/article/view/16792
    » https://www.brazilianjournals.com/index.php/BRJD/article/view/16792
  • Study carried out at the Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil.

Publication Dates

  • Publication in this collection
    06 Dec 2021
  • Date of issue
    2021

History

  • Received
    18 Oct 2021
  • Accepted
    22 Nov 2021
ABRAMO Associação Brasileira de Motricidade Orofacial Rua Uruguaiana, 516, Cep 13026-001 Campinas SP Brasil, Tel.: +55 19 3254-0342 - São Paulo - SP - Brazil
E-mail: revistacefac@cefac.br