Predictability of sentences used in the assessment of speech intelligibility in dysarthria

Abstracts

OBJETIVO: Analisar a preditividade das sentenças utilizadas no protocolo de avaliação da inteligibilidade da fala nas disartrias. MÉTODOS: Foi realizado um estudo transversal com 120 voluntários divididos, aleatoriamente, em quatro grupos de 30 indivíduos. A partir da lista de 25 frases do protocolo, foram elaboradas quatro versões de listas. Em cada uma delas, uma palavra-alvo diferente foi omitida das sentenças. Cada grupo de participantes completou uma versão da lista preenchendo graficamente as lacunas com a primeira palavra que lhes viesse à mente, de modo que cada sentença ficasse coerente. Foram realizadas análises estatísticas para classificar as frases quanto a sua preditividade, comparar a preditividade das palavras em cada sentença e comparar as versões da lista. RESULTADOS: Três sentenças apresentaram alta preditividade; sete, média preditividade; e 15, baixa preditividade. Foram encontradas diferenças de preditividade entre as palavras-alvo em 84% das frases (p<;0,0054). Comparando as versões da lista, constatou-se que a versão 1 diferiu das demais (p<;0,002), mostrando-se menos previsível. CONCLUSÃO: No protocolo de avaliação da inteligibilidade da fala empregado neste estudo há predomínio de sentenças de baixa preditividade, sugerindo que estas podem ser empregadas de maneira confiável na avaliação da inteligibilidade. A análise da inteligibilidade em sentenças por palavras-alvo deve ser usada clinicamente, especialmente quando se conhece a preditividade das sentenças.

Inteligibilidade da fala; Percepção da fala; Fala; Distúrbios da fala; Disartria; Fonoaudiologia

ORIGINAL ARTICLE ARTIGO ORIGINAL

^IMultidisciplinary Program of Hospital Care to Adult and Elderly Health, Universidade Federal de São Paulo - UNIFESP - São Paulo (SP), Brazil

^IIDepartment of Specific Formation, Universidade Federal Fluminense - UFF - Nova Friburgo (RJ), Brazil

^IIIDepartment of Speech-Language Pathology and Audiology, Universidade Federal de São Paulo - UNIFESP - São Paulo (SP), Brazil

Correspondence address

ABSTRACT

PURPOSE: To analyze the predictability of sentences used in the protocol for the assessment of intelligibility of dysarthric speech.

METHODS: A cross-sectional study was conducted in 120 volunteers divided randomly into four groups of 30 individuals.Based on the list of 25 sentences from the protocol, four versions of lists were elaborated. In each version, a different target word was omitted from the sentences. Each group of participants completed a different list version by graphically filling in the blanks with the first word that came to mind, while keeping the sentences coherent. Statistical analyses were carried out to classify sentences according to their predictability, to compare predictability of the words in each sentence, and to compare the different list versions.

RESULTS: Three sentences presented high predictability; seven, average predictability; and fifteen, low predictability. Differences in the predictability of target words were found in 84% of the sentences (p<0.0054). The comparison of list versions revealed that version 1 differed from the others (p<0.002), and was less predictable.

CONCLUSION: Low predictability sentences predominated in the protocol for assessment of speech intelligibility used in this study, suggesting that these sentences can be used reliably for assessing intelligibility. Analysis of intelligibility in sentences based on target words can be used in clinical practice, especially when the predictability of the sentences is known.

Keywords: Speech intelligibility, Speech perception, Speech, Speech disorders, Dysarthria, Speech pathology (Speech, language and hearing sciences)

INTRODUCTION

Reduced speech intelligibility is one of the main manifestations in dysarthric speakers^(1,2). Judgments of intelligibility of speech can be influenced by several factors⁽¹⁾ related to the listener^(3-11), the speaker^(4,5,7-9) or even the assessment instrument^{(1,3-5,8,12-24)}.

With regard to assessment instruments, one such influencing factor is stimulus type⁽⁸⁾. Many studies have identified the effect of using different speech stimuli on intelligibility scores^{(4,5,8,12-15,17,18,23-26)}. Some of these studies have shown that the greater the number of semantic cues available to listeners, the higher the intelligibility scores of speakers^{(4,8,17,18,24)}.

The level of predictability of the sentences used in assessments can also influence intelligibility measures. Predictability can be defined as the redundancy or quantity of semantic content in a sentence⁽⁴⁾. Results of studies investigating the effect of predictability on the measuring of intelligibility indicate that sentences with high predictiveness tend to raise intelligibility scores of speakers compared to sentences with low predictiveness^(4,8,27,28).

The Intelligibility of Speech Assessment Protocol (PAIF)⁽²⁾ is a Brazilian instrument for assessing the intelligibility of speech in dysarthrics. In this protocol, intelligibility is assessed by means of different speech content, including the use of 25 sentences. Scores are based on an orthographic transcription of speech samples and calculated in the conventional manner, according to the percentage of correctly transcribed words.

Since the predictability of the sentences in this protocol is unknown and predictability is a factor influencing the measuring of intelligibility, knowledge of this aspect of the list of sentences can be useful to refine this assessment instrument. Against this background, the aim of the present study was to analyze the predictability of the sentences employed in the PAIF.

METHODS

The study was approved by the Research Ethics Committee of the Universidade Federal de São Paulo (UNIFESP) under process number 1055/09. A cross-sectional study in 120 volunteers (109 women and 11 men), whose mother tongue was Brazilian Portuguese, with a mean age of 21.2 years (SD=3.9) and mean schooling of 12.8 years (SD=1.3), was carried out. Individuals aged less than 18 years, with less than eight years of schooling or familiarized with the Speech Intelligibility Assessment Protocol, were excluded from the study. All participants signed a free and informed consent form (TCLE).

Four different versions of the 25-sentence list of the PAIF⁽²⁾ were devised. In the four versions, different target words were removed from each sentence. The words removed were selected according to their information weight within the sentence. All open-class words were selected, i.e. nouns, verbs, adjectives and adverbs, at the rate of two to four words per sentence, giving a total of 79 words removed (^{Appendix 1}). The closed-class words (conjunctions, articles and prepositions) were not analyzed because they are known to have low levels of predictability⁽¹¹⁾.

Participants were randomly divided into four groups each comprising 30 subjects. The groups filled in the gaps of a different version of the list. Subjects were instructed to complete each sentence with the first word that came to mind while ensuring the resulting sentences were coherent. The task was performed in groups of 20 participants, maximum, in a silent environment.

Answers were analyzed being scored as correct when participants had filled in the blank using the expected target word. The predictability of the words was calculated according to the percentage of correct answers for each target word. The predictability of the sentences was calculated according to the percentage of correct target words per sentence. According to the conventional proposal for intelligibility assessment contained in the PAIF, all of words contained in the phrases should be analyzed to calculate the intelligibility score for sentences. In the present study however, the classification of the level of predictability of each sentence was reached by considering all target words.

Cluster analysis was employed to classify the phrases by predictability level. This statistical technique is used to form groups by similarity, according to the natural relationships that the sample exhibits. The Chi-square test was applied to determine predictability differences in target words of each sentence. Analysis of Variance (ANOVA) was used to compare different versions with one another. Upon detection of differences, Bonferroni's multiple comparisons test was applied. The level of significance considered for all tests was 5%.

In order to assess possible interference of demographic variables (age, schooling and gender) on performance of each group of participants, the ANOVA test was applied for numeric variables, and Chi-square test for categorical variables. The four groups of volunteers were found not to differ for the variables gender or schooling. With regard to age however, a difference was found between the groups designated to lists 3 and 4 (p=0.018).

RESULTS

The sentences comprising the PAIF had 41.7% predictability overall (SD=18.1). Cluster analysis classified the sentences into three levels of predictability: low predictability (less than 50% correct), medium predictability (between 50 and 60% correct) and high predictability (greater than 60% correct (Table 1).

The words in each sentence were compared. Differences among target words in predictiveness level was found for 84% of the sentences (21/25), with at least one target word being more predictable than the other target words in the sentence (p<0.0054) (Table 2).

Comparison of the versions of the list using the ANOVA test identified differences between them (p<0.001)(Table 3). Bonferroni multiple comparisons revealed that list one differed to the others, in that the target words in the version of the list had a lower level of predictability (p<0.002). The version of the list in question is shown in ^{Appendix 1}.

DISCUSSION

Analysis of the predictability of each sentence from the PAIF revealed that, out of the 25 sentences, 15 had low predictability, seven had medium predictability, while only three high predictability. Thus, the majority of the sentences used in the protocol can be applied reliably in intelligibility of speech assessments of arthritic patients. Intelligibility scores based on these sentences would be ideal since the correct decoding of the speech signal by the listener would depend more on acoustic-signal dependent cues (altered in dysarthric speech) than on redundancy of the sentence, which is a signal-independent cue⁽³⁾.

One of the studies examined⁽⁴⁾ proposed an arbitrary criteria for classifying predictiveness of sentences into low (<25% correct), medium (between 25 and 75% correct) and high (>75% correct). In the present study, cluster analysis enabled sentences from the protocol to be grouped, adopting different cut-off criteria (<50%, between 50 and 60%, and greater than 60% correct, for low, medium and high predictiveness, respectively). However, the majority of sentences classified into the high predictability group using this analysis technique had a percentage of correct words which fell below the cut-off criteria for high predictability proposed in the cited study. This confirms the tendency toward lower predictability of sentences on the PAIF.

Subtle differences were observed between the groups of sentences in the cluster analysis (5.6% to 9.5%), particularly among the groups with low and medium predictability. Therefore, the selection of less predictable sentences based on the predictability ranking obtained, as commonly occurs in studies involving this variable^(4,8,27), may prove more suitable for refining the instrument.

Additionally, comparative analysis of the predictability level of target words in each sentence revealed the occurrence of target words with different indices of predictability. Thus, upon applying the PAIF, if speech intelligibility analysis is carried out on sentences focused only on target words, is also possible to select the less predictable open-class words from each sentence, thereby optimizing the assessment instrument.

Comparison of the four versions of the sentence lists from the protocol showed that on version one, target words with low predictability were chosen for most of the sentences. The difference found in version one did not stem from the variables related to the group of listeners, since the four groups did not differ for the variables gender or schooling while a difference for age was found only in groups three and four.

Considering that high semantic predictability can increase sentence intelligibility scores^(4,8,27,28), the use of this less predictable list can render the studied protocol more sensitive for assessing intelligibility. The list can be further refined by substituting sentences and key words with other, less predictable, instances not included in this version.

The studies found in the literature analyzed the effect of sentence predictability on scores of individuals with speech disorders^(4,8,27,28). In general, the procedure used for analyzing the predictability of the sentences was similar to that adopted in the present study. However, in order to classify predictiveness, some of these studies selected the more and less predictable sentences in terms of predictability ranking. Based on the results obtained, it can be concluded that future studies are warranted on alternative ways of measuring sentence intelligibility with the PAIF and that explore the impact of sentence predictability on the scores attained.

CONCLUSION

The Speech Intelligibility Assessment Protocol contains predominantly low predictability sentences, suggesting these can be employed reliably for assessing speech intelligibility. On the majority of sentences, target words differed in terms of degree of predictability, suggesting that selecting these particular words can optimize the assessment instrument. Among the different versions of the list proposed, version one contained words with lower predictability than the other versions, and can therefore be used to increase the sensitivity of the test by performing analysis of intelligibility in sentences based on target words.

ACKNOWLEDGEMENTS

The authors would like to thank the National Council for Scientific and Technological Development (CNPq), for the support provided to carry out this study, under process number 1055/09.

REFERENCES

• 1. Barreto SS, Ortiz KZ. Medidas de inteligibilidade nos distúrbios da fala: revisão critica da literatura. Pró-Fono. 2008;20(3):201-6.
• 2. Barreto SS, Ortiz KZ. Inteligibilidade: efeitos da análise de transcrição e do estímulo de fala. Pró-Fono. 2010;22(2):125-30.
• 3. Lieberman P. Some effects of semantic and grammatical context on the production and perception of speech. Lang Speech. 1963;6(3):172-87.
• 4. McGarr NS. The effect of context on the intelligibility of hearing and deaf children's speech. Lang Speech. 1981;24(Pt 3):255-64.
• 5. Monsen RB. The oral speech intelligibility of hearing-impaired talkers. JSpeech Hear Disord. 1983;48(3):286-96.
• 6. Ellis L, Reynolds L, Fucci D, Benjamin B. Effects of gender on listeners' judgments of speech intelligibility. Percept Mot Skills. 1996;83(3 Pt 1):771-5.
• 7. Finizia C, Lindström J, Dotevall H. Intelligibility and perceptual ratings after treatment for laryngeal cancer: laryngectomy versus radiotherapy. Laryngoscope. 1998;108(1 Pt 1):138-43.
• 8. Garcia JM, Cannito MP. Influence of verbal and nonverbal contexts on the sentence intelligibility of a speaker with dysarthria. J Speech Hear Res. 1996;39(4):750-60.
• 9. Liss JM, Spitzer SM, Caviness JN, Adler C. The effects of familiarization on intelligibility and lexical segmentation in hypokinetic and ataxic dysarthria. J Acoust Soc Am. 2002;112(6):3022-30.
• 10. D'Innocenzo J, Tjaden K, Greenman G. Intelligibility in dysarthria: effects of listener familiarity and speaking condition. Clin Linguist Phon. 2006;20(9):659-75.
• 11. Hustad KC. A closer look at transcription intelligibility for speakers with dysarthria: evaluation of scoring paradigms and linguistic errors made by listeners. Am J Speech Lang Pathol. 2006;15(3):268-77.
• 12. Tikofsky RS, Tikofsky RP. Intelligibility measures of dysarthric speech. J Speech Hear Res. 1964;64:325-33.
• 13. Sitler RW, Schiavetti N, Metz DE. Contextual effects in the measurement of hearing-impaired speakers' intelligibility. J Speech Hear Res. 1983;26(1):30-5.
• 14. Yorkston KM, Beukelman DR. A comparison of techniques for measuring intelligibility of dysarthric speech. J Commun Disord. 1978;11(6):499-512.
• 15. Beukelman DR, Yorkston KM. The relationship between information transfer and speech intelligibility of dysarthric speakers. J Commun Disord. 1979;12(3):189-96.
• 16. Hubbard DJ, Kushner D. A comparison of speech intelligibility between esophageal and normal speakers via three modes of presentation. J Speech Hear Res. 1980;23(4):909-16.
• 17. Yorkston KM, Beukelman DR. Communication efficiency of dysarthric speakers as measured by sentence intelligibility and speaking rate. J Speech Hear Disord.1981;46(3):296-301.
• 18. Ellis LW, Fucci DJ. Magnitude-estimation scaling of speech intelligibility: effects of listeners' experience and semantic-syntactic context. Percept Mot Skills. 1991;73(1):295-305.
• 19. Kempler D, Van Lancker D. Effect of speech task on intelligibility in dysarthria: a case study of Parkinson's disease. Brain Lang. 2002;80(3):449-64.
• 20. Hustad KC, Jones T, Dailey S. Implementing speech supplementation strategies: effects on intelligibility and speech rate of individuals with chronic severe dysarthria. J Speech Lang Hear Res. 2003;46(2):462-74.
• 21. Garcia JM, Crowe LK, Redler D, Hustad K. Effects of spontaneous gestures on comprehension and intelligibility of dysarthric speech: a case report. J Med Speech Lang Pathol. 2004;12(4):145-8.
• 22. Hustad KC. Estimating the intelligibility of speakers with dysarthria. Folia Phoniatr Logop. 2006;58(3):217-28.
• 23. Hustad KC. Effects of speech stimuli and dysarthria severity on intelligibility scores and listener confidence ratings for speakers with cerebral palsy. Folia Phoniatr Logop. 2007;59(6):306-17.
• 24. Bain C, Ferguson A, Mathisen B. Effectiveness of the speech enhancer on intelligibility: a case study. J Med Speech Lang Pathol. 2005;13(2):85-95.
• 25. Bent T, Buchwald A, Pisoni DB. Perceptual adaptation and intelligibility of multiple talkers for two types of degraded speech. J Acoust Soc Am. 2009;126(5):2660-9.
• 26. Donicht G, Pagliarin KC, Mota HB, Keske-Soares M. A inteligibilidade do desvio fonológico julgada por três grupos de julgadores. Pró-Fono.2009; 21(3):213-8.
• 27. Garcia JM, Dagenais PA. Dysarthric sentence intelligibility: contribution of iconic gestures and message predictiveness. J Speech Lang Hear Res. 1998;41(6):1282-93.
• 28. Barreto SS, Ortiz KZ. Speech intelligibility in dysarthric speakers: analysis of intelligibility scores in sentences. Rev CEFAC [Internet] 2010; Supl Esp. [Present at 39th Annual Convention International Association of Orofacial Myology - IAOM; 2010; São Paulo]

Appendix 1

Predictability of sentences used in the assessment of speech intelligibility in dysarthria

Erika Alexandre^I; Simone dos Santos Barreto^II; Karin Zazo Ortiz^III

Publication Dates

• Publication in this collection
  25 July 2011
• Date of issue
  2011

History

• Received
  06 Feb 2011
• Accepted
  09 May 2011