Predictability of sentences used in the assessment of speech intelligibility in dysarthria

Purpose: To analyze the predictability of sentences used in the protocol for the assessment of intelligibility of dysarthric speech. Methods: A cross-sectional study was conducted in 120 volunteers divided randomly into four groups of 30 individuals. Based on the list of 25 sentences from the protocol, four versions of lists were elaborated. In each version, a different target word was omitted from the sentences. Each group of participants completed a different list version by graphically filling in the blanks with the first word that came to mind, while keeping the sentences coherent. Statistical analyses were carried out to classify sentences according to their predictability, to compare predictability of the words in each sentence, and to compare the different list versions. Results: Three sentences presented high predictability; seven, average predictability; and fifteen, low predictability. Differences in the predictability of target words were found in 84% of the sentences (p≤0.0054). The comparison of list versions revealed that version 1 differed from the others (p≤0.002), and was less predictable. Conclusion: Low predictability sentences predominated in the protocol for assessment of speech intelligibility used in this study, suggesting that these sentences can be used reliably for assessing intelligibility. Analysis of intelligibility in sentences based on target words can be used in clinical practice, especially when the predictability of the sentences is known.

completed a different list version by graphically filling in the blanks with the first word that came to mind, while keeping the sentences coherent.Statistical analyses were carried out to classify sentences according to their predictability, to compare predictability of the words in each sentence, and to compare the different list versions.Results: Three sentences presented high predictability; seven, average predictability; and fifteen, low predictability.Differences in the predictability of target words were found in 84% of the sentences (p≤0.0054).The comparison of list versions revealed that version 1 differed from the others (p≤0.002), and was less predictable.Conclusion: Low predictability sentences predominated in the protocol for assessment of speech intelligibility used in this study, suggesting that these sentences can be used reliably for assessing intelligibility.Analysis of intelligibility in sentences based on target words can be used in clinical practice, especially when the predictability of the sentences is known.
The level of predictability of the sentences used in assessments can also influence intelligibility measures.Predictability can be defined as the redundancy or quantity of semantic content in a sentence (4) .Results of studies investigating the effect of predictability on the measuring of intelligibility indicate that sentences with high predictiveness tend to raise intelligibility scores of speakers compared to sentences with low predictiveness (4,8,27,28) .
The Intelligibility of Speech Assessment Protocol (PAIF) (2) is a Brazilian instrument for assessing the intelligibility of speech in dysarthrics.In this protocol, intelligibility is assessed by means of different speech content, including the use of 25 sentences.Scores are based on an orthographic transcription of speech samples and calculated in the conventional manner, according to the percentage of correctly transcribed words.
Since the predictability of the sentences in this protocol is unknown and predictability is a factor influencing the measuring of intelligibility, knowledge of this aspect of the list of sentences can be useful to refine this assessment instrument.Against this background, the aim of the present study was to analyze the predictability of the sentences employed in the PAIF.

METHODS
The study was approved by the Research Ethics Committee of the Universidade Federal de São Paulo (UNIFESP) under process number 1055/09.A cross-sectional study in 120 volunteers (109 women and 11 men), whose mother tongue was Brazilian Portuguese, with a mean age of 21.2 years (SD=3.9)and mean schooling of 12.8 years (SD=1.3),was carried out.Individuals aged less than 18 years, with less than eight years of schooling or familiarized with the Speech Intelligibility Assessment Protocol, were excluded from the study.All participants signed a free and informed consent form (TCLE).
Four different versions of the 25-sentence list of the PAIF (2) were devised.In the four versions, different target words were removed from each sentence.The words removed were selected according to their information weight within the sentence.All open-class words were selected, i.e. nouns, verbs, adjectives and adverbs, at the rate of two to four words per sentence, giving a total of 79 words removed (Appendix 1).The closed-class words (conjunctions, articles and prepo-sitions) were not analyzed because they are known to have low levels of predictability (11) .
Participants were randomly divided into four groups each comprising 30 subjects.The groups filled in the gaps of a different version of the list.Subjects were instructed to complete each sentence with the first word that came to mind while ensuring the resulting sentences were coherent.The task was performed in groups of 20 participants, maximum, in a silent environment.
Answers were analyzed being scored as correct when participants had filled in the blank using the expected target word.The predictability of the words was calculated according to the percentage of correct answers for each target word.The predictability of the sentences was calculated according to the percentage of correct target words per sentence.According to the conventional proposal for intelligibility assessment contained in the PAIF, all of words contained in the phrases should be analyzed to calculate the intelligibility score for sentences.
In the present study however, the classification of the level of predictability of each sentence was reached by considering all target words.
Cluster analysis was employed to classify the phrases by predictability level.This statistical technique is used to form groups by similarity, according to the natural relationships that the sample exhibits.The Chi-square test was applied to determine predictability differences in target words of each sentence.Analysis of Variance (ANOVA) was used to compare different versions with one another.Upon detection of differences, Bonferroni's multiple comparisons test was applied.The level of significance considered for all tests was 5%.
In order to assess possible interference of demographic variables (age, schooling and gender) on performance of each group of participants, the ANOVA test was applied for numeric variables, and Chi-square test for categorical variables.The four groups of volunteers were found not to differ for the variables gender or schooling.With regard to age however, a difference was found between the groups designated to lists 3 and 4 (p=0.018).

RESULTS
The sentences comprising the PAIF had 41.7% predictability overall (SD=18.1).Cluster analysis classified the sentences into three levels of predictability: low predictability (less than 50% correct), medium predictability (between 50 and 60% correct) and high predictability (greater than 60% correct (Table 1).
The words in each sentence were compared.Differences among target words in predictiveness level was found for 84% of the sentences (21/25), with at least one target word being more predictable than the other target words in the sentence (p≤0.0054)(Table 2).
Comparison of the versions of the list using the ANOVA test identified differences between them (p<0.001)(Table3).Bonferroni multiple comparisons revealed that list one differed to the others, in that the target words in the version of the list had a lower level of predictability (p≤0.002).The version of the list in question is shown in Appendix 1.

DISCUSSION
Analysis of the predictability of each sentence from the PAIF revealed that, out of the 25 sentences, 15 had low predictability, seven had medium predictability, while only three high predictability.Thus, the majority of the sentences used in the protocol can be applied reliably in intelligibility of speech assessments of arthritic patients.Intelligibility scores based on these sentences would be ideal since the correct decoding of the speech signal by the listener would depend more on acousticsignal dependent cues (altered in dysarthric speech) than on redundancy of the sentence, which is a signal-independent cue (3) .
One of the studies examined (4) proposed an arbitrary criteria for classifying predictiveness of sentences into low (<25% correct), medium (between 25 and 75% correct) and high (>75% correct).In the present study, cluster analysis enabled sentences from the protocol to be grouped, adopting different cut-off criteria (<50%, between 50 and 60%, and greater than 60% correct, for low, medium and high predictiveness, respectively).However, the majority of sentences classified into the high predictability group using this analysis technique had a percentage of correct words which fell below the cut-off criteria for high predictability proposed in the cited study.This confirms the tendency toward lower predictability of sentences on the PAIF.Subtle differences were observed between the groups of sentences in the cluster analysis (5.6% to 9.5%), particularly among the groups with low and medium predictability.Therefore, the selection of less predictable sentences based on the predictability ranking obtained, as commonly occurs in studies involving this variable (4,8,27) , may prove more suitable for refining the instrument.
Additionally, comparative analysis of the predictability level of target words in each sentence revealed the occurrence of target words with different indices of predictability.Thus, upon applying the PAIF, if speech intelligibility analysis is carried out on sentences focused only on target words, is also possible to select the less predictable open-class words from each sentence, thereby optimizing the assessment instrument.
Comparison of the four versions of the sentence lists from the protocol showed that on version one, target words with low predictability were chosen for most of the sentences.The difference found in version one did not stem from the variables related to the group of listeners, since the four groups did not differ for the variables gender or schooling while a difference for age was found only in groups three and four.
Considering that high semantic predictability can increase sentence intelligibility scores (4,8,27,28) , the use of this less predictable list can render the studied protocol more sensitive for assessing intelligibility.The list can be further refined by substituting sentences and key words with other, less predictable, instances not included in this version.
The studies found in the literature analyzed the effect of sentence predictability on scores of individuals with speech disorders (4,8,27,28) .In general, the procedure used for analyzing the predictability of the sentences was similar to that adopted in the present study.However, in order to classify predictiveness, some of these studies selected the more and less predictable sentences in terms of predictability ranking.Based on the results obtained, it can be concluded that future studies are warranted on alternative ways of measuring sentence intelligibility with the PAIF and that explore the impact of sentence predictability on the scores attained.

CONCLUSION
The Speech Intelligibility Assessment Protocol contains predominantly low predictability sentences, suggesting these can be employed reliably for assessing speech intelligibility.On the majority of sentences, target words differed in terms of degree of predictability, suggesting that selecting these particular words can optimize the assessment instrument.Among the different versions of the list proposed, version one contained words with lower predictability than the other versions, and can therefore be used to increase the sensitivity of the test by performing analysis of intelligibility in sentences based on target words.

Table 1 .
Predictability of sentences and classification of predictability level

Table 2 .
Comparison between words from each sentence