Borg scale: a new method for hypernasality rating

ABSTRACT

Purpose

To investigate the reliability in auditory-perceptual assessment of hypernasality of the Borg centiMax scale and the influence of the speech material on the reliability of two scales.

Methods

Four experienced speech-language pathologists rated hypernasality of 80 audio recordings of patients with repaired cleft palate (40 single-word string and 40 sentences) using the 5-point ordinal scale and the Borg centiMax scale. Intra and inter-rater reliability were calculated for both scales and for both types of speech samples. The comparison between the agreement coefficients of the two speech samples was calculated using the Z test and between the scales was calculated by Spearman correlation coefficient, considering as significant p<0.05.

Results

A very high and statistically significant correlation was found between the Borg centiMax scale and the ordinal scale for both speech samples. Intra- and inter-rater reliability was higher for Borg scale as compared to ordinal scale. Good to excellent intra-rater reliability was found for Borg scale for both speech samples. Poor to excellent intra-rater reliability was found for ordinal scale for both stimuli. Higher inter-rater reliability was demonstrated for Borg scale than ordinal scale for both speech samples. There was a significant difference between the single words string and sentences for intra- and inter-rater reliability using Borg scale, and for inter-rater reliability using ordinal scale.

Conclusion

The Borg centiMax scale showed better intra and inter-rater reliability. Additionally, the speech material comprising of single words string showed better reliability in most of the comparisons for both scales.

Keywords:
Velopharyngeal Insufficiency; Cleft Palate; Speech Disorders; Speech Perception; Speech

RESUMO

Objetivo

Investigar a confiabilidade da escala Borg centiMax como método de avaliação perceptivo-auditiva da hipernasalidade e a influência do tipo de amostra de fala sobre a confiabilidade das avaliações.

Método

Quatro fonoaudiólogas experientes classificaram a hipernasalidade de 80 amostras de fala de pacientes com fissura de palato reparada (40 vocábulos e 40 sentenças) utilizando a escala ordinal de 5 pontos e a escala Borg centiMax. Os índices de concordância intra e interavaliadores foram estabelecidos para ambas as escalas e amostras. A comparação desses índices foi feita pelo teste Z e a comparação entre as escalas foi feita pelo coeficiente de correlação de Spearman (p<0,05).

Resultados

Verificou-se correlação muito alta e significante entre a Escala Borg centiMax e a escala ordinal, para ambas as amostras. Os índices de concordância intra-avaliadores (CCI) para a escala Borg centiMax variaram de excelente a bom e, para a escala ordinal (Kappa), de excelente a pobre, em ambas as amostras. A concordância interavaliadores (CCI) para a escala Borg centiMax variou de excelente a moderada e, para a escala ordinal (Kappa), variou de moderada a pobre, para vocábulos e sentenças. Diferença estatisticamente significante, com melhores índices de concordância intra e interavaliadores para vocábulos, foi obtida com a escala Borg centiMax. Para a escala ordinal, diferença significante entre vocábulos e sentenças foi observada apenas para a comparação interavaliador.

Conclusão

A escala Borg centiMax apresentou melhores índices de concordância intra e interavaliadores. A amostra contendo vocábulos mostrou melhores índices de concordância na maioria das comparações, para ambas as escalas.

Descritores:
Insuficiência Velofaríngea; Fissura Palatina; Distúrbios da Fala; Percepção da Fala; Fala

INTRODUCTION

Hypernasality is one of the most important variables in identifying the speech results on cleft palate treatment⁽¹⁾. The identification and classification of this significant symptom are mainly made using auditory-perceptual assessment, which is considered a “golden standard” method at the speech symptom assessment^(1-3). On considering the principle, that treatment must only be indicated when the problem is identified, the auditory-perceptual assessment continues to be the main tool for clinical decision-making. Besides, it has the advantage of being convenient and economic. However, because it is a subjective method, which depends on the listener’s perception, it is prone to errors and biases, even when performed by expert professionals. Knowing the limitations, that may jeopardize its validity and reliability, is essential for the use of perceptual methods^(1,2).

Specifically, regarding the nasality, the challenge is even higher. This speech symptom was considered, among all perceptive dimensions, the most difficult to judge reliably due to its psychophysics characteristics⁽⁴⁾. Several researchers defend the theory that the nasality is a sensation mentally processed as a prothetic dimension, that is, it differs in terms of change in degrees of quantity^(1,3-6). Thus, the use of appropriate rating scales for the nasality characteristics can improve the reliability of the auditory-perceptual assessment of this symptom, turning it less susceptible to errors and consequently reducing the variations of different perceptual judgments^(1,3,6,7).

Historically, the ordinal scale is the most used for the hypernasality rating, both in researches and clinical practice^(8-10), because it is a more appropriate method for the clinical context. Additionally, the obtained ratings are relatively easy to be compared among different scales and listeners^(9,11). By using the ordinal scale, the listener assigns a score to the evaluated symptom in a linear scale, indicating its degree of severity at a nominal classification order, where the lowest category represents the absence of symptom and the highest, the symptom maximum degree^(7,8). However, the ordinal scale separates the different categories of speech symptoms without quantifying the magnitude of the difference between each category. It occurs that the listeners tend to subdivide, especially the lower end of a scale, into smaller intervals⁽⁵⁾. This way, the psychophysics nature of the nasality suggests that its rating would be favored and, better agreement coefficients among different listeners would be reached, using ratio-based scales^(6,8).

In the ratio-based scales, the listeners freely assign a number that represents the sensorial magnitude of the assessed stimulus. Ratio scale examples are the Direct Magnitude Scale and the Analogical Visual Scale^(1,7).

Another example of ratio scale is the Borg centiMax scale^(12,13). The first Borg scale was developed in the 1960s by the physiologist Gunnar Borg and was called CR10. Its purpose was the classification of the physical effort subjective perception. Over the years, this scale was modified by the author (G. Borg) and by his collaborators experts on the subject, among them Elisabet Borg, who have developed the Borg centiMax scale, also known as the Borg scale, used in the present study. The Borg scale, in all its variations, combines the advantages of ratio scales with the advantages of the category scales for the rating of a symptom degree. The verbal categories are placed in the scale according to the numbers, following a proportion that covers an intensity range biologically natural, in such a way that, for each category exists an amount in a numerical sequence. One of the basic principles of Borg scales is, therefore, to obtain congruence between a category and its proportion in the scale. Thereby, Borg scales supply verbal anchors to the listeners so that they can do their judgements regarding the intensity of a stimulus. The “highest” point (100, in the case of Borg centiMax scale) represents “the maximum intensity of that stimulus already perceived by the individual”. It is still possible to perceive the stimulus as stronger than the maximum, called in the scale as “absolute maximum”, placed a little above, however, without being represented by a value, but by a dot “•”.

Since it was developed, the Borg scale was used in different contexts to assess different sensory perceptions, experiences, and feelings, including colors, taste, odor, physical effort, loudness, noise and easiness to perceive symbols^(6,14,15). Its reliability and validity were demonstrated in different studies and its application was also tested at the clinical diagnosis of pains and on determining perceived effort, including breathing difficulties and fatigue, especially regarding training and rehabilitation tests^(13,16).

In the speech-language pathology area, the Borg scale was recently used to measure vocal effort⁽¹⁷⁾. In this study, the authors investigated whether the CR10 Borg scale could be used as a measurement of perceived vocal effort by the patient himself during the voice therapy and correlated it with another measurement of perceived vocal effort, the item 14 of the Voice Handicap Index (VHI). The authors concluded that CR10 was a clinical tool, easy to use for the perception of vocal effort reduction and that the two scales complement each other because while item 14 of the VHI indicated the frequency of perceived highest effort, the CR10 scale showed the effort severity level.

By considering that hypernasality is defined as a sensation, it is coherent to suppose that Borg centiMax scale can be a good alternative for this symptom rating. In a study performed at the Karolinska Institute, in Stockholm, Sweden, in which this scale was used for the first time for hypernasality rating, the authors showed that Borg scale was the most reliable method among the three studied methods. In this study, hypernasality was assessed in speech samples produced by Swedish-speaker and the listeners’ group involved Brazilian and Swedish speech-language pathologists. According to these authors, the Borg scale allowed the listener the segmentation of different categories of hypernasality in many degrees, which was useful to identify differences in the severity of the symptom in the same category, such as “mild nasality”, for example⁽⁶⁾.

Having in mind the search in the literature for a type of rating scale for hypernasality that presents more reliable results, the contribution of this study for the area is the introduction of a new method of perceptual assessment of this symptom, considered one of the most typical for the cleft palate.

Thereby, the objective of this study was to investigate the reliability of Borg scale (Borg centiMax) by comparing the intra- and inter raters agreement coefficients obtained with the Borg centiMax scale and the ordinal scale of 5 points, used in the clinical practice in the HRAC-USP, for the auditory-perceptual assessment of hypernasality. As a secondary objective, it was intended to investigate the influence of the type of speech sample (single-word strings or sentences) over the reliability of both scales.

METHODS

Speech stimuli

This study was approved by the Research Ethics Committee on Human Beings of the institution (CAAE: 71041917.7.0000.5441). All the patients or those responsible signed the Informed Consent Form.

A total of 120 speech samples were recorded. From these recordings, 20 were excluded because they were used in the listeners’ training and another 20 were excluded for not presenting good audio quality. Thereby, 80 audio recording samples were included in the study (40 containing single-word strings and 40 containing sentences) of 40 patients with repaired cleft palate, associated or not to cleft lip, native speakers of Brazilian Portuguese, of both genders, with age from 10 to 45 years old (average 24 years old). The samples comprised a sequence of 12 words and by a sequence of 12 sentences, containing exclusively oral sounds, followed by high and low vowels.

Procedures

The audio recording was performed into a silent room with good quality equipment and these recordings were stored, edited and later, perceptually analyzed for the rating of hypernasality. Only the recordings with good audio quality were included in the study.

Hypernasality rating

Hypernasality was analyzed by four speech-language pathologists with experience in assessment of individuals with cleft palate. Each listener rated the samples using two distinct scales: ordinal scale of 5 points, being 1 = normal; 2 = mild acceptable; 3 = mild not acceptable; 4 = moderate; 5 = severe; and the Borg centiMax scale (Figure 1). In both scales, the listeners rated the hypernasality according to their internal criteria. It was recommended that the analyses were performed individually, in a silent environment, using their computer and stereo headphones (AKG, model K240 MK II - available for the study). The listeners were advised to listen to the recordings as many times as they deemed necessary and it was emphasized the importance to obey the resting periods during the assessment, to avoid fatigue.

Figure 1
Borg CR Scale (CR100, centiMax)^(12,13)

Each listener analyzed 80 speech samples (40 single-word strings and 40 sentences) with each one of the scales. For the intra-rater agreement calculation, 20% of the total sample was duplicated (16 samples) and it was analyzed twice by each listener within the two scales. This way, the listeners analyzed a total of 192 samples. The samples were available for the listeners by using the virtual disk Google, Google Drive, respecting an interval of 10 days between each analysis.

Data analysis

Both, intra- and inter-rater agreement coefficients were established for both scales and for the two types of speech samples. For the intra-rater agreement calculation, 20% (16) of the sample was duplicated and analyzed a second time by the listeners. For the inter-rater agreement calculation, the four listeners were compared 2-by-2 in each one of the scales. Intra- and inter-raters agreement coefficients for the ordinal scale were determined by using the weighted Kappa coefficient. And, for Borg centiMax scale, the agreement was determined by the Intraclass Correlation Coefficient (ICC), by considering, for both scales, the following interpretation: below 0.40=poor agreement; from 0.40 to 0.59=moderate agreement; from 0.60 to 0.74=good agreement; from 0.75 to 1.00=excellent agreement⁽¹⁸⁾.

It was also established, the correlation between the two scales for both types of speech samples analyzed, by means of the Spearman rank correlation coefficient and taking into account the following interpretation: from 1.00 to 0.90 very high correlation, from 0.90 to 0.70 high correlation, from 0.70 to 0.50 moderate correlation, from 0.50 to 0.30 low correlation and from 0.30 to 0.00 insignificant correlation⁽¹⁹⁾.

The comparison between the two scales was performed by analyzing, in a descriptive way, the intra- and inter-raters agreement coefficients obtained for each one of them.

The comparison between the two types of speech samples was calculated by verifying the intra- and inter-raters agreement coefficients obtained for the words and sentences samples, in both scales, by using the Z test. For the intra-raters calculation, the ratings of the four listeners were gathered totalizing 32 samples and, for the inter-raters calculation, the ratings of six pairs of raters were gathered resulting in 240 samples. For all the comparisons, it was considered p<0.05 as significant.

RESULTS

Intra-rater agreement

For the ordinal scale, the intra-rater agreement coefficient ranged from poor to excellent for the words sample; while for the sentence sample, it was excellent for all the raters. According to the Borg scale, the agreement coefficient ranged from good to excellent for the word sample and moderate to excellent for the sentence sample, as demonstrated in Table 1.

Table 1
Intra-rater agreement at the rating of words and sentences samples using the ordinal scale (Weighed Kappa and its interpretation) and Borg centiMax scale (ICC and its interpretation)

INTRA-RATER AGREEMENT
SPEECH SAMPLE	RATER	ORDINAL SCALE		BORG CentiMax
		Weighed Kappa	Interpretation	ICC	Interpretation
WORDS	1	1.00	Excellent	1.00	Excellent
	2	0.68	Good	0.97	Excellent
	3	0.80	Good	0.99	Excellent
	4	0.38	Poor	0.73	Good
SENTENCES	1	1.00	Excellent	0.42	Moderate
	2	0.81	Excellent	0.95	Excellent
	3	0.82	Excellent	0.99	Excellent
	4	0.83	Excellent	0.93	Excellent

• Caption: ICC = Intraclass Correlation Coefficient

Inter-rater agreement

For the ordinal scale, the inter-raters agreement coefficient ranged from poor to moderate. Regarding the Borg scale, the agreement coefficient ranged from moderate to excellent for the words sample and from moderate to poor for the sentence samples (Table 2).

Table 2
Inter-rater agreement at the rating of words and sentences samples using the ordinal scale (Weighed Kappa and its interpretation) and Borg centiMax scale (ICC and its interpretation)

INTER-RATER AGREEMENT
SPEECH SAMPLE	RATER	ORDINAL SCALE		BORG CentiMax
		Weighed Kappa	Interpretation	ICC	Interpretation
WORDS	1 vs 2	0.33	Poor	0.68	Good
	1 vs 3	0.47	Moderate	0.60	Good
	1 vs 4	0.32	Poor	0.51	Moderate
	2 vs 3	0.50	Moderate	0.89	Excellent
	2 vs 4	0.33	Poor	0.73	Good
	3 vs 4	0.48	Moderate	0.71	Good
SENTENCES	1 vs 2	0.53	Moderate	0.63	Good
	1 vs 3	0.20	Poor	0.43	Moderate
	1 vs 4	0.25	Poor	0.40	Moderate
	2 vs 3	0. 31	Poor	0.71	Good
	2 vs 4	0.33	Poor	0.74	Good
	3 vs 4	0.10	Poor	0.56	Moderate

• Caption: ICC = Intraclass Correlation Coefficient

Comparison between the two scales

It was established the correlation between the two scales considering the samples separately and correlating the assigned values by the four raters (40 samples rated four times by the four raters), using the median values for the ordinal scale and the mean value for Borg scale. The correlation coefficient for the words sample was 0.93 (p<0.001) and, for the sentence samples was 0.94 (p<0.001), revealing a very high and significant correlation between the two scales. Figure 2 shows the coherence between the assessment made by both scales and both samples.

Figure 2
Correlation between Borg centiMax scale (based on the mean) and ordinal scale (based on the median) of the hypernasality scores assigned by the four raters for: (A) word sample (40 samples assessed twice by the four raters); (B) sentence sample (40 samples assessed twice by the four raters)

Comparison between the two samples

For the ordinal scale, significant difference between the two samples was verified only for the inter-raters comparison. For Borg centiMax scale, there was a significant difference between the two samples both for the intra- and inter-raters coefficients (Table 3).

Table 3
Comparison between the single-word stings and sentence samples for the ordinal scale and Borg centiMax scale

	Ordinal Scale		p	Borg centiMax		p
	Words	Sentences		Words	Sentences
	Kappa Coefficient			ICC
Intra-rater	0.70	0.88	0.150	0.96	0.68	0.011*
Inter-rater	0.40	0.28	0.007*	0.67	0.57	0.025*

• Z Test
• ^*
  p<0.05 significant difference

DISCUSSION

This study proposed to investigate the reliability of Borg scale in auditory-perceptual assessment of the hypernasality in individuals with repaired cleft palate, speakers of Brazilian Portuguese and, additionally to investigate the influence of the speech sample type over the assessment reliability. For such purpose, the results obtained with Borg scale were compared to the ones obtained with the ordinal scale of 5 points, used in the clinical practice.

The present study showed better intra- and inter-raters agreement coefficients for Borg scale, in comparison with ordinal scale of 5 points, for both speech sample types (words and sentences), with exception only for the intra-raters agreement at the sentences.

High intra-raters agreement prove the stability and coherence of the judgments of the same listener. The listeners involved in the present study are stable and consistent in their judgments and showed well defined internal standards, probably resulting from the experience acquired in the perceptual assessment of speech symptoms characteristics of cleft palate during the years.

Literature has proved that the listener’s experience is a determining factor to obtain more reliable results in the perceptual assessment^(2,7,20).

In general, good intra-rater agreement coefficients were also reported in other studies performed in the institution and in the international literature, with speech sample similar to the one used in the present study^(11,20-24).

Notwithstanding, well defined internal standards, which grant the stability in their assessment, do not seem to be enough to guarantee reliable results among different raters. To reach high coefficients of agreement among different raters in the perceptual assessment of hypernasality is still a challenge⁽²⁵⁾ and the difficulty is still higher when the hypernasality assessment is made using the ordinal scale^(1,7). In the present study, the inter-raters coefficient using the ordinal scale ranged from poor to moderate for both speech sample types.

These results corroborate the literature that also reports reduced inter-raters agreement coefficients in studies that used the ordinal scale, as an assessment method. Studies performed in the Institution showed coefficients that vary from regular to moderate for several speech samples, including spontaneous conversation^(10,22,23), and in the international literature, moderate coefficients of inter-raters agreement were verified^(26,27). According to some authors, this is a worrying data regarding the speech assessment of individuals with cleft, bearing in mind the importance of the speech resonance as clinical decision guiding⁽²⁸⁾.

Although standard speech samples were used in the present study, elaborated according to international criteria and analyzed by expert professionals, the agreement regarding the degree of hypernasality obtained among the raters was reduced.

On the other hand, different result was obtained using Borg scale. The results showed that the inter-raters agreement coefficient ranged from moderate to excellent for both analyzed samples. In this case, the lowest agreement rate was 0.40 and the highest was 0.89.

The Borg scale was used for the first time to rate hypernasality in a study in Sweden⁽⁶⁾. In this study, by using word samples produced by native Swedish-speaker children with repaired cleft palate, the results showed that Borg scale was the most reliable method when compared to other two methods analyzed in the study.

In the present study, by using two types of standard speech samples of individuals with repaired cleft palate, including Brazilian Portuguese native speakers children and adults, it was proved that Borg scale was the most reliable method when compared with ordinal scale.

It is worth highlighting that the age variable of the individuals, as well as other variables such as sex, type of cleft and former speech therapy, were not controlled, since the main objective of the study was to investigate the reliability of Borg scale as a method for auditory-perceptual assessment of the nasality. The aim of this study was the comparison of the perceptual judgment obtained with this new method and those obtained with the ordinal scale used routinely in the Hospital, performed by different raters in two types of speech samples, not mattering in this case, the patient’s condition.

By selecting the scale can influence the rater’s ability in distinguishing differences of a given symptom and the validity of the different scales depends on the psychophysics nature of the symptom that is being assessed⁽⁸⁾. By considering the hypernasality as a prothetic sensation, scales which use the symptom division or subdivision, such as the ordinal scale, cannot be appropriate to rate, in a reliable way, the nasality⁽⁵⁾. The best results obtained with Borg scale, in the present study, proved that hypernasality is better rated when using ratio-based scales, corroborating other studies^(1,6).

The explanation for this result is, probably, the fact that Borg scale is a method that incorporates the benefits of the ratio scale combined with a category scale; furthermore, its measure scale is based on psychophysics aspects of the human perception. The use of a continuum of numbers from 0 to 100 permits to the listeners to rate the hypernasality and yet, identify different degrees within a particular category. The most classic example is the “mild hypernasality” category^(5,6). Listeners systematically tend to partition the lower end of the scale (such as the “mild” category) into smaller intervals than the upper end. The Borg scale allows the hypernasality segmentation, in a way that a given speech stimulus rated as “mild hypernasal” could be considered as “mild acceptable”, that is, acceptable within the normal standard for the majority of the listeners, what differs, according to the listener’s perception, from the “perceptible” mild nasality. At the ordinal scale, the same stimulus would supposedly be rated as “mild hypernasal”, without the possibility of differentiating it. The interval scale, such as the ordinal scale, does not allow the raters fully express their auditory perception, limiting the rating possibilities⁽⁷⁾.

Regarding the influence of the type of speech sample over scales reliability, the results showed that the agreement coefficients were better for the samples with single-word strings in comparison to the sentence samples, in the majority of the comparisons performed. The speech sample containing a sequence of 12 single words, despite shorter, favored the reliability of the hypernasality assessments, especially when the assessment was done using the Borg scale.

The word sample used in the present study was formed by a sequence of words, or single word string, as called in English. This modality of speech sample was used in the Scandcleft project (an international multicentric project), to assess children’s speech^(3,6,29). The sample is comprised by a sequence of 12 single words, containing exclusively oral sounds, presented as a single speech stimulus, what turns it similar to a short sentence (of approximately 15 seconds), facilitating the rating of different degrees of hypernasality⁽²⁹⁾.

The best reliability obtained with the single-word trings samples, in the present study, can be justified by the extension of the speech stimulus. It is speculated the fact that the sequence of the words is shorter than the sequence of the sentences what may have avoided the influence of other coexisting active and passive speech errors, such as, the nasal air emission or compensatory articulation. According to the literature, in the presence of other speech symptoms, it becomes difficult for the listener to isolate the hypernasality, leading, many times, to its rating as more severe^(2,24,29,30).

A proposal of universal parameters⁽³⁰⁾ for the documentation of the speech results of individuals with cleft palate recommended the use of sentences and single words repetition for the perceptual assessment of hypernasality. The authors suggested that, to assess the hypernasality using words, the samples should contain only high vowels and all the words should contain only one type of pressure consonant per word. And, yet, to be representative of the phonetic system, all the pressure consonants should occur in all the positions of the language occurrence. The sentence samples should contain pressure consonants, not include nasal consonants and include all types of vowels relevant for the language. The sentences should contain a single pressure sound and this one should occur in all the appropriate positions for the language.

The samples used in the present study were elaborated by the “Brazil Cleft” task force based on these guidelines⁽³⁰⁾, to comprise all the Brazilian Portuguese sounds.

The present study contributed to the dissemination of Borg scale as a reliable method of hypernasality assessment. The results of this study prove the high agreement coefficients among different raters using Borg scale, which increases the reliability of the auditory-perceptual assessment, method that continues to be the main indicator of clinical significance of the speech symptoms.

CONCLUSION

Borg centiMax scale showed more reliable intra- and inter-raters agreement results than ordinal scale. Additionally, the speech sample containing single-word strings favored the reliability presenting better intra- and inter-raters agreement in most of the comparisons, in both scales.

ACKNOWLEDGEMENTS

The authors thank the speech-language pathologists that served as raters in this study. To CAPES, for the financial support offered for the performance of the project. And also to Anette Lohmander (Division of Speech-Language Pathology, Karolinska Institutet and Karolinska University Hospital) and Elisabet Borg (Department of Psychology, Stockholm University), for having presented Borg scale to the author RPY.

• Study conducted at Laboratório de Fisiologia, Hospital de Reabilitação de Anomalias Craniofaciais, Universidade de São Paulo – USP - Bauru (SP), Brasil.
• Financial support: CAPES - universial demand.

REFERÊNCIAS

• ¹
  Kent RD. Hearing and believing: some limits to the auditory-perceptual assessment of speech and voice disorders. Am J Speech Lang Pathol. 1996;5(3):7-23. http://dx.doi.org/10.1044/1058-0360.0503.07
  » http://dx.doi.org/10.1044/1058-0360.0503.07
• ²
  Baylis AL, Chapman KL, Whitehill TL. Validity and reliability of visual analog scaling for assessment of hipernasality and audible nasal emission in children with repaired cleft palate. Cleft Palate Craniofac J. 2015;52(6):660-70. http://dx.doi.org/10.1597/14-040 PMid:25322442.
  » http://dx.doi.org/10.1597/14-040
• ³
  Lohmander A, Persson C, Willadsen E, Lundeborg I, Alaluusua S, Aukner R, et al. Scandcleft randomised trials of primary surgery for unilateral cleft lip and palate: 4. speech outcomes in 5-year-olds - velopharyngeal competency and hypernasality. J Plast Surg Hand Surg. 2017;51(1):27-37. http://dx.doi.org/10.1080/2000656X.2016.1254645 PMid:28218551.
  » http://dx.doi.org/10.1080/2000656X.2016.1254645
• ⁴
  Watterson T, Lewis K, Allord M, Sulprizio S, O’Neill P. Effect of vowel type on reliability of nasality ratings. J Commun Disord. 2007;40(6):503-12. http://dx.doi.org/10.1016/j.jcomdis.2007.02.002 PMid:17391692.
  » http://dx.doi.org/10.1016/j.jcomdis.2007.02.002
• ⁵
  Stevens SS. Psychophysics: introduction to its perceptual, neural and social prospects. New York: Wiley; 1975.
• ⁶
  Yamashita RP, Borg E, Granqvist S, Lohmander A. Reliability of hipernasality rating: comparison of three different methods for perceptual assessment. Cleft Palate Craniofac J. 2018;55(8):1060-71. http://dx.doi.org/10.1177/1055665618767116 PMid:29634363.
  » http://dx.doi.org/10.1177/1055665618767116
• ⁷
  Castick S, Knight RA, Sell D. Perceptual judgments of resonance, nasal airflow, understandability, and acceptability in speakers with cleft palate: ordinal versus visual analogue scaling. Cleft Palate Craniofac J. 2017;54(1):19-31. http://dx.doi.org/10.1597/15-164 PMid:28067575.
  » http://dx.doi.org/10.1597/15-164
• ⁸
  Whitehill TL, Lee AS, Chun JC. Direct magnitude estimation and Interval scaling of hypernasality. J Speech Lang Hear Res. 2002;45(1):80-8. http://dx.doi.org/10.1044/1092-4388(2002/006) PMid:14748640.
  » http://dx.doi.org/10.1044/1092-4388(2002/006)
• ⁹
  Brancamp TU, Lewis KE, Watterson T. The relationship between nasalance scores and nasality ratings obtained with equal appearing interval and direct magnitude estimation scaling methods. Cleft Palate Craniofac J. 2010;47(6):631-7. http://dx.doi.org/10.1597/09-106 PMid:20500059.
  » http://dx.doi.org/10.1597/09-106
• ¹⁰
  Medeiros MNL, Fukushiro AP, Yamashita RP. Influência da amostra de fala na classificação perceptiva da hipernasalidade. CoDAS. 2016;28(3):289-94. http://dx.doi.org/10.1590/2317-1782/20162015202
  » http://dx.doi.org/10.1590/2317-1782/20162015202
• ¹¹
  Oliveira ACASF, Scarmagnani RH, Fukushiro AP, Yamashita RP. Influência do treinamento dos avaliadores no julgamento perceptivo da hipernasalidade. CoDAS. 2016;28(2):141-8. http://dx.doi.org/10.1590/2317-1782/20162015163 PMid:27191877.
  » http://dx.doi.org/10.1590/2317-1782/20162015163
• ¹²
  Borg G, Borg E. A new generation of scaling methods: level-anchored ratio scaling. Psychologica. 2001;28:15-45.
• ¹³
  Borg E. On perceived exertion and its measurement [dissertation]. Stockholm: Stockholm University; 2007.
• ¹⁴
  Griep MI, Borg E, Collys K, Massart DL. Category ratio scales as an alternative to magnitude matching for age-related taste and odour perception. Food Qual Prefer. 1998;9(1-2):67-72. http://dx.doi.org/10.1016/S0950-3293(97)00030-X
  » http://dx.doi.org/10.1016/S0950-3293(97)00030-X
• ¹⁵
  Karavatas SG, Tavakol K. Concurrent validity of Borg’s rating of perceived exertion in african-american young adults, employing heart rate as the standard. Internet J Allied Health Sci Pract. 2005;3(1):1-5.
• ¹⁶
  Gerlach Y, Williams MT, Coates AM. Weighing up the evidence-a systematic review of measures used for the sensation of breathlessness in obesity. Int J Obes. 2013;37(3):341-9. http://dx.doi.org/10.1038/ijo.2012.49 PMid:22531088.
  » http://dx.doi.org/10.1038/ijo.2012.49
• ¹⁷
  Van Leer E, van Mersbergen M. Using the Borg CR10 physical exertion scale to measure patient-perceived vocal effort pre and post treatment. J Voice. 2017;31(3):389.e19-25. http://dx.doi.org/10.1016/j.jvoice.2016.09.023 PMid:27887811.
  » http://dx.doi.org/10.1016/j.jvoice.2016.09.023
• ¹⁸
  Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6(4):284-90. http://dx.doi.org/10.1037/1040-3590.6.4.284
  » http://dx.doi.org/10.1037/1040-3590.6.4.284
• ¹⁹
  Mukaka MM. Statistics corner: a guide to appropriate use of correlation coefficient in medical research. Malawi Med J. 2012;24(3):69-71. PMid:23638278.
• ²⁰
  Brunnegård K, Lohmander A, van Doorn J. Comparison between perceptual assessments of nasality and nasalance scores. Int J Lang Commun Disord. 2012;47(5):556-66. http://dx.doi.org/10.1111/j.1460-6984.2012.00165.x PMid:22938066.
  » http://dx.doi.org/10.1111/j.1460-6984.2012.00165.x
• ²¹
  Brandão GR, Freitas JAS, Genaro KF, Yamashita RP, Fukushiro AP, Lauris JR. Speech outcomes and velopharyngeal function after surgical treatment of velopharyngeal insufficiency in individuals with signs of velocardiofacial syndrome. J Craniofac Surg. 2011;22(5):1736-42. http://dx.doi.org/10.1097/SCS.0b013e31822e624f PMid:21959422.
  » http://dx.doi.org/10.1097/SCS.0b013e31822e624f
• ²²
  Scarmagnani RH, Barbosa DA, Fukushiro AP, Salgado MH, Trindade IEK, Yamashita RP. Correlação entre o fechamento velofaríngeo, hipernasalidade, emissão de ar nasal e ronco nasal em indivíduos com fissura de palato reparada. CoDAS. 2015;27(3):267-72. http://dx.doi.org/10.1590/2317-1782/20152014145 PMid:26222944.
  » http://dx.doi.org/10.1590/2317-1782/20152014145
• ²³
  Ferlin F, Yamashita RP, Fukushiro AP. Influência das consoantes de alta e baixa pressão intraoral sobre a nasalidade e nasalância da fala em pacientes com fissura de palato reparada. Audiol Commun Res. 2017;22(0):e1851. http://dx.doi.org/10.1590/2317-6431-2017-1851
  » http://dx.doi.org/10.1590/2317-6431-2017-1851
• ²⁴
  Lee A, Whitehill TL, Ciocca V. Effect of listener training on perceptual judgment of hypernasality. Clin Linguist Phon. 2009;23(5):319-34. http://dx.doi.org/10.1080/02699200802688596 PMid:19399664.
  » http://dx.doi.org/10.1080/02699200802688596
• ²⁵
  Eadie TL, Kapsner-Smith M. The effect of listener experience and anchors on judgments of dysphonia. J Speech Lang Hear Res. 2011;54(2):430-47. http://dx.doi.org/10.1044/1092-4388(2010/09-0205) PMid:20884782.
  » http://dx.doi.org/10.1044/1092-4388(2010/09-0205)
• ²⁶
  Dotevall H, Lohmander-Agerskov A, Ejnell H, Bake B. Perceptual evaluation of speech and velopharyngeal function in children with and without cleft palate and the relationship to nasal airflow patterns. Cleft Palate Craniofac J. 2002;39(4):409-24. http://dx.doi.org/10.1597/1545-1569_2002_039_0409_peosav_2.0.co_2 PMid:12071789.
  » http://dx.doi.org/10.1597/1545-1569_2002_039_0409_peosav_2.0.co_2
• ²⁷
  Persson C, Lohmander A, Elander A. Speech in children with an isolated cleft palate: a longitudinal perspective. Cleft Palate Craniofac J. 2006;43(3):295-309. http://dx.doi.org/10.1597/04-071.1 PMid:16681402.
  » http://dx.doi.org/10.1597/04-071.1
• ²⁸
  Brunnegård K, Lohmander A. A cross-sectional study of speech in 10-year-old children with cleft palate: results and issues of rater reliability. Cleft Palate Craniofac J. 2007;44(1):33-44. http://dx.doi.org/10.1597/05-164 PMid:17214536.
  » http://dx.doi.org/10.1597/05-164
• ²⁹
  Lohmander A, Willadsen E, Persson C, Henningsson G, Bowden M, Hutters B. Methodology for speech assessment in the Scandcleft project--an internacional randomized clinical trial on palatal surgery: experiences from a pilot study. Cleft Palate Craniofac J. 2009;46(4):347-62. http://dx.doi.org/10.1597/08-039.1 PMid:19642772.
  » http://dx.doi.org/10.1597/08-039.1
• ³⁰
  Henningsson G, Kuehn DP, Sell D, Sweeney T, Trost-Cardamone JE, Whitehill TL. Universal parameters for reporting speech outcomes in individuals with cleft palate. Cleft Palate Craniofac J. 2008;45(1):1-17. http://dx.doi.org/10.1597/06-086.1 PMid:18215095.
  » http://dx.doi.org/10.1597/06-086.1

Publication Dates

• Publication in this collection
  02 Dec 2019
• Date of issue
  2019

History

• Received
  05 Dec 2018
• Accepted
  09 Apr 2019