Vocal symptoms in pediatric population: Validation of the Brazilian version of the Pediatric Vocal Symptoms Questionnaire

Abstract

Purpose

To validate the Pediatric Vocal Symptoms Questionnaire (PVSQ) for Brazilian Portuguese.

Methods

The study was approved by the Ethics Committee (758,309). A total of 716 individuals participated in this research, of which 367 were children and adolescents, with and without vocal alteration, aged 6-18 years, and 349 were parents/guardians, who responded to the final version of the instrument. Among the interviewed of this version, 272 participated in the test-retest, and 32, of the sensitivity. Children and adolescents aged 6-18 years responded to the self-evaluation version of the PVSQ, and their parents/guardians to its parental version.

Results

The PVSQ showed acceptable reliability and reproducibility for the Brazilian population and sensitivity to vocal treatment.

Conclusion

The PVSQ was validated for Brazilian Portuguese, being a good instrument of vocal self-evaluation, both in the parental version and in the self-evaluation version.

Keywords
Dysphonia; Validation Studies; Translation (product); Voice Disorders/diagnosis; Questionnaires; Child; Adolescent; Parents

ResumoObjetivo: Validar o Questionário de Sintomas Vocais Pediátrico (QSV-P) para o português brasileiro.

Método

Aprovado pelo Comitê de Ética (758.309). Participaram desta pesquisa 716 indivíduos, dos quais 367 eram crianças e adolescentes, com e sem alteração vocal, entre 6 e 18 anos, e 349 eram pais/responsáveis, que responderam à versão final do instrumento. Entre os respondentes dessa versão, 272 participaram do teste-reteste, e 32, da sensibilidade. As crianças e os adolescentes entre 6 e 18 anos responderam à versão de autoavaliação do QSV-P, e seus pais/responsáveis, à versão parental do QSV-P.

Resultados

O QSV-P apresentou confiabilidade e reprodutibilidade aceitáveis para a população brasileira e sensibilidade ao tratamento vocal.

Conclusão

O QSV-P foi validado para o português brasileiro, sendo um bom instrumento de autoavaliação vocal, tanto na versão parental quanto na versão autoavaliativa.

Descritores
Disfonia; Estudos de Validação; Tradução (produto); Distúrbios da Voz/Diagnóstico; Questionários; Criança; Adolescente; Pais

INTRODUCTION

Vocal symptoms in the pediatric population can be a common occurrence and affect up to 23% of children aged 4-12 years⁽¹⁾, with early manifestations (soon at birth) or late (during development), depending on the etiological⁽²⁾ factor, and be related to organic⁽³⁾, functional⁽²⁾ or organo-functional factors⁽⁴⁾. Such symptoms may be reported by the child, his/her parents/guardians and/or their teachers and perceived by the speech therapist during voice evaluation. The main reports are alterations in vocal quality, fatigue, exertion, loss of vocal extension, lack of control of intensity and frequency and unpleasant sensations related to emission⁽⁵⁾.

Children with vocal disorders say they feel more symptoms than the vocally healthy ones during voice self-evaluation⁽⁶⁾. Pain, throat and cough when speaking and singing, difficulties to read aloud, sing or scream in games and child’s plays were the most mentioned symptoms in one study with schoolchildren and adolescents, who also reported feelings of frustration, anger, embarrassment and voice dissatisfaction⁽⁷⁾.

Contrary to what the traditional literature points out, from the age of 6 years children are able to talk about their symptoms and their vocal alterations^(7-9); however, if the vocal deviation is clear for the parents and the speech therapist, but it is not perceived by the child itself, it is important that the situation be contextualized from the description of the voice and its alterations through age-appropriate stories and approaches⁽¹⁰⁾.

Self-report has been valued in the pediatric voice clinic, and the Pediatric Vocal Symptoms Questionnaire (PVSQ)⁽⁹⁾ is an instrument that strengthens not only the perception of children and adolescents about their voice problem, but also the combination of the parents’ and children’s reports within the clinic^(6,9).

The objective of this study was to validate the Brazilian version of the Questionnaire des Symptômes Vocaux protocol, called the Pediatric Vocal Symptoms Questionnaire (PVSQ), through the cultural and language adaptation of the instrument and psychometric measures of validity, reliability and sensitivity.

METHODS

The study was approved by the Research Ethics Committee of the institution, under Opinion n. 758309. After the cultural equivalence⁽⁹⁾ of the original version of the instrument – Questionnaire des Symptômes Vocaux⁽⁶⁾ – the protocol was applied in its final version to measure the psychometric measures of validation.

The validation process complied with the international standards of the Committee of the Scientific Council of the Association of Medical Outcomes⁽¹¹⁾. Data were collected in two states (São Paulo and Espírito Santo), three cities (São Paulo, Vitória and Vila Velha), three schools (one private and two public) and in speech-therapist and otorhinolaryngology offices, specialized in the care of childhood dysphonia.

The Pediatric Vocal Symptoms Questionnaire⁽⁹⁾ is the only pediatric protocol that addresses self-evaluation and parental evaluation simultaneously, offering a broader overview of voice use and the possible impact of a voice alteration in various aspects of life. It has 31 self-explanatory objective questions, applicable to children and adolescents aged 6-18 years and their parents/guardians, contemplating four domains (spoken voice, sung voice, projected voice and screaming voice). The answers are recorded on a 4 point-numerical scale (0 = never, 1 = sometimes, 2 = almost always and 3 = always), and there is the visual support of circles ranging from small size to very large size, aiming to facilitate the scoring of responses of the younger children⁽⁶⁾. The PVSQ has a single total score obtained by direct sum of the items (maximum score = 38). The original version of the instrument has no cut-off point published so far.

Participants comprised 367 children and adolescents with and without voice complaint and/or vocal alteration and 349 parents/guardians (n = 716). The age of children and adolescents ranged from 6 to 18 years. All parents/guardians signed the Informed Consent Form and responded to the parental version of the PVSQ, and the children and adolescents signed the Consent Term and responded to the self-evaluation version of the said instrument. Parents and children evaluated the vocal quality as excellent, very good (subsequently grouped in the excellent category), good, reasonable or poor (subsequently grouped in the poor category). Among the interviewed, 163 were children and 204 were adolescents, with and without vocal complaint, and 349 were their parents/guardians, whose degree of proximity was close or very close.

The following inclusion criteria were adopted for the group with vocal alteration (GWVA): Brazilian Portuguese as the mother tongue; aged 6-18 years; presence of parental or self-reported vocal complaint and/or vocal quality compatible with voice diverted in the auditory-perceptual evaluation (G₂ or G₃). For the group without vocal alteration (GWOVA), were adopted: Brazilian Portuguese as the mother tongue; aged 6-18 years; absence of parental and self-reported vocal complaint; and vocal quality compatible with adapted voice in the auditory-perceptual analysis (G₀ or G₁). These criteria were ensured through the completion of a questionnaire to identify and characterize the sample. For the group with vocal alteration (GWVA), the following exclusion criteria were adopted: speech or language alterations; speech therapy and upper airway infection during the test and the retest of the PVSQ and during the auditory-perceptual evaluation of the voice; and complaints about psychological/psychiatric alterations that prevented the completion of the questionnaire.

For the group without vocal alteration (GWOVA), we considered the exclusion criteria: speech or language alterations; vocal complaint; vocal problems diagnosed; infection of the upper airways during the test and the retest of the PVSQ; and complaints about psychological/psychiatric alterations that prevented the completion of the questionnaire.

The auditory-perceptual analysis (APA) of the voices was performed by a speech therapist specializing in voice and with expertise in childhood dysphonia. The evaluator was guided, by a written text, to listen to the voices of children and adolescents with and without complaint about vocal alteration; then, the use of a headset was requested, in a comfortable intensity, for a better perception of vocal emission.

The evaluator received the identification of gender and age of each participant to avoid that vocal characteristics expected in the second childhood and adolescence were judged as vocal deviations. Vocal samples were analyzed by counting numbers 1-10 to determine the overall degree of vocal deviation (G) as absent (0), discreet (1), moderate (2) or intense (3). The task of counting was adopted, because it is closer to the customary speech of the children, and the evaluation of the general degree, because it is the clinical measure with greater reliability in the APA, since, in children, the evaluation of the predominant vocal quality varies according to the used task⁽¹¹⁾. A software that draws numbers (Picker) was used to perform the random reproducibility of 20% of the vocal samples totaling 434 files, which were evaluated with 73.61% of intra-rater reliability.

The group with vocal alteration (GWVA) comprised 210 children and adolescents (113 females and 97 males). The group without vocal alteration (GWOVA) comprised 157 children and adolescents (106 females and 51 males). Mean age was 11.50 years for the GWVA and 11.96 years for the GWOVA. Participants of the GWVA and GWOVA groups were similar regarding age (p = 0.175), with a predominance of females (p = 0.008).

In an interval of 2 to 14 days after the initial application of the PVSQ, as suggested by the Scientific Council⁽¹²⁾, 272 participants were summoned to respond again to the instrument for measuring reliability. The overall 136 children and adolescent respondents had a mean age of 12 and 19 years, respectively, with 74 females and 62 males.

Afterwards, 21 children and adolescents were recruited to perform eight sessions of therapy for vocal rehabilitation for measuring the PVSQ sensitivity. Participants should attend a speech therapy session per week and daily perform the vocal exercise plan three times a day. Due to absences and/or low adherence to the proposed treatment, five participants were excluded, which resulted in a final group of 16 children and adolescents who presented vocal quality compatible with a general degree of moderate vocal deviation and a predominantly rough or breathy vocal quality, as well as a laryngological evaluation consistent with behavioral dysphonia, characterized as follows: vocal nodules and double slit (7); mid-posterior triangular slit (2); posterior triangular slit indicative of minimal structural alteration (1); nodular thickening and mid-posterior triangular slit (4); laryngeal hyperconstriction and irregular slit throughout the extension (1); parallel slit in the entire extension (1). Parents and children responded again to the instrument after two months of the speech therapy intervention.

The comparison of participants with and without vocal alteration was performed to verify differences between the studied groups regarding the PVSQ variables (Mann-Whitney test), The internal consistency of the instrument was determined for verification of reliability (Cronbach’s Alpha test), there were possible differences between the test and the retest regarding the variable vocal quality evaluation (Mcnemar test) and other PVSQ variables (Wilcoxon signed-rank test) and the sensitivity of the instrument was measured (Wilcoxon signed-rank test). To analyze the mean scores of the PVSQ in all groups studied (with and without vocal alteration), considering the overall group and the two age groups (schoolchildren and adolescents), the Mann-Whitney test was applied. The Spearman correlation analysis was used to verify whether the vocal evaluation performed by the parents/guardians, in relation to the voice of their children, according to the three categories of analysis (excellent, good and poor), was sensitive to the PVSQ results.

RESULTS

Children and adolescents with vocal alteration and their parents/guardians, when they perceive a loss in vocal quality (bad voice), also identified a higher occurrence of vocal symptoms (Table 1). The PVSQ showed reliability for clinical and scientific use, because the values estimated by Cronbach’s alpha were high (concordance in 92.18% of the tested items), revealing high internal consistency of individual issues and of the overall score of vocal symptoms (p < 0.001), both in self-evaluation and parental evaluation (Table 2). Parents and children similarly perceived vocal quality in the test and retest (Table 3). In addition, the instrument had acceptable level of reproducibility (Table 4) since it presented satisfactory results that can be used in other analyses (Wilcoxon signed-rank test p > 0.05 for most instrument items, showing consistency of response in the test and retest). The PVSQ can also measure the modifications obtained with speech therapy, which was confirmed by the modification of the overall score of vocal symptoms (p < 0.05), in both versions (Table 5).

Table 1
Correlation between vocal quality evaluation and the overall PVSQ score

Variable	VS Score [PVSQ - SE]	VS Score [PVSQ - PE]
SE - GWVA
CC (r)	-0.494	-0.209
p	< 0.001*	0.003*
n	210	197
PE - GWVA
CC (r)	-0.279	-0.725
p	< 0.001*	< 0.001*
n	204	197
SE - GWOVA
CC (r)	-0.313	-0.090
p	< 0.001*	0.269
n	157	152
PE - GWOVA
CC (r)	-0.188	-0.279
p	0.020*	< 0,001*
n	153	152

• ^*
  Significant values (p ≤ 0.050) – Spearman’s correlation test
• Caption: SE = Self-Evaluation; PE = Parental Evaluation; CC: Correlation Coefficient; GWVA = Group with vocal alteration; GWOVA = Group without vocal alteration; n = Number of participants

Table 2
Reliability data of the PVSQ according to Cronbach’s alpha coefficient values of the individual questions and of the overall vocal symptoms score (n = 136)

Item	Test			Retest
	Cronbach’s alpha coefficient	p		Cronbach’s alpha coefficient	p
SE - PE
1 “Do/Does you/your child feel/feels that your/his/her voice gets tired when...”
a “…you/he/she talk/talks, play/plays, talk/talks on the phone... or after that?”	-0.411	0.743		0.620	0.035*
b “... you/he/she read/reads aloud, participate/participates in parties, in the school theater... Or after that? “	0.439	0.137		0.766	0.004*
c “... you/he/she sing/sings, participate/participates in choirs, or sing/sings at Karaoke bars... Or after that? “	0.532	0.077		0.000	0.500
d “...you/he/she play/plays football, dodgeball, run and catch, practise/practises sports... Or after that?”	0.256	0.287		0.564	0.060
2 “Do people ask you/your child to repeat what you/he/she said because of your/his/her voice?”	0.190	0.344		-0.640	0.826
3 “Do/Does you/your child have/has to use force for your/his/her voice to go out?	0.237	0.304		0.613	0.038*
4 “Do/Does you/your child get annoyed because of your/his/her voice?”	-0.019	0.515		0.689	0.015*
5 “Do/ Does you/your child avoid using your/his/her voice when...”
a “…you/he/she talk/talks, play/plays, talk/talks on the phone... because you do not like your voice/because his/her voice is not how he/she would like it to be?”	0.313	0.238		-0.205	0.639
b “... you/he/she read/reads aloud, participate/participates in parties, in the school theater... because you do not like your voice/because his/her voice is not how he/she would like it to be?”	0.158	0.371		0.000	0.500
c “... you/he/she sing/sings, participate/participates in choirs, or sing/sings at Karaoke bars... because you do not like your voice/because his/her voice is not how he/she would like it to be?”	0.184	0.349		-0.113	0.581
d “...you/he/she play/plays football, dodgeball (queimada), run and catch, practise/practises sports... because you do not like your voice/because his/her voice is not how he/she would like it to be?”	0.055	0.457		-0.113	0.581
6 “Do/Does you/your child have/has to use force for your/his/her voice to go out?	0.447	0.131		0.625	0.033*
7 “Have your voice and your child’s voice ever been debauched, mocked, or made a joke of?”	0.686	0.016*		0.425	0.147
8 “Have/Has you/your child ever stayed without speaking because you/he/she were/was voiceless?”	0.324	0.229		0.552	0.066
9 “Do/Does you/your child get angry because of your/his/her voice?”	0.038	0.471		0.461	0.122
10 “Are you/your/your child afraid to harm or worsen your/his/her voice?”	0.693	0.014*		-0.033	0.525
11 “Do/Does you/your child get/gets sore throat when...”
a “…you/he/she talk/talks, play/plays, talk/talks on the phone... or after that?”	0.716	0.010*		-0.421	0.748
b “... you/he/she read/reads aloud, participate/participates in parties, in the school theater... Or after that? “	0.674	0.019*		0.258	0.285
c “... you/he/she sing/sings, participate/participates in choirs, or sing/sings at Karaoke bars... or after that? “	0.241	0.300		0.478	0.110
d “...you/he/she play/plays football, dodgeball (queimada), run and catch, practise/practises sports... or after that?”	-0,294	0,688		0,874	< 0,001*
12 “Do people ask you/your child ‘what is in your voice’?”	0.120	0.404		0.615	0.037*
13 “Do/Does you/your child have to cough or clear the throat when you/he/she talk/talks even without being sick (flu or cold)? (Demonstration by physician/clinician)”	0.480	0.108		0.491	0.101
14 “Do/Does you/your child get hoarse even when you/he/she are/is not sick?”	0.348	0.209		0.067	0.447
15 “Do/Does you/your child have difficulty completing sentences, when you/he/she speak/speaks, because of your/his/her voice?”	0.427	0.146		0.442	0.135
16 “Do/ Does you/your child have to rest the voice when..”
a “…you/he/she talk/talks, play/plays, talk/talks on the phone... or after that?”	0.464	0.119		-0.168	0.616
b “... you/he/she read/reads aloud, participate/participates in parties, in the school theater... Or after that?	0.382	0.181		-0.762	0.858
c	0.139	0.388		0.253	0.290
“... you/he/she sing/sings, participate/participates in choirs, or sing/sings at Karaoke bars … or after that? d	0.042	0.467		0.530	0.078
“...you/he/she play/plays football, dodgeball (queimada), run and catch, practise/practises sports... or after that?” 17 “Do/Does you/your child feel that the throat burns or bothers you even when you are not sick (flu/cold)?”	0.381	0.182		0.582	0.051
18 “Are/Is you/your child sad because of your/his/her voice?”	0.806	0.001*		0.460	0.122
19 “Would you/your child like to change your/his/her voice?”	0.522	0.082		0.558	0.063
VS Score	0.848	< 0.001*		0.904	< 0.001*

• ^*
  Significant values (p ≤ 0.050) – Cronbach’s alpha test
• Caption: SE = Self-Evaluation; PE = Parental Evaluation; A = Subitem of the question referring to the spoken voice; b = subitem of the question referring to the projected voice; c = subitem of the question referring to the sung voice; D = Subitem of the question referring to the screamed voice; VS = Vocal Symptoms

Table 3
Reproducibility data according to the self-evaluation and parental evaluation of vocal quality

Type of evaluation Test %		Retest %			p
		Excellent	Good	Bad
SE
Excellent	1.50	1.50	0.00	0.00	0.081
Good	29.40	3.70	23.50	2.20
Bad	69.10	0.00	6.60	62.50
Total	100.00	30.10	5.10	64.70
PE
Excellent	11.80	5.90	5.90	0.00	0.342
Good	42.60	7.40	28.70	6.60
Bad	45.60	0.00	2.90	42.60
Total	100.00	13.20	37.50	49.30

• Significant values (p ≤ 0.050) – McNemar’s test
• Caption: SE = Self-evaluation; PE = Parental Evaluation

Table 4
Reproducibility data according to the self-evaluation and parental evaluation for the vocal symptoms score of the PVSQ protocol (n = 136)

Pair of variables	M	SD	Min	Max	M	p
SE
1 “Do you feel that your voice gets tired when...”
a “…you talk, play, talk on the phone... or after that?”
Test	0.76	0.76	0.00	2.00	1.00	> 0.999
Retest	0.76	0.76	0.00	2.00	1.00
b “... you read aloud, participate in parties, in the school theater... or after that?”
Test	0.68	0.72	0.00	2.00	1.00	0.046*
Retest	0.71	0.72	0.00	2.00	1.00
c “... you sing, participate in choirs, or sing at Karaoke bars... or after that?”
Test	0.85	0.77	0.00	2.00	1.00	0.414
Retest	0.86	0.75	0.00	2.00	1.00
d “...you play football, dodgeball (queimada), run and catch, practise... or after that?”
Test	0.85	0.85	0.00	2.00	1.00	0.096
Retest	0.89	0.85	0.00	2.00	1.00
2 “Do people ask you to repeat what you said because of your voice?”
Test	0.90	0.78	0.00	2.00	1.00	0.317
Retest	0.90	0.79	0.00	2.00	1.00
3 “Do you have to use force for your voice to go out?”
Test	0.43	0.68	0.00	2.00	0.00	0.014*
Retest	0.47	0.71	0.00	2.00	0.00
4 “Do you get irritated because of your voice?”
Test	0.53	0.77	0.00	2.00	0.00	0.083
Retest	0.55	0.78	0.00	2.00	0.00
5 “Do you avoid using your voice when...”
a “…you talk, play, talk on the phone... because you do not like it?”
Test	0.40	0.71	0.00	2.00	0.00	0.317
Retest	0.41	0.71	0.00	2.00	0.00
b “... you read aloud, participate in parties, the school theater... because you do not like your voice?”
Test	0.42	0.73	0.00	2.00	0.00	0.157
Retest	0.43	0.74	0.00	2.00	0.00
“... you/he/she sing/sings, participate/participates in choirs, or sing/sings at Karaoke bars... Or after that? “
c “... you sing, participate in choirs, or sing at Karaoke bars... or after that?” Test	0.43	0.74	0.00	2.00	0.00	0.157
Retest	0.44	0.75	0.00	2.00	0.00
d “...you play football, dodgeball (queimada), run and catch, practise sports... or after that?”
Test	0.32	0.63	0.00	2.00	0.00	0.317
Retest	0.33	0.63	0.00	2.00	0.00
6 Do you have to use force for speaking?”
Test	0.30	0.59	0.00	2.00	0.00	0.157
Retest	0.32	0.59	0.00	2.00	0.00
7 “Have your voice ever been debauched, mocked, or made a joke of?
Test	0.49	0.73	0.00	2.00	0.00	0.317
Retest	0.49	0.73	0.00	2.00	0.00
8 “Have you ever been without speaking because you were voiceless?”
Test	0.41	0.60	0.00	2.00	0.00	0.317
Retest	0.42	0.60	0.00	2.00	0.00
9 “Do you get angry because of your voice?”
Test	0.35	0.63	0.00	2.00	0.00	0.317
Retest	0.35	0.63	0.00	2.00	0.00
10 “Are you afraid to harm or worsen your voice?”
Test	1,04	0.81	0.00	2.00	1.00	0.157
Retest	1.06	0.81	0.00	2.00	1.00
11 “Do you have a sore throat when ...”
a “…you talk, play, talk on the phone... or after that?”
Test	0.59	0.79	0.00	2.00	0.00	0.025*
Retest	0.63	0.83	0.00	2.00	0.00
b “... you read aloud, participate in parties, in the school theater... or after that? “
Test	0.46	0.70	0.00	2.00	0.00	0.083
Retest	0.49	0.71	0.00	2.00	0.00
c “... you sing, participate in choirs, or sing at Karaoke bars... or after that? “
Test	0.54	0.76	0.00	2.00	0.00	0.083
Retest	0.57	0.76	0.00	2.00	0.00
d “...you play football, dodgeball (queimada), run and catch, practise sports... Or after that?”
Test	0.51	0.76	0.00	2.00	0.00	0.180
Retest	0.53	0.75	0.00	2.00	0.00
12 “Do people ask you ‘what is in your voice’?”
Test	0.37	0.66	0.00	2.00	0.00	0.014*
Retest	0.41	0.69	0.00	2.00	0.00
13 “Do you have to cough or clear the throat when you talk even without being sick (flu or cold)? (Demonstration by physician/clinician)”
Test	0.78	0.75	0.00	2.00	1.00	0.317
Retest	0.79	0.74	0.00	2.00	1.00
14 “Do you get hoarse even when you are not sick?”
Test	0.79	0.75	0.00	2.00	1.00	0.317
Retest	0.79	0.76	0.00	2.00	1.00
15 “Do you have difficulty completing sentences, when you speak, because of your voice?”
Test	0.64	0.75	0.00	2.00	0.00	0.046*
Retest	0.67	0.76	0.00	2.00	0.00
16 “Do you have to rest your voice when...”
a “…you talk, play, talk on the phone... or after that?”
Test	0.74	0.79	0.00	2.00	1.00	0.317
Retest	0.74	0.79	0.00	2.00	1.00
b “... you read aloud, participate in parties, in the school theater... or after that? “
Test	0.65	0.80	0.00	2.00	0.00	0.102
Retest	0.68	0.82	0.00	2.00	0.00
c “... you sing, participate in choirs, or sing at Karaoke bars... or after that? “
Test	0.89	0.85	0.00	2.00	1.00	> 0.999
Retest	0.89	0.85	0.00	2.00	1.00
d “...you play football, dodgeball (queimada), run and catch, practise sports... or after that?”
Test	0.66	0.84	0.00	2.00	0.00	> 0.999
Retest	0.66	0.85	0.00	2.00	0.00
17 “Do you feel that the throat burns or bothers you even when you are not sick (flu/cold)?”
Test	0.71	0.79	0.00	2.00	1.00	0.317
Retest	0.73	0.80	0.00	2.00	1.00
18 “Are you sad because of your voice?”
Test	0.36	0.66	0.00	2.00	0.00	0.014*
Retest	0.40	0.69	0.00	2.00	0.00
19 “Would you like to change your voice?”
Test	0.99	0.86	0.00	2.00	1.00	0.317
Retest	1.00	0.86	0.00	2.00	1.00
VS Score
Test	13.35	9.04	0.00	38.00	11.75	< 0.001*
Retest	26.05	17.74	0.00	74.00	23.00
SE
1 “Does your child feel his/her voice gets tired when …”
a “... he/she talks, plays, talks on the phone ... or after that?”
Test	0.65	0.73	0.00	2.00	0.50	> 0.999
Retest	0.65	0.74	0.00	2.00	0.00
b “... you read aloud, participate in parties, in the school theater... or after that?”
Test	0.56	0.75	0.00	2.00	0.00	0.655
Retest	0.57	0.73	0.00	2.00	0.00
c “... you sing, participate in choirs, or sing at Karaoke bars... or after that?”
Test	0.57	0.78	0.00	2.00	0.00	0.655
Retest	0.57	0.76	0.00	2.00	0.00
d “...you play football, dodgeball (queimada), run and catch, practise sports... or after that?”
Test	0.66	0.82	0.00	2.00	0.00	0.035*
Retest	0.71	0.83	0.00	2.00	0.00
2 “Do people ask your child to repeat what he/she said because of his/her voice?”
Test	0.72	0.75	0.00	2.00	1.00	> 0.999
Retest	0.72	0.76	0.00	2.00	1.00
3 “Does your child have to use force for his/her voice to go out?”
Test	0.38	0.67	0.00	2.00	0.00	0.157
Retest	0.39	0.67	0.00	2.00	0.00
4 “Does your child get irritated because of his/her voice?”
Test	0.27	0.57	0.00	2.00	0.00	0.083
Retest	0.29	0.60	0.00	2.00	0.00
5 “Does your child avoid using his/her voice when...”
a “... he/she talks, plays, talks on the phone ... because his/her voice is not what he/she would like it to be?”
Test	0.18	0.53	0.00	2.00	0.00	0.157
Retest	0.20	0.54	0.00	2.00	0,00
b “... he/she reads aloud, participates in parties, in the school theater... because his/her voice is not how he/she would like it to be?”
Test	0.17	0.51	0.00	2.00	0.00	0.317
Retest	0.18	0.51	0.00	2.00	0.00
c “... he/she sing/sings, participates in choirs, or sings at Karaoke bars... because his/her voice is not how he/she would like it to be?”
Test	0.20	0.53	0.00	2.00	0.00	0.317
Retest	0.21	0.53	0.00	2.00	0.00
d “...he/she plays football, dodgeball (queimada), run and catch, practises sports... because his/her voice is not how he/she would like it to be?”
Test	0.15	0.45	0.00	2.00	0.00	0.157
Retest	0.18	0.49	0.00	2.00	0.00
6 “Does your child have to use force for his/her voice to go out?
Test	0.36	0.65	0.00	2.00	0.00	> 0.999
Retest	0.36	0.65	0.00	2.00	0.00
7 “Has your voice’s son ever been debauched, mocked, or made a joke of?”
Test	0.40	0.70	0.00	2.00	0.00	0.317
Retest	0.41	0.71	0.00	2.00	0.00
8 “Has your child ever been without speaking because he/she was voiceless?”
Test	0.30	0.55	0.00	2.00	0.00	0.317
Retest	0.29	0.55	0.00	2.00	0.00
9 “Does your child get angry because of his/her voice?”
Test	0.32	0.60	0.00	2.00	0.00	0.564
Retest	0.32	0.63	0.00	2.00	0.00
10 “Is your child afraid to harm or worsen his/her voice?”
Test	0.35	0.65	0.00	2.00	0.00	0.317
Retest	0.34	0.65	0.00	2.00	0.00
11 “Does your child get sore throat when...”
a “…he/she talks, plays, talks on the phone... or after that?”
Test	0.36	0.66	0.00	2.00	0.00	> 0.999
Retest	0.36	0.66	0.00	2.00	0.00
b “... he/she reads aloud, participates in parties, in the school theater... or after that?”
Test	0.32	0.63	0.00	2.00	0.00	0.564
Retest	0.32	0.65	0.00	2.00	0.00
c “... he/she sings, participates in choirs, or sings at Karaoke bars... or after that?”
Test	0.34	0.62	0.00	2.00	0.00	0.564
Retest	0.35	0.65	0.00	2.00	0.00
d “...he/she plays football, dodgeball (queimada), run and catch, practises sports... or after that?”
Test	0.31	0.60	0.00	2.00	0.00	0.046*
Retest	0.34	0.64	0.00	2.00	0.00
12 “Do people ask your child ‘what is in your voice’?”
Test	0.26	0.56	0.00	2.00	0.00	0.317
Retest	0.27	0.56	0.00	2.00	0.00
13 “ Does your child have to cough or clear the throat when he/she talks even without being sick (flu or cold)? (Demonstration by physician/clinician)”
Test	0.54	0.72	0.00	2.00	0.00	0.317
Retest	0.54	0.73	0.00	2.00	0.00
14 “Does your child get hoarse even when he/she is not sick?”
Test	0.73	0.76	0.00	2.00	1.00	0.414
Retest	0.74	0.78	0.00	2.00	1.00
15 “Does your child have difficulty completing sentences, when he/she speaks, because of his/her voice?”
Test	0.38	0.61	0.00	2.00	0.00	> 0.999
Retest	0.38	0.61	0.00	2.00	0.00
16 “Does your child have to rest his/her voice when..”
a “... he/she talks, plays, talks on the phone ... or after that?”
Test	0.40	0.69	0.00	2.00	0.00	0.083
Retest	0.43	0.71	0.00	2.00	0.00
b “... he/she reads aloud, participates in parties, in the school theater... or after that?”
Test	0.40	0.69	0.00	2.00	0.00	0.317
Retest	0.39	0.69	0.00	2.00	0.00
c “... he/she sings, participates in choirs, or sings at Karaoke bars”... or after that?
Test	0.41	0.70	0.00	2.00	0.00	0.564
Retest	0.40	0.69	0.00	2.00	0.00
d “...he/she plays football, dodgeball (queimada), run and catch, practises sports... or after that?”
Test	0.43	0.70	0.00	2.00	0.00	0.083
Retest	0.40	0.68	0.00	2.00	0.00
17 “Does your child feel that the throat burns or bothers even when he/she is not sick (flu/cold)?”
Test	0.49	0.70	0.00	2.00	0.00	> 0.999
Retest	0.49	0.70	0.00	2.00	0.00
18 “Does your child get angry because of his/her voice?”
Test	0.35	0.67	0.00	2.00	0.00	0.317
Retest	0.36	0.67	0.00	2.00	0.00
19 “Would your child like to change his/her voice?”
Test	0.56	0.81	0.00	2.00	0.00	0.317
Retest	0.57	0.80	0.00	2.00	0.00
VS Score
Test	8.52	9.95	0.00	36.00	4.50	< 0.001*
Retest	16.55	19.19	0.00	71.00	9.38

• ^*
  Significant values (p ≤ 0.050) – Wilcoxon signed-rank test
• Caption: SE = Self-Evaluation; PE = Parental Evaluation; a = Subitem of the question referring to the spoken voice; b = Subitem of the question referring to the projected voice; c = Subitem of the question referring to the sung voice; D = Subitem of the question referring to the screamed voice; VS = Vocal Symptoms

Table 5
Sensitivity data of the overall score of the PVSQ: Self-evaluation and parental evaluation (n = 32)

Pair of variables	Mean	SD	Min	Max	Median	p
SE
VS Score
Pre-therapy	17.81	7.27	3.50	34.50	16.25	< 0.001*
Post- therapy	6.47	6.09	1.25	25.75	4.13
PE
VS Score
Pre-therapy	15.52	9.68	0.00	36.25	16.38	0.001*
Post-therapy	4.63	4.21	0.00	12.75	5.50

• ^*
  Significant values (p ≤ 0.050) – Wilcoxon test
• Caption: SE = Self-Evaluation; PE = Parental Evaluation; SD = Standard Deviation; Max = Max; Min = Minimum; VS = Vocal Symptoms

Individuals with vocal alteration had more vocal symptoms than individuals who did not report alterations in their voices, both in the parental evaluation and self-evaluation. The vocal symptom score was higher in the self-evaluation than in the parental evaluation (as shown in Table 4), which reinforces the fact that the vocal symptoms are more perceived by the individual, even when it comes to children and adolescents. The experience of living with a voice problem is unique, and the perception of the other may not be sufficiently comprehensive. The perception of vocal symptoms is higher in adolescence, in self-report, when interpersonal aspects become more important, as well as belonging to and communicating with a group^(13,14). Parental evaluation is very similar for children and adolescents.

DISCUSSION

The use of the self-evaluation of vocal quality as an external measure for measuring the validity of a self-evaluation instrument has been adopted in several Brazilian validation studies^(15-17). In a research on the quality of life related to voice in children and adolescents⁽¹⁷⁾, there was a positive correlation between parental evaluation of vocal quality and the scores of the Pediatric Voice-Related Quality-of-Life Protocol (PVRQOL), demonstrating that the perception of a bad voice correlates to a lower quality of life in children and adolescents aged 2-18 years.

The negative correlation between vocal symptoms and evaluation (parents and children) of vocal quality in both the groups with and without voice alteration indicates that one of the symptoms observed is phonatory. Parents and children recognize that, in the presence of a bad voice, there are more vocal symptoms, but the correlation force between these variables is higher in the intra-evaluation than in the inter-evaluation, that is, a child, when assessing his/her vocal quality and vocal symptoms, recognizes a stronger negative relationship than his/her parents do, and the same occurs in parental evaluation. These data reinforce the importance of collecting information between the two respondents.

Only the GWOVA did not correlate with the parental evaluation of vocal symptoms, showing that, when the child or adolescent does not have a vocal problem, the parents have more difficulty to perceive the vocal symptoms. Furthermore, in the presence of a voice alteration, the parents are the informants with the best perception between vocal quality and vocal symptoms, since they presented the highest correlation force (correlation coefficient (r) = -0.72), as shown in Table 1.

The individual analysis of the questions of the PVSQ reinforces the more circumstantial profile of the response of children and adolescents, since seven items presented difference in the test-retest as opposed to the two observed in the parental evaluation, which resulted in higher test-retest difference in the vocal symptoms score in the self-report version, as shown in Table 4. In the self-report, the differences relate to physical, sociofunctional and emotional symptoms; in the parental report, such changes appeared only in the physical item, the most objective domain⁽¹⁸⁾, for which parents have a better perception⁽¹⁷⁾. Analyzing the reproducibility of the vocal symptoms score of the PVSQ, it is observed that: there was difference both in the parental evaluation and self-evaluation; the mean scores were higher in the retest, differently from what occurred in the validation of the original version⁽⁶⁾; the change in the scores reinforces that the contact with the instrument enhances the perception of parents and children regarding the vocal symptoms, by assisting in the comprehension of the experienced difficulties^(7,8), which is very important, even for vocal rehabilitation, since it is only possible to modify behaviors through their perception.

The PVSQ is an instrument that can be used as a revaluation resource in the vocal clinic, since the two versions, whether parental or self-reported, presented sensitivity to the eight sessions of speech therapy administered to 16 children and adolescents with vocal alteration (Table 5). Therefore, the PVSQ corresponds to the proposals of the Patient Reported Outcome Measures (PROMs) to investigate symptoms and problems that may interfere in the life of the individual and in his sense of well-being, to verify the changes resulting from the treatments performed⁽¹⁹⁾ and be a support for clinical monitoring⁽²⁰⁾.

After the speech-language intervention, which did not necessarily correspond to the speech-language discharge, but rather to a service with a predetermined number of sessions, to attend to scientific questions, children and adolescents perceived modifications in 27 PVSQ items, and their parents recognized the changes in 26 questions. Possibly, in a regular clinical care, the protocol may be even more sensitive, at the time of the patient’s actual hospital discharge. As the intervention group covered varied laryngeal alterations and functional and organofunctional dysphonias, it can be affirmed that the PVSQ is sensitive to modifications resulting from a vocal treatment, regardless of the type of dysphonia found. In the self-evaluation, items that did not present post-therapy modification were related to physical aspects (need to repeat what they speak; need to force voice; and sensation of burning or being uncomfortable) and emotional (feeling of anger), while in parental evaluation the lack of modification was present related to sociofunctional issues (avoid using voice in conversations and child’s play; in readings, festivals and theater; and in football, dodgeball (burnt match, sports, etc.), emotional (fear of hurting or worsening voice) and physical (difficulty in completing sentences). The score reduction in most of the PVSQ questions and in the vocal symptoms score demonstrates, therefore, that the protocol can be used as an important resource of vocal monitoring and revaluation.

When the vocal symptoms score of the PVSQ is specifically analyzed, it is possible to observe that the symptoms are more frequent in individuals with vocal alteration and that they are more perceived by self-evaluation (Table 4). The symptoms reported were physical, emotional and sociofuncional, as already pointed out by the literature⁽⁷⁾. The parents, in turn, recognized more vocal symptoms with extrinsic manifestations on the part of the individual, such as alterations in vocal quality, signs of fatigue and vocal effort, which are also the most frequent symptoms in childhood⁽²¹⁾, coming from the children’s own vocal pattern.

Comparing the data of the present study with the validation of the original version⁽⁶⁾, it is observed that the Brazilian average scores are slightly inferior, with the exception of parental evaluation in the group without vocal alteration, which presented itself slightly superior. The greatest difference was observed in the parental evaluation of the group with vocal alteration (Brazilian version = 6.53 and original version = 10.40). The validation in the original version used the composition of three groups (dysphonic, vocally healthy and group with vocal deviation, but without complaint) and observed that the dysphonic group had higher occurrence of vocal symptoms, followed by the group without complaint and with vocal deviation, both in the parental evaluation and self-evaluation⁽⁶⁾. In this study, the composition of the group with dysphonia was not used because the laryngological exam was not performed in all participants (only the intervention group to measure the PVSQ sensitivity underwent functional investigation of the larynx), and the presence of parental or self-reported complaint and/or vocal deviation in the auditory-perceptual evaluation of the voice were adopted as a criterion for the composition of the GWA. These criteria and procedures may have influenced the differences observed in the scores associated with social and cultural factors, since the Brazilian results of the GWA self-evaluation are closer to the group with deviation and without complaint reported in the study of Verduyckt et al.⁽⁶⁾, while the results of the GWA parental evaluation are closer to the dysphonic group. It is believed that the presence of vocal complaint associated with the presence of vocal deviation in the PE potentializes the scores of vocal symptoms of the PVSQ and that a vocal deviation in the PE dissociated from a vocal complaint reduces the score of the instrument, although keeping it superior in relation to individuals without complaint and without vocal deviation.

As important as evaluating the vocal symptoms is to evaluate the frequency of their occurrence⁽⁷⁾. Regardless of etiology, vocal symptoms can be verified by self-evaluation questionnaires⁽¹⁶⁾, which have been considered the cornerstone of a vocal evaluation⁽²²⁾. Although, for the pediatric population, the exclusive parental evaluation⁽⁸⁾ is generally used, the literature points out that children from 6 years of age are able to reflect on their voice problems, providing relevant information about different aspects of vocal quality⁽⁷⁾. Thus, parental evaluation, although very important, should not replace self-evaluation^(7,8), even if the vocal symptoms of dysphonic children are perceived by both parents and the children themselves⁽⁶⁾. Moreover, it is important to consider that discrepancies between the perceptions of children and their parents, regarding social experiences and long-term facts, can make the exclusive parental evaluation an unreliable resource⁽⁶⁾.

Vocal problems should be evaluated in a multimodal way⁽²³⁾ and considering different contexts of voice use. It is known that the reports of parents and children about the vocal symptom do not present general agreement^(6,7), that the child, even recognizing a change in his/her voice, may like it and not point out symptoms⁽⁷⁾ and that there is a poor correlation between the clinic and the perception of the subject who lives with dysphonia^(24-26). The speech-language evaluation and the vocal self-evaluation have different results^(8,17), so they should not be exclusively used. The sum of the information assists in understanding the vocal problem, directs actions, defines therapeutic objectives and measures the effects of therapy in the comparison before and after speech therapy⁽²³⁾.

CONCLUSION

The Questionnaire des Symptômes Vocaux protocol, titled Pediatric Vocal Symptoms Questionnaire, was validated for the Brazilian Portuguese language in parental and self-reported versions. The PVSQ showed acceptable reliability and reproducibility for the Brazilian population and sensitivity to vocal treatment, so it is considered a good instrument of vocal self-evaluation for the aforementioned population. Children and adolescents recognize more vocal symptoms than their parents/guardians, both in the test and in the retest (shown in tables 2 and 4), which demonstrates that the symptom, because it often involves kinesthetic issues, is more well perceived by the individual.

ACKNOWLEDGEMENT

To the Coordination of Improvement of Higher Education Personnel - CAPES - for the financial support granted to this research, an essential condition for its realization.

• Study conducted at Universidade Federal de São Paulo – UNIFESP - São Paulo (SP), Brasil.
• Financial support: CAPES – Bolsa de Doutorado.

REFERÊNCIAS

• ¹
  Carding PN, Roulstone S, Northstone K. The prevalence of childhood Dysphonia: A cross-sectional study. J Voice. 2006;20(4):623-30. http://dx.doi.org/10.1016/j.jvoice.2005.07.004 PMid:16360302.
  » http://dx.doi.org/10.1016/j.jvoice.2005.07.004
• ²
  Martins RH, Ribeiro CB, Fernandes de Mello BM, Branco A, Tavares EL. Dysphonia in children. J Voice. 2012;26(5): 674e.17-e20.
• ³
  Kollbrunner J, Seifert E. Functional hoarseness in children: short-term play therapy with family dynamic counseling as therapy of choice. J Voice. 2013;27(5):579-88. http://dx.doi.org/10.1016/j.jvoice.2013.01.010 PMid:23683805.
  » http://dx.doi.org/10.1016/j.jvoice.2013.01.010
• ⁴
  De Bodt MS, Ketelslagers K, Peeters T, Wuyts FL, Mertens F, Pattyn J, et al. Evolution of vocal fold nodules from childhood to adolescence. J Voice. 2007;21(2):151-6. http://dx.doi.org/10.1016/j.jvoice.2005.11.006 PMid:16504470.
  » http://dx.doi.org/10.1016/j.jvoice.2005.11.006
• ⁵
  Behlau M, Madazio G, Feijó D, Pontes P. Avaliação de voz. In: Behlau M, organization. Voz: o livro do especialista. Rio de Janeiro: Revinter; 2004. vol. 1. p. 85-245.
• ⁶
  Verduyckt I, Morsomme D, Ramacle M. Validation and standardization of the Pediatric Voice Symptom Questionnaire: A double-form questionnaire for dysphonic children and their parents. J Voice. 2012;26(4):129-39. http://dx.doi.org/10.1016/j.jvoice.2011.08.001
  » http://dx.doi.org/10.1016/j.jvoice.2011.08.001
• ⁷
  Verduyckt I, Remacle M, Jamart J, Benderitter C, Morsomme D. Voice-related complaints in the pediatric population. J Voice. 2011;25(3):373-80. http://dx.doi.org/10.1016/j.jvoice.2009.11.008 PMid:20359863.
  » http://dx.doi.org/10.1016/j.jvoice.2009.11.008
• ⁸
  Connor NP, Cohen SB, Theis SM, Thibeault SL, Heatley DG, Bless DM. Attitudes of children with dysphonia. J Voice. 2008;22(2):197-209. http://dx.doi.org/10.1016/j.jvoice.2006.09.005 PMid:17512168.
  » http://dx.doi.org/10.1016/j.jvoice.2006.09.005
• ⁹
  Krohling LL, Behlau M, Verduyckt I. Equivalência cultural da versão brasileira do Questionnaire des Symptômes Vocaux. CoDAS. 2016;28(4):454-8. http://dx.doi.org/10.1590/2317-1782/20162015124 PMid:27409418.
  » http://dx.doi.org/10.1590/2317-1782/20162015124
• ¹⁰
  McAllister A, Sjölander P. Children's Voice and Voice Disorders. Semin Speech Lang. 2013;34(2):71-79. http://dx.doi.org/10.1055/s-0033-1342978
  » http://dx.doi.org/10.1055/s-0033-1342978
• ¹¹
  McAllister A, Brandt SK. A comparison of recordings of sentences and spontaneous speech: perceptual and acoustic measures in preschool children’s voices. J Voice. 2012;26(5):673.e1-e5. http://dx.doi.org/10.1016/j.jvoice.2011.12.013 PMid:22717494.
  » http://dx.doi.org/10.1016/j.jvoice.2011.12.013
• ¹²
  Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al. Assessing health status and quality of life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193-205. http://dx.doi.org/10.1023/A:1015291021312 PMid:12074258.
  » http://dx.doi.org/10.1023/A:1015291021312
• ¹³
  Schoen-Ferreira TH, Aznar-Farias M, Silvares EFM. A construção da identidade em adolescentes: um estudo exploratório. Estud Psicol. 2003;8(1):107-15. http://dx.doi.org/10.1590/S1413-294X2003000100012
  » http://dx.doi.org/10.1590/S1413-294X2003000100012
• ¹⁴
  Francisco MV, Liborio RMC. Um estudo sobre bullying entre escolares do ensino fundamental. Psicol Reflex Crit. 2009;22(2):200-7. http://dx.doi.org/10.1590/S0102-79722009000200005
  » http://dx.doi.org/10.1590/S0102-79722009000200005
• ¹⁵
  Gasparini G, Behlau M. Quality of Life: Validation of the brazilian version of the Voice-Related quality of life (V-RQOL) measure. J Voice. 2009;23(1):76-81. http://dx.doi.org/10.1016/j.jvoice.2007.04.005 PMid:17628396.
  » http://dx.doi.org/10.1016/j.jvoice.2007.04.005
• ¹⁶
  Moreti F, Zambon F, Behlau M. Voice symptoms and vocal deviation self-assessment in different types of dysphonia. CoDAS. 2014;26(4):331-3. http://dx.doi.org/10.1590/2317-1782/201420130036 PMid:25211694.
  » http://dx.doi.org/10.1590/2317-1782/201420130036
• ¹⁷
  Ribeiro LL, Paula KM, Behlau M. Voice-related quality of life in the pediatric population: validation of the Brazilian version of the Pediatric Voice-Related Quality-of-Life Survey. CoDAS. 2014;26(1):87-95. http://dx.doi.org/10.1590/s2317-17822014000100013 PMid:24714864.
  » http://dx.doi.org/10.1590/s2317-17822014000100013
• ¹⁸
  Vance YH, Morse RC, Jenney ME, Eiser C. Issues in measuring quality of life in childhood cancer: measures, proxies, and parental mental health. J Child Psychol Psychiatry. 2001;42(5):661-7. http://dx.doi.org/10.1111/1469-7610.00761 PMid:11464970.
  » http://dx.doi.org/10.1111/1469-7610.00761
• ¹⁹
  Wolpert M. Uses and Abuses of Patient Reported Outcome Measures (PROMs): Potential iatrogenic impact of PROMs implementation and how it can be mitigated. Adm Policy Ment Health. 2014;41(2):141-5. http://dx.doi.org/10.1007/s10488-013-0509-1 PMid:23867978.
  » http://dx.doi.org/10.1007/s10488-013-0509-1
• ²⁰
  Black N. Patient reported outcome measures could help transform healthcare. Br Med J (Clin Res Ed). 2013;346(f167):1-5. http://dx.doi.org/10.1136/bmj.f167 PMid:23358487.
  » http://dx.doi.org/10.1136/bmj.f167
• ²¹
  Tavares ELM, Brasolotto A, Santana MF, Padovan CA, Martins RHG. Epidemiological study of dysphonia in 4-12 year-old children. Braz J Otorhinolaryngol. 2011;77(6):736-46. PMid:22183280.
• ²²
  Dejonckere PH, Bradley P, Clemente P, Cornut G, Crevier-Buchman L, Friedrich G, et al. A basic protocol for functional assessment of voice pathology, especially for investigating the efficacy of (phonosurgical) treatments and evaluating new assessment techniques. Guideline elaborated by the Committee on Phoniatrics of European Laryngological Society (ELS). Eur Arch Otorhinolaryngol. 2001;258(2):77-82. http://dx.doi.org/10.1007/s004050000299 PMid:11307610.
  » http://dx.doi.org/10.1007/s004050000299
• ²³
  Steen IN, MacKenzie K, Carding PN, Webb A, Deary IJ, Wilson JA. Optimizing outcome assessment of voice interventions, II: sensitivity to change of self-reported and observer-rated measures. J Laryngol Otol. 2008;122(1):46-51. http://dx.doi.org/10.1017/S0022215107007839 PMid:17498325.
  » http://dx.doi.org/10.1017/S0022215107007839
• ²⁴
  Woisard V, Bodin S, Yardeni E, Puech M. The Voice handicap Index: correlation between subjective patient response and quantitative assessment of voice. J Voice. 2006;21(5):623-31. http://dx.doi.org/10.1016/j.jvoice.2006.04.005 PMid:16887329.
  » http://dx.doi.org/10.1016/j.jvoice.2006.04.005
• ²⁵
  Karnell MP, Melton SD, Childes JM, Coleman TC, Dailey SA, Hoffman HT. Reliability of Clinician-Based (GRBAS and CAPE-V) and Patient-Based (V-RQOL and IPVI) documentation of voice disorders. J Voice. 2007;21(5):576-90. http://dx.doi.org/10.1016/j.jvoice.2006.05.001 PMid:16822648.
  » http://dx.doi.org/10.1016/j.jvoice.2006.05.001
• ²⁶
  Ugulino ACN, Behlau M. Autoavaliação do Comportamento Comunicativo ao Falar em Público nas Diferentes Categorias Profissionais. In: 22° Congresso Brasileiro de Fonoaudiologia; 2014 Out 8-11; Joiville. Anais. São Paulo: Sociedade Brasileira de Fonoaudiologia; 2014. 4929 p. Sessão de Concorrentes a Prêmio.

Publication Dates

• Publication in this collection
  17 Oct 2019
• Date of issue
  2019

History

• Received
  18 Sept 2018
• Accepted
  10 Jan 2019