ROC curve of the Pediatric Voice Related Quality-of-Life Survey (P-VRQOL)

ABSTRACT

To verify the efficiency and to determine the cutoff values that discriminate children/adolescents with and without vocal complaints, as well as the measures of sensibility, specificity and efficiency of the Brazilian Pediatric Voice-Related Quality-of-Life Survey (P-VRQOL). The participants included 230 parents of children/adolescents of both genders, aged between 2 years and 18 years, with and without vocal complaints that responded the validated Brazilian version of P-VRQOL. The three scores (total, physical and social-emotional) were analyzed by the Receiver Operating Characteristic Curve (ROC curve). The cutoff values, ROC curve and the measures of specificity, sensibility and efficiency varied as the score investigated - total, physical or social-emotional. The total score demonstrated excellent discrimination (efficiency=0.936; specificity=0.991; and sensibility=0.881); the social-emotional score was a reasonable indicator (efficiency=0.794; specificity=0.604; and sensibility=0.983) and the physical score was an excellent sorter (efficiency=0.918; specificity=0.946; and sensibility=0.890). The cutoff values and area under curve were: total score- cutoff=96.25 and AUC=0.98; physical score- cutoff=91.68 and AUC=0.97; social-emotional score cutoff=96.87 and AUC=0.79. The P-VRQOL is an excellent sorter to discriminate children/adolescents with and without vocal complaints. The perception of parents about the presence of vocal problem allows the judge of lower quality of life in 98% of the cases, especially, in P-VRQOL physical domain.

Keywords:
ROC Curve; Quality of Life; Child; Adolescent; Questionnaires; Parents

RESUMO

Verificar a eficiência e determinar a nota de corte do instrumento Qualidade de Vida em Voz Pediátrico (QVV-P), bem como as medidas de sensibilidade, especificidade e eficiência. Participaram 230 voluntários, pais/responsáveis de crianças/adolescentes com e sem queixa vocal, de ambos os gêneros, com idade entre 2 e 18 anos, que responderam a versão brasileira validada do QVV-P, cujos três escores (geral, físico e socioemocional) foram analisados pela Receiver Operating Characteristic Curve (curva ROC). As notas de corte, a curva ROC e as medidas de especificidade, sensibilidade e eficiência variaram conforme o escore investigado - geral, físico ou socioemocional. O escore geral apresentou poder discriminatório excelente (eficiência=0,936; especificidade=0,991; e sensibilidade=0,881), o escore socioemocional teve classificação de rendimento razoável (eficiência=0,794; especificidade=0,604; e sensibilidade=0,983), e o escore físico foi um classificador excelente (eficiência=0,918; especificidade=0,946; sensibilidade=0,890). Foram determinadas as seguintes notas de corte e área sob a curva: escore geral- corte=96,25 e AUC=0,98; escore físico- corte=91,68 e AUC=0,97; e escore socioemocional- corte=96,87 e AUC=0,79. O protocolo QVV-P é um classificador excelente e eficiente na discriminação de crianças e adolescentes com e sem queixa vocal. A percepção dos pais/responsáveis quanto à presença de uma alteração na voz de seu filho possibilita a compreensão do comprometimento na qualidade de vida relacionada à voz, em 98% dos casos, principalmente, no que se refere às questões físicas do QVV-P.

Descritores:
Curva ROC; Qualidade de Vida; Criança; Adolescente; Questionários; Pais

INTRODUCTION

The parental assessment protocols have been highly recommended in the pediatric voice clinic, since the young age of the patient can derail the vocal self-assessment⁽¹⁾. Among the parental assessment procedures, we highlight the quality of life related to voice, enabling better targeting of therapeutic procedures^(2-4).

The Pediatric Voice-Related Quality-of-Life protocol (PVRQOL) with Brazilian validation⁽³⁾ allow the parents to measure the impact of a voice problem in their children quality of life⁽⁴⁾ in a sensitive and reliable manner; it has 10 affirmative questions (4 from socioemotional field of quality of life and 6 from the physical sphere) with application between 2 and 18 years old⁽⁵⁾. Regarding the psychometric measures of the PVRQOL validity⁽³⁾ and considering the increasingly frequent use, we aimed to determine the cutoff points of the instrument and sensitivity, specificity and efficiency measures.

METHODS

Approved by the Ethics Committee (027/11). The study included 230 parents of children/adolescents with and without voice complaints, from both genders, aged between 2 and 18; all of them signed the informed consent form. The collection occurred in speech therapy clinic-school, private and public schools and otolaryngology clinics. The clinical characteristics of the participants are listed in Table 1.

Table 1
Characterization of children/adolescents with and without voice complaint

Group	Female	Male	P-value M x F	Total	Average age	P-value Age	Average score	Minimum score	Maximum score	Median
Group with voice complaint	50	62	0.231	112	9.9	0.874	G=78.65 SE=85.37 P=73.78	22.5	97.5	100
Group without voice complaint	62	56		118	9.8		G=99.05 SE=99.89 P=98.47	90	100	100

• Caption: G=General score; SE= socioemotional score; P=physical score. P-value measured by the Mann-Whitney test

The PVRQOL scores were analyzed by the Receiver Operating Characteristic (ROC curve) that allowed for the determination of the cutoff value by the combination of the greater specificity and sensitivity, verification of efficiency, sensitivity and specificity, allowing the classification of children and adolescents with and without voice complaints by the definition of the area under the curve (area under curve – AUC).

The ROC curve is a binary tool with five degrees of rating: Excellent (0.9 to 1), good (0.8 to 0.9), fair (0.7 to 0.8), poor (0.6 to 0.7), and not discriminating (0.5 to 0.6)⁽⁶⁾. Sensitivity data translated the number of true positives identified by the PVRQOL, compared to all positive who completed the instrument. The specificity data reflect cases false positive compared to all negative.

RESULTS

The groups with and without voice alteration complaints were similar regarding gender (p = 0.231) and age (p = 0.874). The cutoff values, AUC, sensitivity and effectiveness varied according to the investigated score. The overall score showed cutoff values of 96.25, excellent area under the curve, excellent specificity, high sensitivity, and excellent efficiency. The social-emotional domain had cutoff value of 96.87, reasonable AUC, excellent sensitivity, poor specificity and reasonable efficiency. And the physical domain obtained cutoff value of 91.68 and excellent area under the curve, with good sensitivity, excellent specificity and efficiency - see Table 2.

Table 2
Cutoff value, Area under the curve, Sensitivity, Specificity and Efficacy of the PVRQOL protocol

	Cutoff value	Are under the curve	Sensitivity	Specificity	Efficacy
General score	96.25	0.988	0.881	0.991	0.936
Socioemotional score	96.87	0.797	0.983	0.604	0.794
Physical score	91.68	0.971	0.890	0.946	0.918

• Caption: Analysis by ROC curve

DISCUSSION

Data from the ROC curve of the overall score showed that parents/guardians, when recognizing their child voice problem also realize the impact on quality of life in 98% of cases. Moreover, it can be stated that the PVRQOL separates individuals with and without vocal complaints, even without the diagnosis of dysphonia; what sets it as an excellent tool for screening, evaluation and vocal accompaniment.

The excellent specificity showed that PVRQOL only points loss in quality of life regarding the problem of vocal complaint; and the good sensitivity indicated that individuals with a complaint may not be detected, which reinforces the importance of crossing the self-assessment, clinical and laryngeal assessment for the correct diagnosis of dysphonia. As the excellent efficiency ensures that the PVRQOL faithfully measures what it proposes, it is a tool of easy administration and short application⁽⁵⁾.

The parents/guardians recognition of 2 of the 10 questions of the protocol there is a problem, even if small (value of 2 in the likert scale of PVRQOL), indicates a possible change in the child/adolescent voice, since the overall score will be below the cutoff grade; which reinforces the fact that PVRQOL is a central goal protocol voice with excellent specificity and efficiency.

The specific analysis of the PVRQOL domains concludes that the socioemotional domain was not enough to differentiate the individuals, as the AUC was reasonable, demonstrating that it is not a strong parameter for the detection of voice changes. In addition, the excellent sensitivity accompanied by poor specificity and relative efficiency showed that, although it is sensitive to voice complaint is not specific to a voice change.

The physical domain, in turn, showed better data with excellent AUC, good sensitivity and good specificity and efficiency. Thus, however the children/adolescents with vocal complaints are not always identified by this domain, it only recognizes impairment in quality of life before a vocal problem; it is therefore the most specific and robust domain of PVRQOL.

The three protocol scores can be taken into consideration. However, it is recommended attention to the characteristics identified above; it is necessary to analyze, first, the results of the overall score, followed by physical and socioemotional scores. For vocal screening actions, it is recommended, initially, the use of the overall score for criterion of approval or failure, since it has excellent specificity and efficiency.

CONCLUSION

The PVRQOL is an efficient and reliable parental assessment tool with excellent discriminatory power, which can be used in the screening revaluation actions, even if the individual has not been diagnosed with dysphonia. The parents’ perception about the presence of a change in the child's voice allows our understanding of loss of quality of life in 98% of cases, especially from the physical aspects perspective.

• Study carried out at Universidade Federal do Espírito Santo – UFES - Vitória (ES), Brazil.
• Financial support: nothing to declare.

REFERÊNCIAS

• ¹
  Verduyckt I, Remacle M, Jamart J, Benderitter C, Morsomme D. Voice-related complaints in the pediatric population. J Voice. 2011;25(3):373-80. http://dx.doi.org/10.1016/j.jvoice.2009.11.008 PMid:20359863.
  » http://dx.doi.org/10.1016/j.jvoice.2009.11.008
• ²
  Gasparini G, Behlau M. Quality of Life: Validation of the Brazilian Version of the Voice-Related Quality of Life (V-RQOL) Measure. J Voice. 2009;23(1):76-81. http://dx.doi.org/10.1016/j.jvoice.2007.04.005 PMid:17628396.
  » http://dx.doi.org/10.1016/j.jvoice.2007.04.005
• ³
  Ribeiro LL, Paula KM, Behlau M. Voice-related quality of life in the pediatric population: validation of the Brazilian version of the Pediatric Voice-Related Quality-of Life survey. CoDAS. 2014;26(1):87-95. [online] PMid:24714864.
• ⁴
  Connor NP, Cohen SB, Theis SM, Thibeault SL, Heatley DG, Bless DM. Attitudes of children with dysphonia. J Voice. 2008;22(2):197-209. http://dx.doi.org/10.1016/j.jvoice.2006.09.005 PMid:17512168.
  » http://dx.doi.org/10.1016/j.jvoice.2006.09.005
• ⁵
  Boseley ME, Cunningham MJ, Volk MS, Hartnick CJ. Validation of the Pediatric Voice-Related Quality-of-Life Survey. Arch Otolaryngol Head Neck Surg. 2006;132(7):717-20. http://dx.doi.org/10.1001/archotol.132.7.717 PMid:16847178.
  » http://dx.doi.org/10.1001/archotol.132.7.717
• ⁶
  Tape TG. Interpreting diagnostic tests [Internet]. Omaha: University of Nebraska Medical Center; 2015 [citado em 2015 Jun 9]. Disponível em: http://gim.unmc.edu/dxtests/Default.htm
  » http://gim.unmc.edu/dxtests/Default.htm

Publication Dates

• Publication in this collection
  31 May 2016
• Date of issue
  May-Jun 2016

History

• Received
  24 Apr 2015
• Accepted
  16 Aug 2015