Reliability of the Pediatric Quality of Life Inventory - Healthcare Satisfaction Generic Module 3.0 version for the assessment of the quality of care of children with chronic diseases

Abstracts

OBJETIVO: Traduzir e avaliar a confiabilidade do módulo Satisfação com o Serviço de Saúde do questionário Pediatric Quality of Life Inventory versão 3.0 (SSS-PedsQL) para o idioma português e para a cultura brasileira. MÉTODO: Utilizamos a metodologia proposta pelos autores da versão original do questionário: 1) tradução por especialistas bilíngues; 2) tradução reversa para o inglês com avaliação pelos autores da versão original; 3) estudo piloto envolvendo 10 pais (entrevista cognitiva); e 4) avaliação das propriedades de medida do questionário, sendo administrado a um grupo de 60 pais de crianças e adolescentes com doenças crônicas em acompanhamento nos departamentos de pediatria de um hospital-escola. RESULTADOS: Obtivemos alfa de Cronbach entre 0,57 e 0,78 para os domínios do questionário (informação, inclusão familiar, comunicação, habilidades técnicas, necessidades emocionais e satisfação geral), demonstrando uma boa consistência interna. Foi avaliada a reprodutibilidade interobservador em 20% da amostra, com coeficiente de correlação de Spearman de 0,909. Observamos um ótimo escore de satisfação com as equipes multiprofissionais para todos os domínios (> 4,3 pontos), exceto para os domínios "necessidades emocionais", com 3,5 pontos, e "comunicação", com 3,9. CONCLUSÃO: A versão brasileira do SSS-PedsQL é válida e confiável e é útil na mensuração da satisfação global dos pais e/ou cuidadores com o tratamento dos seus filhos.

Doenças crônicas; questionário; qualidade de vida relacionada à saúde; família

ORIGINAL ARTICLE

^IFisioterapeuta. Mestre, Ciências Aplicadas à Pediatria, Universidade Federal de São Paulo - Escola Paulista de Medicina (UNIFESP-EPM), São Paulo, SP, Brazil

^IIPsicóloga. Doutoranda. Mestre, Ciências Aplicadas à Pediatria, UNIFESP-EPM, São Paulo, SP, Brazil

^IIIProfessora adjunta, Disciplina de Alergia, Imunologia e Reumatologia, Departamento de Pediatria, UNIFESP-EPM, São Paulo, SP, Brazil

^IVProfessora associada, Disciplina de Alergia, Imunologia e Reumatologia, Departamento de Pediatria, UNIFESP-EPM, São Paulo, SP, Brazil

^VProfessor adjunto, Disciplina de Alergia, Imunologia e Reumatologia, Departamento de Pediatria, UNIFESP-EPM, São Paulo, SP, Brazil

Correspondence

ABSTRACT

OBJECTIVE: To translate to Brazilian Portuguese and to evaluate the reliability of the Healthcare Satisfaction (HS) Generic Module version 3.0 of the PedsQL inventory for the Brazilian culture.

METHOD: The methods recommended by the authors of the original version of the questionnaire were used: 1) translation by bilingual experts; 2) back translation into English and evaluation by the authors of the original version; 3) a pilot study including 10 parents (cognitive interview); and 4) evaluation of the measurement properties of the questionnaire. The questionnaire was administered to 60 parents of children and adolescents with chronic diseases in key units of the Department of Pediatrics.

RESULTS: Cronbach's alpha ranged from 0.57 to 0.78 for all domains (information, family inclusion, communication, technical skills, emotional needs and overall satisfaction), which confirmed good internal consistency. Test-retest reproducibility was evaluated using 20% of the sample (Spearman correlation coefficient = 0.909). The satisfaction score was good in all areas (> 4.3 points) except in the emotional needs domain.

CONCLUSION: The Brazilian version of the HS-PedsQL is a valid and reliable tool to measure the overall satisfaction of parents and caregivers with the treatment of their children.

Keywords: Chronic disease, inventory, health-related quality of life, family.

Introduction

The importance of ensuring patient satisfaction with the care received during health treatments, and issue of great interest among medical researchers, has been discussed for a long time.^1,2 Some studies^1-5 have raised hypotheses about the main factors associated with satisfaction or dissatisfaction. There is a significant correlation between adherence to treatment, increased adherence and lower chances of changing doctors when the patients are satisfied with the healthcare team. Good relationships and communications between doctor, team and patient are signs of quality and may determine physical and mental health status.^3,5-10

The raison d’être for healthcare organizations is the people for whom these services are provided, that is, the patients. Therefore, several studies^1,3-6,11 have been conducted to find out and understand the wishes and needs of patients and their caregivers. In studies with adults, lack of information has been the complaint most often associated with low satisfaction with care. However, few studies have been conducted to evaluate it in pediatric populations.^3,10,12

Varni et al.⁷ developed a questionnaire to evaluate parental satisfaction with the multidisciplinary care provided to their children, the Healthcare Satisfaction (HS) Generic Module (translated to Portuguese as Módulo de Satisfação com o Serviço da Saúde) of the Pediatric Quality of Life Inventory version 3.0 (HS-PedsQL). A tool capable of measuring treatment satisfaction should be available to find out about family needs. Therefore, we translated, adapted culturally and evaluated the reliability of the HS-PedsQL for use in Brazil.

Material and methods

This study was approved by the Ethics Committee of the institution where it was conducted. All participants signed an informed consent term before interviews and data collection.

The guidelines for translation and validation recommended by the authors¹³ of the original HS-PedsQL were followed according to the steps described below:

Step 1. Independent translation of the original English version of the HS-PedsQL into Brazilian Portuguese and adaptation to the Brazilian culture by three healthcare researchers, followed by a joint reconciliation discussion to produce a combined version of the three translations. At this step, the researchers made all efforts to produce a linguistically and conceptually equivalent translation.

Step 2. The first version in Portuguese was back translated into English by two bilingual translators and then sent to the PedsQL project team, coordinated by Dr. James Varni, in San Diego, CA. The suggestions and comments were accepted and added to the Portuguese version by the researchers during a consensus meeting.

Step 3. Pilot study with cognitive interview was conducted, and the translated version was administered to 10 parents of 2- to 18-year-old children with a chronic disease and currently hospitalized. The responses to the questions were evaluated to detect and correct errors and difficulties introduced along the translation process. The questionnaires were administered in a quiet environment, and the parents were encouraged to point out any difficulties in understanding, which were later discussed by the research team.

Step 4. In this phase, called field study, the questionnaires, already translated and culturally adapted, were administered to a larger group of parents of children with chronic diseases. Participants were recruited during 15 days using a convenience sampling technique. Parents were interviewed in the following hospital units: Pediatric Rheumatology Outpatient Department, Pediatric Nephrology Outpatient Department, Chronic Disease Ward in General Pediatrics, Hemodialysis Unit and Chemotherapy Unit of the Pediatric Oncology Institute. The questionnaires were coded using letters and numbers to ensure confidentiality and participant anonymity.

Participants were parents and caregivers of 2- to 18-year-old children and adolescents with chronic diseases who had been followed up for at least 3 months in our institution. Only the parents and caregivers of patients with a definitive diagnosis were included.

Measurements

Standardized forms were used to collect demographic and clinical data about the patients. To evaluate disease severity, the attending physicians in each unit were contacted and asked to define whether the disease was mild, moderate or severe. The general health condition of the child at the time of the study was assessed by the physician and the caregiver using a visual analog scale in which zero was excellent general condition and 10 was very poor general condition. Socioeconomic data were collected using the criteria issued by the Associação Brasileira de Empresas de Pesquisa (ABEP).¹⁴

The HS-PedsQL⁷ is a 24-item (question) inventory. The items are grouped into six domains associated with healthcare satisfaction: information (five items), family inclusion (four items), communication (five items), technical skills (three items), emotional needs (four items) and overall satisfaction (three items). Parents chose a score from 1 to 5 after being asked whether they were satisfied with each item, in which the scores corresponded to: 1 = never satisfied; 2 = some times satisfied; 3 = often satisfied; 4 = almost always satisfied; and 5 = always satisfied; or they chose "not applicable" (N/A), in which case the item was not scored and not included in the final sum. In the case responses were not given to more than 50% of the items in the questionnaire, that questionnaire was excluded from the study. We decided to administer the questionnaire rather than have participants self-administer it because of the educational level of our population. The interviewer had no previous contact with the patients or parents and was not part of the team responsible for their treatment. For reliability, internal consistency was evaluated using the Cronbach’s alpha coefficient for all the domains, and interobserver reproducibility was tested by having two of the examiners administer the questionnaire to the same subgroup of participants.

Statistical analysis

The HS-PedsQL reliability was determined according to its internal consistency, for which Cronbach’s alpha coefficients (> 0.70) were calculated for the different domains of the module.

According to the Kolmogorov-Smirnov test, data were nonnormally distributed, and the nonparametric Kruskal-Wallis test was used to check the correlation between satisfaction and disease severity. Satisfaction was also correlated with socioeconomic status, and the level of statistical significance was set at p < 0.05.

The Spearman test was used to analyze coherence between data collected to check interobserver agreement, and the level of statistical significance was set at p < 0.05. Means and standard deviations were used for the other analyses.

The Portuguese version of the Statistical Package for the Social Sciences (SPSS) was used for statistical analyses.

Results

Steps 1, 2, and 3. The translation and cultural adaptation of the HS-PedsQL were successful and a final consensus was reached by the local team and the authors of the original version, who authorized its use in this study.

Mean time to administer the HS-PedsQL was 7 minutes. After the pilot study, changes were made that resulted in a better understanding by the parents. One of the questions was classified as difficult to understand by 89.7% of the parents, even after changes: The amount of time spent in helping you son/daughter in his/her return to school, which belonged to the emotional needs domain. In general, parents and caregivers did not understand how the healthcare team might help in their return to school.

Step 4. The preliminary version was administered to parents and caregivers of 60 children with chronic diseases. Of these children, 28 were boys (46.7%). Mean age was 9.6 years (2), and age range was 2 to 18 years. Demographic data of patients and caregivers are shown in Table 1. Of the parents and caregivers in the sample, 49 (81.6%) were mothers, nine were fathers (15%) and two were other caregivers (3.4%); 22 (36.7%) had not finished elementary school, and 39 (65%) did not work out of the home (Table 1). Patient distribution according to treatment unit is shown in Table 2.

Cronbach’s alpha coefficients are shown in Table 3. Reliability and overall internal consistency was 0.786. The calculation of Cronbach’s alpha coefficient for each domain revealed that only the technical skills and family inclusion domains had low internal consistency, at 0.575 and 0.60. All the domains showed good mean satisfaction with the team, except emotional needs, in which the mean was 3.5 points, and communication, with a mean of 3.9.

The analysis of severity revealed that the disease was severe in 54% of the patients; moderate in 28%; and mild in 18%. The results of the Kruskal-Wallis test showed that there was a correlation between severity and satisfaction: patients with a severe disease had a mean satisfaction of 26.47; those with a moderate disease, 37.85%; and those with a mild disease, 30.86% (p = 0.027).

There was no correlation between satisfaction and socioeconomic status (Kruskal-Wallis test, p = 0.0567).

The mean score for general condition of the child at the time of the survey according to the attending physician was 4.6, and the standard deviation, 2.7; according to the caregiver, 3.8 with a standard deviation of 3.1. These findings demonstrated that the general condition of the children was good.

The Spearman correlation coefficient for interobserver reproducibility was almost perfect, at 0.909, for the overall scores assigned to the questionnaires administered to 20% of the sample.

Discussion

The main objective of this study was to translate and validate a questionnaire that measures parents’ and caregivers’ satisfaction with the care provided by a multidisciplinary healthcare team to their children with chronic diseases. Also, it evaluated the main difficulties faced by these parents in their relationship with the healthcare team. Healthcare professionals often do not self-evaluate their capacity of communication with the families, which complicates treatment success.

Several factors are associated with medical treatment efficacy, and when the communication between healthcare professionals and patients is good, there is greater adherence to and continuity of the treatment, as well as lower chances of abandoning treatment and/or changing doctors or institutions.^1,15 However, the practice of self-evaluation is not usual in most healthcare services, particularly in relation to the degree of satisfaction with the care provided.^5,6

In the original HS-PedsQL,⁷ the authors evaluated information, family inclusion, communication, technical skills, emotional needs and overall satisfaction. To evaluate the instrument’s reliability, the authors administered the same questionnaire to the parents of 55 patients with cancer in the Children's Hospital of San Diego and found a satisfactory Cronbach's alpha for HS-PedsQL in general and a range of 0.502 to 0.819 for the various domains, which confirms the internal consistency of the module. In our study, we used the version that was translated and adapted to Brazilian Portuguese in a group of parents or caregivers of children with chronic diseases of different etiologies who were followed up in different units of the Department of Pediatrics, responsible for the care provided to a large number of patients with complex diseases in the city of São Paulo, Brazil, as well as to patients that come from other regions in the state and in the country. In agreement with Varni,⁷ we found an acceptable Cronbach’s coefficient for all domains (from 0.575 to 0.788) and for the entire questionnaire (0.786).

In addition to internal consistency, we evaluated another property of the same measurement, interobserver reproducibility. The HS-PedsQL was administered to 20% of the sample by two observers on the same day at different times, and the Spearman correlation coefficient was 0.909, which is almost perfect.

In our study, we chose to administer the questionnaire to parents and caregivers, and not to use it as a self-administration instrument. Although the language and terms used seem to be understood by most parents, we made that decision because many parents had not completed elementary school (36.6%) and the predominant social class was low (class C – 63.4%) according to the ABEP classification.¹⁴ In a study conducted in our service by Terreri et al.¹⁷ to evaluate the socioeconomic impact of rheumatic fever, they found that 17% of the parents did not know how to read and write.

In general, the level of parental satisfaction was good, with variations according to disease type and severity. More severe diseases involve complex treatments, which may complicate communications with and understanding by the patients and their families.^4,5,18 For example, the level of overall satisfaction was low among parents of children receiving hemodialysis despite the known efforts of the healthcare team, whose work is a quality reference standard in our milieu when compared with other services.¹⁰

Some studies found that healthcare satisfaction is positively associated with overall health condition, that is, patients in good health are more satisfied with the care provided.^2,4 In our study, we also found a correlation between greater disease severity and lower satisfaction with treatment. The HS-PedsQL scores were lower for patients with more severe diseases (53.3% of total) according to the attending physician, and there was a statistically significant difference from patients with mild or moderate diseases (p = 0.027).

The mean scores of the sample as a whole were lower in the communication and emotional needs domains, at 3.9 and 3.5. In the study conducted by Varni et al.⁷, the domain with the lowest score was emotional needs. These important findings reflect several aspects of poor services and dissatisfaction, and the lack of communication by healthcare professionals is one of the most frequent complaints made by patients and their families.¹⁹ Questions and lack of information are the mains causes of lack of adherence to treatment, and the poor communication between healthcare professionals, patients and their families leads to dissatisfactions, insecurity, anxiety and stress.^1,2,4,6,8,19 Some of the duties of healthcare professionals are to provide information about the disease and its treatment, to prepare patients and families before the performance of procedures, to adopt measures to relieve pain and discomfort, to involve the family in the treatment process, and to safeguard the right of the family to make decisions.^8,20 Parental understanding of the disease determines the way the child will experience it and facilitates the work of the multidisciplinary team.

As demonstrated by Varni et al., the administration of the HS-PedsQL was rapid and simple in our study, as it lasted about 7 minutes. The calculation of the overall score was very simple, and no specific handbook or software had to be obligatorily used for that task.

In summary, the results of our study showed that the Brazilian version of the HS-PedsQL is valid, reproducible and reliable, which makes it a useful instrument to detect the level of family satisfaction and diagnose possible failures. These findings may be used to define precise interventions in one or more aspects of healthcare services and to ensure comfort and better assistance for our patients.

Acknowledgments

We thank Professor James W. Varni, from Texas A&M University, and the MAPI-Institute for granting permission to translate and validate the HS-PedsQL inventory into our language and culture and for supporting our work along the several steps of this process.

References

• 1. Lewis CC, Scott DE, Pantell RH, Wolf MH. Parent satisfaction with childrens medical care. Development, field test, and validation of a questionnaire. Med Care. 1986;24:209-15.
• 2. Hall JA, Dornan MC. Meta-analysis of satisfaction with medical care: description of research domain and analysis of overall satisfaction levels. Soc Sci Med. 1988;27:637-44.
• 3. Hays RM, Valentine J, Haynes G, Geyer JR, Villareale N, McKinstry B, et al. The Seattle Pediatric Palliative Care Project: effects on family satisfaction and health-related quality of life. J Palliat Med. 2006;9:716-28.
• 4. Marshall GN, Hays RD, Mazel R. Health status and satisfaction with health care: results from the medical outcomes study. J Consult Clin Psychol. 1996;64:380-90.
• 5. Halal IS, Sparrenberger F, Bertoni AM, Ciacomet C, Seibel CE, Lahude FM, et al. Avaliação da qualidade de assistência primária à saúde em localidade urbana da região Sul do Brasil. Rev Saude Publica. 1994;28:131-6.
• 6. Fitzpatrick, R. Surveys of patient satisfaction: I - Important general considerations. BMJ. 1991;302:887-9.
• 7. Varni JW, Quiggins DJ, Ayala GX. Development of the pediatric hematology/oncology parent satisfaction survey. Child Health Care. 2000;29:243-55.
• 8. Young PC, Wasserman RC, McAullife T, Long J, Hagan JF, Heath B. Why families change pediatricians: factors causing dissatisfaction with pediatric care. Am J Dis Child. 1985;139:683-6.
• 9. Blumenthal D. Part 1: Quality of care - what is it? N Engl J Med. 1996;335:891-4.
• 10. Contro NA, Larson J, Scofield S, Sourkes B, Cohen HJ. Hospital staff and family perspectives regarding quality of pediatric palliative care. Pediatrics. 2004;114:1248-52.
• 11. Barbosa LT. Fatores que afetem a satisfação de pacientes em serviços de saúde: um estudo em setor ambulatorial de hospital. [dissertação]. Natal: Universidade Federal do Rio Grande do Norte; 2004.
• 12. Kloetzel K, Bertoni AM, Irazoqui MC, Campos VP, Santos RN. Controle da qualidade em atenção primária à saúde. I - A satisfação do usuário. Cad Saude Publica. 1998;14:623-8.
• 13. Varni JW. PedsQLTM Translation MethodologyTM. 1998-2007. http://www.pedsql.org/translations.html Acesso: 10/10/2010.
• 14. Associação Brasileira de Empresa de Pesquisa (ABEP). Critério de Classificação Econômica no Brasil. 2004. http://www.abep.org/novo/Content.aspx?ContentID=139 Acesso: 02/10/10.
• 15. Resende MC, Santos FA, Souza MM, Marques TP. Atendimento psicológico a pacientes com insuficiência renal crônica: em busca de ajustamento psicológico. Psicol clin. 2007;19:87-99.
• 16. Klatchoian DA, Len CA, Terreri MT, Silva M, Itamoto C, Ciconelli RM, et al. Quality of life of children and adolescents from São Paulo: reliability and validity of the Brazilian version of the Pediatric Quality of Life Inventory version 4.0 Generic Core Scales. J Pediatr (Rio J). 2008;84:308-15.
• 17. Terreri MT, Ferraz MB, Goldenberg J, Len C, Hilário MO. Resource utilization and cost of rheumatic fever. J Rheumatol. 2001;28:1394-7.
• 18. Ballone GJ. Humanização do Atendimento em Saúde in PsiqWeb. http://www.psiqweb.med.br/site/?area=NO/LerNoticia&idNoticia=251 Acesso: 02/10/10.
• 19. Lemos FA, Lima RA, Mello DF. Assistência à criança e ao adolescente com câncer: a fase da quimioterapia intratecal. Rev Lat Am Enfermagem. 2004;12:485-93.
• 20. Contim D, Chaud MN, Fonseca AS. As questões familiares e a criança e o adolescente com doença crônica: um estudo de revisão bibliográfica. Nursing (São Paulo). 2005;8:267-71.

Reliability of the Pediatric Quality of Life Inventory - Healthcare Satisfaction Generic Module 3.0 version for the assessment of the quality of care of children with chronic diseases

Fernanda M. de Souza^I; Juliana Molina^II; Maria Teresa R. A. Terreri^III; Maria Odete E. Hilário^IV; Claudio A. Len^V

Publication Dates

• Publication in this collection
  12 Mar 2012
• Date of issue
  Feb 2012

History

• Received
  01 Aug 2011
• Accepted
  28 Sept 2011