Acessibilidade / Reportar erro

Net Promoter Score (NPS) as a tool to assess parental satisfaction in pediatric intensive care units

Abstract

Objective

To verify the performance of the Net Promoter Score (NPS) as a tool to assess parental satisfaction in pediatric intensive care units (PICUs).

Methods

The authors conducted an observational cross-sectional multicenter study in the PICUs of 5 hospitals in Brazil. Eligible participants were all parents or legal guardians of PICU-admitted children, aged 18 years or over. The NPS was administered together with the EMpowerment of PArents in THe Intensive Care (EMPATHIC-30), used as the gold standard, and a sociodemographic questionnaire. For analysis, the results were dichotomized into values greater than or equal to the median of the tests. The associations between the 2 tools were evaluated and the distribution of their results was compared.

Results

The parents or legal guardians of 78 PICU-admitted children were interviewed. Of the respondents, 85% were women and 62% were in a private hospital. The median NPS was 10 (IQR, 10-10), and the median EMPATHIC-30 score was 5.7 (IQR, 5.4-5.9). Compared with the gold standard, the NPS had a sensitivity of 100% at all cutoff points, except at cutoff 10, where the sensitivity was slightly lower (97.5%). As for specificity, NPS performance was poorer, with values ranging from 0% (NPS ≥ 5) to 47.4% (NPS = 10).

Conclusions

NPS proved to be a sensitive tool to assess parental satisfaction, but with poor ability to identify dissatisfied users in the sample.

Keywords
Net Promoter Score (NPS); EMPATHIC-30; Quality of health care; Pediatric intensive care units

Introduction

The growing concern of health organizations with users’ well-being has increased the accountability for the quality of care provided by institutions, where the patient or family plays a more active role in the assessment of services. As a result, there has been an increasing demand for tools to assess users’ satisfaction and expectations.11 Latour JM, Hazelzet JA, van der Heijden AJ. Parent satisfaction in pediatric intensive care: a critical appraisal of the literature. Pediatr Crit Care Med. 2005;6:578-84.,22 Donabedian A. Criteria and standards for quality assessment and monitoring. QRB Qual Rev Bull. 1986;12:99-108.

By raising awareness of the importance of parents in promoting child health and well-being,11 Latour JM, Hazelzet JA, van der Heijden AJ. Parent satisfaction in pediatric intensive care: a critical appraisal of the literature. Pediatr Crit Care Med. 2005;6:578-84. institutions have encouraged the continued stay of family members during the child's hospitalization.33 Latour JM, Hazelzet JA, Duivenvoorden HJ, Goudoever JB. Construction of a parent satisfaction instrument: perceptions of pediatric intensive care nurses and physicians. J Crit Care. 2009;24:255-66.,44 Parker MM, Thompson AE, Lumb P, Lacroix J, St Andre A. Co-evolutions of pediatric and adult critical care. Crit Care Med. 2021;49:188-200. In pediatric intensive care units (PICUs), parents’ experiences are often related to the admission of a critically ill child, being perceived as a moment of crisis, in which the needs and stressors involved in the process become apparent.33 Latour JM, Hazelzet JA, Duivenvoorden HJ, Goudoever JB. Construction of a parent satisfaction instrument: perceptions of pediatric intensive care nurses and physicians. J Crit Care. 2009;24:255-66.,55 Frankel LR, Hsu BS, Yeh TS, Simone S, Agus MS, Arca MJ, et al: Criteria for critical care infants and children: PICU admission, discharge, and triage practice statement and levels of care guidance. Pediatr Crit Care Med. 2019; 20:847-87. Therefore, it is important that PICU staff are aware of parents’ expectations and experiences so that the care directed to them can be optimized to meet their needs and increase satisfaction with the care provided.44 Parker MM, Thompson AE, Lumb P, Lacroix J, St Andre A. Co-evolutions of pediatric and adult critical care. Crit Care Med. 2021;49:188-200.,66 Johnson B, Abraham M, Conway J, Simmons L, Edgman-Levitan S, Sodomka P, et al. Partnering with patients and families to design a patient-and family-centered health care system. Institute for Patient-and Family-Centered Care and Institute for Healthcare Improvement; 2008.,77 Schuster MA. Measuring quality of pediatric care: where we've been and where we're going. Pediatrics. 2015;135:748-51.

The Net Promoter Score (NPS), a method developed by Reichheld,88 Reichheld FF. The one number you need to grow. Harv Bus Rev. 2003;81:46-54. is an internationally recognized tool for measuring the level of customer satisfaction and loyalty in different industries.99 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42. Increasingly, NPS is gaining ground in healthcare facilities and being used as a research tool in several specialties.99 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42.,1010 Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: a study of the Net Promoter Score using the friends and family test in England. Int J Geriatr Psychiatry. 2019;34:31-7. The test aims to assess user experience using only one question: “How likely are you to recommend our service to a friend or family?”, which is scored on a scale from 0 to 10.88 Reichheld FF. The one number you need to grow. Harv Bus Rev. 2003;81:46-54. According to the author, respondents scoring a 9 or a 10 are “promoters” of this service. Individuals answering 7 or 8 are considered indifferent (“passives”), whereas those answering 0 to 6 are probably dissatisfied with the service, being defined as “detractors”.88 Reichheld FF. The one number you need to grow. Harv Bus Rev. 2003;81:46-54.,1111 Reichheld F. A pergunta definitiva 2.0: Como as empresas que implementam o net promoter score prosperam em um mundo voltado aos clientes. Rio de Janeiro: Elsevier, 2011.

Due to its simplicity, reliability, and flexibility, NPS is widely used and has revolutionized the way user satisfaction is assessed.99 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42.,1212 Hamilton DF, Lane JV, Gaston P, Patton JT, MacDonald DJ, Simpson AH, et al. Assessing treatment outcomes using a single: question the net promoter score. Bone Joint J. 2014;96-B:622-8. Initially, it was introduced as an alternative to traditional survey methods, which are time-consuming, complicated, and difficult to analyze and understand in practice.1313 Krol MW, Boer D, Delnoij DM, Rademakers JJ. The Net Promoter Score-an asset to patient experience surveys? Health Expect. 2015;18:3099-109. Therefore, the present study's objective was to further investigate the value of NPS in the assessment of parental satisfaction and experiences in PICUs.

Methods

The authors conducted a cross-sectional multicenter study of parents or legal guardians of children admitted to 5 PICUs in Brazil, in the states of Rio Grande do Sul and Sao Paulo. These PICUs admit patients from hospital wards, emergency departments, and operating rooms, as well as those transferred from other hospitals and health care units. The PICUs have between 10 and 19 beds and all have physicians on duty, nurses, psychologists, nutritionists, physiotherapists, and social workers. The hospitals participating in this study are centers that provide care through the publicly funded Brazilian Unified Health System, private health insurance, or out-of-pocket.

The authors used a sample by interest. Eligible participants were all parents or legal guardians of a child admitted to the PICU for at least 24 hours who accompanied their child during PICU stay full-time or part-time. Parents whose son/daughter died in the PICU were not eligible for the study. Parents of a child readmitted to the same PICU more than once were interviewed only once.

Instrument administration

A sociodemographic questionnaire was administered, followed by the NPS. Subsequently, as the gold standard, the EMpowerment of PArents in THe Intensive Care (EMPATHIC-30) was applied by trained researchers. The satisfaction survey was administered within 72 hours of patient discharge from the PICU when they were transferred to a ward.

The Pediatric Index of Mortality 2 (PIM2) was used to assess severity on admission.1414 Slater A, Shann F, Pearson G; Pediatric Index of Mortality (PIM) study group: PIM2: a revised version of the Pediatric Index of Mortality. Intensive Care Med 2003;29:278-85. Patients with a PIM2 greater than or equal to the 75th percentile of the interquartile range of the sample were considered more severe.

For the NPS, the participants were asked to rate on a scale from 0 to 10 how likely they would be to recommend the service. Subsequently, the participants were asked the second question proposed by the test,1111 Reichheld F. A pergunta definitiva 2.0: Como as empresas que implementam o net promoter score prosperam em um mundo voltado aos clientes. Rio de Janeiro: Elsevier, 2011. with the purpose of understanding the reason or reasons for their score: “What is the primary reason for the score you just gave us?”, which is an open-ended question to be answered in the participant's own words. The NPS can be found in the supplementary material.

The authors used the EMPATHIC-30, developed to measure parental satisfaction with the quality of care in PICUs,1515 Latour JM, Duivenvoorden HJ, Tibboel D, Hazelzet JA. The shortened EMpowerment of PArents in THe Intensive Care 30 questionnaire adequately measured parent satisfaction in pediatric intensive care units J Clin Epidemiol. 2013;66:1045-50. which is currently considered a gold standard in the field in several countries around the world.1616 Zhuo-wen, Zhang, Latour, Bo, Guo-ping, Ying, et al. The reliability and validity of the Chinese version of the EMPATHIC-30 questionnaire measuring parental satisfaction in the pediatric intensive care unit. Chin J Nurs, 2015.,1717 Gill FJ, Wilson S, Aydon L, Leslie GD, Latour JM. Empowering parents of Australian infants and children in hospital: translation, cultural adaptation, and validation of the EMpowerment of PArents in the Intensive Care-30-AUS Questionnaire. Pediatr Crit Care Med. 2017;18:e506-13. At present, this is the only questionnaire validated and translated for use in PICUs in the Brazilian population.1818 Lessa AD; Cabral FC, Tonial CT, Costa CA, Andrades GR, Crestani F, et al. Brazilian translation, intercultural adaptation, validity and reliability of EMpowerment of PAs in Intensive Care 30 Questionnaire to Measure Parental Satisfaction in PICU. Pediatr Crit Care Med. 2021;22:e339-48. It is a self-report questionnaire consisting of 30 items divided into 5 domains: information, care and treatment, organization, parent participation, and professional attitude.1515 Latour JM, Duivenvoorden HJ, Tibboel D, Hazelzet JA. The shortened EMpowerment of PArents in THe Intensive Care 30 questionnaire adequately measured parent satisfaction in pediatric intensive care units J Clin Epidemiol. 2013;66:1045-50. Respondents were asked to rate each question on a scale from 1 to 6. For “not applicable” questions, the participants were instructed to assign a score of 0. The questionnaire can be found in the supplementary material.

Data analysis

In order to assess the predictive criterion validity of the NPS, and because there is no cutoff point to define satisfaction/dissatisfaction using the gold standard, participants with an EMPATHIC-30 score greater than or equal to the median of the test in the sample were defined as satisfied. The EMPATHIC-30 total score was calculated by dividing the sum of the scores given to each of the questions by the number of test items. Domain scores were calculated by dividing the sum of the scores given to each of the statements in that domain by the number of items that comprise the domain. The sensitivity and specificity of each NPS cutoff point were calculated based on this classification. A receiver operating characteristic (ROC) curve was used to determine the NPS cutoff point with the best performance, in terms of sensitivity and specificity, in identifying individuals satisfied with the care provided. NPS accuracy was obtained by calculating the area under the ROC curve. Positive and negative likelihood ratios were calculated for each NPS cutoff point. Stata 12.0 was used for all calculations.

Content analysis

The content analysis technique proposed by Laurence Bardin1919 Bardin, L. Análise de conteúdo. Ed.70 (Lisboa). Livraria Martins Fontes:São Paulo, 2011. was used for the treatment of NPS qualitative data. The questions and scope of each of the 5 domains of the EMPATHIC-30 were used as a guiding concept to adapt the material to the purpose of the study. The processing of qualitative data is described in the supplementary material.

Ethical approval

The study was approved by the Research Committees of all 5 participating centers. All respondents provided written informed consent before answering the questionnaires.

Results

The study included 78 PICU-admitted children (Table 1), with a median age of 3.7 (IQR, 0.9-10) years, of whom 53% were male. Children admitted via private health insurance accounted for 68% of the sample, and a large part of the patients (n = 49) were on their first admission to a PICU. Most children (n = 41) had chronic diseases, accounting for 53% of the sample. A total of 26% of patients with a PIM2 greater than 1.25% were classified as “more severe.” The median PICU length of stay was 4 days, with most patients (70%) staying in the PICU for less than 7 days.

Table 1
Characteristics of children, according to independent variables.

Regarding the characteristics of respondents (Table 2), the majority (n = 66) were mothers of the patients, with a median age of 38 years; 56% had completed higher education and more than one-third (36%) had a monthly household income greater than one thousand seven hundred and seventy dollars (average income of Brazilians is around 429 dollars). Most respondents (25%) were women who reported being stay-at-home parents; the remaining respondents reported different occupations, and only 4 refused to report their occupation. Almost all respondents (n = 71) accompanied their child daily in the PICU, staying with the child full-time (24 hours) during hospitalization (78%).

Table 2
Characteristics of respondents, according to independent variables.

The NPS assessment of parental satisfaction showed that 85% of respondents would recommend the study PICU. Only 76 respondents answered the second NPS question. The responses were grouped according to their common characteristics. Subsequently, the concepts of each of the 5 domains of the EMPATHIC-30 were used as final categories. The domains were divided into positive and negative to be in accordance with the intention of the participant's response. Content analysis showed that most participants (n = 50) reported the quality of care and treatment (n = 32) and professional attitude (n = 18) as positive factors. Organization (as a negative factor) (n = 6) was the most commonly mentioned issue when respondents reported the problems that they had encountered in the PICU (n = 10).

The mean NPS score was 9.6 (SD, 0.9), and the median NPS score was 10 (IQR, 10-10). No participant scored less than 5 or equal to 7, and the vast majority (76%) scored a 10 (maximum score) when answering the first NPS question (Table 3). The EMPATHIC-30 total score ranged from 3.2 to 6, with a mean of 5.5 (SD, 0.9) and a median of 5.7 (IQR, 5.4-5.9). The prevalence of individuals completely satisfied with PICU services, i.e., with a maximum EMPATHIC-30 score, was 13% (95% CI, 5.5%-20.5%). All EMPATHIC-30 domains had a mean of 5.0 points or more (Table 3). The median values (6 points) of the ‘information’ and ‘professional attitude’ domains indicate that half or more of the respondents were very satisfied with the service in that category.

Table 3
Measures of central tendency and dispersion of NPS and EMPATHIC-30 total scores and by domains.

Table 4 shows the sensitivity and specificity of each NPS cutoff point, compared with the EMPATHIC-30 total score dichotomized according to the median value as ≥ 5.7 points (n = 40 satisfied respondents) or < 5.7 points (n = 38 dissatisfied respondents). The sensitivity of scores 5-9 to identify respondents satisfied with the care provided in the PICU was 100% (respondents who gave these scores were able to correctly identify all those who scored ≥ 5.7 on the EMPATHIC-30). However, specificity (i.e., the ability to correctly identify those who were dissatisfied) was 0% at cutoff ≥ 5, increasing slowly up to cutoff ≥ 8, then reaching 23.7% and 47.4%, respectively, at cutoffs ≥ 9 and 10. NPS had an accuracy of 72.7% (95% CI, 61.8%-0.82%). At all cutoff points, both positive and negative likelihood ratios were very close to 1, indicating poor test performance in correctly distinguishing between satisfied and dissatisfied service users.

Table 4
NPS sensitivity and specificity compared with EMPATHIC-30 total score.

NPS had similar performance across all 5 domains, with accuracy ranging from 62% to 77%. At all cutoff points, the sensitivity of the NPS to identify satisfied individuals in each of the 5 domains of the EMPATHIC-30 ranged from 90% to 100%. However, specificity was low overall, except at cutoff 10, where it was approximately 40% in the ‘information,’ ‘care and treatment,’ ‘organization,’ and ‘parent participation’ domains, and 60.9% in the ‘professional attitude’ domain. Both positive and negative likelihood ratios were very low, indicating that NPS had a low predictive ability to correctly identify individuals satisfied and dissatisfied, respectively, with the care associated with each EMPATHIC-30 domain in this sample.

Discussion

This is the first study to test the use of NPS as a tool to assess parental satisfaction in PICUs and general pediatrics. No other study was found in the literature that evaluated the sensitivity and specificity of this instrument. In the present analysis, compared with the gold standard (EMPATHIC-30), NPS was highly sensitive at all cutoff points analyzed, but with low specificity. This finding may be due to the fact that all participants scored greater than or equal to 5 on the test, probably reflecting the small sample size. Specificity was slightly higher only at NPS cutoff 10 for the ‘professional attitude’ domain, reaching 60.9%.

Several studies99 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42.,1010 Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: a study of the Net Promoter Score using the friends and family test in England. Int J Geriatr Psychiatry. 2019;34:31-7.,1212 Hamilton DF, Lane JV, Gaston P, Patton JT, MacDonald DJ, Simpson AH, et al. Assessing treatment outcomes using a single: question the net promoter score. Bone Joint J. 2014;96-B:622-8.,2020 Koladycz R, Fernandez G, Gray K, Mariott H. The Net Promoter Score (NPS) for information on clients' experiences in sexual and reproductive health clinics. Glob Health Sci Pract. 2018;6:413-24. using NPS to assess user satisfaction with the provided health services found a satisfaction rate ranging from 60% to 96%. In the present survey, the NPS overall score was 85%, which indicates high user satisfaction with the services provided in the PICU.

Given the increasing use of NPS to assess patient satisfaction with health services and the lack of data in the literature on its validity in this field, some studies have applied it together with other tools in order to allow a comparison of the approaches. In the Netherlands, based on data from patients from 6 hospitals, the NPS was shown to be moderate to strongly correlated with 3 other constructs in the ‘promoters’ and ‘detractors’ categories.1313 Krol MW, Boer D, Delnoij DM, Rademakers JJ. The Net Promoter Score-an asset to patient experience surveys? Health Expect. 2015;18:3099-109. Likewise, Raassens and Haans2121 Raassens N, Haans H. NPS and Online WOM: Investigating the relationship between customers' promoter scores and eWOM behavior. J Serv Res. 2017;20:322-34. also found corresponding scores between promoters and online word-of-mouth (eWOM) behavior. In an outpatient clinic in California, NPS was able to correctly identify the users who were satisfied with the service.99 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42. The present findings are consistent with these results, as the NPS could sensitively detect the parents or legal guardians who were satisfied with the care provided in the PICUs.

Since its creation, the NPS has been questioned for its straightforward way of assessing customer satisfaction, especially when used in the health field. However, some authors99 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42.,1010 Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: a study of the Net Promoter Score using the friends and family test in England. Int J Geriatr Psychiatry. 2019;34:31-7.,2020 Koladycz R, Fernandez G, Gray K, Mariott H. The Net Promoter Score (NPS) for information on clients' experiences in sexual and reproductive health clinics. Glob Health Sci Pract. 2018;6:413-24. consider that this metric provides a broad but valuable view of users’ perceptions, especially when associated with the second descriptive question proposed by the tool. A survey conducted in 18 private hospitals in Brazil,2222 Fiorentino G, Sebastião B, Grass K, Lemos E. Satisfação do paciente nos hospitais privados Brasileiros. Bain & Company Inc, 2016. using the NPS, found the quality of care and staff as the main factor for customer satisfaction. In the present study, the content analysis showed that ‘care and treatment’ was the category most often cited as a positive factor for parental satisfaction, followed by ‘professional attitude.’ This is in line with the results of a study conducted in England,1010 Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: a study of the Net Promoter Score using the friends and family test in England. Int J Geriatr Psychiatry. 2019;34:31-7. which identified that the relationship between care and treatment is an important factor associated with patient satisfaction, being already consolidated as a core feature of most current therapeutic models. Regarding dissatisfaction, ‘organization’ was the category most cited by NPS respondents in the present study. Likewise, on the EMPATHIC-30, the ‘organization’ domain obtained the lowest score compared with the other domains.

In the present study's survey, the authors obtained few critical responses (13%), occasionally with the NPS score being different from the descriptive response. This also occurred in a study of 188 participants in India,2020 Koladycz R, Fernandez G, Gray K, Mariott H. The Net Promoter Score (NPS) for information on clients' experiences in sexual and reproductive health clinics. Glob Health Sci Pract. 2018;6:413-24. which suggested changing the second question of the tool to: “What could be done to improve the service?”. In doing so, the authors believe that users will feel more comfortable in providing feedback, as the question would be asked in a less personal way, thus possibly increasing the rate of responses with negative content as well as suggestions for service improvement.2020 Koladycz R, Fernandez G, Gray K, Mariott H. The Net Promoter Score (NPS) for information on clients' experiences in sexual and reproductive health clinics. Glob Health Sci Pract. 2018;6:413-24.

Initially, some studies questioned whether the NPS question “would you recommend...” would be suitable for the health field.1212 Hamilton DF, Lane JV, Gaston P, Patton JT, MacDonald DJ, Simpson AH, et al. Assessing treatment outcomes using a single: question the net promoter score. Bone Joint J. 2014;96-B:622-8. In the authors’ experience of administering the test, the authors noticed that there were no uncertainties or lack of understanding in relation to answering the question, as the participants always answered it easily. This was also observed in previous surveys conducted in the United States and Europe.99 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42.,1010 Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: a study of the Net Promoter Score using the friends and family test in England. Int J Geriatr Psychiatry. 2019;34:31-7. In addition, when applied to functional illiterate populations, no problems were found in relation to the applicability and acceptability of the tool.2020 Koladycz R, Fernandez G, Gray K, Mariott H. The Net Promoter Score (NPS) for information on clients' experiences in sexual and reproductive health clinics. Glob Health Sci Pract. 2018;6:413-24. These authors have even reported that this was the most answered question in their satisfaction survey, allowing it to reach a larger number of respondents.

For the most part (62% of the sample), data were collected in private hospitals, which already use the NPS as a form of an overall assessment of the institution. The hospitals often send the survey by email days after hospital discharge, which results in a low response rate, especially in pediatric units. This also hinders an accurate assessment of parental satisfaction with care. These data are consistent with those of a study conducted in the United States,1212 Hamilton DF, Lane JV, Gaston P, Patton JT, MacDonald DJ, Simpson AH, et al. Assessing treatment outcomes using a single: question the net promoter score. Bone Joint J. 2014;96-B:622-8. which obtained a response rate of 100% on the NPS when applied at the time of patient discharge, against a response rate of 27% when applied via email or phone call. Therefore, it is important to collect the data from patients while in the hospital, before they are discharged home.

In the present study, most respondents (n = 44) had completed higher education and had a monthly household income greater than or equal to 9 minimum monthly salaries (45%). These factors are associated with a high level of requirement due to high expectations of the service provided.2323 Suda EY, Uemura MD, Velasco E. Assessment of patient satisfaction with physical therapy at a college health clinic in Santo André, SP. Fisioter Pesq. 2009;16:126-31. In the present study, the 2 lowest NPS scores (5 and 6), as well as the 3 lowest EMPATHIC-30 scores (3.8, 3.2, and 3.9), were given by respondents within this profile. However, in general, the parents were satisfied with the care provided in the PICUs. This may be related to the fact that most patients in the present sample were considered less severe and did not require intubation (n = 62). The fragile health status of a patient or family member is associated with a lower level of satisfaction and willingness to recommend services.1010 Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: a study of the Net Promoter Score using the friends and family test in England. Int J Geriatr Psychiatry. 2019;34:31-7.

According to a survey linked to the Brazilian National Association of Private Hospitals,2222 Fiorentino G, Sebastião B, Grass K, Lemos E. Satisfação do paciente nos hospitais privados Brasileiros. Bain & Company Inc, 2016. customers classified as promoters are willing to spend more time going to the hospital of their choice, with a return rate 3 times higher than that of detractors. The survey results also highlighted the importance of recommendations from satisfied users, as 60% of patients reported that the primary reason for choosing the hospital was the recommendation from a physician, family member, or friend.

This study has some limitations, such as the small sample size, which resulted in a partial assessment of NPS performance. Also, this study is subject to a type of information bias, known as courtesy bias. In an attempt to minimize this bias, the interviews were conducted outside the PICU premises. Since the authors didn't interview parents whose children died in the PICU, the authors can consider it as a selection bias.

Compared with EMPATHIC-30, NPS showed high sensitivity to correctly detect users satisfied with PICU care, but with low specificity in identifying dissatisfied users in this analysis. However, both tests obtained similar results regarding the main factors associated with parental satisfaction and dissatisfaction. The authors consider NPS a promising test because it is sensitive in assessing parental satisfaction in the PICU. Nevertheless, the authors believe that studies with larger sample sizes are needed to further assess the value of using NPS in PICUs.

  • Funding
    This study was funded in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Código de Financiamento 001.

Supplementary materials

Supplementary material associated with this article can be found in the online version at doi: 10.1016/j.jped.2022.11.013.

References

  • 1
    Latour JM, Hazelzet JA, van der Heijden AJ. Parent satisfaction in pediatric intensive care: a critical appraisal of the literature. Pediatr Crit Care Med. 2005;6:578-84.
  • 2
    Donabedian A. Criteria and standards for quality assessment and monitoring. QRB Qual Rev Bull. 1986;12:99-108.
  • 3
    Latour JM, Hazelzet JA, Duivenvoorden HJ, Goudoever JB. Construction of a parent satisfaction instrument: perceptions of pediatric intensive care nurses and physicians. J Crit Care. 2009;24:255-66.
  • 4
    Parker MM, Thompson AE, Lumb P, Lacroix J, St Andre A. Co-evolutions of pediatric and adult critical care. Crit Care Med. 2021;49:188-200.
  • 5
    Frankel LR, Hsu BS, Yeh TS, Simone S, Agus MS, Arca MJ, et al: Criteria for critical care infants and children: PICU admission, discharge, and triage practice statement and levels of care guidance. Pediatr Crit Care Med. 2019; 20:847-87.
  • 6
    Johnson B, Abraham M, Conway J, Simmons L, Edgman-Levitan S, Sodomka P, et al. Partnering with patients and families to design a patient-and family-centered health care system. Institute for Patient-and Family-Centered Care and Institute for Healthcare Improvement; 2008.
  • 7
    Schuster MA. Measuring quality of pediatric care: where we've been and where we're going. Pediatrics. 2015;135:748-51.
  • 8
    Reichheld FF. The one number you need to grow. Harv Bus Rev. 2003;81:46-54.
  • 9
    Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, et al. The Use of the Net Promoter Score (NPS) in an allergy clinic and lung clinic: an innovative study on the use of tablet-based tools versus traditional research method. Eval Patient-Relat Results. 2020;11:137-42.
  • 10
    Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: a study of the Net Promoter Score using the friends and family test in England. Int J Geriatr Psychiatry. 2019;34:31-7.
  • 11
    Reichheld F. A pergunta definitiva 2.0: Como as empresas que implementam o net promoter score prosperam em um mundo voltado aos clientes. Rio de Janeiro: Elsevier, 2011.
  • 12
    Hamilton DF, Lane JV, Gaston P, Patton JT, MacDonald DJ, Simpson AH, et al. Assessing treatment outcomes using a single: question the net promoter score. Bone Joint J. 2014;96-B:622-8.
  • 13
    Krol MW, Boer D, Delnoij DM, Rademakers JJ. The Net Promoter Score-an asset to patient experience surveys? Health Expect. 2015;18:3099-109.
  • 14
    Slater A, Shann F, Pearson G; Pediatric Index of Mortality (PIM) study group: PIM2: a revised version of the Pediatric Index of Mortality. Intensive Care Med 2003;29:278-85.
  • 15
    Latour JM, Duivenvoorden HJ, Tibboel D, Hazelzet JA. The shortened EMpowerment of PArents in THe Intensive Care 30 questionnaire adequately measured parent satisfaction in pediatric intensive care units J Clin Epidemiol. 2013;66:1045-50.
  • 16
    Zhuo-wen, Zhang, Latour, Bo, Guo-ping, Ying, et al. The reliability and validity of the Chinese version of the EMPATHIC-30 questionnaire measuring parental satisfaction in the pediatric intensive care unit. Chin J Nurs, 2015.
  • 17
    Gill FJ, Wilson S, Aydon L, Leslie GD, Latour JM. Empowering parents of Australian infants and children in hospital: translation, cultural adaptation, and validation of the EMpowerment of PArents in the Intensive Care-30-AUS Questionnaire. Pediatr Crit Care Med. 2017;18:e506-13.
  • 18
    Lessa AD; Cabral FC, Tonial CT, Costa CA, Andrades GR, Crestani F, et al. Brazilian translation, intercultural adaptation, validity and reliability of EMpowerment of PAs in Intensive Care 30 Questionnaire to Measure Parental Satisfaction in PICU. Pediatr Crit Care Med. 2021;22:e339-48.
  • 19
    Bardin, L. Análise de conteúdo. Ed.70 (Lisboa). Livraria Martins Fontes:São Paulo, 2011.
  • 20
    Koladycz R, Fernandez G, Gray K, Mariott H. The Net Promoter Score (NPS) for information on clients' experiences in sexual and reproductive health clinics. Glob Health Sci Pract. 2018;6:413-24.
  • 21
    Raassens N, Haans H. NPS and Online WOM: Investigating the relationship between customers' promoter scores and eWOM behavior. J Serv Res. 2017;20:322-34.
  • 22
    Fiorentino G, Sebastião B, Grass K, Lemos E. Satisfação do paciente nos hospitais privados Brasileiros. Bain & Company Inc, 2016.
  • 23
    Suda EY, Uemura MD, Velasco E. Assessment of patient satisfaction with physical therapy at a college health clinic in Santo André, SP. Fisioter Pesq. 2009;16:126-31.

Publication Dates

  • Publication in this collection
    16 June 2023
  • Date of issue
    May-Jun 2023

History

  • Received
    30 June 2022
  • Accepted
    24 Nov 2022
Sociedade Brasileira de Pediatria Av. Carlos Gomes, 328 cj. 304, 90480-000 Porto Alegre RS Brazil, Tel.: +55 51 3328-9520 - Porto Alegre - RS - Brazil
E-mail: jped@jped.com.br