Abstract
The objective of this study is to present patient satisfaction results using the Net Promoter Score (NPS). This is a cross-sectional study carried out with microdata from the Continuous National Household Sample Survey of 2022 and which corresponds to the module on Primary Health Care in which the sample carried out was 48,068 guardians of children under 13 years of age in all federation units. The main variables considered were the score given to the health service in the last service in the 12-month period and the main reason for this score. It should be noted that no region of the country achieved a result defined as a Quality Zone (between 51 and 75), concentrating on a category called Improvement Zone (between 1 and 50). The best performance was in the South region (+33) and the worst performances in the Northeast and Southeast regions (+27). The NPS can be an important ally to quickly assess the patient experience in primary health care services in Brazil.
Key words: Primary Health Care; Service Evaluation; Health Services
Resumo
O objetivo desse estudo é apresentar os resultados da satisfação dos pacientes a partir do uso do Net Promoter Score (NPS). Trata-se de um estudo transversal realizado com microdados da Pesquisa Nacional por Amostra de Domicílios Contínua de 2022 e que corresponde ao módulo sobre Atenção Primária à Saúde em que a amostra realizada foi de 48.068 responsáveis por crianças menores de 13 anos em todas as unidades da federação. As principais variáveis consideradas foram a nota atribuída ao serviço de saúde no último atendimento no período de 12 meses e o principal motivo para essa pontuação. Observou-se que nenhuma região do país alcançou um resultado definido como Zona de Qualidade (entre 51 e 75), se concentrando numa categoria denominada Zona de Aperfeiçoamento (entre 1 e 50). O melhor desempenho foi na região Sul (+33) e os piores desempenhos nas regiões Nordeste e Sudeste (+27). O NPS pode ser um importante aliado para avaliar de maneira rápida a experiência do paciente nos serviços de atenção primária à saúde no Brasil.
Palavras-chave: Atenção Primária à Saúde; Avaliação de Serviços; Serviços de Saúde
Resumen
El objetivo de este estudio es presentar los resultados de satisfacción del paciente utilizando el Net Promoter Score (NPS). Se trata de un estudio transversal realizado con microdatos de la Encuesta Nacional Continua por Muestra de Hogares de 2022 y que corresponde al módulo de Atención Primaria de Salud en el que la muestra realizada fue de 48.068 tutores de niños menores de 13 años en todas las unidades de la federación. Las principales variables consideradas fueron el puntaje otorgado al servicio de salud en el último servicio en el período de 12 meses y el principal motivo de este puntaje. Cabe señalar que ninguna región del país logró un resultado definido como Zona de Calidad (entre 51 y 75), concentrándose en una categoría denominada Zona de Mejora (entre 1 y 50). El mejor desempeño se registró en la región Sur (+33) y los peores en las regiones Nordeste y Sudeste (+27). El NPS puede ser un aliado importante para evaluar rápidamente la experiencia del paciente en los servicios de atención primaria de salud en Brasil.
Palabras clave: Atención Primaria en Salud; Evaluación de Servicios; Servicios de Salud
Introduction
The evaluation of Primary Health Care (PHC) services is crucial to measure the efficiency and quality of care provided worldwide. It involves a comprehensive analysis of several aspects, such as access, longitudinality, coordination, and comprehensive services. Countries worldwide implement different approaches and metrics to evaluate their PHC to improve healthcare delivery, identify gaps, and promote equitable access. Evaluation can also assist in policymaking and directing resources to improvement areas. Finding reliable and cost-effective evaluation strategies is an established challenge for PHC.
The Net Promoter Score (NPS) is a summary question whose first publication dates back to 2003, originated by Reichheld (2003), from the Harvard Business Review1. Companies have used the NPS tool to identify loyal customers and evaluate the growth of their service projects2. Due to its success, hundreds of companies in the business sector have started using the NPS tool3,4. Similarly, healthcare and other social sectors have also started incorporating the NPS tool in their clinics and hospitals to assess patient satisfaction with services provided, patient loyalty, and growth2. NPS is a popular survey method used globally and dubbed “The Ultimate Question”.
The instrument has been used to assess patient satisfaction with health services in England, the Netherlands, the United States, and Australia4-9. However, specialist services are the health services most frequently assessed with the NPS. In a recent systematic review, Adams et al.10 observed the use of NPS in hospitals (5 studies), dental services (3 studies), PHC clinics (2 studies), orthopedic centers (2 studies), family planning centers (1 study), and mental health services (1 study). Only 12 studies were included in the research due to the inclusion criteria.
The NPS has been widely used by the National Health System in England since 2012. It was inserted and adapted within the Family Friends Test (FFT) program. It has been applied 25 million times since 2013 and is the world’s most extensive collection of patient experience data. In a summary report published in February 2020, 90% of the patients in general practice services were promoters of this service, while 96% were promoters of dental services11-13.
In Brazil, the NPS was included in the Continuous National Household Sample Survey (PNAD-C), developed by the Brazilian Institute of Geography and Statistics (IBGE) in 202214. The PNAD-C carried out a household survey in more than 200 thousand households, becoming the most significant child health assessment survey ever carried out in Brazil15. This study aims to present patient satisfaction results using the Net Promoter Score (NPS).
Methodological aspects
This cross-sectional study was conducted with microdata from the IBGE’s National Household Sample Survey, which continues in the second quarter of 2022. We used the Primary Child Health Care module, in which the sample consisted of 48,068 people responsible for children under 13 years of age in all 27 federation units.
Implemented in Brazil in 2006, the Integrated Household Survey System (SIPD) aimed to reformulate the IBGE household surveys, providing indicators on the labor market with national coverage and producing baseline information for studying the country’s socioeconomic development. The SIPD is a model for producing household sample surveys in which the planning, execution, analysis, and dissemination of results are coordinated, facilitating meeting new demands and streamlining the resources used. The sample of each survey in this system corresponds to a part or the entirety of a master sample, and the concepts and processes are harmonized between them.
One of the fundamental points of the SIPD is the construction of a sampling structure to fit all household surveys. This structure is the master sample, defined as a set of areal units probabilistically selected from a master registry based on the 2010 Demographic Census, the changes in the Geographic Operational Base, and the National Address Registry for Statistical Purposes (CNEFE).
Subsamples are constructed from the master sample for the several surveys included in the SIPD. The Household Budget Survey, for example, uses a subsample of approximately 40% of the primary sampling units of the master sample. In comparison, the Continuous PNAD uses 100% of the primary sampling units of this sample.
The geographic scope of the Continuous PNAD is the entire National Territory, divided into census tracts of the 2010 Geographic Operational Base, excluding areas with unique features. The target population consists of all people living in permanent private households in the area covered by the survey.
The sampling plan adopted in the Continuous PNAD is a two-stage cluster selection, with stratification of the primary sampling units. In the first stage, the primary sampling units are selected with a probability proportional to the number of households within each defined stratum. The stratification adopted is that defined for the entire SIPD.
In the second stage, 14 occupied permanent private households are selected within each primary sampling unit by simple random sampling of the updated CNEFE. The sample of primary sampling units and households is divided by the three months of a quarter, following the rotation scheme described below.
The Continuous PNAD is collected quarterly, i.e., the total sample of households is collected over three months to produce estimates of the desired indicators at the end of this cycle. In these situations, the sample is planned in such a way as to rotate the selected households, maintaining an overlapping portion between two subsequent reporting periods. In the case of the Continuous PNAD, the sample rotation scheme adopted was the 1-2(5) scheme, which is the most efficient when one of the main interests of the research is changes in quarterly indicators. In this scheme, the household is interviewed for one month and then leaves the sample for two consecutive months, with this sequence repeated five times. It was necessary to define 15 household rotation groups, dividing the sample of primary sampling units into these groups to operationalize this scheme. Five groups are surveyed each month of the quarter, and at the end of the period, the sample is accumulated to produce the indicators.
The Continuous PNAD visits 15,096 primary sampling units spread throughout the National Territory every quarter. Fourteen households are visited in each, totaling 211,344 households per quarter14.
All Module 12 respondents from the Continuous PNAD (Second quarter of 2022) who answered the NPS were considered for the analysis. IBGE’s methodology provides that only those who responded to having consulted a PHC Unit up to 12 months before the interview were invited to answer the NPS question. In other words, 40,106 people of the original total of 48,068 who started responding to the PHC Module (which corresponds to 38 million people in the expanded population) responded that the child received some type of care in the last 12 months (that is, 31.5 million in the universe).
The main variables considered were the score given to the health service in the last appointment in the 12-month period (which generated the NPS indicator) and the main reason for this score.
The Net Promoter Score is a tool that has been used to assess patient experience on an international scale. Using just one question, “On a scale of 0 to 10, how likely are you to recommend a particular service (name of service) to a friend or family member?”, the NPS compares the balance between those who are likely to recommend the services received or not, obtaining a solid indication of the patient’s perception of the service by analyzing whether or not they would encourage other people to use it2,3. Patients responding to the NPS are divided into three categories: promoters (with scores of 9 and 10), neutral/passive (with scores of 7 and 8), and detractors (with scores from 0 to 6). Promoters are patients who are enthusiastic about the service, while detractors represent those who are dissatisfied and evaluate it negatively. Passive/neutral respondents, as the name suggests, are not necessarily dissatisfied nor inclined to actively promote the service provided to them. The final NPS calculation is based on the percentage of participants considered promoters, subtracted by detractors, disregarding the neutral/passive respondents. Thus, the result can range from “-100” to “+100”, where the higher the value, the better the indicator2,3.
To better understand the outcomes, the result was categorized into Excellence Zone, between 76 and 90; Quality Zone, between 51 and 75; Improvement Zone, between 1 and 50; and Critical Zone, between -100 and 016.
The analyses followed three main stages. First, the NPS calculation is presented by state. Next, the main reason for assigning the score for the last service was categorized, excluding the “Other” option. To this end, we opted to present interval estimates with 95% confidence and a 5% level of statistical significance. Finally, the continuous score was categorized as low score (0-6), the “detractors”; a neutral score, the “unaffected” (7 and 8); and high score (9 and 10), the “promoters”, to perform correlation analysis with the Primary Care Assessment Tool (PCATool) score, which the instrument also allowed calculating. The calculation of the PCATool score followed the method published by the Ministry of Health17. All analyses were performed in the statistical program Stata 15.1.
Results
In the 2022 Continuous PNAD, 89.1% of 38 million children under 13 eligible to respond to the PHC module had a father/mother as the head of the family, with a similar distribution among the regions of the country. Furthermore, 31.5 million children under 13 received health care at a PHC unit 12 months before the interview, assigning a score from zero to 10 to this care, which, as we saw previously, allows calculating the Net Promoter Score (NPS) indicator.
This is the most significant Brazilian sample of respondents for this indicator. We observed that no region of the country achieved a result defined as Quality Zone (between 51 and 75), concentrating on a category called Improvement Zone (between 1 and 50). The South achieved the best performance (+33), and the worst was recorded in the Northeast and Southeast (+27 each). Among the Brazilian states, Rio Grande do Sul stands out (+41), with a performance closer to the Quality Zone. Rondônia, Acre, Amapá, Tocantins, Ceará, Rio Grande do Norte, Pernambuco, Alagoas, Bahia, Espírito Santo, São Paulo, Paraná and the Federal District performed below Brazil’s performance (+28) (Graph 1).
Net Promoter Score (NPS) assigned by guardians of children under 13 years of age who used some Primary Health Care service in the last 12 months. Brazil, Great Regions, and Federation Units, 2022.
The main reason given by respondents to justify the NPS score was the performance of the health unit professionals (team) in resolving the problem (35.7% [34.7%-36.7%]), followed by reception - how the child’s guardian was served at the unit (32.4% [31.5%-33.3%]). On the other hand, we observed regional differences. In the North and Northeast, reception stood out against the other reasons. Teamwork was mentioned more often in the South and Southeast (Table 1).
When associating the proportion of people who assigned scores of 9 or 10 (“the health unit promoters”) with the PCATool score (which can also be calculated in the Children’s PHC module of the 2022 Continuous PNAD), we observed a correlation between the highest scores given in the evaluation of the PHC unit and the highest scores obtained when applying the PCATool methodology (Graphs 2 to 5).
Net Promoter Score (NPS) and the association between the score (0 to 10) attributed to the PHC service and the overall PCATool score. Brazil, Great Regions, Federation Units, 2nd quarter/2022. All scores (in ascending order of the “NPS” indicator).
Net Promoter Score (NPS) and the association between the score (0 to 10) attributed to the PHC service and the overall PCATool score. Brazil, Great Regions, Federation Units, 2nd quarter/2022. Score 9 or 10 - “promoters” (by ascending order of the “NPS”) indicator.
Net Promoter Score (NPS) and the association between the score (0 to 10) attributed to the PHC service and the overall PCATool score. Brazil, Great Regions, Federation Units, 2nd quarter/2022. Score 0 to 7 - “detractors” (by ascending order of the “NPS” indicator).
Net Promoter Score (NPS) and the association between the score (0 to 10) attributed to the PHC service and the overall PCATool score. Brazil, Great Regions, Federation Units, 2nd quarter/2022. Score 7 or 8 - “unaffected” (by ascending order of the “NPS” indicator).
Discussion
We observed that no region of the country achieved an NPS defined as a Quality Zone (between 51 and 75), concentrating on a category called Improvement Zone (between 1 and 50). The best performance was in the South (+33), and the worst was in the Northeast and Southeast (+27 each). Among the Brazilian states, Rio Grande do Sul stands out (+41), with a performance closer to the Quality Zone. The main reasons stated by the children’s guardians to give their score (of 0 to 10) were “the health unit’s professional (team) performance in resolving the health problem” and “reception”, which means that health units must pay attention to these characteristics when planning and organizing their primary care services.
We found few studies applying the NPS summary question within the PHC. Although it is employed internationally to measure patient satisfaction in hospitals, dental clinics, and specialized centers, its use in PHC appears to be more widespread in the UK’s National Health System (NHS)10. The NHS reformulated the NPS and called the modified version the Friends and Family Test (FFT), transforming the traditional NPS numerical response scale into a 5-point Likert scale10. The NHS periodically used the FFT, specifically for PHC, and the results for September showed a 92% positive evaluation18.
Studies using the NPS or FFT within the PHC had mixed methodology, focusing on qualitative assessment. The NPS could be used with a broad profile of patients, including less educated adults and children. Moreover, the NPS research showed high patient completion rates, generating a large data volume. On the other hand, some questioned whether the NPS recommendation summary question was appropriate in the healthcare setting, especially if patients have a limited choice of health professionals (access difficulty)5,10,19,20.
Adams et al.10 found that the patient feedback section was the most helpful NPS component. The feedback section in the NPS is well-used by patients, and these comments can help contextualize the quantitative results collected from the NPS survey. For this reason, following a review by the FFT (in 2014), the open-ended question was made mandatory due to the perceived value of these comments. The results of these studies were not found in the comments section. In Brazil, reception and resolution were the most frequent justifications for assigning a response to the NPS question. However, the service speed was pointed out when we looked specifically at the reason for assigning a low score.
Reception is an access practice that aims to restore the relationship of professional-patient solidarity and agility20 to increase equity, the resolution of the first contact, and care speed. Objectively, implementing this tool is represented by the distribution of forms to organize the queue, organizing the ‘walk-in demand’ in the agendas, and the conversation between the team and patients, followed by individual negotiations and clinical care21.
However, despite the significant increase in people registered/linked to teams working in Brazilian PHC services, 74,685.77 people registered in 2018 to 170,816.484 in 202222, and an apparent increase in equity in access to and use of PHC health services for rural areas, the Brazilian northern and northeastern regions, non-whites, schooling, hypertensive patients and diabetic patients, we identified persistent access barriers that characterize this attribute as one of the main challenges for PHC22. Historically, access is the PHC attribute with the worst evaluation in studies using the PCA-Tool23. Thus, we can speculate that the difficulty in assuring access to professional appointments remains one of the main reasons patients attribute a quality evaluation, in the results of this research, linked to the category reception and service speed.
Another reason attributed by patients to performance in the NPS is resolution. The low-resolution rate of PHC in Brazil has been extensively reported in the literature. High rates of referrals to specialties24, erroneous referrals of PHC patients to specialties25, and the slow reduction in hospitalizations for PHC-sensitive conditions, even before the COVID-19 pandemic26, reinforce this perception.
This is the first news, at least to the authors’ knowledge, of a study that associates the highest PCATool scores with the proportion of health unit promoters under the NPS (that is, those who give the highest scores in their evaluations).
Final considerations
Adopting the Net Promoter Score (NPS) to assess the quality of Brazilian PHC services is a significant innovation by IBGE in the 2022 Continuous PNAD, representing the largest household survey on child health assessment ever conducted in Brazil25. Evidence suggests that the NPS can be used as a starting point to understand better the patient’s experience, which can help identify areas that require further investigation and detailed examination6,11,18,19. The IBGE continued to use it in its National Demographic and Health Survey (PNDS), which went to the field in 2023 with a module on access to and use of health services for men and women of childbearing age.
This is the first report, at least to the authors’ knowledge, of a study that associates the highest PCATool scores with the proportion of health unit promoters according to the NPS (that is, those who give the highest scores in their evaluations).
Initially developed to evaluate brands and companies, NPS is now also finding its place in evaluating healthcare services. Previously accused of not being applicable outside of commercial settings, it is now beginning to prove its worth as a simple and valuable way of determining and understanding patient satisfaction. Its simplicity allows its use in settings with few resources or less-educated populations. However, there are still doubts as to whether the Net Promoter Score can be used as the sole method for evaluating services provided in PHC, requiring greater use in studies, comparisons with internationally used PHC scores, and literature dedicated to interpreting its results and comparing/correlating them with other evaluation methods.
References
- 1 Reichheld FF. The one number you need to grow. Harv Bus Rev 2003; 81(12):46-55.
- 2 Alismail A, Schaeffer B, Oh A, Hamiduzzaman S, Daher N, Song HY, Furukawa B, Tan LD. The use of the Net Promoter Score (NPS) in an outpatient allergy and pulmonary clinic: an innovative look into using tablet-based tool vs traditional survey method. Patient Relat Outcome Meas 2020; 11:137-142.
- 3 Koladycz R, Fernandez G, Gray K, Marriott H. The Net Promoter Score (NPS) for Insight Into Client Experiences in Sexual and Reproductive Health Clinics. Glob Health Sci Pract 2018; 6(3):413-424.
-
4 Company B. Companies that use net promoter [Internet]. 2018 [cited 2023 jan 13]. Available from: http://netpromotersystem.com/about/companies-using-nps.aspx
» http://netpromotersystem.com/about/companies-using-nps.aspx - 5 Manacorda T, Erens B, Black N, Mays N. The Friends and Family Test in general practice in England: a qualitative study of the views of staff and patients. Br J Gen Pract 2017; 67(658):e370-e376.
- 6 Wilberforce M, Poll S, Langham H, Worden A, Challis D. Measuring the patient experience in community mental health services for older people: A study of the Net Promoter Score using the Friends and Family Test in England. Int J Geriatr Psychiatry 2019; 34(1):31-37.
- 7 Lawton R, O'Hara JK, Sheard L, Reynolds C, Cocks K, Armitage G, Wright J. Can staff and patient perspectives on hospital safety predict harm-free care? An analysis of staff and patient survey data and routinely collected outcomes. BMJ Qual Saf 2015; 24(6):369-376.
- 8 Leggat SG. Understanding the perspectives of health service staff on the Friends and Family Test. Aust Health Rev 2016; 40(3):299-305.
- 9 Bartram TA. A path analysis study of factors influencing hospital staff perceptions of quality of care factors associated with patient satisfaction and patient experience. BMC Health Serv Res 2017; 17(1):739.
- 10 Adams C, Walpola R, Schembri AM, Harrison R. The ultimate question? Evaluating the use of Net Promoter Score in healthcare: a systematic review. Health Expect 2022; 25(5):2328-2339.
-
11 National Health Service England (NHS). Friends and Family Test data - February 2020 [Internet]. [cited 2023 jan 13]. Available from: https://www.england.nhs.uk/publication/friends-and-family-test-data-february-2020/.
» https://www.england.nhs.uk/publication/friends-and-family-test-data-february-2020 -
12 National Health Service England (NHS). NHS Improvement guidance. Using the Friends and Family Test to improve patient experience [Internet]. 2019 [cited 2023 jan 13]. Available from: https://www.england.nhs.uk/wpcontent/uploads/2019/09/using-the-fft-to-improve-patient-experienceguidance-v2.pdf
» https://www.england.nhs.uk/wpcontent/uploads/2019/09/using-the-fft-to-improve-patient-experienceguidance-v2.pdf -
13 National Health Service England (NHS). NHS England Review of the Friends and Family Test [Internet]. 2014 [cited 2023 jan 13]. Available from: https://www.england.nhs.uk/wpcontent/uploads/2014/07/fft-rev1.pdf
» https://www.england.nhs.uk/wpcontent/uploads/2014/07/fft-rev1.pdf -
14 Instituto Brasileiro de Geografia e Estatística (IBGE). Pesquisa Nacional de Amostra Domiciliar 2022. Instrumento de Coleta de dados [Internet]. Disponível em: https://biblioteca.ibge.gov.br/visualizacao/instrumentos_de_coleta/doc5629.pdf
» https://biblioteca.ibge.gov.br/visualizacao/instrumentos_de_coleta/doc5629.pdf - 15 Pinto LF, Caldas ALFR. Atenção primária à saúde infantil: a maior avaliação de base populacional da história do SUS. Cien Saude Colet 2022; 27(8):3153-3156.
- 16 Reichheld FF. The one number you need to grow. Harv Bus Rev 2003; 81(12):46-54.
- 17 Brasil. Ministério da Saúde (MS). Secretaria de Atenção Primária à Saúde. Departamento de Saúde da Família. Manual do Instrumento de Avaliação de Atenção Primária à Saúde - PCATool - 2020. Brasília: MS; 2020.
-
18 National Health Service England (NHS) [Internet]. 2023 [cited 2023 dez 8]. Disponível em: https://www.egland.nhs.uk/.
» https://www.egland.nhs.uk - 19 Medforth N, Rooksby K. Enabling young service users to provide feedback on their experience: an evaluation of the pilot implementation of children and young people accessible Friends and Family Test in general and dental practices in NHS England South (South Central). Comp Child Adolesc Nurs 2017; 41(1):42-57.
- 20 Sizmur S, Graham C, Walsh J. Influence of patients' age and sex and the mode of administration on results from the NHS Friends and Family Test of patient experience. J Health Serv Res Policy 2015; 20(1):5-10.
- 21 Tesser CD, Norman AH, Vidal TB. Acesso ao cuidado na Atenção Primária à Saúde brasileira: situação, problemas e estratégias de superação. Saude Debate 2018; 42(n. esp. 1):361-378.
-
22 Brasil. Ministério da Saúde (MS). Secretaria de Atenção Primária à Saúde (SAPS) [Internet]. 2023 [acessado 2023 nov 30]. Disponível em: https://egestorab.saude.gov.br/.
» https://egestorab.saude.gov.br - 23 D'Avila OP, Chisini LA, Costa FS, Cademartori MG, Cleff LB, Castilhos ED. Use of Health Services and Family Health Strategy Households Population Coverage in Brazil. Cien Saude Colet 2021; 26(9):3955-3964.
- 24 Souza KOC, Ribeiro CJN, Santos JYS, Araújo DC, Peixoto MVS, Fracolli LA, Santos AD. Acesso, abrangência e resolutividade da atenção básica à saúde no nordeste brasileiro. Acta Paul Enferm 2022; 35:eAPE01076.
- 25 Katz N, Roman R, Rados DV, Oliveira EB, Schmitz CAA, Gonçalves MR, Mengue SS, Umpierre RN. Acesso e regulação ao cuidado especializado no Rio Grande do Sul: a estratégia RegulaSUS do TelessaúdeRS-UFRGS. Cien Saude Colet 2020; 25(4):1389-1400.
- 26 Santos FM, Macieira C, Machado ATG M, Borde EMS, Santos AF. Admissions due to ambulatory care-sensitive conditions (ACSC): an analysis based on socio-demographic characteristics, Brazil and regions, 2010 to 2019. Rev Bras Epidemiol 2022; 25:e220012.
Publication Dates
-
Publication in this collection
21 Oct 2024 -
Date of issue
Nov 2024
History
-
Received
31 Jan 2024 -
Accepted
09 Apr 2024 -
Published
11 Apr 2024