Evaluation of health service providers by consumers through the Analytic Hierarchy Process Method

Abstracts

OBJECTIVE: To assess the quality of services offered by health service providers, according to the perception of consumers. METHODS: A cross-sectional study with 360 consumers from seven health service providers in the metropolitan area of Curitiba, PR, in 2008, was performed. An individual questionnaire measuring the consumers' preferences in relation to six attributes (location of service points; effectiveness of doctors, clinics and hospitals; promptness and kindness when caring for patients and family members; ease of access to the authorization forms for consultations; price; diversity of available doctors, clinics and hospitals) for each one of the analyzed health service providers was carried out. The Analytic Hierarchy Process (AHP), a multiple criteria tool for decision analysis and planning, was used to analyze the responses. RESULTS: The attribute most valued by the consumers was "price". The companies were grouped into two sets, regarding the mentioned attributes: two had lower preference (between 19% and 23%) and five, higher preference (around 10% each). CONCLUSIONS: With this type of research, health service providers could reshape their structures, processes, prices and accredited networks, in order to improve their market strategy.

Health Maintenance Organizations; Patient Satisfaction; Health Services Evaluation; Health Care Quality, Access, and Evaluation

OBJETIVO: Analisar a qualidade dos serviços oferecidos por empresas operadoras de planos de saúde, segundo a percepção de usuários. MÉTODOS: Estudo transversal com 360 usuários de sete operadoras de planos de saúde da cidade de Curitiba, PR, e região metropolitana em 2008. Foi aplicado questionário sobre as preferências dos usuários em relação a seis atributos (localização dos pontos de atendimento; efetividade da ação dos médicos, clínicas e hospitais; rapidez e amabilidade no atendimento; facilidade na liberação de guias; preço; abrangência da rede credenciada) de cada uma das empresas operadoras. Para a análise das respostas foi utilizado o método Analytic Hierarchy Process (AHP, ou Processo Analítico de Hierarquia), ferramenta de análise de decisão e planejamento de múltiplos critérios. RESULTADOS: O atributo mais valorizado pelos usuários foi "preço". As empresas foram agrupadas em dois conjuntos de preferências em relação aos atributos: dos sete planos de saúde, dois apresentaram menor preferência (entre 23% e 19%) e cinco, maior preferência (em torno de 10%). CONCLUSÕES: Com esse tipo de pesquisa, as empresas operadoras de planos de saúde poderiam reformular suas estruturas, processos, preços e redes credenciadas com o objetivo de melhorar seu posicionamento no mercado.

Sistemas Pré-Pagos de Saúde; Satisfação do Paciente; Avaliação de Serviços de Saúde; Qualidade, Acesso e Avaliação da Assistência à Saúde

ORIGINAL ARTICLES

Evaluation of health service providers by consumers through the Analytic Hierarchy Process Method

Evaluación de operadoras de salud por usuarios por el método Analytic Hierarchy Process (AHP)

Dewey Wollmann^I; Maria Teresinha Arns Steiner^{I, II}; Guilherme Ernani Vieira^I; Patrícia Arns Steiner^III

^IPrograma de Pós-Graduação em Engenharia de Produção e Sistemas. Escola Politécnica. Pontifícia Universidade Católica do Paraná. Curitiba, PR, Brasil

^IIPrograma de Pós-Graduação em Métodos Numéricos em Engenharia e Programa de Pós-Graduação em Engenharia de Produção. Setor de Tecnologia. Universidade Federal do Paraná. Curitiba, PR, Brasil

^IIIPrograma de Pós-Graduação em Engenharia de Recursos Hídricos e Ambiental. Setor de Tecnologia. Universidade Federal do Paraná. Curitiba, PR, Brasil

^{Correspondence}

ABSTRACT

OBJECTIVE: To assess the quality of services offered by health service providers, according to the perception of consumers.

METHODS: A cross-sectional study with 360 consumers from seven health service providers in the metropolitan area of Curitiba, Southern Brazil, in 2008, was performed. An individual questionnaire measuring the consumers' preferences in relation to six attributes (location of service points; effectiveness of doctors, clinics and hospitals; promptness and kindness when caring for patients and family members; ease of access to the authorization forms for consultations; price; diversity of available doctors, clinics and hospitals) for each one of the analyzed health service providers was carried out. The Analytic Hierarchy Process (AHP), a multiple criteria tool for decision analysis and planning, was used to analyze the responses.

RESULTS: The attribute most valued by the consumers was "price". The companies were grouped into two sets, regarding the mentioned attributes: two had lower preference (between 19% and 23% ) and five, higher preference (around 10% each).

CONCLUSIONS: With this type of research, health service providers could reshape their structures, processes, prices and accredited networks, in order to improve their market strategy.

Descriptors: Health Maintenance Organizations. Patient Satisfaction. Health Services Evaluation. Health Care Quality, Access and Evaluation.

RESUMEN

OBJETIVO: Analizar la calidad de los servicios ofrecidos por empresas operadoras de seguros de salud, según la percepción de usuarios.

MÉTODOS: Estudio transversal con 360 usuarios de siete operadoras de seguros de salud de la ciudad de Curitiba, Sur de Brasil, y región metropolitana en 2008. Se aplicó cuestionario sobre las preferencias de los usuarios con relación a seis atributos (localización de los puntos de atención; efectividad de la acción de los médicos, clínicas y hospitales; rapidez y amabilidad en la atención, facilidad en la liberación de documentos de aprobación, precio; alcance de la red acreditada) de cada una de las empresas operadoras. Para el análisis de las respuestas fue utilizado el método Analytic Hierarchy Process (AHP, o Proceso Analítico de Jerarquía), herramienta de análisis de decisión y planificación de múltiples criterios.

RESULTADOS: El atributo más valorizado por los usuarios fue "precio". Las empresas fueron agrupadas en dos conjuntos de preferencias con relación a los atributos: de los siete seguros de salud, dos presentaron menor preferencia (entre 23% y 19%) y cinco, mayor preferencia (en torno de 10%).

CONCLUSIONES: Con este tipo de investigación, las empresas operadoras de seguros de salud podrían reformular sus estructuras, procesos, precios y redes acreditadas con el objetivo de mejorar su posicionamiento en el mercado.

Descriptores: Sistemas Prepagos de Salud. Satisfacción del Paciente. Evaluación de Servicios de Salud. Calidad, Acceso y Evaluación de la Atención de Salud.

INTRODUÇÃO

Knowing the profile of consumers, the pattern of service utilization and their different characteristics is the sine qua non for the development of an adequate health care system.⁶

Azevedo³ (1991) stated that the evaluation of health services is an important element in the definition of an adequate and affordable health care system. The author pointed out that the service structure (physical, human, material, instrumental, regulatory and administrative), funding sources, processes (how resources are used) and results (consequences for the consumers) must be evaluated. Bós & Bós⁵ (2004) studied how the economic, social, demographic and epidemiological situation influences the choice of location when caring for the elderly in the public and private systems. According to the study, the establishment where the elderly receive health care depends on both their needs and family resources; in low-income families, the elderly often receive lower priority.

For a health care company, it is essential to understand how their consumers perceive and value its products. Consumers associate with brands and companies, a set of affective and cognitive mental representations, which may be useful to a company when determining its goals and marketing strategies. The brand's public image is closely linked to its identity.

Identity and brand are close but distinct concepts. The identity concerns the definition of personality, while the brand is the perception of this personality. According to Aaker¹ (1996), personality is a set of human characteristics associated with a particular brand. Just as human personalities affect relationships between people, the personality of a brand can build the foundations for the relationship with the customer. Image and identity are inextricably conected.¹ Thus, the brand personality serves to create a strong and lasting relationship with consumers.

Every company should know how its image is perceived by potential consumers and, when necessary, redesign products and services according to this perception. The Analytic Hierarchy Process method (AHP) is effective in this regard. Some studies apply the AHP method to analyze a company's image with its consumers and for other purposes. Blanco^ª(1996) studied the image of the leading banks in Spain and showed a hierarchy of the consumers' preferences, according to attributes considered essential in the Spanish banking system. Costa & Moll⁷ (1999) used the AHP method to study the selection process of sugar cane varieties that should be grown by a plant of the sugar industry to improve productivity results. Munhoz & Castilho^b (2009) used the AHP method to identify and select the best alternative in the acquisition of an information system. Steiner et al¹¹ (2010) evaluated the solid waste management systems in shopping malls in Curitiba, Southern Brasil, identifying the main management practices at each one of them through AHP concepts. Vaidyaa & Kumarb¹³ (2006) carried out extensive research to identify how the AHP method has been applied. They analyzed 150 scientific papers published in prestigious international journals on topics related to products and services, including selection (32 papers), assessment (26 papers), cost-benefit analysis (seven papers), allocation of resources (10 papers), planning and development (18 papers), prioritization (20 papers), decision making (21 papers), prevision (four papers), medicine (five papers) and AHP with Quality Function Deployment (QFD) (seven papers). Of these, 70 were written by North Americans, 27 by Europeans, 50 by Asians and three, by researchers from other continents. Thus, the AHP applicability range and its widespread global use is shown.

The objective of this study was to analyze the quality of services offered by health service providers in a comparative way through the AHP method, and according to the consumers' perception.

METHODS

A cross-sectional study with the seven major health service providers (HP1 to HP7) in the metropolitan area of Curitiba, Southern Brazil, which were responsible for attending 95% of the population in 2008 was carried out. A total of 400 patients, aged 18 to 65 years, were interviewed.

The sample was polietapic (sampling by stages, in the form of decreasing steps, chosen at random) with stratification by neighborhood and number of elements in each stratum. A simple random selection of respondents was carried out; consumers were not questioned about their own health service provider, nor their socioeconomic conditions. The views of respondents who had health plans were considered in the sampling, to identify the companies' strengths and weaknesses, as perceived by the population in Curitiba, as well as to encourage the development of brand positioning strategies in this market.

The aspects analyzed were: 1) how the attributes are ranked by consumers, 2) the consumers perception of the companies in relation to each of the attributes analyzed, and 3) overall perception of companies by the consumers.

Of the 400 interviews, 360 were considered valid, and supervision was conducted by telephone (10%) or personally (17%).

Considering the characteristics of the sample and to ensure consistency for judgment matrices of preferences, the procedure by Silva^c (2007) was adopted in this study, which suggests that the questionnaires consider only one of the attributes (any one), the other values being obtained on its basis. This prevents respondents expressing inconsistent value judgments.

The multicriteria programming through the AHP method is structured for decision making in complex environments, where multiple criteria (or attributes or variables) are considered for prioritization and selection of alternatives. The AHP method, developed in the 1980s by Thomas L. Saaty, is widely used, especially in situations where decisions are made jointly by several people, and where human perceptions and judgments generate long-term repercussions.⁴

The AHP method starts by decomposing the problem into a hierarchy of criteria or attributes more easily analyzable and independently comparable. The next step is a systematic pairwise comparison of the alternatives, according to each of the criteria or attributes. This comparison can be based on actual data or human judgment.⁸

The AHP method transforms the comparisons, often empirical, in numerical values that are processed and compared. By giving weight to each attribute, it is possible to hierarchize them. This ability to convert empirical data into numerical values is the main differential of the AHP over other methods.

According to the method, the pairwise comparison between attributes can be performed in different ways¹². The most used is the scale of relative importance for pairwise comparison, proposed by Saaty^9,10. By assigning values ranging from 1 to 9 to the alternatives, the scale determines, for example, the relative importance of alternative i with regard to alternative j and, reciprocally, the alternative j with regard to alternative i (Table 1).

The use of this scale in the assessment of criteria and/or attributes generates a matrix consolidated with numerical values. The same procedure is used for the evaluation of alternative pairs according to each of the criteria and/or attributes.

These evaluations should be made by each of the K people who are participating in the process of evaluation of alternatives, with the respective weight attribution. It is necessary to establish a single set of matrices (attributes and alternatives per attributes) representing all the evaluation process.

The geometric mean of the values was used, to retain the characteristics of the weights and their reciprocals, according to Aczel & Saaty² (1983). Each element a_ij^c of the consolidated matrices is determined by the following equation:

The values of the consolidated matrices should be standardized in relation to their columns; the relative weights can be calculated between the criteria and/or attributes. These weights are determined by calculating the arithmetic mean of the elements of the lines corresponding to them:

Afterwards, the hierarchy between them can be established, which means obtaining the degree of importance given to each of the criteria and/or attributes.

The same mathematical process can be performed for each of the alternatives according to each attribute. The classification (hierarchization) of the alternatives according to each attribute is expressed by the values of the weights (pa_ij).

To obtain the final result of the analysis, the overall weight of each alternative is determined by calculating the weighted average of the weights of each alternative, according to the various attributes (Equation 3). Therefore, the alternatives are hierarchized, and that with the highest value is selected.

The health service providers were analyzed according to six attributes judged by the consumers as the most important when choosing a health service provider: location of service points (directly impacts the logistics for consumers' transport); effectiveness of doctors, clinics and hospitals (important at the time of attendance); promptness and kindness when caring for patients and family members (related to the consumers anxiety in solving problems) and ease of access to the authorization forms for consultations, examinations, hospitalizations and surgeries (relates to the rights acquired by consumers); price (considered as the quantification of the provided service); diversity of available doctors, clinics and hospitals (which relates to the quality and effectiveness of medical assistance).

The following questions were used in the research (Tables 2 and 3):

"From each pair of attributes listed in Table 2, choose your favorite, according to the scale of preferences." This was intended to identify the most preferred attribute (hierarchize attributes).

"Likewise, indicate, for each pair of companies listed in Table 3, the attribute of your preference, according to the scale." With this question, the company with the highest preference was found (hierarchize companies).

RESULTS

The preferences of health care consumers were analyzed in relation to the attributes (Table 2).

The matrix consistency was ensured by the fact that the judgments of preferences in the columns corresponding to the attributes: effectiveness, promptness, ease of access, price and diversity of available doctors and centers were determined as a function of the values of the attribute "location" (column of the base attribute). Any attribute of the six analyzed could be considered as a base-attribute.

The consolidated matrix was standardized, making it possible to determine the weights for each attribute of companies, by the mean values of each of the lines and their respective percentages. These weights represented the importance that health service consumers gave to each of the attributes used in the study (last column in Table 4).

The attribute "effectiveness" was 4.25 (position (2.1) in matrix) times more dominant than the attribute "location"; "promptness" was 2.75 (position (3.1)) times more dominant than "location ". The attribute " promptness " was 0.65 (position (3.2)) times more dominant that "effectiveness" (2.75 / 4.25) (Table 4).

The most significant attribute for health service consumers was "price" (31.8%); the least significant was "location" (3.9%). The ease of access to the authorization forms and the effectiveness in solving problems were regarded with a certain importance (22.4% and 16.6%, respectively) (last column of Table 4).

The consumers' preferences for the companies with respect to each of their attributes were analyzed. For "location", for example, we obtained the consolidated data matrix (Table 3), which was standardized and the weights determined as a percentage of each company in relation to "location". These weights represented the importance that health service consumers attributed to each company, based on "location" (Table 5).

The company most preferred by the consumers over "location" was HP1 (35.8%). The lower level of preference was linked to HP6 (4.9%). HP2 was the second most preferred (24.7%), HP3 and HP4 were almost at the same preference level (13% and 11%, respectively).

Weights were obtained and the respective percentages of preference of companies in relation to other attributes: effectiveness, promptness, ease of access, price and diversity of available doctors and centers (Table 5).

The company most preferred by consumers in relation to "effectiveness" (the actions of doctors, clinics and hospitals) was HP1 (with 30.3%) and the least favorite was HP7 (4.8%) (Table 5).

The company most preferred by consumers in relation to "promptness" was also HP1 (34.7%) and HP2 was the second most preferred (27.8%). The other companies had the remaining consumers' preference (ranging between 4.7% and 11.6%).

The company most preferred by consumers according to "ease of access" (agility in obtaining authorization forms) was HP2 (32.7%), the least preferred was HP7 (5.2%), while HP1 was the second favorite (31.1%). Companies that have earned the preference of consumers in relation to "price" were HP6 and HP7 (25.4%); the least favorite were HP1 and HP2 (3.5% and 4.1% respectively).

Preference in relation to "diversity of available doctors and centers" (wide range of hospitals, clinics and surgeries; not limited to a few medical services) was for HP1 (36.8%), where the least favorite were HP6 and HP7 (5.9% and 6%, respectively).

After the analysis of preferences for each attribute, we could determine the overall company preference of the consumers, but with the weighted mean of the company preferences with the weights of the preferences of each attribute.

Two sets of companies were identified (Table 5, last line). Companies HP1 and HP2 had a preference of 23.6% and 19.1% of health service consumers, respectively (approximately 43% of preferences). The remaining companies had around 10 % of preference each, totaling about 57%.

DISCUSSION

The results show the viability of the AHP method to analyze the image of health service providers. This method allows assessment of how customers see companies under different lights (attributes considered important to the provision of services).

In this study, the pair-wise comparison was chosen, in order to ensure the consistency of judgment matrices (without making the adjustments recommended by the AHP method). The judgment of all possible combinations was replaced by the judgment of a single attribute (base- attribute as attribute "location") in relation to the others. Relations between the other combinations were determined mathematically according to the relationships of defined proportions in the judgment of base-criterion. In addition to ensuring the consistency of the judgment matrix, this reduced the number of questions relating to the judgments, reducing the time spent with the interviewed consumers.

After placing the companies into two groups, it could be observed that those receiving the lowest preference in the "operational attributes" compensated for these deficiencies with the attribute "price". To set lower prices, they decided to "lower the quality" of their services. This suggests that companies have different market positioning strategies.

In Table 5, it is observed that: 1) the attribute "location" is an important tool for companies HP5, HP6 and HP7 to better define the location and geographic distribution of their points of service; 2) the attribute "effectiveness " is equally important, because it emphasizes the strength of the brand HP1 and the deficiencies of brand HP7 (Table 5); 3) "promptness " in care is very useful to consumers. The importance of this characteristic is evident to the company HP2, and for companies HP6 and HP7, the need to better structure their customer service. Similarly, the interpretation was made for the other attributes.

From the obtained results, companies can decide whether to keep or change their strategies, structures, processes, prices and accredited networks. The preference of consumers is clear and well defined: although "price" was the most important attribute (31.8%, Table 4), the company preferred by consumers is that with the most desired attributes of service quality (HP1 with 23.6%, Table 5).

It is suggested, for future research, in order to refine the study presented here, the analysis of differences in the consumers perceptions, according to the health plan they have. For this reason, studies of Capilheira and Santos⁶ (2006) and Bós & Bós⁵ (2004) may be useful.

REFERENCES

Received: 6/29/2011

Approved: 4/17/2012

The authors declare no conflicts of interest.

• 1. Aaker D. Criando e administrando marcas de sucesso. São Paulo: Futura; 1996.
• 2. Aczél J, Saaty TL. Procedures for synthesyzyng ratio judgements. J Math Psychol. 1983;27(1):93-102. DOI:10.1016/0022-2496(83)90028-7
• 3. Azevedo AC. Avaliação de desempenho de serviços de saúde. Rev Saude Publica. 1991;25(1):64-71. DOI:10.1590/S0034-89101991000100013
• 4. Bhushan N, Rai K. Strategic decision making: applying the analytic hierarchy process. New York: Springer; 2004.
• 5. Bós AMG, Bós AJG. Determinantes na escolha entre atendimento de saúde privada e pública por idosos. Rev Saude Publica. 2004;38(1):113-20. DOI:10.1590/S0034-89102004000100016
• 6. Capilheira MF, Santos IS. Fatores individuais associados à utilização de consultas médicas por adultos. Rev Saude Publica. 2006;40(3):436-43. DOI:10.1590/S0034-89102006000300011
• 7. Costa HG, Moll RN. Emprego do método de análise hierárquica (AHP) na seleção de variedades para o plantio de cana de açúcar. Gest Prod. 1999;6(3):243-56. DOI:10.1590/S0104-530X1999000300009
• 8. Saaty TL. The Analytic Hierarchy Process. New York: McGraw-Hill International; 1980.
• 9. Saaty TL. Método de Análise Hierárquica. WS Silva, tradutor. São Paulo: McGraw-Hill; 1991.
• 10. Saaty TL. Theory and applications of the analytic network process: decision making with benefits, opportunities, costs, and risks. Pittsburgh: RWS Publications; 2005.
• 11. Steiner PA, Braga MCB, Steiner MTA. Hierarchy of Shopping Centers in the City of Curitiba, state of Paraná, Brazil. Int J Comput Sci Netw Secur. 2010;10(5):198-207.
• 12. Triantaphyllou E, Mann SH. Using the analytic hierarchy process for decision making in engineering applications: some challenges. Int J Ind Eng 1995;2(1):35-44.
• 13. Vaidyaa OS, Kumarb S. Analytic hierarchy process: an overwiew of applications. Eur J Oper Res. 2006;169(1):1-29. DOI:10.1016/j.ejor.2004.04.028

a

Blanco MC. Aplicación del analitic process em la medida de la imagen de marca de servicios [tese de doutorado]. Leon: Universidad de Leon; 1996.

b

Gómez-Munhoz DC, Castilho M. Design of a methodology for identifying e selecting the best alternative in acquiring the information system or company. In: Proceedings of the 10th International Symposium on the Analytic Hierarchy Process; 2009 Jul 29- Aug 1. Pittsburgh, USA. Pittsburgh: University of Pittsburgh; 2009

c

Silva DMR. Apliação do método AHP para avaliação de projetos industriais [dissertação de mestrado]. Rio de Janeiro: PUC-RJ; 2007.

Publication Dates

• Publication in this collection
  03 Apr 2013
• Date of issue
  Oct 2012

History

• Received
  29 June 2011
• Accepted
  17 Apr 2012