The Brazilian Unified National Health System: Proposal of a Cost-effectiveness Evaluation Model

Abstract

The Brazilian Unified National Health System (Sistema Único de Saúde [SUS]) is in a prominent position compared to the existing social policies. One of the new tools used by SUS is known as Performance Index of the Unified Health System (Índice de Desempenho do Sistema Único de Saúde [IDSUS]), which is intended to measure the performance of each municipality. Therefore, the aim of this study was to propose a model of cost-effectiveness to compare IDSUS performance against total revenues achieved in Homogeneous Group 2, consisting of 94 municipalities and analysed using data from IDSUS and the System Information of the Public Budget for Health Care (Sistema de Informação do Orçamento Público em Saúde [SIOPS]) for the year 2011. After structuring this data, we carried out descriptive statistical and cluster analysis in order to group similar municipalities in accordance with established variables: IDSUS performance, population and total revenue in health per capita. Even with the division of municipalities into homogeneous groups and after using variables such as population and revenue to regroup them, the results showed there are municipalities with heterogeneous characteristics. Another finding is in the use and intersection of two distinct databases (IDSUS and SIOPS), which allowed for visualizing the impact of health care revenue on the municipalities performance.

Key words:
health policies; SUS; cost-effectiveness; IDSUS

Introduction

One of the great achievements of Brazilian people, even in the 20^th century, was the promulgation of the Constitution of the Federative Republic of Brazil in 1988 that has among others subjects, the social right as citizenship factor. Citizenship is defined as an inclusion of all individuals that share culture, belief, rights and duties for a specific national policy community (Fleury & Ouverney, 2008). Aiming to guarantee the citizenship, the constitutional text brought large mechanisms of social protection, many times understood as heterogeneous, unequal and ineffective, however built from institutions, human resources and stable resources (Cardoso & Jaccoud, 2005).

From the rights achieved in the named Citizen Constitution, the health care social policy implemented in Brazil was the Brazilian Unified National Health System (Sistema Único de Saúde [SUS]) that has as main objectives universality, integrality, equal assistance, community participation and decentralization of single and centralized command. The SUS is also a policy that involves all the parts of the federation, delegating delivery of services duties, supervision and funding for the states, municipality and the Federal District.

Currently, more than 190 million people benefit from SUS and approximately 75% of that population have exclusive dependence on the public health care system (Portal Brasil, 2009). According to data from the Ministry of Health (Portal Brasil, 2009), the health care system operates with 6.1 thousand accredited hospitals, 45 thousand units for primary treatment and performs 2.8 billion ambulatory procedures per year, 19 thousand transplants, 236 thousand heart surgeries, 9.7 million of chemotherapy and radiotherapy procedures and finally 11 million cases of hospitalization.

However, the new guarantees of social protection formalized in the Constitution were not enough to make the health care system homogeneous. On this way, the politic conjuncture of redemocratization and fiscal crisis contributed for the deep stress faced by the SUS, making it difficult to achieve its directives (Pires & Demo, 2006; Tanaka, Drumond, Cristo, Spedo, & Pinto, 2015).

Even facing many issues and relevant challenges, the SUS is consolidated after 25 years as a health care social public policy in Brazil, trying to guarantee the fundamental principles committed in the Constitution of 1988 (Constituição da República Federativa do Brasil, 1988/2016). Thus, evaluation of SUS becomes a fundamental need in conjunction with other public policies, with the objective to measure levels of attendance considering efficiency and effectiveness.

From evaluations made on SUS, it is possible to highlight the ones made on performance and economic evaluations on health care aspect. The performance evaluation is related to quality improvement and monitoring of actions for the health care system, aiming to attend the principles mentioned of efficiency and effectiveness, also with the population health needs (Portaria n. 399, 2006; Tanaka et al., 2015), in addition to provide control and better use of resources set for the system (Costa & Castanhar, 2003; Ferraz, 2010).

In this context is the Performance Index of the Unified Health System (Índice de Desempenho do Sistema Único de Saúde [IDSUS]), synthesis indicator that represents the directives commitment proposed during the creation of SUS and measures the distance between the real and ideal SUS (according to the international accepted standards) (Forte & Nobre, 2014; IDSUS, 2011; Reis, Oliveira, & Sellera, 2012). The index is relatively new and was announced on March 2012, as a set of 24 indicators that measure access to the health services and the effectiveness of this system, so works as a photography of the Brazilian public health care system in the previous years (triennial 2007-2010) and it is presented as a representation of health condition to the municipalities of the country.

This method offers results for all the municipalities of the country, and they were split in homogeneous groups, following the IDSUS classification. The homogeneous groups are needed duo to high heterogeneity of social, economic and population conditions of the Brazilian municipalities. For this study, the research universe chosen is the Homogenous Group 2, composed by 94 municipalities of the country. Justification for the group choice and deep analysis on the classification of the municipalities made by IDSUS will be handled soon, in the methodological aspects.

In addition to performance evaluations already completed, considering the scenario of insufficient resources and demand increase, it is also possible to see how important the economic evaluation on health care is, what consists in analysing two or more strategies or interventions on health care comparing costs and benefits achieved (Drummond, Stoddart, & Torrance, 1987; Elias & Araújo, 2014; Ferraz, 2010; Ribeiro & Polanczyk, 2005).

There are four types of economic analysis on health care according to Drummond, Stoddard and Torrance (1987): cost-minimization, cost-benefit, cost-effectiveness and cost-utility. The cost-minimization is the most simple among the four types because it observes only the lowest cost between two alternatives; the cost-benefit relates both costs and monetary benefits and compares two alternatives; the cost-effectiveness considers not only monetary benefits but also social benefits; finally the cost-utility is the most refined analysis compared to all the others as it compares not only costs and benefits, but also life quality perception of patients (Camelo et al., 2011; Drummond et al., 1987; Ministério da Saúde, 2008). As part of the most used methods are the cost-effectiveness and cost-utility, as they englobe both monetary and clinical benefits. It is possible to note in Brazil, even as a start, utilization of the cost-effectiveness analysis carried out by the Ministry of Health conducted by the Department of Science and Technology (Departamento de Ciência e Tecnologia [DECIT]).

It is emphasized that this study proposes a cost-effectiveness analysis model of the current and active health care public policy in the country. Therefore, health care revenues are observed (cost) using database the System Information of the Public Budget for Health Care (Sistema de Informações sobre Orçamentos Públicos em Saúde [SIOPS]) and the scores achieved from IDSUS (effectiveness) giving as outcome the possible impact produced by the resource on the score achieved.

The purpose of using the cost-effectiveness analysis model is justified because it differs from economic evaluations on health care adopted, which states there are two scenarios that need to be compared, computing other economic variables in the analysis. It is possible to say that this model aims to compare the relations between health care revenue and the IDSUS scores, in order to join two distinct databases and generate a consolidated evaluation.

Utilization of joined data contributes to the efficient management of already scarce resources, being it one of the pillars of public administration. Furthermore, it looks for meeting citizens expectations, who expects from the government better services, reaching the social demands with transparency and integration of all parts of the government (Motta, 2013; Pollitt, 2003).

The Unified Health System (SUS)

In Brazil, access to public health care system as factor of citizenship was established only in 1988 with the promulgation of the Constitution, and hence has been treated as a right of the Brazilian citizen (Conselho Nacional de Secretários de Saúde, 2011; Finkelman, 2002; Santos, Francisco, Faria, & Gonçalves, 2011). This fact occurred through Article 196 (Constituição da República Federativa do Brasil, 1988/2016, p. 118, our translation), that states:

Health is everyone's right and duty of the State, guaranteed through social and economic policies with the objective to reduce risk of disease and other injuries and the universal and equal access to actions and services for his promotion, protection and recovery.

In the context of the organization and management of the Unified Health System, there are basic premises in the Constitution, decentralization of administration, as single command in each government sphere, democratic governance and participation of civil society (Conselho Nacional de Secretários de Saúde, 2011; Pessoto, Ribeiro, & Guimarães, 2015). While SUS fundamental premise is decentralization, an important way of universally attend all population and to manage a complex conjunction of actors, its structure is developed including the three spheres of government (Paim, Travassos, Almeida, Bahia, & Macinko, 2011). The Ministry of Health is primary responsible for the national strategy through planning, control and evaluation of public policies implemented by decentralizing the system (Conselho Nacional de Secretários de Saúde, 2011; Finkelman, 2002; Noronha, Lima, & Machado, 2008). However, it is the state's duty to follow directives established at federal level, in addition to formulate their own policies and contribute in the financing, planning, control and evaluation of SUS in their regions; finally, municipalities have to follow national and state directives as a mission, plan, organize, control, evaluate and finance the system at the municipal scope (Conselho Nacional de Secretários de Saúde, 2011; Finkelman, 2002; Gonçalves, 2014; Noronha et al., 2008).

The main financier of the health care universal model in Brazil is the Union, and the management of resources is a municipality role, which allocates available resources according to the needs of the population. Investment in health care should vary according to the variation of the Gross Domestic Product (GDP) (Produto Interno Bruto [PIB]), in the same year, and the municipalities should apply from 12% to 15% of its revenue in the public health care. Decentralization established after the constitution of 1988 improved participation of the states and municipalities in the implementation and financing of health care policies, making them key elements in providing basic services to the community (Gonçalves, 2014).

For sustainability of SUS and increase in sources of assets, with the objective to match other universal models, the system needs monitoring and evaluation of actions, in order to increase the rationalization of costs, making it more effective (Forte & Nobre, 2014). Dain (2007, p. 1863, our translation) states "health cost is not synonymous to waste of resources. Therefore, health care managers at all levels of the government must ensure the quality of their expenditure and the rationalization and reduction of costs".

Evaluations of Public Health Policies

Public policies are split up according to the following phases: (a) formulation, (b) implementation and (c) evaluation (Paim et al., 2011; Saravia, 2006). Thus, it is possible to understand since its elaboration, objectives and strategies are need to change the reality of the society discussed. The increased relevance given on evaluation, on the another stages of public policies is clearly related to the efficiency of public expenditures and quality of services delivered, in addition to measuring effectiveness of actions of the State and announcement of the results achieved, aiming accountability to the society (Alves & Passador, 2011; Costa & Castanhar, 2003; Ferreira, Najberg, Ferreira, Barbosa, & Borges, 2014).

In the Brazilian territory, a management approach reform, implemented during the Fernando Henrique Cardoso government, driven to combat excessive expenditures and the ongoing fiscal crisis, in the beginning of the nineties, carried out a strong periodization looking for the best performance of the public sector (Abrucio, 2007; Bresser-Pereira, 2010; Motta, 2013). Changes from the called New Public Management (NPM) made evaluation practice of public policies and programs easier, while even timidly, increase of evaluations mechanisms utilization was also influenced by society and put some pressure against the Estate for better public services (Abrucio, 2007; Bresser-Pereira, 2010; Motta, 2013). It is necessary, differently from the adopted NPM view, to observe that the public administration has expectations and objects that diverge from the private organizations management, as follow: to provide citizens a set of related services, integrated that involve groups and multiple intentions, matching social and public interests, much more than a simple combination of services' efficiency and costs reduction (Denhardt & Denhardt, 2000; Motta, 2013; Pollitt, 2003).

Performance Evaluation on Health Care

Looking into public health care, interest in monitoring and evaluating is getting more intensified since the nineties, from publication of the World Health Report 2000 - Health Care Systems: Improving Performance, from the World Health Organization (WHO), which emphasized need to evaluate the SUS (Elias & Araújo, 2014; IDSUS, 2011; Viacava et al., 2004). As there are details in the Brazilian health care system, especially discrepancies between regions, to create a reference guide for Brazil (already used in other countries) becomes a required need (Viacava et al., 2004). Complementary, experience from many countries points a desire for cooperation among the actors involved in formulating health care social policies, in order to create an ample and effective performance evaluation (Murray et al., 2013; Viacava et al., 2004).

Many performance evaluation approaches of the health care system have been tried, but in a fragmented and partial way, and the information collected did not become strategic actions for managers (Portaria n. 399, 2006; Tanaka et al., 2015). Therefore, a need arises to integrate all evaluation methods carried out on SUS, aiming to produce systematic information for adaptation and improvement of the system.

For a complete evaluation of SUS performance, Reis, Oliveira and Sellera (2012) recommend integration of some existent evaluation programs in the context of SUS as follow:

Index of the Unified Health System (IDSUS);

National Program for Evaluation of Health Service (Programa Nacional de Avaliação de Serviços de Saúde [PNASS]);

National surveys for evaluation of the users accesses and satisfaction;

National Program of Access Enhancement and Basic Attention Quality (Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica [PMAQ]).

Considering the proportion and outcome of the performance evaluation programs utilized by SUS and presented by the authors, this study is going to be restricted to only the IDSUS method, timely emphasized through the methodology ahead. It is worth to consider that evaluation as PNASS and PMAQ focus mainly on the user's view in relation to the services provided and the health care system structure, but also with the intention to improve quality and access to the service (Marins & Daher, 2014; Departamento de Atenção Básica, n.d.).

Economic Evaluation on Health

In addition to the SUS performance evaluation, it is possible to see importance and increase of evaluation based on the economic impact produced by a given health policy, aiming efficient use of resources (Elias & Araújo, 2014; Ribeiro & Polanczyk, 2005). For better allocation by managers, the health care economic evaluations are an important aspect for improvement of health care systems; however, Brazil still produces little and has no tradition in the economic evaluation in health care if compared to some countries from Europe (Elias & Araújo, 2014; Murray et al., 2013).

The economic health evaluations are widely utilized in developed countries, especially in Canada, the United States, England and Australia, through institutions that develop analyses for such health care policies (Camelo et al., 2011; Ministério da Saúde, 2008; Murray et al., 2013; Vanni, Luz, Ribeiro, Novaes, & Polanczyk, 2009). In Brazil, it is responsible of the Department of Science and Technology of the Ministry of Health (Departamento de Ciência e Tecnologia do Ministério da Saúde [Decit-MS]) the task to economically analyse health technologies for health care that are incorporated to the management and public health service (Camelo et al., 2011; Elias & Araújo, 2014; Vanni et al., 2009).

Moreover, the primary objective of economic evaluation area is to analyse how to allocate the resources available to health care with the highest possible productivity based on clinical and economic aspects (Vanni et al., 2009). There are four basic types of economic analysis in health care according to Drummond et al. (1987): cost-minimization, cost-benefit, cost-effectiveness and cost-utility.

The cost-minimization is the most simple among the presented methods, which aims to verify between two or more alternatives, the lowest cost for the same clinical outcome, seeking total set of more reduced cost (Drummond et al., 1987).

Regarding the cost-benefit analysis, it is intended to evaluate the monetary benefits of an intervention, reporting both the costs and benefits, in order to compare alternatives and choose the best cost-benefit ratio (Camelo et al., 2011; Drummond et al., 1987).

The cost-effectiveness analysis supports evaluation of costs and consequences of a treatment, program or health care policy, making it complete on evaluation among one or more alternatives (Drummond et al., 1987) - it is usually used because it has as outcome unit both monetary and clinical benefits. These aspects are measured considering years of life saved or complications / hospitalizations prevented (Ministério da Saúde, 2008); therefore, the cost-effectiveness ratio is considered when a limited amount of resources produces more results and benefits units (Ministério da Saúde, 2008).

Finally, the cost-utility, considered a derivation of the cost-effectiveness analysis and has as outcome unit monetary and the clinical benefits measured by life quality of patients. The prime clinical outcome unit utilized in this analysis is the QALY (quality-adjusted-life-year), which considers quality of life perceived by a patient and the survival time attributed to the intervention performed (Drummond et al., 1987; Ministério da Saúde, 2008).

Among the most commonly used analysis are the cost-effectiveness and cost-utility as they englobe both monetary and clinical benefits. The cost-effectiveness analysis is justified as one of the most utilizes, because it is an economic analysis that results in different units of measure and additionally refers to both monetary resource as social benefits (Ministério da Saúde, 2008). However, no data were available as refined and complex as the cost-utility analysis, yet difficult to implement by employing sophisticated measures of effectiveness and outcomes of units based on years of quality-adjusted life (Ministério da Saúde, 2008).

In this context of performance and economic evaluations on health care, as previously pointed out, this research proposes a cost-effectiveness analysis model for public health care policy adopted by SUS, using as variables public revenues for health care and scores achieved by municipalities in IDSUS, which will be specified soon. This model is characterized as a source of information for decision-making, benefiting managers of the health care system, as it joins data from two distinct databases (IDSUS and SIOPS), it also provides an analysis of the possible impacts of health care revenue on the municipalities' performance and the relationship of the possible impacts produced in each financing package adopted by the Ministry of Health.

Methodological Aspects

The study firstly conducted a search in research databases; in this case, they were Scielo, Google Scholar, Scopus, Bank of Theses and Dissertations of USP, Virtual Library for Health (Biblioteca Virtual em Saúde [BVS]), informational websites of the federal government and diverse books. Secondly, national and international material on the subject researched was collected in order to create solid bases and familiarity of the researcher with the theme, which involved subjects according to the keywords used as search parameter: public health policies, the Unified National Health System (SUS), performance evaluation on health care, economic evaluation on health care, cost-effectiveness analysis and IDSUS, published between 2005 and 2014.

Complementally, the second part of the study has mixed nature, being composed of two stages: the first one is intended to study the Performance Index of the Unified Health System (IDSUS), which verified its structure, principal components, presentation approach and results; the second part is the analysis of the results achieved by the municipalities in the Homogeneous Group 2, which were grouped according to similar characteristics, respecting the IDSUS definitions (detailed in the next section) and then regrouped in this research, using cluster analysis (grouping) and techniques of descriptive statistics. After regrouping the municipalities of the Homogeneous Group 2, it was conducted the cost-effectiveness analysis as the proposed model in research and previously mentioned.

Study of the IDSUS

Similarly the indicators used in education that promotes managers the opportunity to understand the positive and negative aspects of the health care system, with the objective to make improvements according to priorities previously established, the IDSUS has similar characteristics and purposes (IDSUS, 2011). Once actions are monitored using indicators, it is possible to follow the established objectives, improvements and issues occurred (Macêdo & Damasceno, 2013). The authors also highlight importance of the information collected from these indicators; it consists privileged information essential for decision making by system managers, as well as being an essential tool in managing the entire organizational system (Macêdo & Damasceno, 2013).

The theoretical fundament used by the IDSUS was the Development Project of a Performance Evaluation Methodology for the Brazilian Health Care System (Projeto de Desenvolvimento de uma Metodologia de Avaliação do Desempenho do Sistema de Saúde Brasileiro [PRO-ADESS]), created from partnership between the Brazilian Association of Post-graduation in Collective Health (Associação Brasileira de Saúde Coletiva [Abrasco]) and Oswaldo Cruz Foundation (Fiocruz) (Viacava et al., 2004). Thus, the IDSUS was designed by selecting some indicators that make up the proposed matrix in PRO-ADESS (IDSUS, 2011), especially the ones of access and effectiveness. Given the monitoring need and evaluation of SUS, the creation of IDSUS aims to establish clear parameters that can lead productive investment of the SUS resource in addition to contribute in increasing the quality and effectiveness of health care services delivered by the state (Forte & Nobre, 2014; IDSUS, 2011). The index also seeks to evaluate the SUS performance in relation to its principles and directives: regionalization, hierarchy, decentralization with single command and tripartite responsibility (IDSUS, 2011; Reis et al., 2012).

Data sources of IDSUS indicators are: National Register of Health Institutions (Cadastro Nacional de Estabelecimentos de Saúde [CNES]); Information System for Treatment Area (Sistema de Informação Ambulatorial [SAI]); Information System for Notification Harm (Sistema de Informação de Agravo de Notificação [Sinan]); Information System of the National Immunization Program (Sistema de Informação do Programa Nacional de Imunização [SI-PNI]); Hospital Information System (Sistema de Informação Hospitalar [HIS]); Information System of Mortality (Sistema de Informação sobre Mortalidade [SIM]); Information System of Born Alive (Sistema de Informação sobre Nascido Vivo [Sinasc]); Brazilian Institute of Geography and Statistics (Instituto Brasileiro de Geografia e Estatística [IBGE]) and Ministry of Social Development and Fight against Hunger (Ministério do Desenvolvimento Social e Combate à Fome [MDS]). All mentioned programs are national information systems, with the participation of states and municipalities, and englobe diverse data on health indicators, however fragmented and with not interaction (IDSUS, 2011; Reis et al., 2012).

Therefore, the IDSUS's purposes are: overall performance evaluation of the municipalities, regions and states; evaluation of access and effectiveness at all levels of care; and diagnosis of deficiencies for improvements actions (Reis et al., 2012). However, it is important to note that Brazilian socioeconomic affect the evaluation of the municipalities; and as consequence, the index considers homogeneous groups of grouped municipalities according to demographic profile, health condition of the population and structure of health care systems (Reis et al., 2012).

Given the presented assumptions, the IDSUS offers the result achieved by scores ranging from 0 to 10, considering all the necessary reflections to deal with the municipalities in different ways, once they present different socio-economic and structural conditions (IDSUS, 2011). Figure 1 shows the composition of the synthesis score presented by IDSUS, it also presents the methodology of analysis of the main components (PCA) utilized in the index structure that according to IDSUS (2011, p. 15, our translation), "constitute the composite indicators or index of potential or obtained access, and effectiveness indices in the distinct levels of health care attention".

Figure 1
Graphic Representation of the Indicators Composition that Constitute the IDSUS

Data Collection and Structuration

Firstly, data from IDSUS available at the Ministry of Health website was used, which refer to the triennial period of 2007 and 2010 and announced in March 2012; therefore, this study uses secondary data. The study universe is the Homogeneous Group 2, composed of 94 municipalities distributed in Brazil. The choice of this group is due to convenience of studying a group with reasonable number of municipalities and that presents high socioeconomic development, medium health care conditions and medium health care structure. It is also highlighted that the comparison between the groups cannot be possible, once the municipalities were split up into homogeneous groups in the IDSUS study for as it presented heterogeneous characteristics among different groups; the differences are related to municipalities with different sizes and health care structures, so it is chosen to compare municipalities belonging to same homogeneous group. The differences among the groups can be seen in Table 1.

Table 1
Municipalities' Distribution per Homogeneous Groups

GROUP	IDSE*	ICS**	IESSM***	QTY MUN.
6	Low	Low	No Structure MAC****	2,183
5	Medium	Medium	No Structure MAC	2,038
4	Low	Low	Low Structure MAC	587
3	Medium	Medium	Low Structure MAC	632
2	High	Medium	Medium Structure MAC	94
1	High	Medium	Medium Structure MAC	29

• Note. Municipalities division into homogeneous groups according to similar characteristics: IDSE, ICS and IESSM. Source: Adapted from CGMA/Demas/SE/MS (2011 as cited in Índice de Desempenho do Sistema Único de Saúde. (2011). Resultados dos indicadores do SUS nos municípios - Dados dos anos de 2008 a 2011 (p. 21). Retrieved June 26, 2012, from http://idsus.saude.gov.br/mapas.html).
• *IDSE - Socioeconomic Index Performance; **ICS - Health Condition Index; ***IESSM - Municipality Health System Structure Index; MAC - High and Medium Care Attention Complexity (specialized, hospital and ambulatory).

From the above figure, it is necessary to indicate the IDSUS parameters utilized for classification and distribution of the municipalities in the homogeneous groups as follow:

Socioeconomic Index Performance (IDSE): according to the IDSUS (2011), it is composed by GDP per capita e percentage of families that benefit from the National Program for Poverty Reduction (Bolsa Família);

Health Condition Index (ICS): it uses the child mortality rate, available in the SIM database, to demonstrate differences among the Brazilian municipalities (IDUS, 2011);

Municipality Health System Structure Index (IESSM): "Municipalities average of reference for access parameters to hospital and ambulatory attention of medium and high complexity" (IDSUS, 2011, p. 21).

After establishing the IDSUS data structure criteria, information of the revenues applied to health care of each municipality was sought. For this, data from database of the SIOPS was used, available on the Ministry of Health website (DATASUS - SUS database). On these systems, information of the different levels of the government are registered and stored, being also the only source of data with detailed information at the municipal level (Medeiros, Albuquerque, Diniz, Alencar, & Tavares, 2014). However, it is important to note that among the 94 municipalities analysed the Homogeneous Group 2 of the IDSUS, Itaperuna (city in the state of Rio de Janeiro) did not declare data for 2011 period, and consequently it was removed from the analysis.

Complementing the Total Revenues for health care information and to build the profile of the municipalities in the sample universe for parallel analysis, data about the population was used - Census 2010 (IBGE, n.d.) and the Municipal Human Development Index (Índice de Desenvolvimento Humano Municipal [IDHM]) (Programa das Nações Unidas para o Desenvolvimento [PNUD], 2013). The choice of these complementary indicators is justified by need to characterize the municipality by its population size and development indicators available.

As already said previously, after building database utilized in the work on the next step, it is stablished approaches to analyse behaviour of the variables into the chosen group. For this descriptive statistics and cluster analysis (grouping) methods are used.

The use of descriptive statistics was ideal to summarize the data about the municipalities' population in conjunction to the total revenue for health care per capita and the IDSUS scores achieved by the municipalities, in addition to be used to build the average of cost-effectiveness analyses. These techniques were also used after the cluster analysis, to summarize the data of the municipalities of each particular group; in this phase, the data analysed was total per capita revenues, population, IDHM and scores of the IDSUS (Tanaka et al., 2015).

For the second phase of data analysis, it was decided to classify the groups according to similar characteristics. Therefore, the chosen tool was the clusters or grouping analysis. Therefore, whether the classification is successful, the objects in the group will be close and those external to the grouping and that are part of another cluster will be distant (Hair, Black, Babin, Anderson, & Tatham, 2006). In cluster analysis, who determines the statistical variable to be utilized is the researcher, making the definition of variables a critical work (Hair et al., 2006).

From the above mentioned assumption, the variables chosen for the segmentation of objects (in this study, the municipalities of Homogeneous Group 2) are: NOTA_IDSUS (score synthesis achieved by the municipality in sum of the 24 indicators) POPULATION (total municipality population) and RECEITA_TOTAL (per capita), which is the division between the total revenue for health care by the municipality population. These variables are justified because the objective of this study is to verify the relationship between total revenues for health care (while observing that the municipalities have different sizes, it is used the total revenue for health care per capita) and the IDSUS scores, through the cost-effectiveness analysis model proposed, which will be described on.

In order to define the similarity among the objects, the measure employed was the quadratic Euclidean distance. To use this measure of similarity, it is recommended to use standardized data (Hair et al., 2006); as the variables used in the study are from different scales and magnitudes, these were converted into standard scores (Z scores), subtracting each variable from average and dividing by the standard deviation.

Then, after defining the similarity measure and clustering method, it is required to find the optimal number of clusters that will be used. For this, it was used the average connection method among the groups and after analysing the composed groups, observing the change variation of the internal heterogeneity and quantity of objects per group, the recommended number for use in the study was of seven groups for the set of 93 municipalities in the Homogeneous group 2. So, after being segmented the group into seven clusters with similar characteristics according to the variables described, the next step was characterized as the model structuring proposed for cost-effectiveness analysis, which is different the analysis already present in the literature, brings peculiarities. As variables for the composition of the cost-effectiveness analysis (Composição da Análise de Custo-benefício [ACE]) from segmentation of the groups, it was used the average revenue divided by the average score of the group in IDSUS. To illustrate the proposed model Figure 2 shows the building steps of this model, as well as the variables involved, in detail.

Figure 2
Detailed Proposed Model of ACE

The cost-effectiveness utilization is considered, economic tool described in the theoretical reference mentioned above, but according to the study proposed model and generically structured as follow:

Results and Discussion

At a first moment, when the IDSUS scores are placed in ascending order, it is possible conclude that the large cities and population over 190,000 people constitute the group that has the worst scores. Possibly, the high demographic concentration and metropolitan regions strongly influence for these municipalities to show lower scores when compared to the scores of the other municipalities, it is suspected, in this period, demand for health care services in the mentioned regions is increased when compared to the other group, decreasing the effectiveness of treatment. It is also possible to see, even superficially, that total revenues per capita on health care for these municipalities are similar to those of the municipalities with better scores.

Regarding the municipalities with best performance, it can be verified that the population ranges from about 38 thousand to over 500 thousand. These are municipalities with large population and some of them belonging to metropolitan areas, however in less number, if compared to municipalities with worst IDSUS performance.

For deeper analyses of the municipalities and for better comparing them, the cluster analysis was used as a way to join them according to similar characteristics proposed by variables: IDSUS score, population and total revenue per capita on health care, as detailed in Table 2.

Table 2
Descriptive Statistic of the Group General Information

Group information summary
	N	Minimum	Maximum	Mean	Standard deviation	Median
Revenue per capita (R$) (Ministério da Saúde, 2011)	93	R$ 195.93	R$ 1,388.82	R$ 489.87	R$ 211.25	R$ 482.20
Population (IBGE, n.d.) nº inhab.	93	38,769	1,221.979	317,452	208,447	263,689
General IDSUS score (IDSUS, 2011)	93	4.19	8.22	5.87	0.76	5.76

• Note. Source: Adapted from Ministério da Saúde. (2011). Sistema de informações sobre orçamentos públicos em saúde. Retrieved February 12, 2013, from http://siops.datasus.gov.br/recdespinfomuncont.php?S=225&UF=12;&Municipio=-1&Item=-1&Fase=35;&Pasta=17;&Tipo=R&Ano=2011&Periodo=12&Ordenacao=Codigo; Instituto Brasileiro de Geografia e Estatística. (n.d.). Censo demográfico 2010. Retrieved February 3, 2013, from http://www.ibge.gov.br/home/estatistica/populacao/censo2010/default.shtm; Índice de Desempenho do Sistema Único de Saúde. (2011). Resultados dos indicadores do SUS nos municípios - Dados dos anos de 2008 a 2011. Retrieved June 26, 2012, from http://idsus.saude.gov.br/mapas.html.

At this moment it is visible that, in addition to the stablished objective by SUS to group the municipalities into homogeneous groups according to defined characteristics (IDSUS, 2011), the revenue for health care was not considered, as there is one municipality in the group with total revenue per capita of R$ 195.93 and at the other side, presenting revenue of R$ 1,388.82. Considering that financing for health care is mentioned as one of the system's bottlenecks (Finkelman, 2002; Ugá & Marques, 2005), this research emphasizes the need and importance to group this information in the Brazilian health care evaluation process.

Another factor considered, only as a complement to the principal variables of the study is the population. According to the Table 2, the group's population is also very diverse, having a minimum number of 38,769 inhabitants and a maximum of 1,221,979. Even when one of IDSUS stratification factors is the municipality population size, the group presents municipalities with very distinct population characteristics, which could influence the quality of access and effectiveness of the system of these places. Additionally, there is a direct influence on the financial support given that the funding comes from the three spheres of the government (Servo, Piola, Paiva, & Ribeiro, 2011).

Finally, still analysing the group using a wide view, it is possible to observe from Table 2 descriptive statistics of the municipalities IDSUS score, in this case, the score vary from 0 to 10 and it is established according to access and effectiveness indicators (IDSUS, 2011). From a minimum of 4.19 and maximum of 8.22 presented, it is possible to see again the heterogeneity of HG2 in relation to accomplishment of the access and effectiveness standards established by the 24 indicators analysed by the index.

Given the heterogeneity of the conditions by the table previously presented for effective comparison, real and fair among the municipalities, it is justified the need for cluster analysis, as detailed in the methodology and described through the next section.

Clusters profile

It can be verified from Table 3 that the group with the largest number of municipalities is the third one, with 31.18% of the total (29 municipalities); the smallest group, the number 7, has only one municipality with about 1.08% of the total. It is also possible to see that there are two groups containing 18 municipalities, with approximately 19.35% of the total in each one. By removing the groups 4 and 7, the groups are close to uniformity in the item quantity of municipalities.

Table 3
Number of Municipalities per Cluster

Cluster	Quantity of municipalities	% of the total
1	11	11.83
2	14	15.05
3	29	31.18
4	2	2.15
5	18	19.35
6	18	19.35
7	1	1.08
Total	93	100,00

• Note. Source: Authors.

Aiming to improve the municipalities' analysis, it was created a profile of each group with additional information of human development (PNUD, 2013) and division the total revenue for health care in the five funding packages established by the Ministry of Health (Portaria n. 399, 2006) - Basic Care (BC), Hospital and Ambulatory Care of Medium and High Complexity (MHC), Health Vigilance (HV), Pharmaceutical Assistance (PA) and SUS Management (SM) - in addition to the variables used in the segmentation (IDSUS score, total revenue per capita and population).

According to Table 4, Group 1 is characterized by having the lowest IDSUS score, the third lowest total revenue for health care, the highest population average and the second worst IDHM average. Looking at the cluster 2, it has the third worst IDSUS score, the lowest revenue for health care, the fourth largest population and the third best IDHM. The group 3 has the fourth worst IDSUS score, the fourth lower revenue for health care, the third largest population and the fourth best IDHM. Cluster 4 is characterized by having on average the second worst IDSUS score, the second largest health care revenue, the second largest population and the worst IDHM of the discussed groups. The cluster 5 presents the second best score in IDSUS, with the third highest revenue, the third smallest population and the second best IDHM. The cluster 6 is characterized by having the third best IDSUS score, the second lowest revenue for health care, lowest population and the third worst IDHM. Finally, the cluster 7, formed by only the city of Barueri (SP), isolated due to have very different characteristics from the other clusters, its profile has the best IDSUS score, highest health care revenue, the second smallest population and the best IDHM.

Therefore, it is possible to verify that the groups 1 and 7 are very distinct, once the group 1 has worse results compared to group 7, according to the analysed variables. Considering the population and the IDHM, it can be suggested that the two factors have play some relationship on the level of service indicators that measure access to and effectiveness of the system proposed by IDSUS, even financing possibly makes influence, due to the fact the worst performance group on average also have the third lowest revenue for health care.

Table 4
Homogeneous Group 2 Municipalities Profile, According to the Clustering

Clusters	IDSUS score	Total Revenue R$ (Ministério da Saúde, 2011)	Population (IBGE, n.d.)	IDH (PNUD, 2013)	Revenue BC (Ministério da Saúde, 2011)	Revenue MHC (Ministério da Saúde, 2011)	Revenue HV (Ministério da Saúde, 2011)	Revenue PA (Ministério da Saúde, 2011)	Revenue SM (Ministério da Saúde, 2011)
1	4.63	399.24	580,584	0.779	74.44	186.05	10.64	10.28	5.35
2	5.55	256.30	299,314	0.810	120.72	34.53	11.88	8.41	11.80
3	5.57	547.02	372,422	0.798	100.00	237.57	11.72	8.92	5.69
4	5.33	1,065.16	425,755	0.777	52.97	284.81	7.68	22.28	0.12
5	6.60	661.65	259,921	0.812	150.99	310.83	14.01	11.74	3.67
6	6.57	349.19	131,953	0.792	167.70	65.90	7.45	16.68	1.06
7	8.22	1,388.82	240,749	0.826	371.32	935.83	33.08	48.28	0.32

• Note. Source: Adapted from Ministério da Saúde. (2011). Sistema de informações sobre orçamentos públicos em saúde. Retrieved February 12, 2013, from http://siops.datasus.gov.br/recdespinfomuncont.php?S=225&UF=12;&Municipio=-1&Item=-1&Fase=35;&Pasta=17;&Tipo=R&Ano=2011&Periodo=12&Ordenacao=Codigo; Instituto Brasileiro de Geografia e Estatística. (n.d.). Censo demográfico 2010. Retrieved February 3, 2013, from http://www.ibge.gov.br/home/estatistica/populacao/censo2010/default.shtm; Programa das Nações Unidas para o Desenvolvimento. (2013). Índice do Desenvolvimento Humano Municipal Brasileiro (Série Atlas do Desenvolvimento Humano no Brasil 2013). Brasília: Autor. Retrieved from http://www.atlasbrasil.org.br/2013/data/rawData/publicacao_atlas_municipal.pdf

Other variables that compose the Table 4 are the financing packages. It is possible to verify that the cluster 4 is the one that makes lowest revenue investment in BC, with average of R$ 52.97 per capita, and the cluster 7 is the one that makes highest investment with average of R$ 371.32 per capita. Regarding MHC financing package, the group 2 is the one that makes the lowest investment with average of R$ 34.53 per capita, and the cluster 7 is the one that makes the highest investment with average of R$ 935.83 per capita. Regarding the HV financing package the cluster 6 is the one that makes lowest investment with average of R$ 7.45 per capita and again the cluster 7 has the highest investment with R$ 33.08 per capita. Regarding PA, the one that makes the lowest investment is the cluster 2, with average of R$ 8.41 per capita, and the one that makes the highest investment is the cluster 7 with average of R$ 48.28 per capita. Finally, regarding the SM, the group that makes the lowest investment is the cluster 7 with average of R$ 0.32 per capita and the one that makes the highest investment is the cluster 2 with average of R$ 11.80 per capita.

It is clear, from this analysis, that there is a large distance related to the financing of the municipalities from HG2. After establishing the clusters, these differences were resolved, allowing better comparison among the municipalities, as there are more homogeneous than in the situation presented by the clustering made by IDSUS.

From the large differences presented among the municipalities related to the variables used in the study (IDSUS score, population and revenue per capita for health care), and with intention to verify the impact these variables make on improving performance and system the public health care system quality, the proposed model of cost-effectiveness analysis provides consolidated data for decision making. The given relevance, already pointed from literature (Reis et al., 2012) is fundamental to the databases integration existent in the SUS.

It is observed that, in this study, data about the system performance (IDSUS, 2011) and public budget for health care (SIOPS, 2011) were concatenated, generating complementary data that can be used by managers as subsidy for planning actions for public health care. The Table 5 highlights the results of the cost-effectiveness analysis (ACE) for the variable total revenue per capita for health care.

Table 5
Cost-effectiveness Analysis for Total Revenue per capita for Health Care

Clusters	ACE (Total)	Population (IBGE, n.d.)	IDH (PNUD, 2013)	Score (IDSUS, 2011)
1	86.30	580,584	0.779	4.63
2	46.16	299,314	0.810	5.55
3	98.15	372,422	0.798	5.57
4	199.84	425,755	0.777	5.33
5	100.33	259,921	0.812	6.60
6	53.16	131,953	0.792	6.57
7	168.96	240,749	0.826	8.22

• Note. Source: Adapted from Instituto Brasileiro de Geografia e Estatística. (n.d.). Censo demográfico 2010. Retrieved February 3, 2013, from http://www.ibge.gov.br/home/estatistica/populacao/censo2010/default.shtm; Programa das Nações Unidas para o Desenvolvimento. (2013). Índice do Desenvolvimento Humano Municipal Brasileiro (Série Atlas do Desenvolvimento Humano no Brasil 2013). Brasília: Autor. Retrieved from http://www.atlasbrasil.org.br/2013/data/rawData/publicacao_atlas_municipal.pdf; Índice de Desempenho do Sistema Único de Saúde. (2011). Resultados dos indicadores do SUS nos municípios - Dados dos anos de 2008 a 2011. Retrieved June 26, 2012, from http://idsus.saude.gov.br/mapas.html

At this point, it is possible to note that the group that presents more cost-effectiveness is the 2, with total revenue of R$ 46.16 for each point reached in IDSUS score; however, it is observed that this group has the second worst IDSUS performance, with an average score of 5.55. It can be said that in this variable, it is attributed to financing the impact produced on performance, as the municipalities of the group are considered medium size and have good performance in IDHM.

Although financing is impact factor for the performance cluster 5, this does not occur in cluster 4, in which, for each point reached in IDSUS score, municipalities have an average total revenue of R$ 199.84, being understood as the cluster of least cost-effective. The factor linked to this impact may be the high population concentration, as this group is the second with most populated municipalities and presenting the worst IDHM with average of 0.777. In order to generate consolidated data and add value to the discussion, the overall analysis of the results obtained by the proposed cost-effectiveness analysis model becomes required. By extending the vision for a global field, it is possible to note that the group 5, formed by about 20% of the municipalities of the sample group (18 municipalities), presents the most consistent results.

Firstly it appears that this group has an IDSUS average, the second best score among the sample of 93 municipalities; on the other hand, health care revenues are at the third position less cost-effective, as compared to total revenue, has an average value of R$ 100.33 for each point reached in IDSUS score.

It is worth to emphasize that even with the higher revenues for health care, there is a consistency regarding the allocation of resources, which can be viewed in the variables basic care and care of medium and high complexity. From them, the use of the resource in this cluster is more efficient in relation to resource use in cluster 7 and to achieve the best score of the cluster, about 1.62 points higher than the cluster 5, it is used in both cases double of the resource value allocated to the same variables, approximately.

This finding consolidates the importance of concatenation of databases available in the public health care system in Brazil. Therefore, the study provides relevant information for the manager's decision making in order to instruct him to better resource allocation and hence, to obtain the best service levels of the indicators observed by IDSUS.

Final Considerations

The public health care system in Brazil, called the Unified Health System (SUS), completed in 2015, 25 years of existence and there is still big and massive efforts to ensure that its directives and principles become effective on all the national territory. For this, many actions of monitoring and evaluation are carried out, even fragmented, and provide an point of view of the current model, which presents fragile points but also progress.

The main objective of this study was a discussion on the situation and sought integration of valuation models, in order to provide consolidated data to managers through a proposal for cost-effectiveness evaluation model that differently from the literature presented in the theoretical reference has peculiarities. Other evaluation models exist in the context of SUS, but this model has as main distinction factor, joining of two different databases in order to provide a more consistent base for the manager decision-making.

Recently, the Ministry of Health adopted the IDSUS, formed 24 indicators and published in March 2012, presenting a picture of the Brazilian public health care system. It measures and summarizes the results of the indicators regarding the access and effectiveness of the system; and hence, it has data from other national databases already available for query. However, it is emphasized that, because it is a national indicator that evaluates municipalities according to pre-established criteria, it was considered extremely important to split the municipalities according to similar characteristics (social, economic and health care aspects), generating as result, six groups with homogeneous characteristics according to the variables used in the index methodology.

In the article, it was possible to conclude that the index englobes data that refers to clinical procedures and processes, but does not cover indicators that measure the efficiency of expenditure and the amount of resources invested in the system. Although financing of the system is a subject constantly discussed, also exposed in the literature as a system bottleneck, it is not considered in the IDSUS.

It is pointed out here a fragility of the index, as the research findings converge to the relationship of direct influence that the efficient management of resources provides to the system performance as a whole. The IDSUS, justified as a recent index incorporated in the agenda of developed measures, must be improved, including in its context data that indicate expenditures and municipal resources applied on health care. Thus, it would be a very effective tool in the analysis of how the resource is spent and how it can be better reallocated, seeking to achieve higher levels of quality during procedures and care attention to the population.

Regarding the application of a model of cost-effectiveness analysis, to relate the performance and resource variables, the paper presented an opportunity for analysis, because the one that is commonly used in the literature englobes economic variables not included in this subject. Therefore, it was justified the proposal for a different model that includes the variables established, in this case, they are the scores averages of the clustering in IDSUS and their respective total revenues and financing package, resulting in an average value of revenue for each score achieved in IDSUS, which provided the discussion with respect to the amounts invested and the impact on the municipalities performance.

Essential as far as the results, limitations of this study should be discussed as follow: the sample universe was delimited to the analysis of the municipalities of Homogeneous Group 2 with 94 municipalities in the country - so it is suggested to study all homogeneous groups. Another limitation is the use of data for the year 2011, since this year had complete data for all variables. In this context, the verification of the other subsequent years and the comparison between the relevant years would be attractive for the construction of a consolidated information database to public managers. Finally, the use of secondary data and the variables established by the researchers may have biases in the obtained results, once the particular health care conditions and the system structure cannot be observed. It is suggested a deeper study, with primary data collection of the municipalities with best and worst performance, to verify the influence of variables understood as determinant in this study.

It can be concluded too that, in addition to the need for higher allocation of resources to the system, primarily it is needed to consider the allocation of these resources in the financing packages established by the Ministry of Health, once it is concluded that municipalities that spend more resources in basic carte attention use fewer resources of the total and reach higher levels of the system performance.

In these terms, the purpose of the study was to indicate the importance of implementing new models of evaluation or the improvement of existing models and provide to the public health care managers, information linked regarding the performance and financial resources provided to the municipality health care in question. Moreover, it was intended to provide an overview of the other municipalities, which could assist in the search for better quality standards if health care and better effectiveness of public expenditure.

References

• Abrucio, F. L. (2007). Trajetória recente da gestão pública brasileira: um balanço crítico e a renovação da agenda de reformas [Edição Especial Comemorativa]. Revista de Administração Pública, 67-86. doi: 10.1590/S0034-76122007000700005
  » https://doi.org/10.1590/S0034-76122007000700005
• Alves, T., & Passador, C. S. (2011). Educação pública no Brasil: condições de oferta, nível socioeconômico dos alunos e avaliação. São Paulo: Annablume.
• Bresser-Pereira, L. C. (2010). Democracia, estado social e reforma gerencial. Revista de Administração de Empresas, 50(1), 112-116. doi: 10.1590/S0034-75902010000100009
  » https://doi.org/10.1590/S0034-75902010000100009
• Camelo, J. S., Jr., Fernandes, M. I. M., Jorge, S. M., Maciel, L. M. Z., Santos, J. L. F., Camargo, A. S., Jr., Passador, C. S., & Camelo, S. H. H. (2011). Avaliação econômica em saúde: triagem neonatal da galactosemia. Cadernos de Saúde Pública, 27(4), 666-676. doi: 10.1590/S0102-311X2011000400006
  » https://doi.org/10.1590/S0102-311X2011000400006
• Cardoso, J. C., Jr., & Jaccoud, L. (2005). Políticas sociais no Brasil: organização, abrangência e tensões da ação estatal. In L. Jaccoud (Org.), Questão social e políticas sociais no Brasil contemporâneo (pp. 181-260). Brasília, DF: IPEA.
• Conselho Nacional de Secretários de Saúde. (2011). Sistema Único de Saúde (Vol. 1) (Coleção Para Entender a Gestão do SUS). Brasília: CONASS.
• Constituição da República Federativa do Brasil de 1988. (2016). Alterações determinadas pelas Emendas Constitucionais de Revisão nos 1 a 6/94, pelas Emendas Constitucionais nos 1/92 a 91/2016 e pelo Decreto Legislativo no 186/2008 [Coordenação de Edições Técnicas]. Brasília: Senado Federal.
• Costa, F. L., & Castanhar, J. C. (2003). Avaliação de programas públicos: desafios conceituais e metodológicos. Revista de Administração Pública, 37(5), 969-992. Retrieved from http://bibliotecadigital.fgv.br/ojs/index.php/rap/article/view/6509/5093
  » http://bibliotecadigital.fgv.br/ojs/index.php/rap/article/view/6509/5093
• Dain, S. (2007). Os vários mundos do financiamento da saúde no Brasil: uma tentativa de integração. Ciência e Saúde Coletiva, 12, 1851-1864. doi: 10.1590/S1413-81232007000700008
  » https://doi.org/10.1590/S1413-81232007000700008
• Denhardt, R. B., & Denhardt, J. V. (2000). The new public service: serving rather than steering. Public Administration Review, 60(6), 549-559. doi: 10.1111/0033-3352.00117
  » https://doi.org/10.1111/0033-3352.00117
• Departamento de Atenção Básica. (n.d.). Programa Nacional de Melhoria do Acesso e da Qualidade da Atenção Básica (PMAQ). Retrieved May 26, 2015, from http://dab.saude.gov.br/portaldab/ape_pmaq.php
  » http://dab.saude.gov.br/portaldab/ape_pmaq.php
• Drummond, M. F., Stoddart, G. L., & Torrance, G. W. (1987). Methods for the economic evaluation of health care programs. Oxford: Oxford University Press.
• Elias, F. T. S., & Araújo, D. V. (2014). How health economic evaluation (HEE) contributes to decision-making in public health care: the case of Brazil. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen, 108(7), 405-412. doi: 10.1016/j.zefq.2014.08.021
  » https://doi.org/10.1016/j.zefq.2014.08.021
• Ferraz, M. B. (2010). Economia da saúde e sua inserção no sistema de saúde. In A. C. Bliacheriene & J. J. Santos (Orgs.), Direito à vida e à saúde: impactos orçamentários e judiciais (pp. 128-142). São Paulo: Atlas.
• Ferreira, V. D. R. S., Najberg, E., Ferreira, C. B., Barbosa, N. B., & Borges, C. (2014). Inovação em serviços de saúde no Brasil: análise dos casos premiados no concurso de inovação na administração pública federal. Revista de Administração Pública, 48(5), 1207-1227. doi: 10.1590/0034-76121602
  » https://doi.org/10.1590/0034-76121602
• Finkelman, J. (Org.). (2002). Caminhos da saúde pública no Brasil. Rio de Janeiro: Fiocruz.
• Fleury, S., & Ouverney, A. M. (2008). Política de saúde: uma política social. In L. Giovanella (Org.), Políticas e sistema de saúde no Brasil (pp. 23-64). Rio de Janeiro: Fiocruz .
• Forte, L. M., & Nobre, F. C. (2014). Avaliação do índice de desempenho do SUS (IDSUS) nos municípios do RN com vistas à definição de estratégias de gestão. Revista de Gestão em Sistemas de Saúde, 3(1), 82-93. doi: 10.5585/rgss.v3i1.114
  » https://doi.org/10.5585/rgss.v3i1.114
• Gonçalves, S. (2014). The effects of participatory budgeting on municipal expenditures and infant mortality in Brazil. World Development, 53, 94-110. doi: 10.1016/j.worlddev.2013.01.009
  » https://doi.org/10.1016/j.worlddev.2013.01.009
• Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Análise multivariada de dados. Porto Alegre: Bookman.
• Índice de Desempenho do Sistema Único de Saúde. (2011). Resultados dos indicadores do SUS nos municípios - Dados dos anos de 2008 a 2011. Retrieved June 26, 2012, from http://idsus.saude.gov.br/mapas.html
  » http://idsus.saude.gov.br/mapas.html
• Instituto Brasileiro de Geografia e Estatística. (n.d.). Censo demográfico 2010. Retrieved February 3, 2013, from http://www.ibge.gov.br/home/estatistica/populacao/censo2010/default.shtm
  » http://www.ibge.gov.br/home/estatistica/populacao/censo2010/default.shtm
• Macêdo, F. F. R. R., & Damasceno, M. M. J. (2013). Análise de desempenho do sistema único de saúde na região Nordeste do Brasil. Revista de Contabilidade do Mestrado em Ciências Contábeis da UERJ, 18(3), 20-36.
• Marins, M., & Daher, D. V. (2014). O programa nacional de melhoria do acesso e qualidade da atenção básica (PMAQ-AB): relato de experiência. Revista Enfermagem Profissional, 1(2), 331-337. Retrieved from http://www.seer.unirio.br/index.php/enfermagemprofissional/article/view/3720/pdf_1414
  » http://www.seer.unirio.br/index.php/enfermagemprofissional/article/view/3720/pdf_1414
• Medeiros, K. R., Albuquerque, P. C., Diniz, G. T. N., Alencar, F. L., & Tavares, R. A. W. (2014). Bases de dados orçamentários e qualidade da informação: uma avaliação do Finanças do Brasil (Finbra) e do Sistema de Informações sobre Orçamentos Públicos em Saúde (Siops). Revista de Administração Pública, 48(5), 1113-1133. doi: 10.1590/0034-76121483
  » https://doi.org/10.1590/0034-76121483
• Ministério da Saúde. (2008). Avaliação econômica em saúde: desafios para a gestão no Sistema Único de Saúde [Série A. Normas e Manuais Técnicos]. Brasília, DF: Editora MS. Retrieved from http://bibspi.planejamento.gov.br/bitstream/handle/iditem/152/livro_aval_econom_saude.pdf?sequence=1
  » http://bibspi.planejamento.gov.br/bitstream/handle/iditem/152/livro_aval_econom_saude.pdf?sequence=1
• Ministério da Saúde. (2011). Sistema de informações sobre orçamentos públicos em saúde. Retrieved February 12, 2013, from http://siops.datasus.gov.br/recdespinfomuncont.php?S=225&UF=12;&Municipio=-1&Item=-1&Fase=35;&Pasta=17;&Tipo=R&Ano=2011&Periodo=12&Ordenacao=Codigo
  » http://siops.datasus.gov.br/recdespinfomuncont.php?S=225&UF=12;&Municipio=-1&Item=-1&Fase=35;&Pasta=17;&Tipo=R&Ano=2011&Periodo=12&Ordenacao=Codigo
• Motta, P. R. D. M. (2013). O estado da arte da gestão pública. Revista de Administração de Empresas, 53(1), 82-90. doi: 10.1590/S0034-75902013000100008
  » https://doi.org/10.1590/S0034-75902013000100008
• Murray, C. J., Richards, M. A., Newton, J. N., Fenton, K. A., Anderson, H. R., Atkinson, C., Bennett, D., Bernabé, E., Blencowe, H., Bourne, R., Braithwaite, T., Brayne, C., Bruce, N. G., Brugha, T. S., Burney, P., Dherani, M., Dolk, H., Edmond, K., Ezzati, M., Flaxman, A. D., Fleming, T. D., Freedman, G., Gunnell, D., Hay, R. J., Hutchings, S. J., Ohno, S. L., Lozano, R., Lyons, R. A., Marcenes, W., Naghavi, M., Newton, C. R., Pearce, N., Pope, D., Rushton, L., Salomon, J. A., Shibuya, K., Vos, T., Wang, H., Williams, H. C., Woolf, A. D., Lopez, A. D., & Davis, A. (2013). UK health performance: findings of the Global Burden of Disease Study 2010. The Lancet, 381(9871), 997-1020. doi: 10.1016/S0140-6736(13)60355-4
  » https://doi.org/10.1016/S0140-6736(13)60355-4
• Noronha, J. C., Lima, L. D., & Machado, C. V. (2008). O Sistema Único de Saúde - SUS. In L. Giovanella (Org.), Políticas e sistema de saúde no Brasil (pp. 365-394). Rio de Janeiro: Fiocruz .
• Paim, J. S., Travassos, C., Almeida, C., Bahia, L., & Macinko, J. (2011). O sistema de saúde brasileiro: história, avanços e desafios. The Lancet, 377(9779), 11-31. doi: 10.1016/S0140-6736(11)60054-8
• Pessoto, U. C., Ribeiro, E. A. W., & Guimarães, R. B. (2015). O papel do Estado nas políticas públicas de saúde: um panorama sobre o debate do conceito de Estado e o caso brasileiro. Saúde Sociedade, 24(1), 9-22. doi: 10.1590/S0104-12902015000100001
  » https://doi.org/10.1590/S0104-12902015000100001
• Pires, M. R. G. M., & Demo, P. (2006). Políticas de saúde e crise do Estado de Bem-Estar: repercussões e possibilidades para o Sistema Único de Saúde. Saúde e Sociedade, 15(2), 56-71. doi: 10.1590/S0104-12902006000200007
  » 10.1590/S0104-12902006000200007
• Pollitt, C. (2003). Joined‐up government: a survey. Political Studies Review, 1(1), 34-49. doi: 0.1111/1478-9299.00004
• Portal Brasil. (2009, October 31). SUS democratiza o acesso do cidadão aos serviços de saúde. Retrieved February 14, 2013, from http://www.brasil.gov.br/sobre/saude/atendimento/sus
  » http://www.brasil.gov.br/sobre/saude/atendimento/sus
• Portaria n. 399, de 22 de fevereiro de 2006. (2006). Divulga o pacto pela saúde 2006 - Consolidação do SUS e aprova as diretrizes operacionais do referido pacto. Retrieved from http://bvsms.saude.gov.br/bvs/saudelegis/gm/2006/prt0399_22_02_2006.html
  » http://bvsms.saude.gov.br/bvs/saudelegis/gm/2006/prt0399_22_02_2006.html
• Programa das Nações Unidas para o Desenvolvimento. (2013). Índice do Desenvolvimento Humano Municipal Brasileiro (Série Atlas do Desenvolvimento Humano no Brasil 2013). Brasília: Autor. Retrieved from http://www.atlasbrasil.org.br/2013/data/rawData/publicacao_atlas_municipal.pdf
  » http://www.atlasbrasil.org.br/2013/data/rawData/publicacao_atlas_municipal.pdf
• Reis, A. T., Oliveira, P. T. R., & Sellera, P. E. (2012). Sistema de avaliação para a qualificação do Sistema Único de Saúde (SUS). Revista Eletrônica de Comunicação, Informação e Inovação em Saúde, 6(2). doi: 10.3395/reciis.v6i2.543
  » https://doi.org/10.3395/reciis.v6i2.543
• Ribeiro, R. A., & Polanczyk, C. A. (2005). Avaliação de tecnologia em saúde: entendendo as fronteiras dos ensaios clínicos e metanálises. Revista da Sociedade de Cardiologia do Rio Grande do Sul, 14(6), 32-35. Retrieved from http://sociedades.cardiol.br/sbc-rs/revista/2005/06/Artigo10.pdf
  » http://sociedades.cardiol.br/sbc-rs/revista/2005/06/Artigo10.pdf
• Santos, L. M., Francisco, J. R. S., Faria, A. M. B., & Gonçalves, M. A. (2011, setembro). Controle por resultados na alocação de recursos na atenção básica em saúde das microrregiões do Sudeste do Brasil. Anais do Encontro Nacional da Associação Nacional de Pós-Graduação e Pesquisa em Administração, Rio de Janeiro, RJ, Brasil, 35 .
• Saravia, E. (2006). Introdução à teoria da política pública. In E. Saravia & E. Ferrarezi (Orgs.), Políticas públicas - Coletânea, volume 1 (pp. 21-42). Brasília, DF: ENPAP.
• Servo, L. M. S., Piola, S. F., Paiva, A. B., & Ribeiro, J. A. (2011). Financiamento e gasto público de saúde: histórico e tendências. In C. Melamed & S. F. Piola (Orgs.), Políticas públicas e financiamento federal do Sistema Único de Saúde (pp. 85-108). Brasília, DF: IPEA .
• Tanaka, O. Y., Drumond, M. Jr., Cristo, E. B., Spedo, S. M., & Pinto, N. R. D. S. (2015). Uso da análise de clusters como ferramenta de apoio à gestão no SUS. Saúde e Sociedade, 24(1), 34-45. doi: 10.1590/S0104-12902015000100003
  » https://doi.org/10.1590/S0104-12902015000100003
• Ugá, M. A., & Marques, R. M. (2005). O financiamento do SUS: trajetória, contexto e constrangimentos. In N. T. Lima, S. Gerschman, F. C. Edler, & J. M. Suárez (Orgs.), Saúde e democracia: história e perspectivas do SUS (pp. 193-233). Rio de Janeiro: Fiocruz .
• Vanni, T., Luz, P. M., Ribeiro, R. A., Novaes, H. M. D., & Polanczyk, C. A. (2009). Avaliação econômica em saúde: aplicações em doenças infecciosas. Cadernos de Saúde Pública, 25(12), 2543-2552. doi: 10.1590/S0102-311X2009001200002
  » https://doi.org/10.1590/S0102-311X2009001200002
• Viacava, F., Almeida, C., Caetano, R., Fausto, M., Macinko, J., Martins, M., Noronha, J. C. de, Novaes, H. M. D., Oliveira, E. dos S., Porto, S. M., Silva, L. M. V. da, & Szwarcwald, C. L. (2004). Uma metodologia de avaliação do desempenho do sistema de saúde brasileiro. Ciência & Saúde Coletiva, 9(3), 711-724. doi: 10.1590/S1413-81232004000300021
  » https://doi.org/10.1590/S1413-81232004000300021

Publication Dates

• Publication in this collection
  2016

History

• Received
  22 Dec 2014
• Reviewed
  10 Sept 2015
• Accepted
  11 Sept 2015