Acessibilidade / Reportar erro

The effects of time on the performance of local government: the “Pact for Health Care” Program

Abstract

This study evaluated the impact of the time a Brazilian local government stays as member of the program “Pacto pela Saúde” (Pact for Health) - by signing a Management Agreement -, and its efficiency to provide primary health care for the population. The research observed the period from 2008 to 2012. The program was an initiative of the Federal Government operated by municipalities through the Management Agreement and aimed to improve healthcare policy management adopting a results-based managerial approach. The program was in place between 2006 and 2012 and was operated by 4,587 local governments (80 percent of the Brazilian municipalities). The research question guiding the study was ‘What was the effect of the time of a local government in the program (in years) on the efficiency of health care delivery to local populations? A quasi-experimental research design was adopted, estimating a dose-response model with generalized propensity score. An efficiency indicator of the primary care policy (IDEAB) was estimated via principal component analysis, based on the targets recommended by the program. The results of the dose-response model showed that the number of years in the Management Agreement had a positive and statistically significant impact on the efficiency of health care delivery in participating municipalities. For each additional year in the agreement, IDEAB increased by an average of 0.011 to 0.019 units. Therefore, the results suggest that establishing targets are important for the governance of the Brazilian health care policy.

Keywords:
Pacto pela Saúde; health care policy impact assessment; municipal primary health care policy; dose-response model with generalized propensity score

Resumo

Este estudo avaliou o impacto do tempo de adesão ao termo de compromisso de gestão (TCG), no âmbito do programa Pacto pela Saúde, sobre o nível de eficácia da política municipal de atenção básica, no período de 2008 a 2012. O TCG objetivou aprimorar a governança de política de saúde pelos entes federados, com especial atenção à gestão por resultados. O programa vigorou no Brasil entre 2006 e 2012, tendo recebido a adesão de 4.587 municípios (80% do total). Esta pesquisa buscou responder à seguinte questão: “qual foi o efeito causal do tempo de participação (em anos) no programa Pacto pela Saúde sobre o nível da eficácia da política local de atenção básica, para os municípios participantes?”. Para tanto, adotou-se um desenho de pesquisa quase experimental, mediante estimação de um modelo de dose-resposta com escore de propensão generalizado. Estimou-se, via análise de componentes principais, um indicador de eficácia da política de atenção básica (IDEAB), tendo como referência as metas preconizadas pelo programa. Os resultados da estimação da função de dose-resposta evidenciaram que o tempo de adesão ao Pacto pela Saúde teve impacto positivo e estatisticamente significativo sobre o nível de eficácia das políticas de atenção básica nos municípios participantes. Para cada ano adicional de permanência da política, o IDEAB aumentou, em média, entre 0,011 e 0,019 unidades. Portanto, os resultados sugerem que as metas importam para a governança de política de saúde municipal brasileira.

Palavras-chave:
Pacto pela Saúde; avaliação de impacto de políticas de saúde; política de atenção básica municipal; modelo de dose-resposta com escore de propensão generalizado

Resumen

El presente estudio evaluó el impacto del tiempo de membresía al Término de Compromiso de Gestión (TCG) sobre el nivel de efectividad de la política municipal de salud en Brasil, de 2008 a 2012. El TCG fue parte del programa Pacto por la Salud, y tenía como objetivo mejorar la gobernanza de la política de salud por parte de los estados federados, con especial atención a la gestión basada en resultados. El programa se ejecutó en Brasil entre 2006 y 2012, y fue adoptado por 4.587 municipios (80 por ciento del total). Esta investigación buscó responder a la siguiente pregunta: ¿Cuál fue el efecto causal del tiempo de participación en el programa (en número de años) sobre la efectividad de la política de atención primaria para los municipios participantes? Para ello, se adoptó un diseño de investigación cuasiexperimental, estimando un modelo de dosis-respuesta con puntaje de propensión generalizada. Se estimó un indicador de efectividad de la política de atención primaria (IDEAB) a través del análisis de componentes principales, con base en los objetivos recomendados por el programa. Los resultados de la estimación de la función dosis-respuesta mostraron que el número de años en el programa Pacto por la Salud tuvo un impacto positivo y estadísticamente significativo en el indicador de efectividad de la política de atención primaria para los municipios participantes. Por cada año adicional en la política, el IDEAB aumentó en un promedio de 0.011 a 0.019 unidades. Por lo tanto, los resultados sugieren que los objetivos son importantes para la gobernanza de la política de salud municipal brasileña.

Palabras clave:
Pacto por la Salud; evaluación de impacto de las políticas de salud; política municipal de atención primaria; modelo de dosis-respuesta con puntaje de propensión generalizado

1. INTRODUCTION

The improvement of Brazilian public health policy, with the aim of making it more efficient and effective, can be considered one of the main challenges of the Unified Health System (Sistema Único de Saúde - SUS) today (National Council of Secretaries of Health [CONASS], 2015Conselho Nacional de Secretários da Saúde. (2015). A Gestão do SUS (Para Entender a Gestão do SUS). Recuperado dehttp://www.conass.org.br/biblioteca/pdf/A-GESTAO-DO-SUS.pdf
http://www.conass.org.br/biblioteca/pdf/...
; Noronha, Lima, & Machado, 2012Noronha, J. C., Lima, J. D., & Machado, C. V. (2012). O Sistema Único de Saúde (SUS). In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 365-393) Rio de Janeiro, RJ: Ed. Fiocruz .; Ocké-Reis, 2012Ocké-Reis, C. O. (2012). SUS: o desafio de ser único. Rio de Janeiro, RJ: Ed. Fiocruz .). In order to achieve efficiency and effectiveness, the Ministry of Health (MH) has coordinated two major movements regarding the National Health Policy in recent years: a) the decentralization of resources to federal entities; and b) the focus on primary care (Law no. 8,142, 1990; MS Ordinance no. 399, 2006; MS Ordinance no. 2,488, 2011).Portaria MS n. 2.488, de 21 de outubro de 2011. (2011). Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes e normas para a organização da Atenção Básica, para a Estratégia Saúde da Família (ESF) e o Programa de Agentes Comunitários de Saúde (PACS). Brasília, DF. These two movements are in line with international recommendations, which emphasize the role of health production technology focused on primary care and the governance of health care policy (Lorenzoni, Murtin, Springare, Auraeen, & Daniel, 2018Lorenzoni, L., Murtin, F., Springare, L. S., Auraeen, A., & Daniel, F., (2018, April). Which policies increase value for money in health care? (OECD Health Working Papers, No. 104). Paris, France: OECD.; Organisation For Economic Co-Operation And Development [OECD], 2010Organisation For Economic Co-Operation And Development. (2010). Value for money in health spending (OECD Health Policy Studies). Paris, France: Autor .; World Health Organization [WHO], 2008World Health Organization. (2008). The world health report 2008: primary health care (now more than ever). Geneva, Switzerland: Autor .).

Regarding the decentralization of resources, the 1996 basic operational standard established by SUS (MS Ordinance No. 2.203, 1996Portaria MS n. 2.203, de 5 de novembro de 1996. (1996). Norma Operacional Básica do SUS. Brasília, DF.) assigned greater responsibility to municipalities in the provision of health services, and increased direct transfers of resources from the federal government (Conass, 2015Conselho Nacional de Secretários da Saúde. (2015). A Gestão do SUS (Para Entender a Gestão do SUS). Recuperado dehttp://www.conass.org.br/biblioteca/pdf/A-GESTAO-DO-SUS.pdf
http://www.conass.org.br/biblioteca/pdf/...
). In addition, after the standard’s establishment, municipalities mobilized more of their resources for health (Piola, Paiva, Sá, & Servo, 2013Piola, S. F., Paiva, A. B., Sá, E. B., Servo, L. M. S., (2013, julho). Financiamento público da saúde: uma história à procura de rumo (Texto para Discussão IPEA, n. 1846). Brasília, DF: Instituto de Pesquisa Econômica Aplicada.)1 1 It should be mentioned that Constitutional Amendment No. 29 (2000), which made it possible to combine the revenues of municipalities (15%), states and the Federal District (12%) and the federal government (the amount allocated for health in the previous year plus the nominal variation of gross domestic product [GDP]) for the health sector, has contributed to increasing the entities’ participation in health expenditures. . If, on the one hand, greater protagonism of municipalities in conducting health policy through the decentralization of SUS meant closer proximity to the preferences and reality of local health conditions and demands, on the other hand, it raised concerns regarding the technical capacity and transparency in running and supervising health policy, especially in less populous municipalities, where supposedly there would be less technical training at work (Saltman, Bankauskaite, & Vrangbaek, 2007Saltman, R. B., Bankauskaite, V., Vrangbaek, K., (Orgs.). (2007). Decentralization in health care: strategies and outcomes (European Observatory on Health Systems and Policies Series). New York, NY: McGraw-Hill.; Vo, 2010Vo, D. H. (2010). The economics of fiscal decentralization. Journal of Economic Surveys, 24(4), 657-679.).

The second recent movement in Brazil’s health policies concerns the emphasis on primary care2 2 By definition, primary care, also known as primary health care (PHC), comprises an individual’s first contact with health care services (first level outpatient care), which should be easily accessible and cover most common diseases and conditions, as well as immunizations. Moreover, it consists of a health care model that emphasizes family health and health promotion (Giovanella & Mendonça, 2012). . On this pillar, Family Health Strategy (FHS) stands out as the priority strategy to organize primary care under SUS (MS Ordinance no. 399, 2006Portaria MS n. 399, de 22 de fevereiro de 2006. (2006). Divulga o Pacto pela Saúde 2006 - Consolidação do SUS e aprova as Diretrizes Operacionais do Referido Pacto. Brasília, DF.). It has made primary care through SUS comprehensive, thus becoming the center of the health care model. FHS was so quickly assimilated and adopted by Brazilian municipalities that in 1994 there were 150 teams throughout the country, and by 2014 there were over 37,000 of them (Ministry of Health [MS], 2016Ministério da Saúde. (2016). Sistema de Informações em Saúde. Recuperado dehttp://datasus.saude.gov.br/informacoes-de-saude/tabnet
http://datasus.saude.gov.br/informacoes-...
). However, the literature reports that the way these teams conducted the implementation of FHS was quite heterogeneous in providing disease prevention and health promotion services, which motivated a broad national debate to increase the scope of primary care in municipalities (Giovanella & Mendonça, 2012Giovanella, L., & Mendonça, M. H. M. (2012). Atenção primária à saúde. In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 493-545). Rio de Janeiro, RJ: Ed. Fiocruz .).

Bearing the two movements (decentralization of resources and primary care focus) as mottos, the Health Pact program was launched in Brazil in 2006 (MS Ordinance no. 399, 2006Ministério da Saúde. (2016). Sistema de Informações em Saúde. Recuperado dehttp://datasus.saude.gov.br/informacoes-de-saude/tabnet
http://datasus.saude.gov.br/informacoes-...
). It aimed to improve the governance of health policy by the federated entities, with special attention to results-based management. Each municipality had to materialize the pact by signing a management commitment term (MCT). The idea behind its signature was to set up incentive and accountability mechanisms for municipalities to improve their provision of primary care services by defining goals based on nationally established health indicators. (MS, 2014Ministério da Saúde. (2014). Caderno de diretrizes, objetivos, metas e indicadores: 2013-2015 (2a ed.). Brasília, DF: Autor.).

The signing of the MCT under the Health Pact program was in force in Brazil from 2006 to 2012, and gathered 4,587 municipalities (about 80% of them). Despite the program’s potential to improve basic health care policy management practices, no studies were found in the literature that evaluated its impact on the participating municipalities. Studies on impact assessment of health policies in Brazil tend to focus on three main areas: a) decentralization (Rocha, Orellano, & Nishijima, 2016Rocha, F. F., Orellano, V., & Nishijima, M. (2016). Health spending autonomy and infant mortality rates: a matter of local administrative capacity?. The Journal of Developing Areas, 50, 293-309.; Santos, Nascimento, & Camara, 2017Santos, R. G., Nascimento, S.P., & Camara, M. R. G.. (2017). The influence of the pact for health on the expenditure on the financing of public health on the view of the Brazilian State. Planejamento e Políticas Públicas, 49, 109-131.); b) social determinants of health (Rocha, Nishijima, & Peixoto, 2013; Soares, 2007Soares, R. (2007). Health and the evolution of welfare across Brazilian municipalities. Journal of Development Economics, 84(2), 590-608.); and c) health interventions (Hone, Rasella, Barreto, Atun, Majeed, & Millett, 2017Hone, T., Rasella, D., Barreto, M., Atun, R., Majeed, A. & Millett, C. (2017). Large reductions in amenable mortality associated with Brazil’s primary care expansion and strong health governance. Health Affairs, 36(1), 149-158.; Rocha & Soares, 2010).

In view of this gap, the purpose of this article is:

  • To estimate the effect of program participation time (in years) on the level of effectiveness of local primary health care policy among participating municipalities between 2008 and 2012.

This time cutoff was due to the availability of data by the MS Health Information System (2016Ministério da Saúde. (2016). Sistema de Informações em Saúde. Recuperado dehttp://datasus.saude.gov.br/informacoes-de-saude/tabnet
http://datasus.saude.gov.br/informacoes-...
). To answer this question, we used a quasi-experimental research design, allowing to control for selection bias, considering that participation in the policy was non-random.

The methodological procedure of this study consisted of two stages. First, the primary care effectiveness indicator (PCEI) was estimated through a multivariate analysis of the main components. The variables that composed the synthetic indicator were selected based on the agreement of goals between the federal government and municipalities. In the second stage, dose-response models were estimated using generalized propensity score (Imbens, 2000Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87 (3), 706-710.; Lechner, 2001Lechner, M. (2001). Identification and estimation of causal effects of multiple treatments under conditional Independence assumption. In M. Lechner, & F. Pfeiffer(Eds.), Econometric evaluation of labour market policies(ZEW Economic Studies, 13, pp. 43-58). Heidelberg, Germany: Physica.), which made it possible to identify to what extent program participation time (in years) impacted the PCEI. Because of its a quasi-experimental design, the dose-response model minimizes confounding effects resulting from non-random assignment of the policy, reducing selection bias in estimating the causal impact (Imai & Van Dyk, 2004Imai, K., & Van Dyk, D. A. (2004). Causal inference with general treatment regimes: generalizing the propensity score. Journal of the American Statistical Association, 99(467), 854-866.; Murname & Willet, 2011Murname, R. J., & Willet, J. B. (2011). Methods matter: improving causal inference in educational and social science research. Oxford, England: Oxford University Press.).

This article is divided into 5 sections, including this introduction. Section 2 addresses the literature on primary care and health policy governance agenda, following recommendations from international organizations and economic rationality. This section also presents characteristics of the Health Pact program. Section 3 describes the methodology of the study, both in terms of the database compiled by the authors and the methods used. The results, which were analyzed in light of the adherence of municipalities to the policy, the presence of selection bias and the econometric models of impact assessment, are presented in section 4. Finally, section 5 summarizes the findings and contributions of the study, and it also suggests a policy agenda.

2. LITERATURE

2.1 Contemporary health policy agenda: primary care and governance

Health policy, in general, is associated with the organization and coordination of national health systems (NHS). According to Fleury and Ouverney (2012Fleury, S., & Ouverney, A. M. (2012). A política de saúde: uma política social. In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 25-57). Rio de Janeiro, RJ: Ed. Fiocruz.), NHSs comprise the set of actions and health services provided to the population, according to their needs, with the perspective of reaching goals, objectives or principles. Within this scope, NHS policy covers 4 elements a) resource allocation; b) provision of health services; c) management; and d) regulation. Among the four, service delivery (health promotion, especially) and emphasis on the governance (management) of health systems stand out the most in contemporary health policy (Tulchinsky & Varikova, 2010Tulchinsky, T. H., & Varikova, E. A. (2010). What is the “New Public Health”? Public Health Reviews, 32(1), 25-53.; WHO, 2000World Health Organization. (2000). The world health report 2000: health systems - improving performance. Geneva, Switzerland: Autor., 2008World Health Organization. (2008). The world health report 2008: primary health care (now more than ever). Geneva, Switzerland: Autor .).

As for the provision of health services, primary care is considered the preferred health care model in NHS framework. It is argued that it presents greater resolution capacity and lower cost when compared to medium and high complexity assistance (WHO, 2008World Health Organization. (2008). The world health report 2008: primary health care (now more than ever). Geneva, Switzerland: Autor .).

The effectiveness of primary care policies is primarily due to the fact that it reduces the demand for care in health facilities: since this model fosters health promotion, individuals receive better guidance concerning health care and health education, which makes them more able to take care of their own health and that of their family. Thus, by offering a service that is closer to the community’s imminent needs, primary care delivers a more adequate service to the user, increasing its resolution and minimizing the burden of hospital systems and medium and high complexity services (Starfield, 2002Starfield, B. (2002). Atenção primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasília, Brasil: Unesco.).

Another factor that justifies the effectiveness of primary care policies is the emphasis on general practitioner care (also called “family doctors”). These professionals monitor the individuals of a given community (or neighborhood) over time, which allows them to gain better knowledge of their reality and health history. General practitioners usually prescribe the most appropriate medications and curative or preventive procedures, avoiding duplication and unnecessary procedures, such as exams, drugs and surgeries, increasing resolution and reducing costs (Giovanella & Mendonça, 2012Giovanella, L., & Mendonça, M. H. M. (2012). Atenção primária à saúde. In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 493-545). Rio de Janeiro, RJ: Ed. Fiocruz .; Starfield & Shi, 2002Starfield, B. (2002). Atenção primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasília, Brasil: Unesco.).

In summary, the main mechanisms by which primary care improves community health are: a) more and better education for families about risks, behaviors and primary health care; b) prevention or early detection of diseases; c) community engagement in immunization campaigns; and d) longitudinal monitoring of families, allowing more time for contact and greater knowledge of their health issues (Riley, 2007Riley, J. C. (2007). Low income, social growth, and good health: a history of twelve countries. Berkeley, CA: University of California Press.).

Some empirical studies have sought to test the efficiency and effectiveness of the primary care model. Starfield and Shi (2002Starfield, B. (2002). Atenção primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasília, Brasil: Unesco.) assessed the primary care performance of 13 industrialized countries, in terms of health outcomes and operating costs, and classified their NHS into 3 levels of emphasis on primary care (weak, moderate and strong). The authors found that the greater the emphasis on primary care, the lower the cost of the health system. In addition, countries with weaker primary care infrastructure have worse health outcomes in terms of child mortality rate, birth weight and life expectancy loss due to illness and disability. The authors also point out that a minimum level of investment on health is required in order for the primary care policy to achieve satisfactory results.

In terms of Brazil, the evidence also confirms the primary care high resolution hypothesis. Hone et al. (2017Hone, T., Rasella, D., Barreto, M., Atun, R., Majeed, A. & Millett, C. (2017). Large reductions in amenable mortality associated with Brazil’s primary care expansion and strong health governance. Health Affairs, 36(1), 149-158.) analyzed the impact of the FHS teams’ coverage rate and of a synthetic health governance indicator on primary-care-sensitive mortality rates in 1,622 Brazilian municipalities. Based on a quasi-experimental regression method with fixed effects for municipalities, the authors showed that between 2000 and 2012, the FHS showed positive results in reducing primary-care-sensitive mortality for municipalities with good levels of health governance.

Regarding the second central aspect of contemporary health policy, there is the governance of health systems, which advocates for the importance of analyzing cost-effectiveness of health actions (WHO, 2000World Health Organization. (2000). The world health report 2000: health systems - improving performance. Geneva, Switzerland: Autor.). The World Health Organization (WHO) believes that involving the coordination and supervision of all NHS functions, improvements in health governance have direct and indirect effects on health outcomes. As a direct effect, health promotion and care services are better supplied. Indirect effects include an increase in health professionals’ productivity, immunization coverage and the implementation of intersectoral policies.

Empirical studies have assessed the impact of health governance components on policy outcomes: it is clear that health expenditure is only associated with better outcomes in countries with good governance, i.e., where budget formulation and policy implementation and monitoring are effective (Lorenzoni et al. 2018Lorenzoni, L., Murtin, F., Springare, L. S., Auraeen, A., & Daniel, F., (2018, April). Which policies increase value for money in health care? (OECD Health Working Papers, No. 104). Paris, France: OECD.; Rajkumar & Swaroop, 2008Rajkumar, A. S., & Swaroop, V. (2008). Public spending and outcomes: does governance matter? Journal of Development Economics, 86, 96-111.), and also those where health expenditure rationalization does not compromise service delivery (OECD, 2015).

The following section explores how the Health Pact program articulated these two core elements in Brazil’s health policy agenda, with an emphasis on primary care and health governance.

2.2 Health policy in Brazil and the Health Pact program

Brazilian health policy, since the implementation of SUS in 1990, has been set up by means of operational norms aimed at organizing the national health system. As seen in the previous section, the decentralization of health service delivery, a fundamental pillar of the new health policy, occurred slowly via operational norms and ministerial ordinances (Ouverney, 2014Ouverney, A. M. (2014). A construção da municipalização da saúde: estratégias normativas. In S. Fleury, Democracia e inovação na gestão local de saúde (pp. 81-120). Rio de Janeiro, RJ: Ed. Fiocruz .). According to Conass (2015), these norms defined competencies and conditions for each entity, so they could take on new attributions within SUS and qualify to receive funds from the Federal Government.

Between 1990 and 2016, 7 regulations were issued, of which 4 were basic operational norms (1991, 1992, 1993 and 1996), 2 health care operational norms (2001 and 2002) and the Health Pact (2006). Amidst these, the 1996 basic operational norm for SUS (MS Ordinance no. 2,203, 1996Portaria MS n. 2.203, de 5 de novembro de 1996. (1996). Norma Operacional Básica do SUS. Brasília, DF.) was the one that represented a leap towards the process of municipalization, by creating new management conditions for municipalities, enabling the less populous ones to receive regular transfers of resources from the Federal Government (Conass, 2015Conselho Nacional de Secretários da Saúde. (2015). A Gestão do SUS (Para Entender a Gestão do SUS). Recuperado dehttp://www.conass.org.br/biblioteca/pdf/A-GESTAO-DO-SUS.pdf
http://www.conass.org.br/biblioteca/pdf/...
).

Recently, the Health Pact and its update by the National Policy of Primary Care (updated by MS Ordinance No. 2,436, 2017Portaria MS n. 2.436, de 21 de setembro de 2017. (2017). Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes para a organização da Atenção Básica, no âmbito do Sistema Único de Saúde (SUS). Brasília, DF.), consolidated primary care as a health care model in the country, incorporating policy intersectoriality, health system management and planning by subnational entities, and connecting it to the regionalization of service provision (Conass, 2015Conselho Nacional de Secretários da Saúde. (2015). A Gestão do SUS (Para Entender a Gestão do SUS). Recuperado dehttp://www.conass.org.br/biblioteca/pdf/A-GESTAO-DO-SUS.pdf
http://www.conass.org.br/biblioteca/pdf/...
).

Specifically, in relation to the Health Pact, two main innovations stand out. The first refers to the movement for results-based management within SUS. Municipalities began to commit to certain health goals that were signed and sealed by the entities. According to Conass (2015Conselho Nacional de Secretários da Saúde. (2015). A Gestão do SUS (Para Entender a Gestão do SUS). Recuperado dehttp://www.conass.org.br/biblioteca/pdf/A-GESTAO-DO-SUS.pdf
http://www.conass.org.br/biblioteca/pdf/...
), the Health Pact improved the demand for more efficient health system management by the municipalities, which are now considered health managers responsible for providing primary care.

The second innovation proposed by the Health Pact referred to the abolition of an old document - called “management conditions” - as an instrument to entitle the municipality for health policy. This document was replaced by the MCT, which sealed the Health Pact between the federal government, states and municipalities, listing the responsibilities and attributions inherent to governmental spheres in conducting and managing the health system. The MCT contemplated objectives, goals, and monitoring and assessment indicators, established based on national and state priorities, negotiated among the entities, in the various instances of SUS alliance. Based on these priorities, municipalities developed their own goals, considering their health situation. The goals were then approved by the respective municipal health councils and later included in the municipal health plans. The MCT was in force between 2006 and 2012, and gathered 4,587 municipalities during that period (82.43% of Brazilian municipalities at the time). It should be noted, however, that the policy did not foresee any accountability mechanism for meeting goals and indicators.

Following Decree no. 7.508 (2011Decreto n. 7.508, de 28 de junho de 2011. (2011). Regulamenta a Lei n. 8.080, de 19 de setembro de 1990, para dispor sobre a organização do Sistema Único de Saúde - SUS, o planejamento da saúde, a assistência à saúde e a articulação interfederativa, e dá outras providências. Brasília, DF.), a new term format was established for health policy governance, the Organizational Contract for Public Health Action (OCPHA), which preserved the general characteristics of the MCT, represented by the sealed goals and objectives. Adherence to OCPHA, however, was mandatory for all municipalities. In addition, OCPHA placed greater emphasis on regional planning, expanding the role of health regions and creating regional inter-managerial committees to support this planning. Since it is a universal policy, OCPHA was not the object of this study.

Having presented the Health Pact program, the next section discusses the methodology of this study in terms of the data, variables and methods employed.

3. METHODOLOGY

3.1 Data

Data on municipalities’ participation in the Health Pact program were obtained by the authors upon request to the Tripartite Intergovernmental Commission of the Ministry of Health (2016Ministério da Saúde. (2016). Sistema de Informações em Saúde. Recuperado dehttp://datasus.saude.gov.br/informacoes-de-saude/tabnet
http://datasus.saude.gov.br/informacoes-...
), who sent the list of municipalities that signed the MCT per year. It should be noted that once a municipality had joined the program, there was no possibility of withdrawal from it, therefore, the time period between signing the term of adherence and the end of the program gives us a precise measure of its exposure.

For a panel analysis of municipalities, it is important to establish a standard territorial division, given that in the period under study (2006-2012), new municipalities were created from the dismemberment of others. To this end, the official Brazilian territorial division of the year 2010 (according to the Brazilian Institute of Geography and Statistics [IBGE]) was used as the basis for this study, making municipalities in the analyzed period compatible. Based on our compatibility, by 2012, 4,587 municipalities had joined the Health Pact program by signing the MCT3 3 The list of municipalities included in the analysis can be requested by email. .

The data for the construction of the PCEI were extracted from the MS Health Information System (2016), under the Sealed Objectives and Goals section (MS, 2014Portaria MS n. 2.203, de 5 de novembro de 1996. (1996). Norma Operacional Básica do SUS. Brasília, DF.). The system provides information for the period from 2008 to 2012, which is therefore the period of empirical analysis.

To estimate the causal effect of program participation time on PCEI, selection bias control was required, otherwise our estimates would not be reliable. Thus, we used explanatory variables based on sociodemographic information collected in the 2000 and 2010 Demographic Census, which were compiled in the Atlas of Municipal Human Development (UNDP, FJP and IPEA). In addition, we used a measure of the annual municipal economic activity provided by IBGE. The next subsection summarizes the variables employed in this study.

3.2 Variables

The literature on impact assessment breaks the types of variables down into 3: a) impact variable or policy variable; b) result variable; and c) control variables or confounding variables. The impact variable for the empirical exercise of this article, denoted by t, is discrete quantitative and indicates how long, in years, municipalities participated in the Health Pact program. The policy result variable, denoted by Y, is not observed and reflects the different dimensions of primary health care provision under the responsibility of Brazilian municipalities. Thus, Y, or the PCEI had to be estimated based on variables sealed by the municipalities. The use of a synthetic indicator is advantageous in that it allows for latent aspects, which could not be captured by a single variable, to be covered and taken into account.

In order to develop the PCEI, the first stage was to select the relevant variables. Bearing all 67 indicators of the Health Pact policy goals in mind (in terms of morbidity, mortality, coverage and municipalities’ resources for health care), the 29 that were regarded as primary-care related were selected. However, the information available for these 29 variables did not always cover all the municipalities. Moreover, some variables did not present variability between municipalities. Given the circumstances, 5 variables (presented in Box 1) were used to construct the PCEI.

Box 1
Variables included in the effectiveness indicator of primary care policy

To estimate the effect of program participation time on PCEI, it was necessary to control for selection bias, so causal interpretations would not be invalidated. The choice of confounding variables (or control variables) was grounded on the findings of health economics and public policy assessment literature. According to the causality model, variables must be associated with both the impact variable and the result variable. Therefore, we selected variables that reflected the socioeconomic conditions of the municipality, as reported by the Human Development Atlas (UNDP, FJP, IPEA) for the years 2000 and 2010. In addition, we used an annual variable, the municipal gross domestic product (GDP) per capita, and its monetary values were deflated by the 2013 Broad National Consumer Price Index (BNCPI), to reflect real terms. Box 2 presents the control variables employed in this study.

Box 2
ControlVariables for Impact Assessment

3.3 Method

3.3.1 Calculating the primary care effectiveness indicator

Once we had the result variables, according to Box 1, the synthetic indicator was calculated using the principal component analysis methodology. Unlike other studies, we chose to use all five principal components to calculate the synthetic indicator, and they were weighted to reproduce the total system variability (Jolliffe, 2002Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). New York, NY: Springer.). Upon the calculation of the weights, the PCEI was calculated for each municipality and year, according to the following equation:

Y i j = θ 1 j S 1 i j + θ 2 j S 2 i j + θ 3 j S 3 i j + θ 4 j S 4 i j + θ 5 j S 5 i j (1)

where S 1ij is the first principal component of the municipality i in the year j and θ 1j is the weight for the year j for the first principal component. The municipality’s i PCEI in the year j, denoted by Y ij , is, therefore a quantitative-continuous variable, ranging on a scale from 0 to 1, where values closer to 1 indicate greater effectiveness of the primary care policy 4 4 The indicator was statistically validated, according to Jolliffe (2002) and OECD (2008). Results are available upon request. .

3.3.2 Estimation of the dose-response model with generalized propensity score

The assessment of causal effect of municipalities’ program adherence time on PCEI was performed through the dose-response model with generalized propensity score (Imbens, 2000Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87 (3), 706-710.; Lechner, 2001Lechner, M. (2001). Identification and estimation of causal effects of multiple treatments under conditional Independence assumption. In M. Lechner, & F. Pfeiffer(Eds.), Econometric evaluation of labour market policies(ZEW Economic Studies, 13, pp. 43-58). Heidelberg, Germany: Physica.). This method is based on Rubin’s Potential Results model (1974Rubin, D. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688-701.), which proposes the interpretation of causal effects by comparing potential results.

For a description of the dose-response model5 5 This model is also known in the literature as multiple treatment model, dose-effect model, or dose-response function (Guardabascio & Ventura, 2013). , it was originally based on the design to assess the effect of participation in a program. In this case, there are two states of treatment: a) the individual participates in the policy (treated); or b) the individual does not participate in the policy (non-treated). However, in many situations, the research question of interest is centered on the effect of treatment dose on a response of interest (dose-response) for those who participated in the policy. Thus, the dose may correspond to an ordinal and discrete quantitative variable (e.g.: time in years) or continuous quantitative variable (e.g.: monetary value of the benefit of a public policy).

The discrete case dose-response model was proposed by Lechner (2001Lechner, M. (2001). Identification and estimation of causal effects of multiple treatments under conditional Independence assumption. In M. Lechner, & F. Pfeiffer(Eds.), Econometric evaluation of labour market policies(ZEW Economic Studies, 13, pp. 43-58). Heidelberg, Germany: Physica.) and the continuous case dose-response model was proposed by Imbens (2000Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87 (3), 706-710.). In this study, the point was to assess the effect of a municipality’s participation time in the Health Pact (dose) and the PCEI (response), that is, the discrete case.

Formally, and based on Rubin’s Potential Results model (1974Rubin, D. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688-701.), each municipality i is considered to have a treatment value t, a set of confounding variables X, and a potential PCEI result Y i (t). According to Lechner (2001Lechner, M. (2001). Identification and estimation of causal effects of multiple treatments under conditional Independence assumption. In M. Lechner, & F. Pfeiffer(Eds.), Econometric evaluation of labour market policies(ZEW Economic Studies, 13, pp. 43-58). Heidelberg, Germany: Physica.), in dose-response models where dose is a discrete variable, treatment may take on values T = {1,..., Q}, where the municipality is exposed to a particular level of treatment t ? T. The estimate of interest is the mean causal effect of the treatment t on the mean outcome of the dose-response function. In the case of two doses of treatment, t and s, there would be the expected effect of t on Y, rather than the effect of s, for the same individual:

θ ( t ) = E [ Y i ( t ) - Y i ( s ) ] (2)

Given the fundamental problem of causal inference, we cannot observe the effect of two different doses for the same municipality at the same time, and this possibility is expressed as follows:

θ ( t ) = E ( Y i t - Y i s | T = t ) = E ( Y i t | T = t ) - E ( Y i s Y i s | T = t ) (3)

Where in the component E(Y i s | T = t) is estimated counterfactually. The extension of the average treatment effect of equation 3 to more than two treatment doses is done by generalization, in which comparisons are made between pairs of treatments t and s. As each participating municipality received one type of treatment, other treatments were not observed and, therefore, estimated through counterfactuals. According to Imbens (2000Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87 (3), 706-710.) and Lechner (2001Lechner, M. (2001). Identification and estimation of causal effects of multiple treatments under conditional Independence assumption. In M. Lechner, & F. Pfeiffer(Eds.), Econometric evaluation of labour market policies(ZEW Economic Studies, 13, pp. 43-58). Heidelberg, Germany: Physica.), the counterfactual properties proposed by Rubin (1974Rubin, D. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688-701.) and Rosenbaum and Rubin (1983Rosenbaum, P., & Rubin, D. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.) remain, with some refinement, in dose-response models6 6 In dose-response models it is assumed that the treatment (T) presents independence pairwise with each of the potential outcomes. This assumption is called weak unconfoundedness. In binary models, it is assumed that the treatment is independent of the entire set of potential outcomes (strong unconfoundedness). .

To estimate the counterfactual situation in the presence of multiple doses of treatment, municipalities were paired using the generalized propensity score. This method is an extension of the binary case propensity score proposed by Rosenbaum and Rubin (1983Rosenbaum, P., & Rubin, D. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.) to accommodate multiple treatments and eliminate selection bias. The selection bias, which generates confounding effects in the estimation of the causal impact, occurs when observable variables X are associated with both the treatment dose t and the outcome variable (Angrist & Pischke, 2014Angrist, J. D., & Pischke, J.-S. (2014). Mastering ‘metrics: the path from cause to effect. Princeton, NJ: Princeton University Press.; Gertler, Martinez, Premand, Rawlings, & Vermeersch, 2016Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J. (2016). Impact evaluation in practice (2nd ed.). Washington, DC: The World Bank.).

According to Imai and Van Dyk (2004Imai, K., & Van Dyk, D. A. (2004). Causal inference with general treatment regimes: generalizing the propensity score. Journal of the American Statistical Association, 99(467), 854-866.), the dose-response model with generalized propensity score consists of estimating, first, the conditional probability of a municipality receiving a particular level of treatment t, conditional on the confounding variables X.

r ( t , x ) p r ( T = t | X = x ) (4)

Similarly to the assumptions on pairing by propensity score for binary treatments, the balancing property must also be satisfied. That is, for a stratum with values similar to r(t,x), it is assumed that the probability that T = t does not depend on the value of X,

X p r ( T - t ) | r ( t , x ) (5)

This balancing assumption, along with the assumption of unconfoundedness (i.e., that all variables relevant for selection bias are being considered), implies that the assignment to treatment is independent, conditional on the generalized propensity score. If the assumption of unconfoundedness is satisfied, given the covariates, we have

Y ( t ) T | X t T (6)

Thus, in this study, for any period of participation in the Health Pact program, t, we have

Dessa maneira, neste estudo, para qualquer tempo de participação no programa Pacto pela Saúde, t, tem-se que

f T { t | r ( t , X ) , Y ( t ) } = f T { t | r ( t , X ) } (7)

Where the generalized propensity score allows to eliminate any bias associated with differences in covariates X. Thus, the results of the mean treatment effect can be estimated by conditioning the dose-response function to the generalized propensity score:

E { Y ( t ) } = E { β ( t , r ( t , X ) ) } (8)

Where

β ( t , r ( t , X ) ) = E { Y ( t ) | r ( t , X ) = } = E { Y | T = t , r ( T , X ) = r } (9)

According to Imbens (2000Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87 (3), 706-710.), the implementation of the dose-response method with a generalized propensity score involves 3 steps:

  1. .The score is estimated r(t,X) according to equation 4;

  2. .The conditional expectation expressed in equation 9 is estimated; and,

  3. .The mean response for the treatment level is estimated as the mean of the estimated conditional expectation β(t,r(t,X)), with the mean covariate distribution.

The implementation of the dose-response model with pairing by propensity score was performed through the “DoseResponse” package, available in Stata 13 software (Bia & Mattei, 2008Bia, M., & Mattei, A. (2008). A Stata package for the estimation of the dose-response function through adjustment for the generalized propensity score. The Stata Journal, 8(3), 354-373.; Mattei & Bia, 2009Mattei, A., & Bia, M. (2009). DoseResponse: Stata module to estimate dose-response function through adjustment for the generalized propensity score. Boston, MA: Boston College Department of Economics.).

4. RESULTS

This section presents the results of empirical tests to assess the effect of time of participation in the Health Pact program on PCEI. Section 4.1 displays municipalities’ adherence to the program, with emphasis on the analysis by greater regions. Section 4.2 presents analyses of correlations and statistical tests that attest to the presence of selection bias in the program, to the extent that confounding variables related to the socioeconomic level of the municipality are associated with different exposure times to the program, as well as to different levels of the PCEI. Finally, section 4.3 presents the results of the dose-response model with generalized propensity score.

4.1 Municipalities’ adherence to the Health Pact program

In this section, we report the rates of adherence of municipalities to the Health Pact by signing the MCT. Table 1 shows the growing number of municipalities that joined the Health Pact in the period from 2006 to 2012 by greater geographical region. During the course of the program, 4,587 municipalities joined in, which represents 82.43% of all Brazilian municipalities. The Midwest and the Southeast were the regions with the largest adherence (98.5% and 96.6%, respectively). The North was the region with the lowest adherence (66.8%). Thus, the analysis of the causal effect controlling for any selection bias was deemed pertinent, since adherence was non-random and, as indicated in Table 1, more prevalent among the most developed regions of the country.

Table 1
Number of municipalities that signed the management commitment term by greater region per year (2006-2012)

There were also tests of difference in the PCEI means between municipalities according to the country’s greater regions. The municipalities in the Southern region had the highest PCEI level while North and Northeast presented the worst PCEI scores 7 7 The results of the F test (and significance level) to analyze the differences between the groups of municipalities, by greater region: a) did not sign the MCT; b) adhered for 2 years; and c) adhered for 4 years, were, respectively: i) intragroups: did not adhere (90.5; 1%); adhered for 2 years (33.6; 1%); adhered for 4 years (30.6; 1%); ii) intraregions: North (2.8; 10%); Northeast (8.2,5%); Southeast (3.2, 5%); South (7.9.5%); and Midwest (0; not significant). . This analysis was also performed between groups of municipalities according to population size, and it showed that small municipalities (up to 30,000 inhabitants) and municipalities with populations ranging from 300,001 to 500,000 inhabitants boasted the highest PCEI scores8 8 The results of the F test (and significance level) to analyze the differences between the groups of municipalities: a) did not sign the MCT; b) adhered for 2 years; and c) adhered for 4 years were, respectively: i) intragroups: did not adhere (19.8; 1%); adhered for 2 years (7.7; 1%); adhered for 4 years (8.9; 1%); and ii) intragroups by size: up to 10 thousand inhabitants (5.4; 1%); from 10,001 to 30,000 inhabitants (4.8; 1%); and from 300,001 to 500 thousand inhabitants (4.1; 5%). .

The data revealed the presence of a positive and statistically significant correlation between adherence to the Health Pact and PCEI9 9 The estimated Pearson correlation between PCEI and dose was 0.2903, statistically significant at 1%. . Thus, municipalities that had been in the policy for the longest period were also those with the best PCEI scores. However, we needed to control for confounding biases in this relationship, which we describe in the following section.

4.2 Selection bias: correlation between result variable and socioeconomic indicators

Prior to impact assessment, we performed statistical tests to verify the presence of selection bias in the sample. Selection bias, which leads to confounding effects on causal impact estimation, occurs when observable variables are associated with both treatment dose t and result variable Y.

Regarding the result variable Y and its correlation with socioeconomic variables, every year presented a positive and statistically significant correlation between the PCEI and the following variables: a) GDP per capita; b) life expectancy; c) municipal human development index10 10 Pearson correlation coefficients between PCEI and: GDP per capita (0.2340, significant at 1%); life expectancy (0.5210, significant at 1%); Municipal HDI (0.5123, significant at 1%). (HDI). Regarding the correlation between treatment dose t and socioeconomic variables, it was found that, although magnitudes were low compared to correlations with PCEI, there was a positive and statistically significant correlation between treatment dose and the following variables: a) GDP per capita; b) life expectancy; c) municipal HDI11 11 Pearson correlation coefficients between PCEI and: GDP per capita (0.0941, significant at 1%); life expectancy (0.2653, significant at 1%); Municipal HDI (0.3198, significant at 1%). .

4.3 Impact Assessment

This subsection presents the estimation results of dose-response models with generalized propensity score, which provide the causal effect of the dose of intention to treat the Health Pact program, on the PCEI, minimizing the confounding effects resulting from the selection bias. It should be emphasized that the estimator of interest is the intention-to-treat (ITT) type, because, although the signature of the MCT implied that the municipality would have to commit to a results-based management, there were no accountability and compliance mechanisms. Therefore, the causal inference population in this study are the 4,587 municipalities that joined the policy.

The estimation of the dose-response model with generalized propensity score occurred in two stages. In the first stage, the generalized propensity score was estimated from a regression, where the dependent variable is the time the municipality participated in the program (in years) and the independent variables are the socioeconomic indicators that can generate the selection bias (Table 2). A Poisson regression was used, since the dependent variable is positive and refers to a count data (Cameron & Trivedi, 2005Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics: methods and applications. New York, NY: Cambridge University Press.).

In the second stage, a generalized linear regression was estimated, whose dependent variable is the PCEI and the independent variable is the generalized propensity score, obtained by the predicted value of the first stage equation (Guardabascio & Ventura, 2013Guardabascio, B., & Ventura, M. (2013). Estimating the dose-response function through the GLM approach. Roma, Italy: Italian National Institute of Statistics.).

Also, it was tested by the robustness of the results on impact assessment of different specifications of the generalized propensity score and the construction of the result variable according to reference period (Box 3). It is worth mentioning that, although the basic data present a panel structure, for the estimation of the dose-response model with generalized propensity score, it was necessary to obtain a synthesis indicator for the exposure period.

Box 3
Versions of the dose-response model with a generalized propensity score for the impact of adherence time to the Health Pact on the indicator of primary care policy effectiveness

The first stage estimation results, the generalized propensity score, according to the model’s set of independent variables are presented in Table 2. As can be seen, most of the variables present a statistically significant coefficient. It is noteworthy that, in the estimation of propensity score models, the choice of confounding variables reflects a theoretical construction (which variables lead to selection bias, i.e., affect both the time of adherence to the policy and the result of interest) and, thus, the coefficients should not necessarily be interpreted. The goal was to have the best model to match municipalities at different levels of treatment, as presented in the methodological section.

Table 2
Estimation coefficients of the generalized propensity score considering a Poisson distribution by model type. Dependent variable: years of adherence to the Health Pact

Having estimated the first stage of the model, where the generalized propensity score was obtained, the results of the dose-response model estimation were presented, controlling for the propensity score obtained in the first stage (Table 3). Consequently, the mean effect of time of participation in the Health Pact program on PCEI is unfolded according to three specifications of the dependent variable.

The results show that the impact did not vary considerably in magnitude according to the type of specification, which demonstrates its robustness. On average, for each additional year of adherence to the Health Pact, PCEI increased by 0.011 to 0.019 points, depending on the model.

It should also be noted that the coefficients for the generalized propensity score in the dose-response function were statistically significant at 1%, as recommended in the literature (Guardabascio & Ventura, 2013Guardabascio, B., & Ventura, M. (2013). Estimating the dose-response function through the GLM approach. Roma, Italy: Italian National Institute of Statistics.).

Table 3
Estimation of the dose-response function model, considering response (PCEI) and dose (years of adherence to program) following a Poisson distribution.

Figure 1, in panels “a”, “b” and “c”, presents the graphs of the dose-response function estimates for the three estimated models. The x-axis presents the treatment level (1 to 7 years), while the y-axis shows the expected PCEI value according to exposure to the treatment level. The confidence interval for the 95% function is also shown in Figure 1. The models in Figure 1 present a slight parabola format, which means a small decrease in the PCEI’s expectancy between the first and second year of adherence, an increase in the PCEI’s expectancy between the third and fourth year of adherence and an exponential growth trend of the PCEI from the fourth year on.

This behavior of the dose-response function may represent at least two situations. The first suggests that, in the early stages of adherence, having signed the MCT imposed some administrative changes to municipalities’ health sector, such as having to readjust activities and actions, hire new health professionals, train health teams, which may have called for an adaptation period, when the goals for the first two years of adherence were not met. One example of this was the coverage expansion of Oral Health Teams, which, in turn, had a direct impact on one of the variables that make up the PCEI.

Figure 1
Dose-response function estimated by model

The second situation may reflect the fact that municipalities need a minimum amount of time to internalize the changes caused by participation in the Health Pact and gain experience in health management to ensure effectiveness. In this sense, the exponential part of the graph, starting at four years of adherence to the program, may be showing that the Health Pact produces positive effects on effectiveness from the fourth year on (after a period of adaptation to changes and accumulated experience with primary care technology).

5. FINAL CONSIDERATIONS

The main objective of this study was to assess the causal effect of policies with health goals on an indicator of management effectiveness of primary care services. The focus of the analysis was the assessment of the Health Pact, through the signature of the MCT (MS Ordinance no. 399, 2006), which was in force from 2006 to 2012, and gathered a total of 4,587 Brazilian municipalities. Such adherence was quite remarkable in the Southeastern and Midwestern regions, where more than 95% of municipalities signed the MCT.

To calculate the causal effect of the Health Pact, the PCEI was developed using a multivariate technique of principal components. When analyzing PCEI, there was a positive correlation with adherence time to the Health Pact, that is, the longer the adherence time, the higher the PCEI. In addition, municipalities that adhered to the Health Pact presented higher PCEI compared to municipalities that did not.

By means of the PCEI, an econometric dose-response model with generalized propensity score was estimated, according to Imbens (2000Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87 (3), 706-710.) and Lechner (2001Lechner, M. (2001). Identification and estimation of causal effects of multiple treatments under conditional Independence assumption. In M. Lechner, & F. Pfeiffer(Eds.), Econometric evaluation of labour market policies(ZEW Economic Studies, 13, pp. 43-58). Heidelberg, Germany: Physica.). This model proved to be the most appropriate to assess the effect of the policy, as not only does it consider multiple treatments (dose, represented by years of MCT signature), it adequately addresses possible self-selection problems, i.e., non-random participation in the program.

The results of the dose-response function estimation showed that adherence time to the Health Pact had a positive and statistically significant impact on the effectiveness levels of primary care policies of participating municipalities. That is, for each additional year of policy permanence, the effectiveness indicator improved by 0.011 to 0.019 points on average, i.e., municipalities were better able to meet the sealed goals.

In other words, such positive impact demonstrates that goals do matter for the governance of Brazilian municipal health policy. In this sense, guided by explicit and well-defined objectives, municipalities begin to guide their actions in a clearer, better planned and systematized way in order to meet these objectives.

The estimated coefficients for the dose effect, which were low in magnitude, seem to conclude that the impact of the policy on the effectiveness indicator was a modest one. However, it can be argued that one cannot lose sight of the fact that there was no specific commitment mechanism in regard to the Health Pact, which makes such results even more relevant.

Thus, the positive coefficient for causal impact denotes that the Health Pact was effective in bringing about commitment and systematic planning of health actions by municipal administrations, which resulted in an improvement in management effectiveness. However, one can argue that such result could have had greater magnitude if: a) some conditionality had been claimed by the MH; b) there had been greater participation by municipal health councils in supervising the work of municipal administrations; and c) intersectoral policies related to municipal health had been encouraged (basic sanitation, environmental management, health education).

When analyzing the dose-response function curve, it seems clear that municipalities with up to 2 years of adherence, on average, showed a drop in PCEI, while municipalities with more than 3 years of adherence were contemplated with increasing PCEI values. The assessment of this curve demonstrates that there is possibly a period of adaptation for municipal management to meet their goals -- whether they be team expansion, hiring and training of health workers and registration, or monitoring families -- which may lead to short term loss of policy effectiveness.

However, as municipalities internalize these changes and gather experience, there are gains in effectiveness. The exponential part of the graph suggests that the positive effects produced by participation in the program on the effectiveness of primary care policy among the participating municipalities occurs from the fourth year onward.

Finally, it should be noted that smaller municipalities (up to 30 thousand inhabitants) obtained better PCEI levels than larger municipalities. This data corroborates the evidence found in studies on public health, which show that by placing priority on territorialized health actions, the primary care model favors smaller municipalities, characterized by having better conditions to structure and carry out this type of action (Giovanella & Mendonça, 2012Giovanella, L., & Mendonça, M. H. M. (2012). Atenção primária à saúde. In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 493-545). Rio de Janeiro, RJ: Ed. Fiocruz .).

This article contributes to the literature by assessing the impact of the Health Pact program in an unprecedented way, based on a quasi-experimental design. In addition, it adds to the theoretical literature by systematizing the state of the art in goal-oriented health policies, as well as disclosing how municipal health production responds to policies with a focus on accountability and management by result.

Therefore, it can be assumed that the signature of the MCT was an important step on the road to improving the governance of primary care policy, which according to WHO (2000World Health Organization. (2000). The world health report 2000: health systems - improving performance. Geneva, Switzerland: Autor., 2008World Health Organization. (2008). The world health report 2008: primary health care (now more than ever). Geneva, Switzerland: Autor .), is a mandatory premise for better health outcomes. Thus, the research results suggest the maintenance of the policy and its due improvement, such as the creation of conditionalities and more effective goal monitoring, through the use of a composite indicator, such as the PCEI, which comprised specific elements of primary care.

REFERÊNCIAS

  • Angrist, J. D., & Pischke, J.-S. (2014). Mastering ‘metrics: the path from cause to effect Princeton, NJ: Princeton University Press.
  • Bia, M., & Mattei, A. (2008). A Stata package for the estimation of the dose-response function through adjustment for the generalized propensity score. The Stata Journal, 8(3), 354-373.
  • Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics: methods and applications New York, NY: Cambridge University Press.
  • Comissão Intergestores Tripartite. (2016). Informações gerais Recuperado de http://www.saude.gov.br/gestao-do-sus/articulacao-interfederativa/comissao-intergestores-tripartite
    » http://www.saude.gov.br/gestao-do-sus/articulacao-interfederativa/comissao-intergestores-tripartite
  • Conselho Nacional de Secretários da Saúde. (2015). A Gestão do SUS (Para Entender a Gestão do SUS). Recuperado dehttp://www.conass.org.br/biblioteca/pdf/A-GESTAO-DO-SUS.pdf
    » http://www.conass.org.br/biblioteca/pdf/A-GESTAO-DO-SUS.pdf
  • Decreto n. 7.508, de 28 de junho de 2011 (2011). Regulamenta a Lei n. 8.080, de 19 de setembro de 1990, para dispor sobre a organização do Sistema Único de Saúde - SUS, o planejamento da saúde, a assistência à saúde e a articulação interfederativa, e dá outras providências. Brasília, DF.
  • Emenda Constitucional n. 29, de 13 de setembro de 2000 (2000). Altera os arts. 34, 35, 156, 160, 167 e 198 da Constituição Federal e acrescenta artigo ao Ato das Disposições Constitucionais Transitórias, para assegurar os recursos mínimos para o financiamento das ações e serviços públicos de saúde. Brasília, DF.
  • Fleury, S., & Ouverney, A. M. (2012). A política de saúde: uma política social. In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 25-57). Rio de Janeiro, RJ: Ed. Fiocruz.
  • Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J. (2016). Impact evaluation in practice (2nd ed.). Washington, DC: The World Bank.
  • Giovanella, L., & Mendonça, M. H. M. (2012). Atenção primária à saúde. In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 493-545). Rio de Janeiro, RJ: Ed. Fiocruz .
  • Guardabascio, B., & Ventura, M. (2013). Estimating the dose-response function through the GLM approach Roma, Italy: Italian National Institute of Statistics.
  • Hone, T., Rasella, D., Barreto, M., Atun, R., Majeed, A. & Millett, C. (2017). Large reductions in amenable mortality associated with Brazil’s primary care expansion and strong health governance. Health Affairs, 36(1), 149-158.
  • Imai, K., & Van Dyk, D. A. (2004). Causal inference with general treatment regimes: generalizing the propensity score. Journal of the American Statistical Association, 99(467), 854-866.
  • Imbens, G. W. (2000). The role of the propensity score in estimating dose-response functions. Biometrika, 87 (3), 706-710.
  • Instituto Brasileiro de Geografia e Estatística. 2016. Produto interno bruto dos municípios Recuperado de https://www.ibge.gov.br/estatisticas/economicas/contas-nacionais/9088-produto-interno-bruto-dos-municipios.html?=&t=o-que-e
    » https://www.ibge.gov.br/estatisticas/economicas/contas-nacionais/9088-produto-interno-bruto-dos-municipios.html?=&t=o-que-e
  • Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). New York, NY: Springer.
  • Lechner, M. (2001). Identification and estimation of causal effects of multiple treatments under conditional Independence assumption. In M. Lechner, & F. Pfeiffer(Eds.), Econometric evaluation of labour market policies(ZEW Economic Studies, 13, pp. 43-58). Heidelberg, Germany: Physica.
  • Lei n. 8.142, de 28 de dezembro de 1990 (1990). Dispõe sobre a participação da comunidade na gestão do Sistema Único de Saúde (SUS) e sobre as transferências intergovernamentais de recursos financeiros na área da saúde e dá outras providências. Brasília, DF.
  • Lorenzoni, L., Murtin, F., Springare, L. S., Auraeen, A., & Daniel, F., (2018, April). Which policies increase value for money in health care? (OECD Health Working Papers, No. 104). Paris, France: OECD.
  • Mattei, A., & Bia, M. (2009). DoseResponse: Stata module to estimate dose-response function through adjustment for the generalized propensity score Boston, MA: Boston College Department of Economics.
  • Ministério da Saúde. (2014). Caderno de diretrizes, objetivos, metas e indicadores: 2013-2015 (2a ed.). Brasília, DF: Autor.
  • Ministério da Saúde. (2016). Sistema de Informações em Saúde Recuperado dehttp://datasus.saude.gov.br/informacoes-de-saude/tabnet
    » http://datasus.saude.gov.br/informacoes-de-saude/tabnet
  • Murname, R. J., & Willet, J. B. (2011). Methods matter: improving causal inference in educational and social science research Oxford, England: Oxford University Press.
  • Noronha, J. C., Lima, J. D., & Machado, C. V. (2012). O Sistema Único de Saúde (SUS). In Giovanella, L., Escorel, S., Lobato, L. V. C., Noronha, J. C., Carvalho, A. I., (Orgs.), Políticas e sistema de saúde no Brasil (2a ed., pp. 365-393) Rio de Janeiro, RJ: Ed. Fiocruz .
  • Ocké-Reis, C. O. (2012). SUS: o desafio de ser único Rio de Janeiro, RJ: Ed. Fiocruz .
  • Organisation For Economic Co-Operation And Development. (2008). Handbook on constructing composite indicators: methodology and user guide Paris, France: Autor.
  • Organisation For Economic Co-Operation And Development. (2010). Value for money in health spending (OECD Health Policy Studies). Paris, France: Autor .
  • Organisation For Economic Co-Operation And Development. (2015). Fiscal sustainability of health systems: bridging health and finance perspectives Paris, France: Autor .
  • Ouverney, A. M. (2014). A construção da municipalização da saúde: estratégias normativas. In S. Fleury, Democracia e inovação na gestão local de saúde (pp. 81-120). Rio de Janeiro, RJ: Ed. Fiocruz .
  • Piola, S. F., Paiva, A. B., Sá, E. B., Servo, L. M. S., (2013, julho). Financiamento público da saúde: uma história à procura de rumo (Texto para Discussão IPEA, n. 1846). Brasília, DF: Instituto de Pesquisa Econômica Aplicada.
  • Programa das Nações Unidas para o Desenvolvimento Humano (2013). Índice de desenvolvimento humano municipal brasileiro (Atlas do Desenvolvimento Humano no Brasil). Brasília, Brasil: PNUD, IPEA, FJP. Recuperado de http://atlasbrasil.org.br/2013/pt/download/
    » http://atlasbrasil.org.br/2013/pt/download/
  • Portaria MS n. 399, de 22 de fevereiro de 2006 (2006). Divulga o Pacto pela Saúde 2006 - Consolidação do SUS e aprova as Diretrizes Operacionais do Referido Pacto. Brasília, DF.
  • Portaria MS n. 2.203, de 5 de novembro de 1996 (1996). Norma Operacional Básica do SUS. Brasília, DF.
  • Portaria MS n. 2.436, de 21 de setembro de 2017 (2017). Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes para a organização da Atenção Básica, no âmbito do Sistema Único de Saúde (SUS). Brasília, DF.
  • Portaria MS n. 2.488, de 21 de outubro de 2011 (2011). Aprova a Política Nacional de Atenção Básica, estabelecendo a revisão de diretrizes e normas para a organização da Atenção Básica, para a Estratégia Saúde da Família (ESF) e o Programa de Agentes Comunitários de Saúde (PACS). Brasília, DF.
  • Rajkumar, A. S., & Swaroop, V. (2008). Public spending and outcomes: does governance matter? Journal of Development Economics, 86, 96-111.
  • Riley, J. C. (2007). Low income, social growth, and good health: a history of twelve countries Berkeley, CA: University of California Press.
  • Rocha, F. F., Orellano, V., & Nishijima, M. (2016). Health spending autonomy and infant mortality rates: a matter of local administrative capacity?. The Journal of Developing Areas, 50, 293-309.
  • Rocha, F. F., Nishijima, M., & Peixoto, S. G. D. (2013). Primary health care policies: investigation on morbidity. Applied Economics Letters, 20, 1046-1051.
  • Rocha, R., & Soares, R. (2010). Evaluating the impact of community-based health interventions: evidence from Brazil’s Family Health Program. Health Economics, 19, 126-158.
  • Rosenbaum, P., & Rubin, D. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55.
  • Rubin, D. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688-701.
  • Saltman, R. B., Bankauskaite, V., Vrangbaek, K., (Orgs.). (2007). Decentralization in health care: strategies and outcomes (European Observatory on Health Systems and Policies Series). New York, NY: McGraw-Hill.
  • Santos, R. G., Nascimento, S.P., & Camara, M. R. G.. (2017). The influence of the pact for health on the expenditure on the financing of public health on the view of the Brazilian State. Planejamento e Políticas Públicas, 49, 109-131.
  • Soares, R. (2007). Health and the evolution of welfare across Brazilian municipalities. Journal of Development Economics, 84(2), 590-608.
  • Starfield, B., & Shi, L. (2002). Policy relevant determinants of health: an international perspective. Health Policy, 60, 201-218.
  • Starfield, B. (2002). Atenção primária: equilíbrio entre necessidades de saúde, serviços e tecnologia Brasília, Brasil: Unesco.
  • Tulchinsky, T. H., & Varikova, E. A. (2010). What is the “New Public Health”? Public Health Reviews, 32(1), 25-53.
  • Vo, D. H. (2010). The economics of fiscal decentralization. Journal of Economic Surveys, 24(4), 657-679.
  • World Health Organization. (2000). The world health report 2000: health systems - improving performance Geneva, Switzerland: Autor.
  • World Health Organization. (2008). The world health report 2008: primary health care (now more than ever). Geneva, Switzerland: Autor .
  • 1
    It should be mentioned that Constitutional Amendment No. 29 (2000), which made it possible to combine the revenues of municipalities (15%), states and the Federal District (12%) and the federal government (the amount allocated for health in the previous year plus the nominal variation of gross domestic product [GDP]) for the health sector, has contributed to increasing the entities’ participation in health expenditures.
  • 2
    By definition, primary care, also known as primary health care (PHC), comprises an individual’s first contact with health care services (first level outpatient care), which should be easily accessible and cover most common diseases and conditions, as well as immunizations. Moreover, it consists of a health care model that emphasizes family health and health promotion (Giovanella & Mendonça, 2012).
  • 3
    The list of municipalities included in the analysis can be requested by email.
  • 4
    The indicator was statistically validated, according to Jolliffe (2002) and OECD (2008). Results are available upon request.
  • 5
    This model is also known in the literature as multiple treatment model, dose-effect model, or dose-response function (Guardabascio & Ventura, 2013).
  • 6
    In dose-response models it is assumed that the treatment (T) presents independence pairwise with each of the potential outcomes. This assumption is called weak unconfoundedness. In binary models, it is assumed that the treatment is independent of the entire set of potential outcomes (strong unconfoundedness).
  • 7
    The results of the F test (and significance level) to analyze the differences between the groups of municipalities, by greater region: a) did not sign the MCT; b) adhered for 2 years; and c) adhered for 4 years, were, respectively: i) intragroups: did not adhere (90.5; 1%); adhered for 2 years (33.6; 1%); adhered for 4 years (30.6; 1%); ii) intraregions: North (2.8; 10%); Northeast (8.2,5%); Southeast (3.2, 5%); South (7.9.5%); and Midwest (0; not significant).
  • 8
    The results of the F test (and significance level) to analyze the differences between the groups of municipalities: a) did not sign the MCT; b) adhered for 2 years; and c) adhered for 4 years were, respectively: i) intragroups: did not adhere (19.8; 1%); adhered for 2 years (7.7; 1%); adhered for 4 years (8.9; 1%); and ii) intragroups by size: up to 10 thousand inhabitants (5.4; 1%); from 10,001 to 30,000 inhabitants (4.8; 1%); and from 300,001 to 500 thousand inhabitants (4.1; 5%).
  • 9
    The estimated Pearson correlation between PCEI and dose was 0.2903, statistically significant at 1%.
  • 10
    Pearson correlation coefficients between PCEI and: GDP per capita (0.2340, significant at 1%); life expectancy (0.5210, significant at 1%); Municipal HDI (0.5123, significant at 1%).
  • 11
    Pearson correlation coefficients between PCEI and: GDP per capita (0.0941, significant at 1%); life expectancy (0.2653, significant at 1%); Municipal HDI (0.3198, significant at 1%).
  • [Translated version] Note: All quotes in English translated by this article’s translator.

Publication Dates

  • Publication in this collection
    10 Jan 2020
  • Date of issue
    Nov-Dec 2019

History

  • Received
    27 Dec 2018
  • Accepted
    03 Oct 2019
Fundação Getulio Vargas Fundaçãoo Getulio Vargas, Rua Jornalista Orlando Dantas, 30, CEP: 22231-010 / Rio de Janeiro-RJ Brasil, Tel.: +55 (21) 3083-2731 - Rio de Janeiro - RJ - Brazil
E-mail: rap@fgv.br