Evaluation models and Brazilian health reform: a qualitative-participatory approach

Abstracts

Ao longo dos últimos anos, registra-se um crescente interesse por propostas avaliativas não tradicionais e mais abrangentes nos movimentos de reforma sanitária em curso na América Latina. Este estudo teve por objetivo analisar potencialidades do enfoque avaliativo qualitativo-participativo ante o desafio de fortalecer as reformas sanitárias na região, particularmente aquelas que se configuram progressistas, como o caso brasileiro. Há necessidade de se avaliarem as reformas sanitárias rigorosa e permanentemente, sobretudo a incongruência do emprego de modelos normativos para avaliar sistemas de saúde baseados nos princípios de universalização, integralidade, humanização e gestão democrática. Além da demanda por estratégias e instrumentos de avaliação, a reforma sanitária brasileira requer a adoção de propostas e práticas avaliativas fundamentadas em outros paradigmas distintos daquele ainda hegemônico no âmbito da avaliação em saúde. Advoga-se pela utilização de modelos emergentes de avaliação, tais como os de corte qualitativo-participativo.

Reforma sanitária; Serviços de Saúde; Pesquisa Interdisciplinar; Pesquisa Qualitativa; Planos e Programas de Pesquisa em Saúde; Avaliação da Pesquisa em Saúde

COMENTÁRIOS

Evaluation models and Brazilian health reform: a qualitative-participatory approach

Modelos evaluativos y reforma sanitaria brasilera: enfoque cualitativo-participativo

Maria Lúcia Magalhães Bosi^I; Francisco Javier Mercado-Martinez^II

^IDepartamento de Saúde Comunitária. Faculdade de Medicina. Universidade Federal do Ceará. Fortaleza, CE, Brasil

^IIDepartamento de Salud Pública. Centro Universitario de Ciencias de la Salud. Universidad de Guadalajara. Guadalajara, Jalisco, México

^{Correspondence}

ABSTRACT

Throughout the last years, there has been a growing interest in ongoing assessment proposals in Latin America, which are more far-reaching and not traditional. The aim of this study was to analyze the potential of qualitative-participatory evaluation in view of the challenge of strengthening health reforms in the region, particularly those considered progressive, such as the Brazilian case. There is the need to assess health reforms in a rigorous and permanent way, especially the incongruity when using normative models to evaluate health systems based on principles of universality, comprehensiveness, humanization and democratic management. In addition to the demand for assessment instruments and strategies, the Brazilian health reform requires the adoption of evaluation proposals and practices that are founded on other paradigms, distinct from the hegemonic one, in the sphere of health assessment. It is recommended that emerging evaluative models be used, such as those with a qualitative-participatory approach.

Descriptors: Health Care Reform. Interdisciplinary Research. Community-Based Participatory Research. Qualitative Research. Health Research Plans and Programs. Health Research Evaluation.

RESUMEN

A lo largo de los últimos años, se registra un interés creciente por propuestas evaluativas no tradicionales y más amplio en los movimientos de reforma sanitaria en curso en la América Latina. Este estudio tuvo por objetivo analizar potencialidades del enfoque evaluativo cualitativo-participativo frente al desafío de fortalecer las reformas sanitarias en la región, particularmente aquellas que se configuran como progresistas, como el caso brasilero. Hay necesidad de evaluar las reformas sanitarias de forma rigorosa y permanente, sobretodo la incongruencia del empleo de modelos normativos para evaluar sistemas de salud basados en los principios de universalización, integralidad, humanización y gestión democrática. Además de la demanda por estrategias e instrumentos de evaluación, la reforma sanitaria brasilera requiere la adopción de propuestas y prácticas evaluativas fundamentadas en otros paradigmas distintos de aquel hegemónico en el ámbito de la evaluación en salud. Se defiende la utilización de modelos emergentes de evaluación, tales como los de corte cualitativo-participativo.

Descriptores: Reforma sanitária. Servicios de Salud. Investigación Interdisciplinaria. Investigación Cualitativa. Planes y Programas de Investigación en Salud. Evaluación de la Investigación en Salud.

INTRODUCTION

Since the 1990s, diverse initiatives have been created to consolidate the health sector reform in several Latin American countries, including Brazil. Inspired by different ideals, the initiatives have been developed to improve the coverage and quality of health services, to promote practices based on equality and comprehensiveness, as well as to encourage the democratization and social participation in the health system. In this context, while some reforms were directed towards the privatization and profit-making of health services, others were concerned with universal and free access to health care. Such positions are part of wider polarizations, which are the consequence of changing relations among the government, society and the market.²

During this period, there was also a growing interest in emergent evaluation practices and proposals, especially those favoring a qualitative-participatory approach.^9,10 A key aspect of such proposals, besides their anti-positivist position, is their strong interest in the inclusion of dimensions that are traditionally excluded from the hegemonic model of evaluation. Among these are concepts such as alterity, subjectivity and the involvement of stakeholders in the evaluation process In such context, the purpose of this paper is to explore the contributions that non-traditional evaluative perspectives, especially those based on the qualitative-participatory approach, could bring to strengthen health reforms in Latin America. It focuses particularly on the ongoing Brazilian reform process.

With regard to the debate on evaluation models and types, authors in this study contend that certain aspects of the health system are unique and constitute the theoretical, legal and technical-operational foundation of the respective reforms. Hence, it equally serves as the basis for their evaluative process. In this sense, a health system founded on a broader conception of health (universal access to health care, comprehensiveness and humanization)¹ has specificities that should be incorporated in any initiative aimed at its assessment, especially when it refers to the quality of the system.⁶ In other words, values underlying the medical system expected by the Brazilian health movement should "interact in a friendly way" with the models through which it is evaluated. For this reason, it seems unacceptable to assess health projects that intend to be "democratic" and "inclusive", when using models founded on conceptions and methodologies that prioritize dimensions such as effectiveness and efficacy, among other so-called objective outcomes, once they do not show the values inspiring the above mentioned movement.

Certainly, it seems essential to emphasize such premise in the current agenda of the health sector, taking into account the consolidation of the Brazilian health reform, especially the Sistema Único de Saúde (SUS - National Health System).¹ The indiscriminate use and inappropriate interchange of different concepts and types of evaluation - and the resulting adherence to their conflicting presuppositions with the argument of reforms - create simulacra that institute, in the discursive-symbolical fields, initiatives considered as qualitative, participatory or quality (in its full sense) evaluation.⁵ As reported by some authors, such traditional evaluative approaches and practices, despite their use and institutionalization, have been the object of frequent criticism, using different theoretical-conceptual, methodological, political and cultural arguments.^8,10 Moreover, the hegemonic models applied in Brazil and other Latin American countries are the result of the imposition of specific forms of evaluation; proposals that have been conceived by first world centers of academic, economic and ideological power.⁴ Thus, by recognizing the limitations of the hegemonic model and often making use of critical thinking, new proposals and experiences have emerged in the region, resulting from the search for evaluative models that are more participatory and interested in recovering perspectives and proposals from stakeholders. Consequently, distinct communities have emerged as collective participants and they have been critical not only to define health priorities but also to implement and evaluate differently health programs and services.

ABOUT THE QUALITATIVE-PARTICIPATORY APPROACH

Qualitative participatory evaluation may have a role in the consolidation of progressive health reforms in the region, including the Brazilian one. This approach consists in a set of theoretical-methodological initiatives that focus on issues such as alterity, subjectivity and meanings, among other things. Such initiatives are found to be of little interest for advocates of traditional models; nevertheless, they are key components when it comes to recovering the diversity of points of view and suggestions about how to organize health systems committed to the local needs, based on dialogue and negotiation among stakeholders with different interests, as it is usually observed in the daily life of the medical system. It comprises a diversity of proposals that question those models focused on efficacy, cost reduction; and resource optimization; instead, they introduce as core elements issues such as democratization, participation and dialogue; in addition to the inter-subjectivity component found in these initiatives, as part of wider social and health processes.

In view of what has been mentioned here, certain questions seem to be central to assess the Brazilian health reform: What type of evaluation has been performed to understand the advances, regressions and difficulties experienced by health programs and policies? What evaluative model is found to be more organic in relation to programs committed to universality, justice and social control? What criteria are used to ensure "success" or "failure" of the health reform movement?

ASSESSING THE REFORM: WHAT MODEL?

The previous questions illustrate the political dimensions inherent to the emergent evaluative models and seem opportune when, against "triumphal" discourses of the Brazilian health reform, some analyses critically appraise its scope and warn about the challenges that remain ^3,7,11,13

Thus, in addition to the importance of the same monitoring and evaluative process, current debate and challenges force the analysis of criteria, technologies and instruments that legitimize the "successful discourse" or its opposite. As a result, this exercise oriented towards "evaluating the evaluations" could provide critical elements, highly valued in the Brazilian health movement. Consequently, the evaluation has a strategic role in the construction of the reform, in view of the risk of going back to the ideology of reform.

In this way, before judging the health reform, one should question which dimensions are being considered; what voices and interests are listened to or excluded, what evidence is used to authorize the decisions made. Lastly, one should ask who finally decides whether the Brazilian health reform is working and what the criteria adopted for this judgment are.

As a consequence, several challenges emerge as regards this topic. The creation of a culture of evaluation is not the main concern. It is the construction of evaluative models that could guide the debate and operate as instruments of management and transformation in the sense of consolidating the advances already achieved in relation to "greater directions".¹² From this perspective, the criteria used to evaluate the health reforms, including the Brazilian one, must not come from decision-makers or "experts" exclusively; instead, they must equally involve health professionals, users of health services, and patients, among others.

Many of the above considerations adopt a critical position on many instruments and technologies used in the traditional evaluation in the sense that the latter usually restricts or prevents listening to others and having a dialogue. However, despite the predominance of such paradigm in the region, an effort is being made by state agencies towards the incorporation of the qualitative-participatory approach, both in the discourse and in the practice of evaluating programs on a local and national level.⁵ The gradual incorporation of subjective aspects in the evaluation process accounts for the growing interest in quality issues, although still not institutionalized in the technologies used. Such interest, on the other hand, does not prevent the identification of methodological and theoretical-conceptual problems in these models.⁶ In fact, it should be emphasized the need to question whether the current evaluative models, on which the evaluation of the Brazilian health reform is founded, are focused on principles of equity, comprehensiveness and humanization of health care. Despite ideals and discourses, most assessment experiences in the health sector in the region show the influence of the punitive-normative-quantitative tradition. In this case, the use of numbers invites one to consider whether they can account for the achievement of the principles that have inspired the reforms and, in the specific case of Brazil, the SUS principles, directives and presuppositions. In this sense, it could be appropriate to recover certain ideas and assumptions from the qualitative-participatory approach for such construction.

In the current stage of the health reform, for example, there is a debate in relation to primary health care, understood as the key strategy to reorganize the system.¹³ However, achieving such goal implies dialogue and the recognition of alterity or otherness, especially when considering the provision of health care to populations in situations of risk. As it is known, these populations are usually immersed in different cultures and codes, thus requiring new methods and instruments that enable their voices to be listened to. Thus, a system open to the needs, values and rights of such populations, the construction of management models and the qualification of health professionals, oriented to the solutions of different voices and demands, is required. These are goals that are beyond the reach of traditional instruments and measurements of the formal evaluative logic.

To discuss about humanization, equality and comprehensiveness in the health system, among other key concepts that are connected to subjectivity, implies the inclusion of the concept of quality using a multi-dimensional, qualitative and participatory conception. Thus, although it is possible to carry out an evaluation focused on quality, the qualitative-participatory approach has greater scope, because it is the one that necessarily includes stakeholders involved in health actions, their subjective demands, values, feelings and desires. This approach allows the recognition of conflicts inherent to several processes that promote subjectivity, thus enabling the construction of evaluation models centered on such principles.

FINAL CONSIDERATIONS

In this paper, authors tried to argue that the health reforms in Latin America, including the Brazilian one, involve issues such as being receptive to different social participants: the quality of inter-personal relationships in the organization of services; the ethics of health care and the care process; and the democratic management, among other dimensions. Such reforms also allude invariably to the use of conceptual and methodological approaches which enable the understanding of phenomena from a multi-dimensional and historical perspective. They require concepts and technologies capable of recovering the intensity of human processes and relations that usually remain hidden in the spreadsheets aiming to quantify them, according to mainstream models of evaluation.

The power of transformation of emergent evaluation is unquestionable, particularly as a way to strengthen social control and the democratic and participatory management in the SUS. Contradictorily, so is its ability to adopt existing practices. In this sense, the practice that encourages the participation of all stakeholders must be effective, as well as other forms of interaction and dialogue. The evaluation process inexorably shows a logic of power that precedes it; thus the qualitative-participatory approach has a place in the consolidation of the Brazilian health reform.

REFERENCES

Received: 12/19/2008

Approved: 2/3/2010

Supported by the Brazilian Ministry of Health/ Fundação Cearense de Apoio ao Desenvolvimento Científico e Tecnológico (Process N. 046/07).

The authors declare that there are no conflicts of interest.

• 1. Andrade LOM, Barreto ICHC, organizadores. SUS passo a passo: história, regulamentação, financiamento e políticas nacionais. São Paulo: Hucitec; 2007.
• 2. Ase I. La Descentralización de Servicios de Salud en Córdoba (Argentina): Entre la Confianza Democrática y el Desencanto Neoliberal. Salud Colect. 2006;2(2):199-218.
• 3. Belmartino S. Los valores vinculados a equidad en la reforma de la atención médica en Argentina. Cad Saude Publica. 2002;8(4):1067-76. DOI:10.1590/S0102-311X2002000400012
• 4. Bosi MLM, Mercado FJ, organizadores. Avaliação qualitativa de programas de saúde: enfoques emergentes. Petrópolis: Vozes; 2006.
• 5. Bosi MLM, Uchimura KY. Avaliação da qualidade ou avaliação qualitativa do cuidado em saúde? Rev Saude Publica. 2007;41(1):150-3. DOI:10.1590/S0034-89102007000100020
• 6. Bosi MLM, Uchimura KY. Avaliação qualitativa de programas de saúde: contribuições para propostas metodológicas centradas na integralidade e na humanização. In: Bosi MLM, Mercado-Martinez FJ, organizadores. Avaliação qualitativa de programas de saúde: enfoques emergentes. Petrópolis: Vozes; 2006. p.87-117.
• 7. Campos GWS. O SUS entre a tradição dos sistemas nacionais e o modo liberal-privado para organizar o cuidado à saúde. Cienc Saude Coletiva. 2007;12(Supl):1865-74. DOI:10.1590/S1413-81232007000700009
• 8. Estrella M, Gaventa J. Quien da cuenta de la realidad? Monitoreo y evaluación participativa: revisión bibliográfica. Brighton: Institute of Development Studies; 2001. (IDS Material de Trabajo, 70).
• 9. Guba E, Lincoln Y. Fourth generation evaluation. Newsbury Park: Sage; 1989.
• 10. Mercado-Martínez FJ, Tejada-Tayabas LM, Springett J. Methodological issues in emergent evaluations of health programs: lessons from Iberoamerica. Qual Health Res 2008;18(9):1277-88. DOI:10.1177/1049732308321755
• 11. Paim JS, Teixeira CF. Configuração institucional e gestão do sistema único de saúde: problemas e desafios. Cienc Saude Coletiva. 2007;12(Supl):1819-29. DOI:10.1590/S1413-81232007000700005
• 12. Santos NR. Desenvolvimento do SUS, rumos estratégicos e estratégias para visualização dos rumos. Cienc Saude Coletiva 2007;12(2):429-35. DOI:10.1590/S1413-81232007000200019
• 13. Temporão JG. A tensão entre o ideário reformista e a proposta real em construção de uma política de saúde no Brasil. Rev Bras Saude Matern Infant. 2007;7(1):9-13. DOI:10.1590/S1519-38292007000100001

Publication Dates

• Publication in this collection
  11 June 2010
• Date of issue
  June 2010

History

• Accepted
  03 Feb 2010
• Received
  19 Dec 2008