Realist evaluation for programs and services in the health area: an integrative review of the theoretical and methodological literature

Objective: to identify and analyze the concepts of the realist evaluation and the methodologies recommended for its development in the health area. Method: an integrative review, which included theoretical and methodological studies published in the following databases: COCHRANE Library, EVIPNet, Health Systems Evidence, LILACS, PDQ-Evidence, PubMed, Rx for Change, and SciELO, in addition to Teses-CAPES and Google Scholar, for the gray literature. The mediation category underlay the analysis. Results: 19 references were included, published between 1997 and 2018. It is an innovative proposal to direct the process of evaluating health programs, interventions, and/or policies, with the democratic participation of the parties involved, such as users, workers, managers and researchers; it proposes to elaborate theories about what works, for whom, in what context, and how. The mediation category indicated the need for these theories not to be restricted to the micro-context, but to incorporate the elements of the social macro-structure to which they are connected. Conclusion: It is indicated that the realist evaluation is to be conducted in 21 stages. It takes into account qualitative and procedural methods, which makes it powerful for understanding human and social relationships in the context analyzed. Theories that come from evaluating the functioning of the programs analyzed have greater explanatory chances if they are built by reference to the social totality.


Introduction
The evaluation of complex health interventions, as is the case with the public health policies and programs, is considered a challenge, especially given the assumption that it must organically monitor the creation and implementation of these interventions (1) .
The World Health Organization stresses the importance of evaluations based on the human rights principles and advocates the involvement of the socalled stakeholders (parties who are interested in the changes promoted by the programs and policies), including beneficiaries, as they can contribute to a better understanding of the processes that bring about changes in a given reality (2) .
Complex interventions are often informed by elements of experience and are dependent on the resources of those who make health decisions. On the other hand, it is known that assumptions about the success of this type of intervention must be better understood through evaluation processes, since they occur concretely within the scope of social relationships, allowing to ascertain the plausibility of the intervention and assist the evaluators in deciding what should be prioritized (3) .
Procedural monitoring, with access by the team of evaluators to intermediate outcomes, is considered essential in the evaluation of complex health programs (4) .

The evaluations indicated for complex interventions
involve stages that range from the identification of health needs to the design and implementation of the programs and policies. In addition, it is necessary that the evaluation process takes into account the discussion of the political priorities and considers collecting data at diverse moments, to capture changes over time (3) .
The traditional models for assessing public programs and policies do not have such attributes. Coming from predominantly positivist frameworks, they focus on the interests of the organizations and institutions, the logic of human resources, and the cost-benefit ratio. In this perspective, the structure is vertical, with the evaluator playing a central role and being generally external to the context of the evaluation (5) .
The realist evaluation aims to overcome this traditional approach. In this sense, it involves qualitative and quantitative components, based on theory, to promote the implementation of policies and programs in specific contexts (6) . The realistic review, which shares the same theoretical perspective with the realist evaluation, argues that the best evidence must come from a theoretically oriented and locally situated process (7) .
A literature review mapped the concepts of the realist evaluation as applied to health systems research, based on primary studies that used this evaluation methodology. The authors recommend greater clarity regarding the definitions of mechanisms and context, two elements considered structural in the realist evaluation, in addition to the outcome (8) .
Thus, this review aims at identifying and analyzing the concepts of the realist evaluation and the recommended methodology for its implementation in the health area.

Method
The question in this review is the following: What are the concepts and stages of the realist evaluation (6) Presenting outcomes; (7) Interpreting and discussing outcomes; and (8) Disclosing outcomes (10) .
In this sense, once the first phases that demanded After identifying the studies, the references were selected by title and abstract, full reading, inclusion/exclusion, and data extraction by the team of four reviewers, with at least two of them working independently. Data extraction was performed using an instrument composed of the following items: (1) Title; The evaluation of the methodological quality of the studies included was not carried out, as the object of study was of the theoretical type, and there was no inclusion of empirical studies.
From the point of view of collective health, the theoretical approach of historical and dialectical materialism was adopted in this study (11) , with the essential conceptual elements of the realist evaluation being analyzed according to the mediation category.
The authors of this article declare that they have no conflict of interest.

Results
428 references were identified from data sources, manual searches, and references of the references, as shown in Figure 1.
After excluding duplicate and unavailable publications and selection by title and abstract, 47 references were analyzed in full. In total, 19 references were included, as shown in Figure 2.  It develops a contextual understanding which reveals the mechanisms that generate different outcomes. It seeks to understand what types of evaluations are useful to which audiences, under which political conditions. The realist evaluation can be applied using the outcomes directly in making decisions about programs; using the outcomes to influence the way a program or its effects are seen; or using the outcomes to justify the decision about the programs. 3 Mark, Henry, Julnes (1998) Did not describe other data (14) The emerging realist evaluation estimates the effects of programs and assesses activities, such as which social values are served by certain programs. Thus, it seeks to identify the mechanisms underlying the effects of the program, the conditions under which these mechanisms operate, and the types of individuals for whom they operate. 4 Tilley (2000) Did not describe other data (15) It provides information to support public policy decision making and implementation. The realist evaluation question comprises the following: What works, for whom, and in what circumstances? Based on an understanding of how the measures will produce varying impacts on different circumstances, it is believed that the policy maker will be better prepared to decide which policies to implement under which conditions.

Kazi, Rostila (2002)
England Evaluation studies (16) It incorporates the main mechanisms, contexts, and components of the programs in the evaluation process. One of the main contributions of the realist evaluation paradigm is the concept of inserting the evaluator in the organizational process/structure. The realist evaluation considers the influence of the social relationships and organizational structures that make up the open system on the program's outcomes. As it is an open system, it is necessary to identify some regularities, that is, characteristics, factors, and mechanisms that lead to better or worse outcomes, and to identify the conditions under which the causal mechanisms would be activated to produce the outcomes. 6 Pawson, Tilley (2004) Did not describe other data (17) An evaluation guided by theory, with an explanatory search. In the evaluation process, the theories are tested with the purpose of refining them. The authors highlighted the need to understand the nature of the programs and how they work, considering the following: 1) The nature of the programs and how they work; 2) Basic concepts for understanding programs that involve mechanism, context, and results. 3) Strategies and methods of the realist evaluation; 4) Presentation and use of the outcomes of the realist evaluation by the policy maker, in order to understand the issues related to the policy, practice, and organizational limits involved in the implementation of a program. It is strongly linked to the Emancipatory Practice Development programs, supporting effective research questions that will test research outcomes and inform the possibility of transferring mechanisms in different contexts. The result depends on the context as it interferes with the mechanisms. It is based on the principles of realism and seeks to apprehend what is true (mechanisms that may or may not trigger), real (events that may or may not be observable but that exist), and empirical (evidence of experiences and observations made). The explanations mainly require interpretations of qualitative data to discover the reasoning and circumstances of the actors in specific contexts, not in their abstraction, which necessarily involves the participation of stakeholders and the identification of the local history. It requires the theories and/or assumptions implicit in a program to be made explicit in order to determine what and how to assess. It implies identifying theoretical assumptions, resources, and activities of the program, related to the mechanisms that lead to short, medium, and long-term outcomes. 9 Keller, et al. (2009) Sweden Business (20) It must be applied by those who wish to plan and implement innovations considering the receptive context for change, the readiness of the system for innovation, power relations, and the external socio-political context. After the innovation has been implemented, the realist evaluation can be applied to provide an explanation of the outcome patterns, depending on the mechanisms and on the contextual constraints.
Kontos, Poland (2009) Canada Rehabilitation (21) The critical realist evaluation derives from critical realism, an approach that is advocated for implementing evidence-based innovation in health. The evaluation is inherent to the implementation process and not something out of place to measure outcomes. The context and the process are considered, that is, the conditions that promoted or hindered the changes. These can be assessed by combining quantitative and qualitative data, which will promote an understanding of why the intervention worked, for whom, and under which circumstances. 11 Coryn, Noakes, Westine (2011) USA Evaluation (22) Five principles for a theory-oriented evaluation: 1) Theory-oriented evaluations/evaluators should create a plausible program theory; 2) Theory-oriented evaluations/evaluators should create and prioritize evaluation questions around a program theory; 3) The theory of the program should be used to guide the planning, design, and execution of the evaluation considering relevant contingencies; 4) Theory-oriented evaluations/evaluators must measure the constructs postulated in the program theory; 5) Theory-oriented evaluations/evaluators must identify breaks, side effects, determine the effectiveness (or ineffectiveness) of the program, and explain cause and effect associations between the theoretical constructs.  (23) It starts from the precept that social organization occurs in the form of systems. Social systems are open: the elements can enter and leave the system. As a result, any event has many causes and, at the same time, can have many consequences. It also means that every result of a program is the result of multiple causes. The findings are likely to focus on a subset of mechanism-contexts-outcomes. In general, they can indicate that: • a specific intervention works separately; • it is implemented in different ways; • it is more effective with some groups than with others; • it will have a greater use in one location than in another; • it has intentional and unintentional consequences; • its effects are likely to last. 13 Pawson, Manzano-Santaella (2012) United Kingdom Sociology and Social Politics (24) Guided by theory and capturing the outcomes of all the interventions, aiming to identify what works, for whom, in which circumstances, under what aspects, for how long, and why. The complex range of outcomes must be explained to verify the program's effectiveness. Another objective of the realist evaluation is to improve the programs, distinguishing the effectiveness of the implementation (effective or ineffective). 14 Luskin, Ho (2013) USA Did not describe (25) Social improvement must be considered as the objective of the collective evaluations so that program developers, participants, policy makers, and the general public can make decisions about programs and policies.

15
Souza (2013) Singapore Education (26) It is possible to consider a social program as the input that will reconfigure or differently activate the underlying causal mechanisms within pre-existing social structures to generate change or a different potential within the action context. An action context comprises aspects of structure, culture, agency, and relationships.

16
Manzano (2016) United Kingdom Did not describe (27) The research process will begin by creating theories, which will be tested, refined and re-tested and, in this iterative process, the understanding of the real world is also refined.

Wong, et al. (2016)
United Kingdom Primary Care (28) It must explain the underlying theories of a program, developing clear hypotheses about how and for whom the programs can work. It implies collecting data, not only about the impacts of the program or about the program implementation processes, but also about specific aspects of the context that can affect the intended and unintended outcomes of the program and about the specific mechanisms that may be creating changes. The evaluation must be properly described: what it consists of, who the target is, who provides it, what the geographic reach is, what is expected to be achieved, and so on. The data collected must include information about program impacts and implementation processes, specific aspects of the program context that can affect the program outcomes and how those contexts shape the specific mechanisms that may be creating changes. When seeking information from the participants, it is assumed that different participants have different perspectives, information, and understandings about how programs should work and whether they actually work. The realistic methodology is well suited to the study of Community-Based Participatory Research. General and specific limitations of the realist evaluations must be explained so that the readers can interpret the findings based on them. Points or limitations imposed by any changes made to the evaluation processes must also be reported and described.

18
Wong, et al. (2017) United Kingdom Primary Care (29) It collects specific aspects of the context that may impact on the intended and unintended outcomes of the program, and the specific mechanisms that may be changing the outcomes.

19
Wong (2018) United Kingdom Primary Care (30) It is initiated by the construction of a theory, that is, an explanation of how, why, for whom, in which contexts, and on which basis an intervention is designed to "work". The result of any phenomenon is derived from the context and from the (C+M=O) mechanism. Realist evaluations are research methodologies that explicitly and consistently link the context to the outcomes and set out to address complexity issues.  (22) .
The CMO configuration has the potential to introduce a broad and complete picture of what is happening, in order to elucidate the essential elements that enabled the use, or not, of interventions, programs, and/or public policies (13) , assuming that Rev. Latino-Am. Enfermagem 2020;28:e3255.
Another innovative aspect is the purpose of understanding the configuration of outcome patterns obtained by implementing the interventions. The realist evaluation studies aimed to present the changes resulting from the implementation of interventions and how such measures were produced and introduced to modify the context and balance of the underlying mechanisms (15,20) .
In order to understand the investigated reality, the realist evaluation must begin with theories, which will be tested and refined in a cyclical and iterative way, being structured in the form of proposals about how the mechanisms occurred in contexts to produce outcomes (6,(17)(18)(19)21,27) . In this sense, programs/ interventions/policies are evaluated based on the changes produced in the individuals, subgroups, and contexts involved, in addition to identifying the social and cultural resources that are necessary to sustain the changes (6) . For the realist evaluation, it is necessary use it (22) .
In this way, the evaluation allows for a circuit that can be guided by different strategies for understanding the reality under study (6,12,19,21) . The choice of the data collection methods must be guided by the theory, in order to test the assumptions/theories and to unveil the patterns and regularities of the program, through observations, data collection, data analysis, among others (6,27) . In complex programs, random sampling, randomized clinical trials or quasi-experimental projects may not be able to identify elements that interfere with the participation of those involved in the program, because the intervention has unexpected processes that cannot be predicted in advance for statistical purposes and do not capture outcomes and contextual elements (20,27) .
In this sense, it is necessary to include the evaluator in the process, who has the task of understanding and testing the theory studied (16,19) . This believing that public discussion informs society for making socially responsible decisions. Therefore, the evaluation is not a condition for deciding on the merit of a social policy, but it supports democratic decisionmaking processes (14,25) .
The critical realist evaluation derives from critical realism, an approach that is advocated for implementing evidence-based innovation in health. In this perspective, it is highlighted that the evaluation is inherent to the implementation process and not something out of place to measure outcomes. The critical realist evaluation emerges more explicitly to show that it is part of the broader implementation process considering the conditions that promoted or hindered the changes (21) .
In short, the realist evaluation has the potential to support decision-making and the creation of public policies, as the outcomes are presented according to the different contextual realities of pre-existing social structures, to generate a change that includes structures, culture, and social relationships (13,15,26) .  Using the studies included and the analysis category, the specific guidelines for each stage were described, considering that the stages can occur in an overlapping and iterative way. 15. Details of the participants (28) 16. Main results (28) 17. Theory Test (17) Discussion and final considerations (28) 18. Summary of the findings (28) 19. Strengths, limitations, and future directions (28) 20. Discussion and main conclusions (28) 21. Funding and conflict of interests (28)  findings and conclusions (28) ;

Introduction:
4. Presentation of the study theme: Explain the purpose of the evaluation with secondary data (28) ;

Theory of the program:
Describe the initial program theory (or theories) that underpin the program, policy, or initiative (28) ;

Evaluation questions, objectives, and focus:
Indicate the question(s) and specify the objectives for the evaluation. Describe whether and how the program theory was used to define the scope and focus of the evaluation (28) ;

Justification for using the realist evaluation:
Explain why a realist evaluation approach was chosen and (if relevant) adapted (28) ; 13. Data analysis: Describe and justify which method of analysis was used, how the program theory was developed, supported, refuted and refined, and whether the analysis changed during the evaluation (28) .
There is not just one suitable analytical method, as it depends on the theories proposed and on the availability of data (28) . In the realistic program theory, different outcome patterns are expected to exist for different groups or contexts within the program, and the analysis tests these theories (23) ; 14. Ethical approval: Indicate whether the realist evaluation required and obtained ethical approval from the relevant authorities, providing details as appropriate.
Explain the reason if it is not necessary to conduct the evaluation (28) .

Results:
15. Details of the participants: Report (if applicable) who participated in the evaluation, details of the data they provided, and how the data was used to develop, support, refute or refine the program theory (28) ;

Main results: Present the main results, linking
them to the contexts, mechanisms, and configurations of outcomes. Show how they were used to develop, test, or refine the program theory (28) ; interest of the evaluators (28) .
The aforementioned stages, integrated from the studies gathered in this review, allow guiding the development, as well as the preparation, of the final report of a realist evaluation (17,23,28) .

Discussion
It was assumed in this work that the potential of the realist evaluation lies in its ability to capture the interactions inherent to the CMO complex.
This type of evaluation highlights that the outcomes found in a given evaluation process are not linearly transferable to other realities, as they depend on the interaction of particular social processes (23) .
The interactions, as analyzed by the mediation category in this work, represent the articulation between the parts of a complex totality and, at the same time, the movement between the singularity and the totality, reflection processes about a certain reality, captured by its essence and not only in the appearance realm (32) .
Mediation constitutes the ontology of the social being that is based on the own movement of the categories of reality, and not on logical ideal concepts, being present in the sociability of the social being.
Therefore, it is sustained in the perspective of the relationship between man and nature and, in this way, the transformation of nature by man (work) is a condition of human existence (31) . In the realist evaluation, part of the underlying mechanisms, which can interfere with the outcomes of a given project, depends on the reflection of this social being on the proposed intervention and on the reality in which it is inserted.
This field of mediations takes shape in the particularity in which the dialectic between the universal and the singular occurs. It is in this mediation field that the singular facts are related to the laws of universality, which is configured from the reality of the singular.
The individual, being the smallest unit of the social totality, has infinite variations and, therefore, has great complexity and particularities (31) .
The particular represents the expression of the categories of mediation between the singular individuals and society (31) . In the realist evaluation, the context is located in the particular dimension since, depending on its conditions, there will be or there will not be the activation of underlying mechanisms, which may techniques, and strategies that the subject becomes aware of in order to penetrate the plots of reality as a possibility to transform it (32) .
Regarding the limitations, realist evaluations of property demarcation and education programs in penitentiaries, initially described by the creators of the realist evaluation, were not based on theoretical frameworks or totalizing theories that explained what the root is of the problems that lead to the creation of these programs. Nor was it discussed, for example, how society is organized and the mode of capitalist production, which are generators of social inequalities. Thus, in its origin, the realist evaluation showed certain pragmatism when looking for cause and effect relationships (6) .
Other limitations concern the philosophical dimension of critical realism, which emphasizes that the production of several authors who follow the critical realistic theory is restricted to non-historical abstraction by not using any tools that make it possible to understand how objective social structures can be transformed and how to carry out the transformation. Thus, it states that critical theorists do not always make assumptions about the object of investigation when analyzing social relationships, which makes the starting point of investigation arbitrary. In this perspective, the causal mechanisms are self-referenced and the structures only exist as a result of human behavior and that the causal powers would thus be relational (33) .
Such limitations are still under discussion, even by one of the main theorists of critical realism -Roy Baskar -who has sought to add the dimensions of historicity and totality, and the dialectic, considering the contradictions of reality (34) . It has been Rev. Latino-Am. Enfermagem 2020;28:e3255.
From the milestones of the mediation category, however, there is a need for these theories to be elaborated incorporating the elements of the social macrostructure to which the mechanisms are connected.
For collective health, this has essential implications for research and health policies because taking social totality as a reference expands the explanation about reality and, consequently, the possibilities for transformation.