Policy on Integrative Health Practices of Federal District in Brazil: Evaluability Study

Integrative and Complementary Practices in Health include different health care forms and have been present for more than decades in health services in the Federal District, where the implementation assessment of your policy is not completely defined. To assist in understanding reality, an Evaluability Study was carried out collaboratively, involving discussion with stakeholders, questionnaires and consultation of documents. The policy components were identified, organized and modeled. A simplified-logical-model and one of the operational-logical-models, validated by the participants, are presented in this article. For policy evaluation, the initial aspects of the modeled logical implementation chain were prioritized. It was identified as fundamental the promotion of the actors’ knowledge about the set of resources to the implementation of the Integrative Practices services and about the related intersectoral responsibilities. Formative, qualitative and collaborative assessments are suggested. The modeling carried out in the studies of this policy, shed light on the study of similar ones.


Introduction
Traditional and Complementary Medicine (TCM) originate from various parts of the world and have their use recommended by the World Health Organization (WHO) 1 .In Brazil, they are called Integrative and Complementary Health Practices (Pics), a term that refers to complex medical systems and other therapeutic resources that contribute to the global promotion of human care, including the expansion of individuals' co-responsibility for health 2 .Since 2001, the Federal District (DF) has defined Integrative Health Practices (PIS) as the set of technologies that address human health in its multidimensionality, promoting its role as protagonist, co-responsibility, emancipation, freedom and ethical attitude, to promote, maintain and recover one's health 3 .
The National Policy on Integrative and Complementary Practices (PNPIC) for the Unified Health System (SUS) in Brazil was published in 2006, guiding the implementation of these practices in the country and sharing responsibilities with state management, but without defining or standardizing its evalua-tion2.Partial national monitoring takes place through computerized systems 4 , but municipalities are responsible for local monitoring of implementation and results 5 .
In 2015, 527,953 individual consultations were registered in the Brazilian Primary Health Care information system in Pics, in 1,362 municipalities 6 .However, the growth and expansion of Pics in the country does not mean institutionalization and sustainability of the PNPIC, without which there is fragility and instability of supply and difficulty in monitoring and evaluation to contribute to safety and quality 7 .
The multifactorial context of mismatches in the implementation of policies for integrative and complementary practices in health 7- 12 , added to the lack of definition of ideal mechanisms for evaluating these practices 4, [13][14][15][16] , makes studies necessary to better understand their operating logic and seek viable strategies for such evaluations.
The Evaluability Study (EA) has the potential to assist planning, contributing to the feasibility of proposed actions in the field of public health, helping to develop realistic objectives and to provide quick and low-cost feedback on implementation.It does not intend to generate evidence on effectiveness, but to identify the logic of the program, in addition to mismatches in the intervention 17 .
The EA is of particular value to managers who wish to promote organizational learning, which can serve as an instrument to transfer a program to new leadership, and to anyone planning an evaluation 18 .Also, by generating understanding of the mechanisms through which the intervention achieves its results, it may indicate necessary adjustments in objectives, activities or resources, increasing its potential to meet the needs of society 17, 19 .
With more than three decades of history since the beginning of the institutionalization of these practices in public health in its territory, the DF can be considered a pioneer in Brazil 20 .In the last quadrennium of management, 80% of the Primary Care Managements in the DF offered PIS 21 .The District Policy for Integrative Health Practices (PDPIS) -as the object of this study -enabled the translation of a reality that can contribute to the states of the federation.
Due to its strong potential to contribute to the implementation of Pics policies, the EA presented in this article was carried out, with the objective of expressing the PDPIS operating logic and identifying aspects of its implementation for monitoring and evaluation.This publication follows the quality standard suggested by Baratieri et al. 22 .

Materials and methods
The PDPIS 23 EA was carried out using a mixed research method, using document consultation, research meetings, a policy strategy spreadsheet and a questionnaire, following the steps described in box 1. Box

Checking plausibility, validating and adjusting the design
The set of information obtained and organized in a logical way must be plausible and in accordance with the reality encountered.
Analysis, adjustment and validation of the elaborated policy logicmodels.
2nd and 3rd Meeting with the GERPIS team.Available documents.Questionnaires.Logic-models.

Exploring possibilities for evaluation and monitoring
Interest and feasibility of monitoring or evaluation.Carrying out evaluability with policy stakeholders was a dynamic process, with information being aggregated, validated and adjusted throughout its development 17 .EA interactions allowed for adjustments in the logical design and evaluation intentions, based on the repeated use of research resources and the review of information obtained at each stage 17, 19 .EA development took place between 2020 and 2021.
In the first stage of the research -focused on involving the interested parties -the context of the study was verified, consulting the available public documents.The DF has Brasília as its only municipality, with around three million inhabitants.The State Department of Health of the Federal District (SES-DF) has central and regionalized management, with no municipal health departments.Each of the seven Health Regions has a Superintendent as the highest authority and it is up to the regional management to act on the implementation of the PIS policy, as well as other health policies.The network has an exclusive PIS basic unit and two Live Pharmacies.
The Management of Integrative Health Practices (GERPIS), central management unit of SES-DF, was responsible for creating and updating the PDPIS.During the study, it was the unit responsible for promoting its implantation and implementation in the SUS-DF, including related educational processes.
The PDPIS, published in 2014 after public consultation, appreciation and approval by the SES-DF Management Collegiate and by the DF Health Council, included: Acupuncture; Art therapy; Self massage; Phytotherapy and Medicinal plants; Hatha Yoga; Homeopathy; Lian Gong in 18 therapies; Anthroposophical Medicine and Therapies; Meditation; music therapy; Reiki; Shantala; Tai Chi Chuan; and Integrative Community Therapy.In 2019, the following were also included in the DF health network: Laya Yoga, Stress Reduction Technique®, Ayurveda 3, 24 .In 2022, Auriculotherapy 25 .
Each PIS had one or more public servants acting as District Technical Reference (RTD), linked to GERPIS, which contributed to the development of clinical management processes, with the qualification of PIS facilitators and with permanent education 26, 27 .The health units had the figure of the PIS Facilitator, a qualified professional who organizes, supervises, informs, executes and monitors the implementation of the PIS individually or in groups 28 .
In a meeting, the study and concepts used were presented to the GERPIS team.The research questionnaire was applied and validated, three participating Health Regions were randomly selected, information was obtained for selection and contact with other actors involved, and the groups of study participants were intentionally defined, based on the main people involved in the implementation of this policy 29 .
Care professionals and central and regional managers of the SES-DF participated in the study, in addition to members of the Health Council of the DF.User representatives on the DF Health Council were the only stakeholders external to SES-DF.Stakeholders exclusively linked to secondary or tertiary care were not consulted, as primary care has historically been emphasized in the implementation of PIS 1-3 .
For the second stage of the research -clarifying the intention of the policy and starting the modelling -the GERPIS group provided documents from the sector.In a meeting, the objectives and guidelines of the PDPIS were identified and aligned, following the views of the participants.The other stakeholders involved answered the survey questionnaire.
Official monitoring reports of the 2016-2019 District Health Plan and regulations, which are publicly accessible, were consulted.Also GERPIS internal documents, from 2020, containing its work processes 30 and records of meetings for the preparation of a new text for the PDPIS.
In view of the third stage -exploring the reality of PDPIS execution -, in addition to the documents and questionnaires, GERPIS professionals expressed their views on each of the 72 PDPIS strategies.A spreadsheet was prepared for this purpose, containing: execution, feasibility, expected results, how to measure results, pertinence in the guideline to which they belong and in relation to the policy objectives, clarity, which operational axis they could be part of, among the proposed axes, activities developed and comments.
In the fourth stage of the research -checking plausibility and adjusting the design -, in light of Leviton 17 , Thurston and Ramaliu 18 , Wholey 19 and Rossi, Lipsey and Freeman 31 , the elements were rearranged and some original texts of the policy were modified, in order to elaborate the first complete version of the models intervention logic.
This process was based on the detailed material obtained.No information was discarded, so the most frequent ones served as the basis for modeling.The consultation of documents and regulations confirmed and facilitated the organization of contributions from the group of people involved.
The responses obtained in the previous steps were organized in an Excel spreadsheet for frequency analysis.The only open question in the questionnaire ratified information obtained.The proposed operational axes were used for classifying and aligning the strategies, as some generated doubts when reading or were too complex.
Some policy strategies integrated the model guidelines, others remained as strategies, and most were shifted as actions in each strategy.Five actions carried out that were not included in the PDPIS text were included, and twelve strategies were discarded by consensus on their non-implementation.
The Simplified Logical Model (MlogS) explained the relationship between the fundamental elements of the PDPIS, while the Operational Logical Models (MlogAt), one for each policy strategy, explained the execution of activities related to its implementation identified during the study (figures 1 and 2).
The second and third meetings with the GERPIS group allowed for a deeper discussion on the models under construction, which were analyzed and validated by the group, after presenting the results of the questionnaires.There was a consensus suggestion for adjustments to better describe the reality of policy implementation.
Modeling was improved to increase emphasis on three aspects of policy implementation considered by the GERPIS group to be more relevant than represented in the proposed design: educational activities, community participation, and sharing information about PIS.
The fifth and sixth stages of the study -exploring possibilities -were initiated in the last meetings.Priorities and feasibility of monitoring and evaluating the PDPIS were discussed, based on the most frequent responses in the survey questionnaire and the logical models validated by the GERPIS group, as well as on the challenges for implementation and monitoring.All the prepared material was made available so that the participants of this group could register other answers or ideas, closing the stages of the research.

Results
Of the total number of study participants, 38 answered the questionnaire, 25 (65.8%) of whom were female.The complete level of education of the majority was specialization (52.6%), followed by higher education (26.3%), master's (13.2%) and high school (7.9%).Restricting the analysis only to those who participated in the group of PIS facilitators, 52.9% had a postgraduate degree and 11.8% had no higher education.
The 10 years tenure of 77.1% of the 35 SES participants' and their diversified professional background help to understand the institutional reality.The three user representatives on the Health Council were participants outside the SES.
GERPIS was made up of administrative technicians, a manager, RTD holders and collaborators of the PIS modalities, a senior civil servant responsible for monitoring and a multiprofessional postgraduate resident.The lead author of this study was part of the team as RTD.
During the stakeholder consultation, there was a lack of clarity in some guidelines and strategies in the policy text, with repetition of ideas.The document referred to the context at the time of its preparation, requiring updates, and presented a structure with little connection between its components, with strategies that were not hierarchical, prioritized, or clearly related to objectives or results.
3 guidelines were outlined in the first modeling with the GERPIS group.Later, one on professional education in PIS was added.The final version of the proposed MlogS of the PDPIS (figure 1) presents four guidelines, which define the lines followed in the operationalization, each one with an objective and two strategies for the construction of action plans.This is the first known modeling of the PDPIS elaborated in 2014.In the dynamics of the model's operation, the products of the strategies are also feeding and feedback resources of the model and the desired results can exist if there is joint achievement of objectives.Purpose of the PDPIS (major objective): to promote, using the PIS, a health culture, with autonomy and empowerment of people and communities for self-care, in the SuS-DF, increasing the satisfaction of health professionals and users, the rationalization of health actions and the resolvability of the system.Impact of PDPIS: Contribution of PIS to a culture of health, with autonomy and empowerment of people and communities for self-care, increased rationalization of health actions and system resolution, satisfaction of health professionals and users, in the SuS-DF.

Strength
Source: Own elaboration.
The text of the PDPIS generically predicted the budget guarantee as a strategy for its institutionalization. Funding had been carried out by various sources, but nonspecific and with little definition, which is why MlogS does not make it explicit in the first guideline, translating it as an absent resource in the logic of the policy's operation, despite the trend of change in this scenario, which will require updates of the logic-model.
The word strategy came to represent the main sets of actions aimed at certain results, implying a reduction of the 72 in the text to only 08 strategies in the model.While the policy follows its purpose as a major objective, the strategies follow the specific objectives in each guideline.The design of the current functioning of each strategy, with the actions developed, even if partially, irregularly or in an initial phase, and their expected effects was represented in eight MlogAt, exemplified in figure 2.

Measurement of results
Regular offer of qualification courses for health workers in the SES-DF network.Number of events, participants and PIS actions or services carried out in public or philanthropic services outside the SES-DF.
The text of the policy only mentioned GERPIS for PIS management, care, teaching and research, without making explicit the responsibilities of other stakeholders, which weakens intersectoral actions inside and outside the SES-DF.However, growing movements of decentralization of management focused on PIS, such as standardization, action of supporters at the three levels of care and expansion of a basic unit exclusively for PIS, strengthen its implementation and monitoring.
The partnerships for the management or implementation of the PDPIS were also explicit, when 76% of the respondents in the different groups indicated the existence of, on average, 3.5 partnerships each, in their scope of action, mainly with schools or universities, community groups and other health units.
As for PIS services, they follow the same institutional norms used in the management of other health policies implemented in the DF.However, it did not seem clear to the participants of this study how the PIS were organized in terms of decentralization of management and responsibilities, in order to guarantee intersectoral commitment and the different resources needed to implement the PDPIS.
Regarding monitoring, the District Health Plans (PDS) included PIS offer indicators and the 2019 Annual Management Report indicated 100% achievement of the objective regarding the strengthening of PDPIS actions in the last quadrennium 21 .However, in the opinion of the GERPIS team, such indicators were limited to absolute numbers that do not translate or explore enough aspects of reality and the inadequacy of Pics codes in the SUS information systems leads to underreporting of services 32 .
Despite the limitations, the team highlighted that the indicators played a fundamental role in mapping SES-DF Network services and allowing the planning of new services.By integrating the PDS, they contributed to discussions on priorities in health management and to the expansion of PIS information, from senior management to the community.
Following it has favored the recording of other important data for monitoring, on the facilitators, number of assistance, intersectoral and educational actions, among others, thus existing a partial qualitative monitoring.Current monitoring is consistent with the proposed MlogS guidelines, despite not sufficiently supervising the expected effects in each of them, as explained by the GERPIS group.In assessing the evaluability, the plausibility of the program is central to determine its potential to be evaluated, since it indicates whether it is feasible to achieve the proposed objectives with the activities foreseen in the continuous causal chain of the logical model 17,33,34 .

SAÚDE DEBATE
Stakeholders agreed, for the most part, on the main elements of the policy, which allowed modeling, linking them in a logical way.With the first four objectives of the PDPIS original text, 95% or more of the participants agreed totally or partially; with the fifth, 87%.No participant strongly disagreed.All declared themselves interested in implementing the PDPIS.
Another relevant data from the study was that 73.7% of the participants were at that time or had already been users of any of the PIS available by the SUS in the DF.This result corroborates the monitoring data, demonstrating implementation and access to PIS in the service network.The interest in implementing other PIS within its scope of action was registered by 73.6%, in a similar percentage for the different groups consulted, and the lack of interest may be linked, in the opinion of the GERPIS group, to the current implementation management limitations, such as operationalization of services, monitoring and technical follow-up.
Almost 80% of the participants stated that the community helps with the implementation, however 50% agreed totally or partially that there are professionals in their area or workplace who voluntarily make it difficult.
Several participants worked in the same places, so this aspect needs to be further explored to be better understood.
The lack of support from managers was emphasized by participants in the questionnaires and meetings, with implications for the availability of resources and the functioning of services.Part of the participants reported never having been a PIS user or practitioner, which can be associated with a lack of knowledge, which is also a necessary resource for the implementation of the policy and can be a factor that hinders its implementation, increasing the importance of the policy.
The implementation of the PDPIS requires the availability of knowledge for the network, which, despite not being so explicit in the original text, not being uniform among the PIS modalities and not reaching all units, was part of the actions developed over the years, favoring the maintenance of activities with PIS in the network, which was confirmed by all data collection mechanisms.
Of the five activities added to the logical models in relation to the text of the PDPIS, all refer to the dissemination of information and knowledge about PIS.The model becomes plausible as these actions are included.
The response options provided for prioritization of evaluation by the participants were grouped into three groups: precursory aspects of the implementation of PIS services; aspects related to the functioning of these services; and aspects related to the expected effects with the implementation of the PDPIS (table 1).Source: Own elaboration.
The first set predominated in the general total of answers, with emphasis on the availability of resources and training or professional regulation for PIS.However, this did not have a uniform response among the consulted groups, as it was not prioritized by groups of central managers.
Only for the GERPIS group did the second set of aspects predominate in the answers, but the operationalization and improvement of PIS services was the third most prioritized option for evaluation by the participants in general, registered by all groups.
Apparently, the central management of PIS perceives itself to be in hold of issues that are precursors to the implementation of services that the rest of the stakeholders involved in this study are not.This reinforces the need for education on the use of PIS as care technologies.At the same time, monitoring does not seem to be enough to provide central management with sufficient information about the operation of network services, in order to improve management.
The knowledge and perception of the functioning of the PIS services, the mechanism of action and the rationale for these practices, seems to be a critical point in the implementation of the PDPIS.Without understanding and acting on this node, the implementation and execution of services becomes difficult, without which their purpose cannot be achieved.
It is noteworthy that the development of the study allowed expanding the vision of managers and others involved with the PIS on the policy and its implementation 20 , as well as its results fostering communication and knowledge for all who access them.
The evaluative questions, the last stage of the EA carried out, must translate the desire and need for understanding of the actors, in addition to being able to show directions that add value to the evaluated object, recognizing its results or illuminating its limitations 35 .Two central questions were identified: • Do the actors involved know and exercise their responsibilities in providing the necessary resources for the implementation of PIS in light of the institutional norms of the SES-DF?
• Do public servants qualified to provide assistance with PIS and their managers feel that they have the intellectual resources needed to perform the services?

Discussion
The centralized responsibility in GERPIS and the little definition of financial resources are aspects identified for improving the implementation of the PDPIS, which can be favored with this study.However, the implementation has been maintained and expanded over time, with a network of actors internal and external to the SES-DF, including users and those who work in a technical and educational way.
The structure of the PDPIS text included almost all the elements described by the Ministry of Health36 as necessary to fulfill its purpose.This study enabled the recognition and modeling of its constituent elements, advancing in relation to the necessary logical organization for monitoring and evaluation.
The PDPIS guidelines identified in this study coincide approximately with the components described for the Recife Municipal Policy for Integrative Practices 37 .They "express achievement ideals and guide strategic and priority choices" 38(104) .They indicate lines of action to be followed or paths of action for the elaboration of plans, programs, projects and activities that will make the policy operational, serving as a foundation for the actions of managers 36 .
The guideline on guaranteeing access to homeopathic, anthroposophic, medicinal plants and herbal medicines, in the original text, despite the growth and appreciation of the two units of Live Pharmacy implemented, was not feasible, due to the unavailability of resources, technologies and structure, aspects mentioned by Mendes et al. 39 .
To explore aspects of evaluation, monitoring or improvement of the policy, it is necessary to consider its innovative proposal in the current national context of health services and its implementation limitations, with clear weaknesses in monitoring.Even so, the offer and monitoring of these practices in the DF stand out, considering that the data available in Brazil on Pics is scarce and even the offer is still timid, despite the positive effects for users and services that use them 15 .
The logical design constructed and presented here can support other planning instruments, which can be part of or derived from the text of the policy, favoring the existence of parameters, criteria and indicators for planning, monitoring and evaluating its execution, in addition to specifying quantitative and qualitative goals, that may or may not be included in official government plans.
Considering that a policy is a general planning instrument, eliminating the need to quantify objectives, targets and resources 36 , the use of more targeted and detailed management instruments than the policy itself would allow for easy readjustment and faithful representation of the implementation reality over the years, without the need to adjust the core of the policy, its purpose and its guidelines.The set of modeled information allows a first look at the implementation reality, but requires periodic updates to represent reality.Likewise, the modeling of educational actions with managers and professionals may be able to assist in decision-making for future adjustments, with the potential to qualify the response of the health system 40 .
The modeled intervention is comprehensive and complex, provoking questions and reflections that require flexibility and adjustments to the particularities of each context involved 41, 42 .In this sense, the generated models were able to logically represent the main components for the operationalization of the policy under study, but could not bring about many advances in relational aspects that interfere with its implementation.
The human aspects need to be explored, as the interprofessional relationships themselves can affect and be positively affected by the implementation of Pics in health units, for a complex of reasons, including the hierarchy of powers in the traditional training of professionals in contrast to the greater horizontality in the dissemination and use of knowledge of Pics 11 .
Despite the specific Pics policies in the country, the implementation often occurs independently by the interested professionals themselves 10, 43 , lacking resources for its realization.For this reason, studying Pics management can be a tendency to support the growth and quality of the offer 43 , since the practice implemented, financed and developed directly by professionals, without management support, would not characterize a State policy, generating instability in the offer 7 .
Even with a good level of education -65.8% of participants reporting a specialization or master's degree -specific training for Pics is one of the limitations to the expansion process.There is little national regulation, insertion of the subject in teaching is incipient and experiences in permanent education in Pics are local 9,10 .Access to inputs is also fundamental for the quality of services and the resolvability of health care 12, 14 .
It is known that there is a political and technical lack of preparation of health professionals to work with Pics in the SUS, being necessary, for its consolidation in the network, educational processes that train professionals in line with the SUS guidelines and with the principles of Collective Health 15, 44 .The training of professionals for Pics in the SUS should receive special attention, so that there is responsibility and understanding of its potentialities and limits, avoiding trivialization or misuse 43 .This is not restricted to PIS.Other studies on health promotion identified the need to strengthen formal education and expand articulated intersectoral actions 45 , in addition to the desire to expand material and immaterial resources to consolidate processes 46 .
Thus, with the purpose of answering questions and gaps in knowledge about the intervention, its operation and results, the importance of formative evaluations is highlighted, seeking to improve the program in the course of its execution and with the involvement of the actors 42,47, 48 .
It is suggested to focus on the set of resources necessary for the operation of PIS services in the health network, so that managers and other actors also appropriate intellectual resources on PIS, obtaining knowledge about the functioning of these services in the network, in order to complement their health care management framework.

Final considerations
Carrying out the study made it possible to clarify and align the elements of the PDPIS in a coherent way, in the current context, with the consensus of the interested parties involved.This makes it possible to explore aspects to be evaluated, monitored or improved.
The participation of those interested in the construction and validation of the modeling carried out facilitates the understanding and execution of elementary aspects of the logical chain of implementation of the policy throughout the network, which can have repercussions on improving its operation in the short, medium and long term.
In addition, the generated logical models can be adjusted whenever there are changes in the scenario, as well as be used for planning, executing, monitoring and improving the implementation of the PDPIS, in a network, to achieve its purpose.
The modeling of the district policy makes it possible to advance towards studies of each PIS and its operationalization in the Health Regions of the DF.Other levels of health care and the context outside the SES -limitations of this study -could also be more easily explored.
It is suggested to encourage the set of resources needed to implement the PDPIS together with conventional health services, with emphasis on knowledge about PIS and related sectoral and intersectoral responsibilities.
The elaboration of action plans derived from the policy can be a good opportunity to agree on these responsibilities, favoring decentralized actions to meet local needs.It is hoped that this article can contribute to the development of PIS in the country.

Figure 1 .
Figure 1.Simplified logical-model of the PDPIS 2014, from the Evaluability Study of the policy.Federal District, 2021

Figure 2 .STRATEGY 4 :
Figure 2. Operational logical model of strategy 4 of the PDPIS 2014, from the Evaluability Study of the policy.Federal District, 2021 OBJECTIVE: Educar profissionais para implantação e uso qualificado de PIS no SUS do DF.STRATEGY 4: Qualification of professionals to apply contemporary and traditional scientific knowledge of PIS.Resources • People with technical, traditional, educational and PIS management knowledge, responsible for activities • Material, technological and human resources • Establishment of criteria for qualification in PIS • Political and management support and interest Activities Product Results for each action | RIO DE JANEIRO, V. 47, N. 137, P. 116-132, AbR-JuN 2023

Number of participants, selection criteria and forms of participation in the Evaluability Study, DF, June 2021 Participants Expected No. Effective No. Selection criteria and strategy Form of participation
Box 2. Number of participants, selection criteria and forms of participation in the Evaluability Study, DF, June 2021

Effective No. Selection criteria and strategy Form of participation
3.0000.5240,this research project was approved by the Research Ethics Committees of the Sergio Arouca National School of Public Health and the Health Sciences Teaching and Research Foundation, in the Federal District.
Figure 2. Operational logical model of strategy 4 of the PDPIS 2014, from the Evaluability Study of the policy.Federal

Table 1 .
Aspects indicated by the 38 stakeholders as priorities for evaluation in relation to the implementation of the PDPIS SAÚDE DEBATE | RIO DE JANEIRO, V. 47, N. 137, P. 116-132, AbR-JuN 2023