Implementation of the Pharmaceutical Services Strategic Component supply chain: Building the evaluationABSTRACT
This article presents the building process of an evaluative study aimed at analyzing the implementation of the Pharmaceutical Services Strategic Component (CESAF) supply chain. This funding component of Brazil’s pharmaceutical policy targets individuals with diseases whose control is strategic for the Unified Health System (SUS), including some classified as neglected diseases. To guide the evaluation, three models were developed: the Pharmaceutical Supply Chain Theoretical Model; the CESAF’s Operation Theoretical Model; and the Logic Model for Analyzing the Implementation of the CESAF Supply Chain. Both the modeling and the evaluation matrix were developed based on a literature review and refined through a panel of experts. The final matrix comprised 74 indicators, organized into five dimensions: planning and management, forecasting, procurement, storage, and distribution. The proposed evaluative framework is useful for identifying contextual factors that may affect the availability of medicines at SUS dispensing units. The tools developed fill a gap in the literature.
KEYWORDS
Pharmaceutical services; Drugs, essential; Health evaluation; Neglected diseases
RESUMO
O objetivo do artigo é apresentar o processo de construção de um estudo avaliativo para análise da implementação da cadeia de abastecimento farmacêutico do Componente Estratégico da Assistência Farmacêutica (Cesaf ). Esse componente de financiamento da Assistência Farmacêutica destina-se a pessoas com doenças cujo manejo é estratégico para o Sistema Único de Saúde (SUS), algumas consideradas negligenciadas. Para construção da avaliação, foram desenvolvidos três modelos: Modelo Teórico da Cadeia de Abastecimento Farmacêutico; Modelo Teórico de funcionamento do Cesaf; e Modelo Lógico de Análise da Implementação da Cadeia de Abastecimento Farmacêutico do Cesaf. A modelagem e a Matriz de Análise e Julgamento da avaliação foram elaboradas a partir da revisão da literatura e da apreciação por painel de especialistas. A matriz final foi composta por setenta e quatro indicadores, organizados em cinco dimensões: planejamento e gestão, programação, aquisição, armazenamento e distribuição. O modelo avaliativo construído mostrou-se útil para identificar as características do contexto que podem interferir no abastecimento de medicamentos às unidades dispensadoras do SUS. As ferramentas apresentadas buscam suprir lacuna da literatura.
PALAVRAS-CHAVES
Assistência farmacêutica; Medicamentos essenciais; Avaliação em saúde; Doenças negligenciadas
ABSTRACT
This article presents the building process of an evaluative study aimed at analyzing the implementation of the Pharmaceutical Services Strategic Component (CESAF) supply chain. This funding component of Brazil’s pharmaceutical policy targets individuals with diseases whose control is strategic for the Unified Health System (SUS), including some classified as neglected diseases. To guide the evaluation, three models were developed: the Pharmaceutical Supply Chain Theoretical Model; the CESAF’s Operation Theoretical Model; and the Logic Model for Analyzing the Implementation of the CESAF Supply Chain. Both the modeling and the evaluation matrix were developed based on a literature review and refined through a panel of experts. The final matrix comprised 74 indicators, organized into five dimensions: planning and management, forecasting, procurement, storage, and distribution. The proposed evaluative framework is useful for identifying contextual factors that may affect the availability of medicines at SUS dispensing units. The tools developed fill a gap in the literature.
KEYWORDS
Pharmaceutical services; Drugs, essential; Health evaluation; Neglected diseases
RESUMEN
El objetivo de este artículo es presentar el proceso de construcción de un estudio evaluativo para el análisis de la implementación de la cadena de abastecimiento farmacéutico del Componente Estratégico de la Asistencia Farmacéutica (CESAF). Este componente de financiamiento de la Asistencia Farmacéutica está dirigido a personas con enfermedades cuyo manejo es estratégico para el Sistema Único de Salud (SUS), algunas consideradas desatendidas. Para la construcción de la evaluación, se desarrollaron tres modelos: el Modelo Teórico de la Cadena de Abastecimiento Farmacéutico; el Modelo Teórico de Funcionamiento del CESAF; y el Modelo Lógico de Análisis de la Implementación de la Cadena de Abastecimiento Farmacéutico del CESAF. El modelo y la matriz de análisis y juicio evaluativo se realizaron a partir de la revisión de la literatura y de la valoración por panel de expertos. La matriz final estuvo compuesta por setenta y cuatro indicadores, organizados en cinco dimensiones: planificación y gestión, programación, adquisición, almacenamiento y distribución. El modelo evaluativo construido se mostró útil para identificar las características del contexto que pueden influir en el abastecimiento de medicamentos a las unidades dispensadoras del SUS. Las herramientas presentadas buscan cubrir una laguna existente en la literatura.
PALABRAS CLAVE
Servicios farmacéuticos; Medicamentos esenciales; Evaluación en salud; Enfermedades desatendidas
Introduction
In Brazil, despite advances in pharmaceutical policies1 and the pursuit of integrated Pharmaceutical Services (AF), problems related to the planning, procurement, storage, or distribution of medicines can still be found across the various AF funding components, which can affect the availability of essential medicines2–6.
More than 15 years after the creation of the AF7 financing components, the Basic Component of Pharmaceutical Assistance (CBAF) and the Specialized Component of Pharmaceutical Assistance (CEAF) have received more attention in the scientific literature compared to the Strategic Component of Pharmaceutical Assistance (CESAF). However, this component had the worst results related to drug availability (< 40%), according to data from nationally applied research4.
CESAF is regulated by Ordinance GM/MS No. 4,114/20218 and covers a wide range of diseases of epidemiological importance9. It includes drug treatment for the control of tuberculosis, leprosy and focal endemic diseases such as malaria, leishmaniasis and Chagas disease, among others. It also includes antiretroviral drugs from the STI/AIDS Program and the National Program for the Prevention and Control of Viral Hepatitis10. Therefore, it includes users considered neglected or with poverty-related illnesses11. Medicines that require high investment in manufacturing and also some monopolies9.
The lower-than-expected availability of certain medicines at dispensing units4 points to weaknesses in the implementation process of CESAF’s pharmaceutical supply chain (CAbFar), which involves actions by all three levels of government8.
Fragile pharmaceutical supply chains can hinder access to medication-based treatment and jeopardize health12,13. In the public sector, this problem may stem from a combination of factors such as inadequate funding, a lack of incentives for maintaining stock, an inability to accurately forecast demand, issues regarding supply and pharmaceutical market regulation, the diversion of medicines to the private sector, or inefficient logistics and distribution systems13,14. To mitigate these barriers, there has been a rise in research aimed at developing tools and strategies to strengthen the CAbFar, thereby informing policy formulation and improving health system performance15,16.
Thus, it is pertinent to conduct an implementation analysis of CESAF’s medicine supply chain. Understanding the factors influencing this implementation can help ensure that the availability of essential medicines within CESAF is effectively achieved.
To evaluate an intervention, it is important to define and model it by clarifying its relevant elements, such as objectives and goals, inputs, activities, outputs, target audience, and outcomes17,18. Modeling involves explaining the often complex links that unfold over time among structures, processes, and outcomes18.
This article aims to present the process of designing an evaluative study to analyze the implementation of CESAF’s CAbFar, that is, its modeling and the development of the Analysis and Judgment Matrix (MAJ), thereby creating instruments that provide a theoretical foundation for the evaluation.
Material and methods
An evaluative study of a descriptive and exploratory nature was conducted, addressing the process of modeling and constructing the evaluation’s MAJ.
Given the complexity of evaluating CESAF’s CAbFar and proceeding from the premise that the evaluation focuses on two distinct interventions (CAbFar and CESAF), three models were developed: a) the Pharmaceutical Supply Chain Theoretical Model (MT-CAbFar); b) the CESAF’s Operation Theoretical Model (MT-CESAF); and c) the Logic Model (MLog) for the Implementation Analysis of CESAF’s Pharmaceutical Supply Chain.
The development of the MT-CAbFar aimed to identify, based on the literature, the essential activities comprising the CAbFar and the key factors influencing its effective operation. The MT-CESAF sought to understand CESAF from the perspective of the responsibilities of the various levels of government. The MLog aimed to integrate these two components by establishing how CESAF’s CAbFar should be implemented, illustrating the relationships between available inputs and planned activities to generate and deliver the expected outcomes.
The intervention’s theoretical models and the MLog were developed based on a literature review. These models served as the foundation for creating the MAJ, which includes indicators for inputs, activities, outputs, and immediate outcomes of the dimensions identified as priorities during the modeling process, found in scientific and grey literature, as well as in regulations and legislation.
The initially proposed Mlog for evaluation and the MAJ were reviewed by an expert panel comprising key stakeholders (managers, pharmacists, or other professionals) committed to developing logistics activities within the scope of CESAF, as well as researchers experienced in AF.
The invitation included the Informed Consent Form and information regarding the methodology to be employed. Participants were sent a working document for prior review and familiarization with the research topic. This contained the Mlog and MAJ representations.
The consensus meeting was held via Zoom and attended by five experts. Consulted literature indicated no consensus regarding the ideal number of experts; however, consensus techniques involving face-to-face interaction19,20 require highly structured interactions and fewer than 15 participants20.
Consensus on the Mlog and MAJ items was reached synchronously, involving discussion and the completion of the working document during the meeting, following the approach used by Pereira21. Before the meeting concluded, participants were given the opportunity to express their views on the topics discussed and offer suggestions.
The participants’ comments and suggestions underwent descriptive analysis to serve as a basis for potential additions or modifications22.
To foster a participatory evaluation, staff members from the Pharmaceutical Care Management Coordination unit of the Superintendence of Pharmaceutical Care and Strategic Supplies (CGAF/Safie), responsible for managing CESAF at the state level in Rio de Janeiro – the setting for this evaluation – were invited to join a follow-up consensus meeting. However, a change in management during the data collection period made it impossible to continue their involvement without disrupting the research process. Consequently, the research team reconvened to make the necessary adjustments and ensure the evaluation could proceed.
This research was approved by the CEP-Ensp under Opinion No. 6,501,402 on 11/10/2023.
Results and discussion
Theoretical Model of the Pharmaceutical Supply Chain
To prepare the MT-CAbFar, the following questions were asked: What are pharmaceutical supply chains and what activities are defined in the literature as essential? What are its goals? What factors can influence its proper functioning? Answering these questions was crucial to understanding the theory about how CAbFar works, providing support to understand how it should be structured and organized and, also, which aspects of the context should be analyzed in the evaluation study.
Five documents presenting CAbFar models and seeking to understand their key drivers in developing countries were selected and analyzed. Among the selected documents, one was a guide developed by the United States Agency for International Development (USAID)23 for managing the supply chain of health products, while the others were scientific articles13,24–26. Information regarding the documents is summarized in table 1, in chronological order of publication.
Documents selected to underpin the theoretical model of the pharmaceutical supply chain, based on essential activities, objectives, and key influencers of the pharmaceutical supply chain in developing countries
Analysis of the texts reveals that, despite the use of varying terminology for CAbFar, its core activities and objectives, there is a consensus that this intervention is not limited merely to warehouse and fleet management. Indeed, the conceptualization appears to be expanding over time.
In the first publication analyzed23, the authors chose to use the terms ‘supply chain’ and ‘logistics’, often interchangeably, throughout the text. They view the CAbFar as operationally constituted by a logistics cycle encompassing the key activities of product selection, quantification and procurement, inventory management, storage, distribution, and customer service. Quality monitoring must be integrated into every activity of the logistics cycle, addressing not only product quality but also the quality of the work performed. For the authors, the goal is to ensure that customers – both immediate recipients and the end-users seeking health services – receive the right product, in the appropriate quantity and quality, at the right place and time, and at a suitable price. Thus, the effective functioning of supply chains enhances the impact of public health programs by improving the quality, effectiveness, and efficiency of care.
Yadav13 views the CAbFar as a complex combination of public, private, and non-governmental institutions engaged in programming, procurement, and distribution activities. He incorporates the concepts of availability and coverage into its objectives to ensure not only quality and efficiency but also equity in the treatment provided to the population.
Throughout their text, Fialho and Martins24 use the term ‘public sector supply chain’. They view the CAbFar as part of a complex web of organizations whose roles are subject to social and political demands from both the community and the government. They identify resource allocation decisions and logistics functions as fundamental to supply chain performance. Although they do not explicitly define the essential logistics activities, their data collection instrument comprises variables related to programming, storage, distribution, and dispensing – elements that make up the AF cycle27. In this study, the ultimate goal of the CAbFar of the evaluated program is to ensure the provision and receipt of medicines, thereby making them available for rational use by the population.
Sunyoto et al.25 view the pharmaceutical supply chain as an ecosystem of organizations, people, technology, activities, information, and resources that come together to ensure the most efficient product delivery from the point of manufacture to the end user, thereby guaranteeing access to diagnosis, prevention, and treatment. They add the supply of inputs, production, and selection to the essential activities previously described by other authors. Consequently, they discuss issues such as production capacity, the existence of sole sources for certain medicines, and barriers related to Research & Development (R&D) – factors that can affect the supply-demand relationship described by Chaves et al.14.
It is worth noting that Sunyoto et al.25 also include ‘Service and utilization’ among the essential activities of the CAbFar, although they do not discuss them further in the article. Perhaps because, in the countries studied, the barriers arise during the preceding activities. In any case, they provide the most comprehensive description of CAbFar activities compared to the other authors mentioned.
Furthermore, for Sunyoto et al.25, the supply chain is situated within the concept of access proposed by Frost and Reich28, taking into account the dimensions of adoption, affordability, quality, and availability.
Finally, Subramanian’s26 perspective on the CAbFar encompasses all the previously described concepts, including policies, human resources, information, and technology, to supply and ensure products of appropriate quality from the supplier to the customer. While the specific core activities are not explicitly defined, this question can be addressed by analyzing the earlier publications, particularly Sunyoto et al.25.
According to Subramanian26, a robust pharmaceutical supply chain must ensure sustainable healthcare that is both geographically and economically accessible. This reflects the adoption of concepts regarding healthcare sustainability, geographic accessibility, and affordability. Geographic accessibility was implicit in the document prepared by USAID23, which described the need for medicines to be in the ‘right place’ for customers. Affordability had been mentioned by Sunyoto et al.25.
For Subramanian26, the objectives of a CAbFar should not be limited to immediate outcomes. Therefore, the author advocates expanding the supply chain’s objective to include long-term improvements in quality of life and overall well-being.
‘Well-being’ is included in the World Health Organization’s (WHO) expanded concept of health and can be defined as:
A positive state experienced by individuals and societies. Similar to health, it is a resource for daily life and is determined by social, economic, and environmental conditions29(10).
According to the WHO, human health and well-being are also interconnected with sustainable development29.
Since a society’s well-being is linked to the equitable distribution of resources, Subramanian26 points out that achieving a state of well-being may be more difficult for more vulnerable populations. For this reason, he suggests that countries create a favorable environment by proposing an integrated model of CAbFar facilitators and their role in improving overall well-being through sustainable healthcare.
From Subramanian’s perspective26,30, achieving sustainability in CAbFar requires – beyond consideration of economic, social, and environmental aspects – prioritizing stakeholder collaboration, continuity of care (measured by health outcomes), and the monitoring of quality initiatives regarding products, services, and processes.
Regarding the facilitators for achieving this sustainability, in addition to the ‘investment’ category described by the WHO29, Subramanian26 suggests the categories of policies, people, and practices, as well as systems and data.
Other publications13,23–25 also present factors that may influence the proper functioning of the CAbFar (table 1). Although there is no consensus regarding all the terms used, most of the items are already included, in some way, in the categorization established by Subramanian26. This demonstrates the comprehensiveness of that model regarding the objectives and potential contextual influences on the CAbFar.
At the national level, there are few studies analyzing the factors that affect CAbFar operations within public health programs. This gap in the Brazilian literature underscores the importance of the study by Fialho and Martins24, which warrants special attention in the modeling process.
Fialho and Martins24 sought to explain how institutional elements and network design influence the supply chain logistics performance of the Farmácia de Minas Program. Using structural equation modeling, the authors demonstrated that institutional context factors – particularly those of a regulatory and cognitive nature – can affect decisions regarding capacity and resource allocation. Regarding network design, the way facilities were organized proved decisive in translating institutional elements into performance outcomes24.
At the end of the article, Fialho and Martins24 suggest that future research consider variables linked to instability resulting from changes in government and the lack of coordination among the three levels of government. In the Brazilian context, the ‘Communication and Coordination’ aspect – which appears in the model proposed by Sunyoto25 – also proves to be a significant influence on CAbFar. Poor communication and coordination can lead to a loss of trust among the various stakeholders and compromise the supply of medicines25.
Based on the analysis performed (table 1), the MT-CAbFar was developed for the Brazilian context (figure 1).
Here, the CAbFar is understood as the set of connections among all components directly involved in logistics, both private and public sector actors across various levels of management, who perform distinct roles based on arrangements previously agreed upon by managers. The tensions and coordination limitations discussed in the literature can be further explored through the proposed assessment. Operationally, the CAbFar generally comprises the following activities: Research and Development (R&D), Registration, Production, Selection, Programming, Procurement, Storage, Distribution, and Utilization.
The black arrows surrounding these activities signify the need for coordination among them. When these operations fail, health interventions prove ineffective – even if R&D activities are incentivized, medicines are produced to expected quality standards and selected based on appropriate technical criteria, and patients receive care in suitable facilities from specialized professionals31. It is worth noting that R&D, Registration, Production, and Utilization activities operate at different levels compared to the others, as they depend on decisions that often extend beyond the health system itself.
The objective of CAbFar is understood to be, in the short term, ensuring the availability of quality medicines, in the right quantities, at the right time, and in the right place, thereby contributing to healthcare that is sustainable, geographically accessible, and financially viable for both the user and the health system. Within the Brazilian Unified Health System (SUS), CAbFar is expected to function effectively so that medicines are available to users free of charge. In this model, sustainability is understood as both the pharmaceutical supply chain’s ability to remain functional and its contribution to healthcare.
Furthermore, the chain is influenced by variables from internal and external contexts, encompassing categories such as people and practices, systems and data, institutional elements, network design, investments, policies, and regulation. These aspects must be integrated through constant, robust communication and coordination among the various levels and stakeholders to ensure quality of life and general well-being in the long term.
CESAF’s Operation Theoretical Model
To better understand the functioning of CESAF, an analysis was conducted of Ordinance GM/MS No. 4.114/2021, which regulates the standards and actions regarding access to medicines and supplies for programs managed under this component8. This ordinance outlined the responsibilities of the Federal Government, states, the Federal District (DF), and municipalities, as well as the interrelationships between them. These responsibilities were classified into the following categories: Financing; Selection; Programming; Procurement; Storage and Distribution; Dispensing; Planning and Management; and Pharmacovigilance. Based on this systematization, it was possible to develop a theoretical model of CESAF’s operation (figure 2).
Strategic Component of Pharmaceutical Services’s Operation Theoretical Model, based on the responsibilities of federal entities, in accordance with Ordinance GM/MS No. 4,114 of December 30, 2021
BNAFAR – National Database on Pharmaceutical Services Actions and Services; DF – Federal District; Rename – National List of Essential Medicines; PCDT – Clinical Protocols and Therapeutic Guidelines; PNAB – National Primary Health Care Policy; PNAF – National Pharmaceutical Services Policy; PNM – National Medicines Policy; PNVS – National Health Surveillance Policy; PNAES – Specialized Health Care Policy.
Funding, selection, programming, and procurement within the scope of CESAF are centralized at the federal level, represented by the Ministry of Health (MS); the medicines and supplies involved must be listed in the National List of Essential Medicines (RENAME), Clinical Protocols and Therapeutic Guidelines (PCDT), and other official government technical documents.
Although programming is the responsibility of the MS, this activity must be carried out in a coordinated and comprehensive manner by AF, Health Care, and Health Surveillance – following a bottom-up, integrated approach from the local to the federal level, and must also encompass pharmacovigilance activities8. Therefore, the organization and execution of CESAF activities must align with the provisions of the National Policies on Medicines, Pharmaceutical Services, Primary Health Care, Health Surveillance, and Specialized Health Care8,32. The double-headed arrows on the right side of figure 2 indicate that this coordination should guide managers at all levels in operationalizing other processes, particularly regarding the generation of information for programming, such as epidemiological data and historical consumption figures.
Once the procurement process has been executed and concluded, the MS must receive and store the medicines and supplies in a suitable location and subsequently distribute them to the states and the Federal District. These entities, in turn, must also have appropriate facilities to receive and store, prior to distributing the items to the municipal public health network. The same criteria apply to municipalities and the DF, which are responsible for storing and distributing supplies to health units, thereby ensuring a steady pharmaceutical supply across the entire public network. Subsequently, the dispensing of medicines and supplies is to be carried out under the responsibility of the municipal level and the DF8.
Additionally, the training of managers and health professionals, as well as activities related to the planning, control, evaluation, and systematic monitoring of the organization and execution of CESAF, must be conducted through cooperation among the MS, states, municipalities, and the DF8. These activities have been categorized as ‘Planning and Management’. The double-headed arrows on the left side of figure 2 also represent this need for coordination among the different levels.
The National Database of Pharmaceutical Services Actions and Supplies within the SUS (BNAFAR), managed by the MS, is a fundamental element for ensuring the effective coordination of information regarding the stock, distribution, dispensing, and procurement needs for the medicines and supplies covered by CESAF8.
Finally, through the coordination of activities across different levels, CESAF will be able to fulfill its objective – as outlined in Ordinance No. 4.114/2021 – of helping to ensure equitable access to medicines and supplies for the prevention, diagnosis, treatment, and control of diseases and health conditions that are endemic, epidemiologically significant, or socioeconomically impactful, or that affect vulnerable populations, within the scope of SUS strategic health programs8.
Logic Model for the Implementation Analysis of the CESAF Pharmaceutical Supply Chain
The Mlog for the implementation analysis of CESAF’s CAbFar (figure 3) was constructed by integrating the two theoretical models previously presented, utilizing the structure of the logic model developed by Pereira21.
CESAF – Strategic Component of Pharmaceutical Services; RENAME – National List of Essential Medicines; PCDT – Clinical Protocols and Therapeutic Guidelines; HR – Human Resources; ITS – Information Technology System.
The Mlog is a visual framework representing the relationship between available inputs and planned activities, designed to enable an intervention to achieve expected outputs and outcomes33,34. Outputs are directly linked to activities—meaning each activity has an immediate consequence. Outcomes are the changes the intervention aims to bring about; they can be subdivided into intermediate outcomes (short- and medium-term) and final outcomes or impacts (long-term)33.
Beyond demonstrating the causal relationship between these elements, a useful Mlog must acknowledge the influence of external contextual factors on the intervention’s ability to produce these outcomes34.
The ‘R&D’ and ‘Production’ dimensions found in the MT-CAbFar were not included in the evaluation’s Mlog, as they fall outside the scope of the present study.
‘Selection’ is also absent from the Mlog, based on the premise that this dimension is clearly defined for CESAF, given that in Brazil, the RENAME encompasses the medicines funded by this component. However, the existence of a ‘list of selected medicines and treatment protocols described in the PCDT’ is included as an input within the ‘Programming’ dimension.
The ‘Utilization’ dimension was also excluded, as it encompasses prescribing, dispensing, administration, monitoring, and patient adherence35. Consequently, it covers other factors – such as users’ socioeconomic characteristics, perceived needs, beliefs, and preferences – alongside barriers to access, none of which fall within the scope of this research36.
Therefore, the operational activities of CAbFar to be evaluated include the dimensions of Programming, Procurement, Storage, and Distribution—also known as the logistical stages of the AF cycle35.
The ‘Pharmacovigilance’ dimension of the MT-CESAF framework was also not considered, as it is not a specific activity of CAbFar. However, the Mlog framework lists ‘Programming carried out in a bottom-up and integrated manner’ as one of the expected outputs of the ‘Programming’ dimension.
Based on Pereira21, the ‘Planning and Management’ dimension appears as a cross-cutting element relative to the other dimensions. Actions taken within this dimension affect Programming, Procurement, Storage, and Distribution. Standardizing workflows for managers, holding regional workshops, and fostering human resource training are some examples of items included under ‘Planning and Management’.
Regarding the context, the same categories previously described in the MT-CAbFar were considered.
During the consensus meeting, it was agreed to maintain the dimensions proposed for the Mlog, as well as the context categories, without changes. The inputs, activities, outputs, and outcomes were deemed sufficient and appropriate for describing the CESAF’s CAbFar; nevertheless, some modifications were suggested.
The discussion held during the consensus meeting and the working documents submitted by the experts led to seven changes to inputs, four to activities, and five to outputs described in the evaluation’s Mlog. No changes were suggested regarding intermediate outcomes or final outcomes.
Evaluation Analysis and Judgment Matrix
In the field of health assessment, the MAJ consists of criteria, indicators, and standards. This is important for more clearly defining which aspects should be the focus of the assessment37.
The construction of the matrix for the implementation analysis of CESAF’s CAbFar took into account inputs, activities, outputs, and immediate outcomes, corresponding to each aspect of the Mlog (figure 3).
Initially, seventy indicators were developed, distributed across five dimensions and forty-seven evaluation aspects. Existing literature served as the basis for the semantic structuring, calculation methods, and judgment criteria for the indicators.
It is worth noting that the evaluation aspects are the items described in the Mlog. Each evaluation aspect may encompass more than one indicator. Conversely, some evaluation aspects found in the Mlog are not described in the MAJ due to limitations regarding the data sources used. These aspects were addressed in an exploratory or analytical manner during the implementation evaluation.
The subsequent stage of developing the MAJ involved revising its initial structure to incorporate proposals from the expert panel and other necessary adjustments.
All experts deemed the indicators suitable for measuring the implementation level of CESAF’s CAbFar. However, the MAJ required revision to align with changes to the Mlog suggested during the consensus meeting. Consequently, three new assessment aspects and four indicators were incorporated, and modifications were made to eight existing indicators. The development of the Mlog and the MAJ is an interdependent process; thus, modifying one necessitates a review of the other21.
No changes were proposed regarding the judgment criteria or the calculation methods for the indicators, as the experts considered them appropriate.
After gaining access to the database intended for the implementation assessment and better understanding the process flow of CESAF’s CAbFar across the three management levels, the need to adjust certain indicators became apparent. The inability to access some initially anticipated data sources – such as BNAFAR – It also entailed the need for adjustments.
To address this issue, the research team – comprising public health and AF researchers – convened and, by consensus, made the necessary modifications. In total, one item was excluded, four were added, and twenty-two were modified within the assessment matrix.
To enable comparison across analysis dimensions and management levels, the team also recognized the need to assign a weight to each indicator. The research team determined that the weight calculation should take into account the type and total number of indicators within each analysis dimension. Initially, weights were assigned as follows: input indicators received a weight of 2; activity and output indicators received a weight of 3; and outcome indicators received a weight of 5. The final weight for each indicator was obtained by dividing the initial weight by the number of indicators of the same type within each analysis dimension. For dimensions lacking outcome indicators, a weight of 5 was assigned to activity and output indicators (table 2). Following these adjustments, each analysis dimension could yield a maximum of 100 points.
The matrix was designed so that the degree of implementation could be assessed based on the classifications proposed by Cosendey et al.38: critical (< 25%), incipient (25–49%), unsatisfactory (50–75%), and acceptable (> 75%). The final matrix can be found in Santos39.
Conclusions and final considerations
Modeling the intervention made it possible to demonstrate the relationship between inputs, activities, outputs, and expected outcomes regarding the implementation of CESAF’s CAbFar. This was crucial for developing an evaluation model capable of generating sufficient data to describe the organization and structure of the intervention, thereby allowing for an assessment of the extent to which CESAF’s CAbFar has been implemented at the federal, state, and municipal management levels. The resulting evaluation model proved useful in identifying contextual factors that may affect the supply of medicines – particularly those from CESAF – to SUS dispensing units.
Due to a change in management at CGAF/Safie, the number of participants in the expert consensus panel for Mlog and MAJ was lower than expected, and there was a lack of effective participation from the stakeholders who would be involved in the evaluation in the future. Nevertheless, the consensus process drew upon the prior experience of expert researchers in the field of AF and a former manager who had worked on the implementation of CESAF’s CAbFar initiative in the state of Rio de Janeiro.
Mlog and MAJ had to be adapted to the data sources available to the evaluation team, which required significant additional effort to make the evaluation feasible.
Although some indicators are common to all three management levels, others differ to reflect the distinct activities performed at each level. Consequently, there were variations in the number of feasible indicators across the input, activity, output, and outcome categories for the different management levels; this issue was addressed through the application of weighting.
These limitations do not prevent Mlog (figure 3) and MAJ from underpinning other studies focused on CESAF. On the contrary, they fill a gap in the literature and are adaptable, as are the theoretical models presented. In the case of MT-CAbFar (figure 1), its use can be extended to study other components of AF financing.
In light of the above, this article has described products with the potential to underpin future studies, including the methodological process of constructing the evaluation itself and the challenges encountered along the way, as well as the models presented here.
Data availability:
The research data are contained in the manuscript itself
-
Financial support:
This article was published and translated with the support of the Graduate Program in Public Health at ENSP/FIOCRUZ, using funds from CAPES/PROEX
References
-
1 Bermudez JAZ, Esher A, Osorio-de-Castro CGS, et al. Assistência farmacêutica nos 30 anos do SUS na perspectiva da integralidade. Ciênc saúde coletiva. 2018;23(6):1937-1949. DOI: https://doi.org/10.1590/1413-81232018236.06022018
» https://doi.org/10.1590/1413-81232018236.06022018 -
2 Costa EA, Araújo PS, Pereira MT, et al. Technical issues and conservation conditions of medicines in the primary health care of the Brazilian Unified Health System. Rev saúde pública. 2017;51(Supl 2):12s. DOI: https://doi.org/10.11606/S1518-8787.2017051007106
» https://doi.org/10.11606/S1518-8787.2017051007106 -
3 Gerlack LF, Karnikowski MGO, Areda CA, et al. Management of pharmaceutical services in the Brazilian primary health care. Rev saúde pública. 2017;51(Supl 2):15s. DOI: https://doi.org/10.11606/S1518-8787.2017051007063
» https://doi.org/10.11606/S1518-8787.2017051007063 -
4 Nascimento RCRM, Álvares J, Guerra AA, et al. Availability of essential medicines in primary health care of the Brazilian Unified Health System. Rev saúde pública. 2017;51(Supl 2):10s. DOI: https://doi.org/10.11606/S1518-8787.2017051007062
» https://doi.org/10.11606/S1518-8787.2017051007062 -
5 Rover MRM, Faraco EB, Vargas-Peláez CM, et al. Acesso a medicamentos de alto preço: desigualdades na organização e resultados entre estados brasileiros. Ciênc saúde coletiva. 2021;26(11):5499-5508. DOI: https://doi.org/10.1590/1413-812320212611.27402020
» https://doi.org/10.1590/1413-812320212611.27402020 -
6 Bueno MAM, Simões TC, Luz TCB. Diferenças na disponibilidade de medicamentos prescritos na Atenção Primária: evidências do Projeto Prover. Ciênc saúde coletiva. 2022;27(3):1191-1203. DOI: https://doi.org/10.1590/1413-81232022273.38782020
» https://doi.org/10.1590/1413-81232022273.38782020 -
7 Ministério da Saúde (BR). Portaria nº 399/GM, de 22 de fevereiro de 2006. Divulga o Pacto pela Saúde 2006 – Consolidação do SUS e aprova as Diretrizes Operacionais do referido Pacto [Internet]. Diário Oficial da União, Brasília, DF. 2006 fev 23 [acesso em 2025 ago 6]; Seção 1:43-51. Disponível em: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2006/prt0399_22_02_2006.html
» https://bvsms.saude.gov.br/bvs/saudelegis/gm/2006/prt0399_22_02_2006.html -
8 Ministério da Saúde (BR). Portaria nº 4.114, de 30 de dezembro de 2021. Dispõe sobre as normas e ações para o acesso aos medicamentos e insumos de programas estratégicos, sob a gestão do Componente Estratégico da Assistência Farmacêutica (CESAF), no âmbito do SUS [Internet]. Diário Oficial da União, Brasília, DF. 2022 jan 3 [acesso em 2025 ago 6]; Seção 1:38. Disponível em: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2022/prt4114_03_01_2022.html
» https://bvsms.saude.gov.br/bvs/saudelegis/gm/2022/prt4114_03_01_2022.html -
9 Santos MBP, Luiza VL, Oliveira MA. Panorama das doenças do Componente Estratégico da Assistência Farmacêutica: dados epidemiológicos de 2016 a 2020 e oferta de medicamentos. Physis. 2025;35(2):e350227. DOI: https://doi.org/10.1590/S0103-73312025350227pt
» https://doi.org/10.1590/S0103-73312025350227pt -
10 Ministério da Saúde (BR). Relação Nacional de Medicamentos Essenciais: Rename 2024 [Internet]. Brasília, DF: Ministério da Saúde; 2025. [acesso em 2025 fev 8]. 265 p. Disponível em: https://www.gov.br/saude/pt-br/composicao/sectics/rename/rename-2024
» https://www.gov.br/saude/pt-br/composicao/sectics/rename/rename-2024 -
11 Santana RS, Lupatini EO, Leite SN. Registro e incorporação de tecnologias no SUS: barreiras de acesso a medicamentos para doenças da pobreza? Ciênc saúde coletiva. 2017;22(5):1417-1428. DOI: https://doi.org/10.1590/1413-81232017225.32762016
» https://doi.org/10.1590/1413-81232017225.32762016 -
12 World Health Organization. Equitable access to essential medicines: a framework for collective action. (Report WHO/EDM/2004.4). [Internet]. Geneva: World Health Organization; 2004 [acesso em 2022 dez 10]. Disponível em: https://apps.who.int/iris/handle/10665/68571
» https://apps.who.int/iris/handle/10665/68571 -
13 Yadav P. Health product supply chains in developing countries: diagnosis of the root causes of underperformance and an agenda for reform. Health Syst Reform. 2015;1(2):142-154. DOI: https://doi.org/10.4161/23288604.2014.968005
» https://doi.org/10.4161/23288604.2014.968005 -
14 Chaves LA, Osorio-de-Castro CGS, Caetano MC, et al. Desabastecimento, uma questão de saúde pública global: sobram problemas, faltam medicamentos [Internet]. In: Portela MC, Reia LGC, Lima SML, organizadores. Covid-19: desafios para a organização e repercussões nos sistemas e serviços de saúde. Rio de Janeiro: Editora Fiocruz; 2022 [acesso em 2022 maio 28]; p. 103-115. Disponível em: https://books.scielo.org/id/kymhj/06
» https://books.scielo.org/id/kymhj/06 -
15 Eller AJ, DiDomizio EE, Madden LM, et al. Strengthening systems of care for people with or at risk for HIV, HCV and opioid use disorder: a call for enhanced data collection. Ann Med. 2022;54(1):1714-1724. DOI: https://doi.org/10.1080/07853890.2022.2084154
» https://doi.org/10.1080/07853890.2022.2084154 -
16 Saeed G, Kohler JC, Cuomo RE, et al. A systematic review of digital technology and innovation and its potential to address anti-corruption, transparency, and accountability in the pharmaceutical supply chain. Expert Opin Drug Saf. 2022;21(8):1061-1088. DOI: https://doi.org/10.1080/14740338.2022.2091543
» https://doi.org/10.1080/14740338.2022.2091543 - 17 Mendes MF, Cazarin G, Bezerra LCA, et al. Avaliabilidade ou pré-avaliação de um programa. In: Samico I, Felisberto E, Figueiroa AC, et al., organizadores. Avaliação em saúde: bases conceituais e operacionais. Rio de Janeiro: MedBook; 2010. p. 57-64.
- 18 Champagne F, Brousselle A, Hartz Z, et al. Modelizar as intervenções. In: Brousselle A, François C, Contandriopoulos AP, et al., organizadores. Avaliação: conceitos e métodos. Rio de Janeiro: Editora Fiocruz; 2011. p. 61-74.
-
19 Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376-380. DOI: https://doi.org/10.1136/bmj.311.7001.376
» https://doi.org/10.1136/bmj.311.7001.376 -
20 Fink A, Kosecoff J, Chassin M, et al. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74(9):979-983. DOI: https://doi.org/10.2105/AJPH.74.9.979
» https://doi.org/10.2105/AJPH.74.9.979 -
21 Pereira NC. Análise da implementação dos serviços farmacêuticos na Atenção Primária brasileira. [tese]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz; 2019 [acesso em 2022 jan 8]. p. 129. Disponível em: https://arca.fiocruz.br/items/0a357e24-dc92-47a3-a786-c942cec2446d
» https://arca.fiocruz.br/items/0a357e24-dc92-47a3-a786-c942cec2446d -
22 Vitorino SAS. Avaliação da implantação da vigilância alimentar e nutricional na atenção primária em saúde: estudos de caso em municípios de pequeno porte de Minas Gerais. [tese]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz; 2017 [acesso em 2022 jan 8]. p. 329. Disponível em: https://arca.fiocruz.br/items/f02ef13e-3af5-4d9b-a09a-3169f570986b
» https://arca.fiocruz.br/items/f02ef13e-3af5-4d9b-a09a-3169f570986b -
23 United States Agency for International Development (USAID). The logistics handbook: a practical guide for the supply chain management of health commodities [Internet]. 2nd ed. Arlington (VA): USAID, DELIVER Project, Task Order 1; 2011 [acesso em 2022 jan 8]. Disponível em: https://www.humanitarianlibrary.org/sites/default/files/2019/04/Logistics%20handbook-USAID.pdf
» https://www.humanitarianlibrary.org/sites/default/files/2019/04/Logistics%20handbook-USAID.pdf -
24 Fialho RCN, Martins RS. Elementos institucionais e desempenho da logística de uma rede pública de assistência farmacêutica. Rev Adm Publica. 2016;50(5):819-841. DOI: https://doi.org/10.1590/0034-7612146817
» https://doi.org/10.1590/0034-7612146817 -
25 Sunyoto T, Potet J, den Boer M, et al. Exploring global and country-level barriers to an effective supply of leishmaniasis medicines and diagnostics in eastern Africa: a qualitative study. BMJ Open. 2019;9(5):e029141. DOI: https://doi.org/10.1136/bmjopen-2019-029141
» https://doi.org/10.1136/bmjopen-2019-029141 -
26 Subramanian L. Enabling health supply chains for improved well-being. Supply Chain Forum Int J. 2020;21(4):229-236. DOI: https://doi.org/10.1080/16258312.2020.1776091
» https://doi.org/10.1080/16258312.2020.1776091 -
27 Marin N, Luzia VL, Osório-de-Castro CGS, et al., organizadores. Assistência farmacêutica para gerentes municipais [Internet]. Rio de Janeiro: OPAS/OMS; 2003. [acesso em 2022 set 26]. 373 p. Disponível em: http://www6.ensp.fiocruz.br/visa/?q=node/4863
» http://www6.ensp.fiocruz.br/visa/?q=node/4863 -
28 Frost LJ, Reich MR. Creating access to health technologies in poor countries. Health Aff (Millwood). 2009;28(4):962-973. DOI: https://doi.org/10.1377/hlthaff.28.4.962
» https://doi.org/10.1377/hlthaff.28.4.962 -
29 World Health Organization. Health promotion glossary of terms 2021 [Internet]. Geneva: World Health Organization; 2021 [acesso em 2023 maio 7]. 44 p. Disponível em: https://www.who.int/publications/i/item/9789240038349
» https://www.who.int/publications/i/item/9789240038349 -
30 Subramanian L, Alexiou C, Nellis JG, et al. Developing a sustainability index for public health supply chains. Sustain Futures. 2020;2:100019. DOI: https://doi.org/10.1016/j.sftr.2020.100019
» https://doi.org/10.1016/j.sftr.2020.100019 - 31 Osorio-de-Castro CGS, Luiza VL, Castilho SR, et al., organizadores. Assistência farmacêutica: gestão e prática para profissionais da saúde. Rio de Janeiro: Editora Fiocruz; 2014. 767 p.
-
32 Ministério da Saúde (BR). Portaria GM/MS nº 1.604, de 18 de outubro de 2023. Institui a Política Nacional de Atenção Especializada em Saúde (PNAES), no âmbito do Sistema Único de Saúde [Internet]. Diário Oficial da União, Brasília, DF. 2023 out 20 [acesso em 2025 ago 6]; Edição 200; Seção 1:87. Disponível em: https://www.gov.br/saude/pt-br/composicao/saes/legislacao/portaria-gm-ms-no-1-604-de-18-de-outubro-de-2023/view
» https://www.gov.br/saude/pt-br/composicao/saes/legislacao/portaria-gm-ms-no-1-604-de-18-de-outubro-de-2023/view - 33 Bezerra LCA, Cazarin G, Alves CKA, et al. Modelagem de programas: da teoria à operacionalização. In: Samico IC, Felisberto E, Figueiroa AC, et al., organizadores. Avaliação em saúde: bases conceituais e operacionais. Rio de Janeiro: MedBook; 2010. p. 65-78.
-
34 Petersen D, Taylor EF, Peikes D. The logic model: the foundation to implement, study, and refine patient-centered medical home models [Internet]. Rockville (MD): Agency for Healthcare Research and Quality; 2013 [acesso em 2025 jul 31]. AHRQ Publication No. 13-0029-EF. 12 p. Disponível em: https://pcmh.ahrq.gov/page/logic-model-foundation-implement-study-and-refine-patient-centered-medical-home-models
» https://pcmh.ahrq.gov/page/logic-model-foundation-implement-study-and-refine-patient-centered-medical-home-models - 35 Oliveira MA, Bermudez JAZ, Osório-de-Castro CGS. Assistência farmacêutica e acesso a medicamentos. Rio de Janeiro: Editora Fiocruz; 2007. 97 p.
-
36 Luiza VL. Acesso a medicamentos essenciais no Rio de Janeiro [tese]. Rio de Janeiro: Escola Nacional de Saúde Pública, Fundação Oswaldo Cruz; 2003 [acesso em 2022 jan 8]. p. 227. Disponível em: https://arca.fiocruz.br/items/4c6bce87-e70b-461d-b6af-55b1c0952daf
» https://arca.fiocruz.br/items/4c6bce87-e70b-461d-b6af-55b1c0952daf - 37 Natal S, Alves KA, Felisberto E, et al. Interpretação e análise das informações: o uso de matrizes, critérios, indicadores e padrões. In: Samico IC, Felisberto E, Figueiroa AC, et al., organizadores. Avaliação em saúde: bases conceituais e operacionais. Rio de Janeiro: MedBook; 2010. p. 89-107.
-
38 Cosendey MAE, Hartz ZMA, Bermudez JAZ, et al. Validation of a tool for assessing the quality of pharmaceutical services. Cad Saúde Pública. 2003;19(2):395-406. DOI: https://doi.org/10.1590/S0102-311X2003000200006
» https://doi.org/10.1590/S0102-311X2003000200006 - 39 Santos MBPS. Análise de implementação da cadeia de abastecimento farmacêutico do Componente Estratégico da Assistência Farmacêutica [tese]. Rio de Janeiro: Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz; 2025. p. 298.
Edited by
-
Editor in charge:
Leonardo Vidal Mattos, Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro (Rio de Janeiro/RJ), Brasil. Lattes: http://lattes.cnpq.br/0343170437330734, Orcid: https://orcid.org/0000-0003-4800-0010, e-mail: leonardomattos@iesc.ufrj.br
Source: Prepared by the authors, adapted from Subramanian
Source: Prepared by the authors.
Source: Prepared by the authors, based on Pereira