Monitoring and evaluation of Primary Health Care attributes at the national level: new challenges

Five new challenges were brought to the federal management of SUS from the establishment of the Primary Health Care Secretariat (SAPS) in May 2019, as follows: a) to expand people’s access to health facilities; b) to define a new financing model from health outcomes and efficiency; c) to define a new model of provision and training of family and community doctors for remote areas; d) to strengthen clinic and multi-pro-fessional teamwork; e) to expand computerization of health facilities and use of electronic medical records. This essay discusses these elements in light of a new evaluation model that also guides a new process of financing the Brazilian Primary Health Care (PHC). It builds on the correction of distrib-utive distortions, and also seeks to guide greater effectiveness and efficiency in public investment and quality of service provided to the population. The proposal for a new PHC evaluation and financing model was elaborated through studies of the best international examples and discussion with representatives of the National Council of State Health Secretaries (CONASS) and the National Council of Municipal Health Secretaries (CONASEMS), and with technical support from the World Bank. found in the results achieved among them, show the need to establish a new financing process that values user performance, quality, and satisfaction concerning the services provided. This essay presents the initiatives of the Primary Health Care Secretariat and the challenges for the implementation of a new model for monitoring and evaluating the attributes of primary health care, in line with the new PHC financing model.

introduction Primary Health Care (PHC) is the basis of the largest universal health systems in the world, and the citizen's gateway to the health system. It is also responsible for the integration and coordination of the necessary care. Several studies have shown that PHC can solve about 85% of community health problems 1 , using adequate technological density, and avoiding unnecessary interventions, ensuring greater patient safety. When organized under the logic of its attributes, PHC positively affects people's health, such as, for example, providing greater and better access to services; higher quality of care; greater preventive focus; early diagnosis and treatment of health problems; and reduction of unnecessary and potentially harmful specialized care 2 .
These characteristics help to achieve better health for people and sustainability for the health system. Guiding health systems towards strong PHC brings more efficiency and, mainly, ensures better results in people's health. Strong primary care is essential for a robust health system 3 .
However, unlike other health care environments with a focus on the use of dense technologies, because of its far-reaching action and focus on clinical diagnosis from the knowledge of professionals, PHC usually shows great variations in the ability to intervene in people's health problems, and leads to different results achieved by PHC in universal health coverage systems, which requires creating instruments to equalize it, and also increases the challenges of monitoring and evaluating its results as a Public Health Policy. A study conducted in 31 countries showed the intricate nature of Primary Health Care and the need to consider multidimensional aspects to assess its impact on people's health 4 .
While there were several good examples of PHC-centered health systems around the world at the time SUS was established, with proper monitoring and evaluation models, the migration of the centrality of the system to this care environment in the SUS occurred continuously, which can be observed with a brief historical retrospective. The first major PHC structuring program (Family Health Program, 1994) emerges only four years into the Organic Law of SUS (1990). In 1996, the new federal financing model (NOB 96) is established, which set a regular and universal mechanism for transferring resources to municipalities, conditioned to population size and the organization and provision of services in primary care, namely, the Primary Care Baseline (PAB) and the practice of monitoring, control, and evaluation in the SUS, overcoming the traditional mechanisms, centered on the billing of services produced, and valuing the results resulting from programs with epidemiological criteria and quality performance 5 . In 1998, the Ministry of Health published the manual for the organization of primary care 6 , and the theme of monitoring and evaluation begins to have more relevance for SUS managers. The first primary care policy (PNAB) was established in 2006. The National Program for the Improvement of Access and Quality (PMAQ) was established in 2011, and aimed to encourage managers and teams to improve the quality of health services provided to citizens of the territory, through better access and quality of Primary Care.
Although it has increased resources for Brazilian PHC, the PMAQ has been a very controversial program since its implementation. Some reports show it can induce changes, with adjustments both in the physical structure and in the service process, with modifications pointed out after the program was implemented, especially in the organization of work, concerning the material resources and infrastructure of the Family Health Strategy (ESF) and in the organization of the records 7 .
If, on the one hand, somehow, the aspects monitored by the program helped to drive improvements in the physical installations of the facilities, and brought the discussion about planning and organization of services, the difficulties in monitoring and evaluating health indicators (results presented to managers only when the team score was issued) and the large number of variables involved in the evaluation process, made the program the target of much criticism by city managers. Complicated and still hardly understood by many municipal managers, its methods caused 1,025 administrative appeals in the second cycle, all questioning the results of the assessment.
In its third cycle (2015-2017), it employed six external evaluation instruments, called "Modules" in phase 2, none of which used scientifically validated evaluation instruments that allowed comparisons with other countries. Phase 2 Module contains 1,039 questions distributed into: I -Observation in the primary care facility -316; II -Interview with a primary care team professional -257; III -Interview with primary care facility users -165; IV -Interview with a NASF professional -98; V -Observation at the primary care facility for Oral Health -136; and VI -Inter-view with an Oral Health Team professional -67. Its longitudinality is limited to three-yearly panels. Moreover, in the third cycle, the large sample of more than 150,000 users of health units is not statistically representative; that is, it has no external validity. However, on the other hand, it is an essential intentional sample for future studies, but not for the daily routine and necessary monitoring of the direction of a system financed with public resources, which requires transparency and accountability.
With the advent of the PMAQ, improvements were implemented in the supervision and evaluation of the work of the "Family Health Strategy" teams, with emphasis on the establishment of the variable performance financial incentive, which is a Quality Component of the Variable Primary Care Baseline (Variable PAB) 8 . However, many factors must still be analyzed as weaknesses, such as the choice of monitoring indicators, criteria for team adherence, characteristic of voluntary program adherence, low frequency of evaluations, dependence on contracts with universities and low reflection of the periodic evaluation with teams' daily health production, among others.
Another aspect refers to the selection and adherence process of the teams, with the possible bias of managers, who start to privilege the teams with better conditions for good results to receive incentives, to the detriment of others, for certification, not consistent with a global reality 9 . The latter are no longer evaluated, generating an insufficient snapshot of PHC's reality in the country.
A central problem in inducing the evaluation process and, consequently, improving the quality of health care, concerns how PHC is financed in the country. We can summarize that most of the resources concern four elements: the transfer based on the update of the resident population in the municipalities, as per the IBGE (fixed PAB), the transfer per team registered with SCNES (part of the variable PAB, which disregards the duplicate, triplicate or multiple count of people, in the old "A Sheets", and the inefficient national management in promoting the removal of these duplicate registers), the transfer to induce other strategies/programs, such as the school health program, Health Gym, Better at Home, among others, and the professional provision of community health workers (ACS, mentioned as provision since it is the only professional category for which the federal government fixed a salary baseline and transfers 95% of this amount, regardless of the results achieved). With the removal of the PMAQ, it can be said that the federal financing of PHC is mostly based on information self-reported by municipal managers. On the other hand, it is essential to note that despite criticism, the PMAQ strengthened the culture of assessment and pay-for-performance in the country.
Given the above, it was necessary to prepare a proposal for a new PHC monitoring, and evaluation model, which can effectively induce an improvement in the quality of Primary Health Care in the country, based on a process that is (1) continuous, (2) simple-to-apply, (3) more transparent, (4) of gradual and progressive complexity, (5) in line with the best international experiences, and (6) centered on people's needs.
While the public health system was structured as early as the post-war period in some countries, as is the case of England, the most potent direction for PHC as the core of the system started only in the late 1970s. The most significant structural changes in health systems occur in the 1980s in most of these countries, and the PHC evaluation processes undergo a series of adaptations in their models, partly through the learning shown in the results of research on these models, partly due to the constant need to readapt to new realities 10,11 . This is a point highlighted in the models of other countries, especially concerning the indicators used: the constant need for change in the evaluation process, which is justified for three main reasons: (1) changes in the population's epidemiological processes, (2) changes in the organization of health services, including the network structure and referral/counter-referral; and (3) search for the results of the indicators, which, while improving what is monitored, tends to generate a deterioration in what is not the focus of the assessment. Thus, the indicators must be modified so that new processes are incorporated into the teamwork.
Another significant difference that directly affects the evaluation model is that, in general, the federal government is the organizer, contractor and administrator of PHC services to citizens (such as England and Portugal, albeit with differences in structures and administrative sub-structures between these countries) 12,13 in most countries, unlike Brazil, where the federal government does not administer or contract services, which is the role of the 5,570 municipalities. In short, it can be said that there are 5,570 PHC administrators in the country, and while territorial dimensions are enormous, with many cultural, economic and social diversities, the heterogeneities found in the results achieved among them, show the need to establish a new financing process that values user performance, quality, and satisfaction concerning the services provided.
This essay presents the initiatives of the Primary Health Care Secretariat and the challenges for the implementation of a new model for monitoring and evaluating the attributes of primary health care, in line with the new PHC financing model.

Methods
A strategic vision and focus on results are essential requirements to strengthen government performance and increase the impact of public policies on social reality 14 . Intensive monitoring of government programs and actions can add value to public management and improve efficiency in the provision of public services 15 . The evaluation of the implementation of a policy involves the selection of supplies, process, and product indicators, and investigates the transformation of supplies used in processes and products. On the other hand, the evaluation of results analyses whether the result and impact indicators are in line with the goals and qualitative research, such as that of user satisfaction 16 .
One of the very relevant points in the formulation of the indicators is the establishment of a direct relationship with the objectives intended by the programs, since when formulating programs and actions, provision should be made for the organization of procedures for the collection and handling of specific, reliable information in all phases of the implementation cycle, allowing the construction of monitoring and evaluation indicators 17 .
In the adapted view of Bonnefoy and Armijo 18 and Jannuzzi 19 , the indicators can be: a) Supply indicators -directly related to human, material, financial, and other resources to be allocated and used in government actions, such as the number of doctors per thousand inhabitants and the per capita health expenditure, for example. b) Process indicators -intermediate measures that translate the efforts made to achieve the results, such as the percentage of attendance of a specific target audience and the percentage of released financial resources. c) Product indicators -they measure the achievement of physical goals or deliveries of products or services to the Program's target audi-ence, such as the percentage of children vaccinated against the established physical goals. d) Result indicators -measurements that "express, directly or indirectly, the benefits resulting from the actions undertaken in the context of the Program, and are particularly important in the context of results-oriented public management. Examples are the morbidity (diseases) rates, coefficient of maternal mortality". e) Impact indicators -they are comprehensive and multidimensional and are related to society as a whole. They measure the effects of medium and long-term government strategies. In most cases, they are associated with sector and government objectives.
Jannuzzi 20 mentions that the criteria for choosing indicators can be divided into two distinct groups: 1) Essential properties -they are those that any Program indicator must show, and should always be considered as choice criteria, regardless of the phase of the management cycle of the Program (Planning, Implementation, Evaluation, and other).
They are: a) Validity -the ability to represent the reality that one wants to measure and modify; b) Reliability and simplicity -easy to obtain, build, maintain, communicate, and understand by the general (internal or external) public.
2) Complementary properties: a) Sensitivity -the ability of an indicator to reflect timely the changes resulting from the interventions carried out; b) Disaggregability -the capacity for the regionalized representation of sociodemographic groups, considering that the territorial dimension is itself an essential component in the implementation of public policies; c) Economicity -the indicator's ability to be obtained at moderate costs; d) Stability -the ability to establish a stable historical series that allow monitoring and comparisons; e) Measurability and auditability.
The methods to be used for national monitoring and evaluation of PHC attributes took into account: 1) The new challenges of the Primary Health Care Secretariat (SAPS), and, particularly, expanding people's access to family health facilities and strengthening the clinic and teamwork; 2) The selection of process and result indicators as defined in the Ordinance establishing the Previne Brasil 21 Program, art. 12-D, which establishes that the following categories of indicators must be observed for the payment-for-performance: I -process and intermediate results of the teams, II -health results; and III -global PHC results. Yet, in its sole paragraph, it states that the indicators should also consider the clinical and epidemiological relevance, availability, simplicity, low cost of obtaining, adaptability, stability, traceability, and representativeness; 3) Indicators that are directly related to the intended objectives, such as strengthening the clinic and teamwork; 4) Indicators that have a known data source based on nationally-based information systems for their calculation; 5. Discussion and consensus between the three levels of management to choose the indicators, namely, federal, state, and municipal. To comply with its competencies and commitments that seek to face structural challenges, among which: (i) the expanded people's access to family health facilities, (ii) the definition of a new financing model based on health results and efficiency, (iii) the definition of a new model for the provision and training of family and community doctors for remote areas, (iv) the strengthening of the clinic and multidisciplinary teamwork, and (v) the expanded computerization of PHC facilities and the use of electronic medical records; the Primary Health Care Secretariat of the Ministry of Health, following international experiences, started the construction of a new evaluation model that could induce a process of improving people's health results, guide greater efficiency in public investment and quality of the service provided, increase the transparency of the monitoring and evaluation processes with managers and professionals, and establish a continuous and uninterrupted period for monitoring the results of all health teams.
Several international evaluation methods were reviewed [23][24][25][26] for the construction of this model, focusing on universal health systems with better results and higher organizational similarity with the Brazilian people, originating an own Evaluation Model (Chart 1). However, the provision of health services by municipalities is not a common practice in the world -in fact, in a few countries, the municipality is considered a federated entity, usually not having full financial and administrative freedom as in Brazil.
As one of the strategies, we decided to focus on the use of secondary data mainly, but not exclusively, from the Primary Health Care Infor-mation System (SISAB). This system was created in 2013 to replace the Primary Care Information System (SIAB), with the main advantage of transmitting individualized data, as opposed to only aggregated data from the previous system. While the registries could be appropriately organized individually in the municipalities, the federal bases of the SIAB did not support this type of storage, there was no unequivocal citizen identification, and only numbers were stored, since, at the time of its construction, the computational power, storage and data transmission structure available were compatible with a disaggregated model. The SISAB can be fed either by the systems provided by the Ministry of Health -eSUS PEC, electronic medical record system, and CDS, data entry model from manual recording on paper sheets -or by any proprietary/commercial system that connects to the data centralizer and transmitter, also provided by the Ministry. The use of SISAB in this evaluation process allows for a broad scope, since the health teams already send data periodically, given the legal obligation for the Ministry of Health to finance the teams. In this sense, Presidential Decree No. 9,723/2019 adds a vital element for cleaning the database and facilitating the unequivocal registration of citizens when considering the Individual Taxpayer Registration Number (CPF) as an identifier to be included in all federal databases 27 , something that had already been happening as binding element of the National Health Card (CNS), but is now being promoted.
However, the process of criticizing the data sent had to be improved to realize the use of SISAB in this model, which would allow: 1) a data feed closer to the different types of existing health teams, considering a substantial variation in their establishment than those defined in the Ministry's Ordinances; and 2) informing municipal managers better of inconsistencies or errors in registration in the electronic medical record systems used, providing an understanding of problems and possible corrections.
The first point is mainly due to the current rules of the National Registry of Health Establishments System (SCNES), which forces the change in the team model from the momentary establishment, and not in the work process. As an example, the lack of a professional nurse assigned to the team entails the removal all other professionals from the team and their direct allocation to the facility, even if the work process remains unchanged, only undermined by the temporary shortage of a professional. Consequently, the Ministry of Health does not receive the production from this team, and it is impossible to calculate indicators in this granularity since the production of the professionals will be counted only in the facility.
Thus, the new model should be based on the teams' ability to provide access to the population. According to data from the National Supplementary Health Agency (ANS), reinforced by those found in the National Health Survey (PNS-2013) 28 , about 3 out of 4 Brazilians depend exclusively on the SUS as a health plan for the direct care activities of health services. However, as of the writing of this paper, just over 90 million Brazilians were duly registered in the national primary care databases (SISAB). Part of the population is likely to have received care even with inexact or incomplete records. However, considering that the worst scenario is precisely found in the largest municipalities, the lack of technological apparatus or connectivity cannot be considered as a determining factor. Federative units such as São Paulo, Rio de Janeiro, and the Federal District, the three largest GDPs per capita in the country, are among the five cities with the lowest proportion of SISAB registrations compared to the total population. When disregarding the population covered by supplementary health, only São Paulo improves the situation but remains in the lower half of the ranking.
The cadastral list is essential to measure the number of people who are under the responsibility of PHC at some point in the territory. After the registration incentive phase, a study on how long the registration can be considered inactive from the moment that the PHC user does not receive any type of care by the health team is in the making.
The "active" users would then start to compose the real register of people of a particular team whose responsibility is to ensure longitudinal, comprehensive, and coordinated care within the health care network. This process is intended to be used as a basis for population assignment to teams, with a transition period between the number of registrations and the active popula- The IBGE uses perennial methods, allowing longitudinal analysis, makes use of nationally and internationally validated instruments such as, for example, the Primary Care Assessment Tools (PCAT), allowing the incorporation and comparison of the national assessment with the international setting. This cycle of evaluative research will allow the construction of an extensive database that can support researchers to evaluate programs and policies on a national, regional, state, and sometimes local scale (for the capitals of Brazil, Metropolitan, and "inland" regions of each federative unit). Moreover, linking demographic databases with administrative data will allow the construction of several cohorts.

Monitoring and assessment
Priority will be given to the use of the Primary Care Health Information System for (SISAB) database. This will allow the univocal identification of citizens and their PHC demands through the Brazilian Individual Taxpayer Registration Number (CPF) and also their assigned team (ESF/eAP, using the INE-CNES code).
The monitoring proposal will be monthly, with analysis and dissemination of the results of the indicators every four months at the level of granularity of the Family Health or PHC team. Indicators that enable national/international comparability, inducing quality of care, will be used. Duplicate entries will be managed with integration with other national databases of the Federal Government.
Source: Elaborated by the authors. tion. It is essential to highlight that this does not interfere in the reference of the territory, nor the population-based territorial actions carried out by the teams, but it will undoubtedly encourage adjustments in places where the population does not use the local health service, promoting greater balance in the work of the teams. Moreover, it will give citizens the option of choosing their primary care provider, recovering free will as a major principle of life in society in the interaction of people with the SUS. Currently, in making efforts to make people the center of health systems, the free choice of PHC provider is one of the indicators selected by the Organization for Economic Cooperation and Development to measure this objective.

Selection of indicators
Furthermore, a regular, continuous, and qualified process of monitoring and evaluating indicators that will monitor important but still deficient PHC points will be initiated, considering the current possibilities of the database structure. Among these will be elements of maternal and child care, preventable diseases, PHC-sensitive hospitalizations and care for chronic diseases, elements in which low-cost but relevant technical training interventions generate enormous impacts on the health system and the life of the general population. The selected indicators will be based on their clinical and epidemiological relevance, process indicators, and intermediate results of the ESF, health outcome indicators, and global PHC indicators. For this set of indicators, monitoring will be carried out every four months (same periodicity as other SUS management instruments), with granularity at the team level, with gradual targets that consider the current stage of each health team and weighted values corresponding to the difficulty of reaching the indicator.
At first, seven payment-for-performance-related indicators ( Table 2) were selected and agreed at the 10 th Regular Meeting of the Tripartite Commission (CIT) 29 for 2020, while other indicators are being discussed and evaluated for use in 2021 and 2022, as provided for in Ordinance GM/MS N° 3,222, of December 10, 2019 30 .
Performance indicators will be given marks in comparison with the current situation of the municipality, followed every four months, and compared with the target agreed between the federated entities, but always above the values observed to improve the results. The related monitoring indicators will not generate transfers to the municipalities but will help to understand the results obtained in the performance indicators, either because they are causes or their consequences, or because they are closely related. The selection of indicators considered the current database model of SISAB in such a way that the majority can be calculated in a recent historical series down to the team level -as disaggregated as possible.
It is also important to mention the creation of information panels that will be made available for the use of health managers and professionals for the monthly and continuous monitoring of health indicators and the registration base of each team.

evaluation from the individual perspective
Also noteworthy is the use of instruments to assess the quality of care and patient experience (in population-based surveys), with international and national recognition and validation, such as the Primary Care Assessment Tool (PCATool) 31 , the Patient-Doctor Relationship Questionnaire (PDRQ-9) 32 , a questionnaire that assesses the doctor-patient relationship from the perspective of the patient in the context of PHC, and the Net Promoter Score (NPS) 33 , which has already been used in several areas, including private health, such as the most uncomplicated way to assess customer experience and fidelity. These instruments and their global indicators will be incorporated in 2022 in the Ministry of Health's group of routine monitoring indicators.
The success of this model relies on the possibility to measure the performance achieved, using metrics that are clear, feasible for the local reality and public. Ideally, they should be accurate and timely indicators to the desired performance criterion, sensitive to variations in team performance, and resistant to manipulation or fraud 42,43 .
Moreover, effective governance arrangements are an essential prerequisite for the success of any program and require support mainly in their implementing phase for their success. In this sense, SAPS technical teams will support the municipalities with the most significant difficulties in achieving good performance, both for improving the management of the clinic and the entire work process of the teams.
The participation of municipalities is mandatory in this new payment-for-performance model of the Brazilian PHC, thus including all the teams linked to PHC services.

Final considerations
The new PHC evaluation model proposed by the Ministry of Health seeks to include monitoring and evaluation at the base of the financing process. Furthermore, it aims to be more straightforward, more transparent, and continuous than the model currently adopted, with a short set of indicators of increasing introduction and progressive complexity, giving health managers and professionals the time to adapt. To this end, a se- ries of changes are being made to databases and capitation systems, generating a higher capacity for data analysis at all levels of management. This model is based on international PHC experiences, notably the United Kingdom and Portugal, observing their successes and errors perceived over decades, incorporating the well-demonstrated need to financially encourage the maintenance of an active user base as a system beacon, minimizing the risk of unwanted effects arising from the selection of specific indicators. This is how Brazil starts to incorporate into the PHC guidelines what is more concrete in PHC assessment in the world, advancing safely and adequately structured, always with the citizen at the core of the system, and valuing efficiency in public spending.