Possible ways for Public Health Surveillance practices evaluation

This is an evaluative and qualitative study that proposes to investigate self-assessment evaluation as a device to analyze Health Surveillance practices through a questionnaire built by researchers, adapted from the Self-Assessment of Improved Access and Primary Care Quality (AMAQ) and available on the FORMSUS platform. Forty-one Health Surveillance workers and managers of a large municipality from São Paulo State evaluated the realms of “management”, “teamwork” and their respective sub-realms. Two categories were created to analyze the results: “Management” and “Team” in dialogue with references from Management, Evaluation and Health Surveillance. Most “management” and “teamwork” sub-realms were deemed satisfactory. Self-assessment evaluation through an applied evaluation tool was shown to be a powerful resource for the analysis of Health Surveillance practices in combination with other devices adopted by the Unified Health System (SUS). Unlike usual evaluation processes guided by quantitative markers, this self-assessable evaluative process included subjects and enabled the possibility of incorporating a new look at itself to the way Health Surveillance is carried out and support future management contracts between workers and managers.


Introduction
This study investigated self-assessment as a device for analyzing Health Surveillance practices.A self-assessable questionnaire was applied to managers and workers of the Health Surveillance Department (DVS) of the Municipal Health Secretariat of Guarulhos-SP, adapted from the Self-Assessment of Improved Access and Primary Care Quality (AMAQ), an integral part of the Primary Care Access and Quality Improvement Program (PMAQ) of the Ministry of Health (MS) 1,2 .
This study was based on the evidence of evaluative processes and work contracts between managers and workers in that municipality, such as the PMAQ and the Health Surveillance Actions Qualification Program (PQA-VS) 3,4 .
In general, health surveillance actions have been evaluated from results achieved, with reference to agreed goals among the federated entities.Epidemiological indicators are normally used to demonstrate the capacity to control disease or illness, or even a number of inspections, in the case of health surveillance 3,5 .
Other references were sought in this investigation in an attempt to understand evaluation as a formative process, where opinions of the subjects of practices are taken into account and can also be incorporated as management of surveillance work 6,7 .
The PMAQ has raised debates and investigations, with notes of correctness and misconceptions of this type of evaluation, where the transfer of resources is conditioned to the adequacy of quality parameters 8,9 .
In any case, the self-assessment component is understood as an important starting point for the development of the PMAQ, since it allows the recognition of positive and problematic realms of work organization process and teams' management.We identified, from a pedagogical dynamic, critical nodes that hinder the development of health actions in the territory, as well as gains by interventions implemented.AMAQ was based on several other evaluation tools used in Brazil and in other countries 1,2 .
Some authors have addressed the study of evaluative processes that, if guided by the inclusion and participation of the various stakeholders involved, become devices for institutional change 6,10 .Furtado 6 argues it is necessary to include the different conceptions from the viewpoint of the groups involved with a program or service, since common sense and traditional evaluation do not cover all aspects of the complex initiatives that address social and health problems.
Health surveillance is a vast field of knowledge and practices of the Brazilian Unified Health System (SUS) and the most recent legal framework, Ordinance Nº 1,378 11 defines it as: a continuous and systematic process of collecting, consolidating, analyzing and disseminating data on health-related events, aiming at the planning and implementation of public health measures for the protection of the population's health, prevention and control of risks, diseases and illnesses, as well as health promotion.
The aforementioned ordinance states that "health surveillance actions cover the entire Brazilian population" and defines its knowledge and practices cores aimed at the surveillance of communicable diseases, chronic non-communicable diseases, among other diseases and health conditions.It does not specifically discriminate the core of epidemiological and environmental surveillance; it also refers to "health surveillance of risks arising from the production and use of products, services and technologies of interest to health" 11 .
This definition is close to the formulation of Public Health Surveillance, a term used in other countries to denominate "new epidemiological surveillance", in order to avoid confusion with Epidemiology, a discipline that provides the basis for the practice of various "surveillance" activities 12,13 .
Law Nº 8,080 14 defined the concepts of epidemiological surveillance and health surveillance, which are already consolidated practices and blend with the Brazilian Public Health history 15 .At that moment of implantation of the SUS, some Brazilian Public Health authors formulated a new model of care, a new health practice, "health surveillance", which in some way incorporated practices of health surveillance and epidemiological surveillance intervening on causes, risks and damages, as proposed in the model [16][17][18] .
This polysemy of similar terms in Brazil generates a confusion between managers and workers who, since the onset of the SUS, have tried to institutionalize health surveillance in the states and municipalities, as a field, with its different knowledge and practice cores 19 which include epidemiological surveillance, health surveillance, worker's health surveillance and environmental health surveillance 15,20,21 .This is a terrain of conceptual disputes and models, especially when it comes to Health Surveillance.This has been studied by some authors, who consider it Public Health practice, with its well-defined object, articulated with other "surveillance" 22,23 activities.Others have considered it as a "neighboring territory" to the new Public Health Surveillance, as conceived in Brazil since the creation of the MS Health Surveillance Secretariat in 2003 24 .
In any case, the MS, when publishing Ordinance Nº 1,378 11  The aforementioned Working Group prepared a preliminary "Base Document" 26 , which was the object of discussion in the Health Surveillance Course, held at the XXIX Congress of Municipal Health Secretaries of the State of São Paulo in 2015, organized by the Council of Municipal Health Secretaries (COSEMS) 27 .One of the representatives of the MS submitted the proposal of the WG to expand the concept of health surveillance, incorporating the issue of "health regulation, integrality of actions and integration of surveillance" 28 .
Despite the elaboration of the Base Document, discussion on the Health Surveillance Policy in the country has not progressed throughout 2015.In May 2016, the MS published Ordinance GM/MS Nº 1017 of May 11, 2016 with the purpose of proposing guidelines for the formulation of the PNVS and the strengthening of health surveillance programs and actions 29 .In 2016, the Ministry also published Resolution Nº 535 of August 19, 2016, in which it defines the regulation of the National Conference on Health Surveillance to be held in November 2017 30 .This is a rich moment for debate, in which managers, workers and users will be able to discuss concepts, the Policy, models and practices of health surveillance.
While content of the Base Document 26 is preliminary, especially with regard to the principles, guidelines and organizational lines of the future Policy, as well as other health surveillance documents and legal frameworks served as inspiration for the construction of the sub-realms found in the questionnaire prepared for this study.

Methodology
This is an evaluative qualitative study using a questionnaire elaborated for the self-assessment of managers and workers about content associated with health surveillance practices, which was conducted in Guarulhos, a municipality in the Metropolitan Region of São Paulo and the second largest city in the state 4 .
Study subjects were DVS workers and managers.Of the 202 professionals invited by email, 41 (20%) accepted, namely, 13 of the Zoonosis Control Center, 1 of the Death Verification Service, 9 of Epidemiological Surveillance, 2 of the Board of Executive Officers; 3 of the Public Health Laboratory, 12 of Health Surveillance and 1 municipal manager.
The tool was adapted from the Family Health Strategy Quality Improvement Assessment (AMQ and AMAQ) and other health service evaluation tools, used and validated nationally and internationally 1,2 .The evaluative research should include different visions and values, facilitate and broaden the use of evaluation resources, consider the inevitable political character of research in general and of evaluation in particular, and enable those involved with the evaluated program or service 6 .In addition, it aims to produce knowledge that can guide decisions about feasibility, availability of time and resources to be applied in certain areas or sectors 10 .
The self-assessment questionnaire was conceived from four realms: municipal management, health surveillance management, surveillance service management and health surveillance team (team profile and work process), which are divided into 14 sub-realms and these, in issues (standards) that cover what is expected in terms of health surveillance quality.There were 110 questions, 50 of them addressed to managers, 28 to both managers and workers and 32 to workers, according to the competences of management, coordination and health surveillance teams.
The tool included closed and open questions, validated in a pre-test phase by DVS managers and a manager of the Municipal Health Secretariat.It was answered online, on the FormSUS platform (a public use service provided by DATASUS to create forms on the WEB), where respondents triggered the contents of the Informed Consent Form and signaled their agreement to participate in the research.Data were transposed onto an Excel database for results consolidation.
For closed questions, workers or managers assigned a grade to the standards expected for the sub-realms.These grades were consolidated based on a scale of values that represent the standards for questions or statements regarding the expected quality of health surveillance structure, processes and results.Scores ranged from zero to ten for each standard and the scale of values ranged from "very unsatisfactory", to "unsatisfactory", "fair", "satisfactory" and "very satisfactory", according to the mean scores assigned by the set of respondents.
In addition, respondents had available space to record their opinions about the item evaluated, which defined the qualitative nature of the study.These open responses were analyzed through the thematic content analysis technique, which, according to Minayo 31 and Bardin 32 is about discovering the meaning cores underpinning a communication, whose existence or frequency meant something to the object studied.Realms were taken as units of analysis and sub-realms as thematic units.Regarding the analysis phase of the results, two analytical categories were created: "Management" -subdivided into municipal management of health surveillance and surveillance services and "Team".
The Research Ethics Committee, State University of Campinas approved this study.

Results and discussion
Results show a satisfactory appreciation of health surveillance practices: Next, we will show the analysis and discussion of the results from the "Management" and "Team" categories, in dialogue with some theoretical frameworks of Health Management and Evaluation and the legal frameworks of the SUS and health surveillance.

Management
Municipal Management -Sub-realms A to D. Results refer to all municipal management actions to ensure that health surveillance fulfills its role of protecting the health of citizens, ascribed by SUS legal frameworks 11,33 .
Issues regarding the role of municipal management in implanting and implementing health surveillance and its competence in coordinating the municipal component of the National Health Surveillance bring to the fore the need to discuss a health surveillance policy for the country.Establishing a policy's concept, values, principles, guidelines and priorities can facilitate management's work by orienting and clarifying its generally relevant and complex functions.The formulation of a policy must respond to the interests of the various social groups and classes to meet the social need identified at a particular historical and political moment 34 .In the case of the Health Surveillance Policy, there are various interests in dispute and it may become clearer to managers of what health surveillance we are talking about: the one that is closer to a concept of Public Health Surveillance or Epidemiological Surveillance, as defined in Ordinance Nº 1,378 11 , or a broader concept that encompasses all the surveillance activities in the same field?
Managers are expected to carry out analyses of the health situation centered on damages, risks, socio-environmental conditionants and determinants, ensuring the protection of the population's health, risk, diseases and illnesses prevention and control, as well as health promotion, tasks not exclusive to surveillance "expert" professionals, but also to other stakeholders in health services, particularly Primary Care 35 .
The managers' self-assessment was positive regarding the management's ability to prepare surveillance actions in public health emergencies; to coordinate, monitor and evaluate the sentinel strategy in the hospital setting and develop strategies for risk and disease surveillance in an articulated way in the health care network, since surveillance actions, even if coordinated regionally or centrally in the municipalities, execute and support actions in the territories.It was also positive to ensure health surveillance's involvement in thematic networks, as well as to ensure strategies for integrating practices and surveillance work process.These new management functions of implanting and ensuring coordination in the new thematic networks are challenging 36 .
A recent study on the implementation of the Rede Cegonha revealed difficulties in integrating health surveillance practices and women's health in the region of Campinas.Fragmentation is a key point pointed out by health surveillance managers, both internally and externally 37 .Investments in management devices that value collective spaces, co-management and matrix support are strategies to reduce the "estrangements" and lack of knowledge among areas underpinning the SUS, among them health surveillance 38,39 .Patient safety can be one of the health surveillance provisions to the health services of the municipal network, in line with continuing education.
Work management topics relate to the policy of recognizing workers, such as a career development plan and rewarding teams, whether for performance, achieving agreed goals or workers' protection.Structural problems, some aggravated by the decisions of governments in charge, added to the recently announced aggravated financing of the SUS 40 create an uncomfortable setting for any municipal or health surveillance manager, as expressed by some: It is necessary to ensure the sufficient quantity of employees to maintain quality service provision.There is a work overload.
Even more robust funding for health surveillance will not be able to address these structural and institutional gaps in the various social policies.Still, leaving these problems public and analyzing them in democratic and participatory management can be a way of seeking local solutions.
The theme of participation, social control and user satisfaction was rated as "very satisfactory".They considered that management promotes and assures citizens access to information and participation in the formulation, implementation and evaluation of health policies, considering the right to health and citizenship.
From the viewpoint of health surveillance, communication and dialogue with society are often confrontational, particularly in health surveillance bodies, since the interests of the market and the State are confronted, with the latter having a health protection role 23 .A recent study addressed the media during conflict with dwellers due to health surveillance interventions in the control of visceral leishmaniasis in dogs, by indicating euthanasia as one of the measures directed at diseased animals 41 .Conflicts should not be barriers in this communication.The First National Conference of Health Surveillance may be another step in this approach between managers, workers and users.
Health Surveillance Management -Subrealms E to G. The exercise of Institutional Support was reported by managers as management technology to help teams set groups, analyze their own work and build interventions.Oliveira and Campos 39 mention that institutional support is a "methodology that is characterized by reformulating traditional management mechanisms, by adopting an interactive analytical and operational stance that would complement and transform the way of performing managerial roles as that a coordination".
It is interesting to analyze whether the practice of institutional support persists, because together with other devices such as continuing education, it seems to strengthen management, according to stakeholders of the self-assessment.
Managers considered monitoring and evaluation practice to be satisfactory, despite institutional limits.They use evaluative processes as a management tool; adopt strategies that strengthen the feeding and use of information systems; organize and adopt monitoring actions in the area of surveillance; promote the discussion of results and encourage the use of information by health surveillance teams.They also consider that they comply with the pacts established with other federative entities, according to results achieved, such as good vaccine coverage and advances in some programs such as STI, AIDS and Tuberculosis 4 .
The use of information and its incorporation into the daily routine of managers and teams, including monitoring and evaluation are crucial to health surveillance practice.It can be said that it is the core, which subsidizes planning, underpins and guides health interventions.It remains to be seen how information is used: only in annual planning and "late macro-diagnostics" 42 , sporadic and bureaucratic monitoring and evaluation, or is it really part of the daily life of health surveillance managers and workers to direct work, geared to capture and meet the health needs of the population?In this self-assessment process, it was possible to perceive the difference in the managers' and workers' perspective, especially those of health surveillance bodies, who report the very scarce use of information, monitoring and evaluation to subsidize daily actions.
Management of the Surveillance Service -subrealms H to J were answered by managers and workers who recognize the physical and equipment infrastructure as a deficit, which hinders the performance of daily activities: We serve customers in front of all professionals, in the hallway or in the meeting room, when not in use.
Aspects related to the storage, availability and sufficiency of supplies, immunobiologicals and medicines for the development of surveillance actions and the process of dispensing and controlling vaccines, laboratory kits and medicines by health services were also evaluated.
Stakeholders pointed out the manager's difficulties in providing working conditions related to structural issues.Nevertheless, the performance of surveillance actions as a whole often performed individually and with the professionals' own resources was satisfactory.
Factors discussed in this topic allow us to reflect on how health surveillance municipal management can provide working conditions for the development of increasingly complex actions under its responsibility, thus requiring adequate infrastructure to address daily and emergency situations 43 , which requires adequate funding, besides local administrative and managerial issues that make it possible to perform necessary acquisitions and maintenance.Several initiatives have been identified over the last few years to strengthen and qualify health surveillance management and financing, such as, for example, VI-GISUS stages I and II 44,45 .
Brito's 46 evaluation study on the VIGISUS Project points out health surveillance's small governance to intervene in administrative and managerial aspects of the municipal government.It is necessary to discuss the organizational capacity to execute the financing, which presupposes support among the federative entities, considering bureaucratic issues to streamline investments in the sector.
Regarding information systems, points related to the quality of the information produced, knowledge of the teams about the current systems, feeding and strategies adopted for its strengthening, as well as the use and dissemination of the information produced were addressed.
While self-assessment was satisfactory, the observations made point out issues for reflection on the work process, both of managers and teams.In addition to local specificities, this is a challenging topic to health surveillance.There are many existing health information systems, some unique to health surveillance.Information systems on vital events, live births (SINASC) and mortality (SIM) are important for the development of surveillance activities and, according to the organization of municipal secretariats, are allocated to health surveillance and other sectors 47,48 .
Statements revealed work fragmentation in the use of each information system, distancing and certain lack of knowledge regarding the information databases of vital events.On the use of information by management, they ask: What monitoring?What evaluation?
In relation to the use of data and information produced by surveillance and its use in PHC facilities: Information produced by surveillance is not always used in PHC facilities... nor in other health services... Some workers point out that managers demand production, the "pile of processes on the desk" and surveys carried out, instead of information to direct servicing health needs: The matter of concern in my sector is the number of processes performed.
The availability of several information systems, lack of standardization of data collection and management, problems with connectivity and training difficulties are known to the MS, translated in the Information and Computing Policy, with a view to overcome these problems and advance in the wide use of information in the SUS 48 .
Health information systems facilitate data collection, storage and organization to enable monitoring and analysis to identify health risks and problems in order to allow the knowledge of the health status of a given population 49 , which denotes the importance of production and use of information in the field of health surveillance as a subsidy for decision-making at various levels of the federation.

Team
Sub-realms K, L, M, N, evaluated by workers.Regarding the Continuing Education and qualification of the health surveillance teams, workers indicated a "satisfactory" result, despite some insufficiency in meeting the competencies' requirements and professional skills development, as teams' complementary training, participation in refresher courses and the use of education devices and distance matrix support for the qualification of care provided to users was not well evaluated, despite recognizing management efforts: Refresher and qualification courses are not offered to all health surveillance professionals [...] I understand that there are few spots, which makes it difficult to improve work and qualification...The Organization of Work Processes was deemed "fair".Questions addressed work based on priorities, risk classification and vulnerabilities, involvement of society in the planning of actions, access to citizen, intersectoral coordination, action on determinants and conditionants, monitoring and evaluation and holding regular meetings.
Conceptual aspects that underpin surveillance practices subsidize this evaluation and can be object of continuing education, also evaluated as "fair" by professionals.Teams' poor knowledge on these topics is noted:

Some professionals do not even know what social determinants are
A challenge for health surveillance managers is to organize a way of working at risk in the territory, epidemiological indicators, planning, mon-itoring and evaluation, as well as aspects related to management strategies.According to workers, there is a lack of space for reflection on work, notwithstanding advances in the team's work when they participate in the planning: There is much to improve, but action planning with increased employee participation setting priorities has raised the quality of the surveillance work process.
Reconciling demands and responsibilities, often with deadlines to meet them, with moments of study, reflection, planning and evaluation are herculean, yet essential tasks.
The sub-realm Comprehensive Health Care obtained a "satisfactory" result with regard to the questions about health protection and promotion, prevention and control of risks, illnesses and diseases at the various levels of care.We analyzed health education, alert and response actions to outbreaks and events of importance in health, inspections, monitoring, issuing health license, reception and compliance with notifications, reports and claims, active search, control of reservoirs, hosts and vectors, vaccination and surveys.
It is necessary to think about the need for harmonization of knowledge, understanding the meaning of activities and a coordinated work with the other sectors of the Health Secretariat and other government agencies, with health surveillance matrix support to the other points of care of the network in order to strengthen actions, as mentioned in the results related to managers.Professionals value educational actions and improved channels of communication with the regulated sector, even if it is to increase the reporting space.They are uncomfortable with the rationale of "fire-extinguishing" surveillance work.They believe that education and prevention actions are incumbent upon primary care, with the support of health surveillance.
Participation, Social Control and User Satisfaction were deemed "satisfactory".We evaluated particularities about addressing social problems of greater local expression, joint actions with the community and debates about local health problems and the availability of channels of communication with users.
Finally, aspects regarding the tool used in the self-assessment study and the most relevant results found in the study will be highlighted.
The instrument brought a "formative" component by incorporating into each sub-realm a set of explanations or citations of some concepts located in recent legal milestones of the SUS, such as care network and thematic networks, National List of Health Actions and Services (RENASES), Institutional Support, continuing education, among others.That is, even if the respondent was not familiar with the concept, it was possible to look at the practice and reflect on whether or not the themes submitted were already part of their daily repertoire.It is necessary to think of a more synthetic tool, and it is necessary to review the need for some sub-realms to be evaluated by both managers and workers.
Self-evaluation revealed positive points and critical nodes of health surveillance practices such as the incorporation of the use of information in the daily life of managers and workers, the fragmentation of practices within health surveillance and externally in the health care network, as well as structural and organizational problems to which professionals are subjected.Workers and managers have had different perceptions on these issues.
There is a lack of space for dialogues with other points of care of the health network, both in the construction of common projects of actions in the territory and technical or matrix support.
On the one hand, managers comply with interfederative pacts and rarely use information in their daily routines, and on the other, workers are immersed in their universe of technical work, not reflective and with structural and organizational gaps.At the same time, they produce a lot of information that seems to be restricted to the realm of top management or to reference specialists for certain diseases.
Taking ownership of the information produced by the teams can be a mobilizer for man-agers and workers towards producing a formative evaluation practice.It is believed and hoped, as Onoko-Campos and Furtado 7 argue, that "the involvement of stakeholders be constant and active and the evaluator play a facilitating role.It is hoped that the evaluation process will allow participants to assimilate the skills to better understand and use the results, as well as to engage or conduct new evaluations".

Conclusions
The questionnaire built by researchers for the self-assessment of health surveillance managers and workers was sufficient and innovative when opening space to their statements and reflections.Much in the same way as the AMAQ tool, it included elements of formative evaluation in its content, showing concepts related to the universe of the SUS and health surveillance, even to those unfamiliar with the subject.
The research pointed to the self-assessment as a relevant element of the systematic evaluation of the activities developed by managers and workers.Results must necessarily be challenged to understand the critical nodes and positive aspects of the practice, with a view to making changes and future management contracts.Self-assessment can be considered as yet another management device, as well as collective spaces, continuing education and institutional support.
The self-evaluation proved to be a powerful tool for the analysis of the practices and, unlike the usual evaluation processes based on quantitative indicators, it included the subjects and incorporated a look at the ways of doing Health Surveillance.

Chart 1 .
Sub-realms by standards of quality, score, result and classification.