evaluation of the performance of Tuberculosis Control Programs in Brazil and Spain : an integrative review of the literature

This is an integrative review of the literature, which sought to locate evidence regarding the evaluation of the performance of tuberculosis control programs in the Brazilian and Spanish contexts. Articles indexed in databases, search directories and virtual libraries produced in Brazil and Spain from 2002 to 2013 were selected. A total of 17 articles (9 Brazilian and 8 Spanish) were included for analysis. The intention was to identify program evaluation concepts, methodologies and indicators for TB control program evaluation in both countries. In the Spanish context, a concern with results-oriented management and the attempt to develop broader indicators for the assessment of a series of health programs, including those related to tuberculosis control, was identified. The Brazilian references showed greater proximity to the classic methods of evaluation research, and a greater variety of objects of study regarding the activities expected in a control program. The findings highlight the complexity of program evaluation activities, which involve multiple and diverse stakeholders, and point to the need for integration of epidemiological and operational indicators in tuberculosis.


introduction
For those involved in public health, scrutinizing the performance of ongoing measures undertaken through programs and public policies is an act that is both continuous and extremely necessary.Taking into consideration social, political and economic interests, this type of examination is done in order to attain an efficient and effective model for the provision of health services.It should be one that meets the needs of individuals and communities 1 .
In Brazil, over the 26 years of the National Health System (SUS), the need for evidence that can feed into the process for making the system better 2,3 has resulted in many discussions on the use of evaluation programs in health care.These discussions have been taking place in governmental institutions and in academic and scientific circles 4,5 .This has been the case for other countries that opted for a national health system such as in Spain that provides universal health coverage as part of their national health system.Their system is similar to the Brazilian health system in that reforms where changes in health care were linked to the process of redemocratization after a military dictatorship 6 .
It is necessary to remember that health care sector reforms in both of these countries came about through concerted mobilized actions which even today fuels tensions between government bodies and society 7,8 .This in turn puts a spotlight on the need for mediation between the two parties which then feeds into the public policy formulation process in health care 8 .
The recent successive cuts to Spanish social programs due to the economic crisis 9 and the barriers people face when using SUS (in the pursuit of complying with certain principles and directives) 2,10 has left many assessors in a difficult position in relation to their roles.It also reinforces the permanent existence of social control measures that dictate end user outcomes.The road to deal with the challenges identified by evaluations that have been done, is long.In order for information from these evaluations to aid in service management and programs, it is necessary to tackle: the different concepts arising from the area, the lack of evaluative processes that have become institutionalized and the multiple dimensions within the challenges which are comparative to those in other social spheres 11 .
Health care has produced organized responses to deal with problems.With this in mind, we have put a spot on Tuberculosis (TB) which is a disease that has well established methods for diagnosis and treatment.However the management of TB from a health system point of view requires coordinated actions amongst different parties which include prevention and control 12 .
The epidemiological indicators for TB and treatment outcomes from the two countries show the need to analyze the response capability of the Programs for the Control of Tuberculosis (PCT) in the context of their respective health systems.In Brazil TB continues to be the main cause of death for those infected with HIV/AIDs.It is also the fourth cause of death by an infectious disease among the general population 13 with an incident rate of 35.8 cases per 100,000 inhabitants in 2012.The percentage of those cured among newly diagnosed cases was 67.7% in 2011 13 .In Spain the incident rate for TB was 13.0 cases per 100,00 inhabitants in 2012.The percentage of those cured among newly diagnosed cases was only 73,3% in 2011 14 .In Spain 46.3% of registered cases of TB in the same period were cases of foreigners the majority of whom came from: South America, Eastern Europe and the northern Africa.The immigrant's living conditions and their general precarious circumstances are elements which present difficulties for the public authorities to control the disease in their country 14 .
Both in Brazil and Spain the decentralization and strengthening of the Primary Attention to Health Coordinator role, considered as part of the strategy to effect reform in the health system, established a context that influenced the specific issue of controlling TB.That coupled with health care monitoring are being developed.
The operational complexity in the control of a diseases such as TB and the evidence concerning the monitoring of the performance of health programs are potentially very interesting for researchers and managers in both countries.This article seeks to carry out a bibliographical study on the evaluations of the TB control program based on the literature produced in Brazil and Spain.The choice of these two countries was based on: the similarities of their respective public national health systems, the common challenges encountered in service management over the years, and the development of programs to control diseases.

Material and Methods
This work is an integrated review of the literature whose main purpose is to deepen our understanding of a specific phenomenon looking at both its strengths and weaknesses.The is a rational and critical analysis of the given purpose 15 .
The integrated review consists of an ordered and systematic analysis of the results obtain from literature.It also suggests the development of six steps 15 : 1) identifying the key investigative question that needs to be answered and which has a structured theme to be both researched and documented.Also we needed to identify descriptors and key-words that could be searched for in the literature; 2) establishing search strategies and defining criteria for what material would be included and excluded; 3) categorization of the selected information and defining information of interest to be reviewed; 4) critical analysis of the final body of work; 5) interpretation of the findings; 6) presentation of the reviewed work.
For this study the question that needed to be addressed was: "How has the issue of performance in relation to the control programs for TB been approached in the technical/scientific literature in Brazil and Spain?" In this study the following was reviewed: original scientific research, topical and critical articles written in this area, literature on TB, studies of cases and accounts of TB, monographies, dissertations and thesis.We also reviewed: institutional publications (reports, and plans), protocols and practical clinical guides (practical guidelines) produced in Brazil and Spain, literature published in Portuguese, Spanish and English and that was published in the last decade (2002-2013).Information from the following databases was used: CINAHL (Cumulative Index for Nursing and Allied Health Literature), Medline (Medical Literature Analysis and Retrieval System Online), Lilacs (Literatura Latino-Americana e do Caribe em Ciências de Saúde) e SCOPUS (Web of Science).The virtual library SciELO (Scientific Electronic Library Online) also was used as a relevant source for searches.The search engine Google Scholar (Google Acadêmico) complemented searches as it allowed for the identification of material of interest not available in the sources mentioned above.
The choice of search descriptors was done before the start of this study through looking through terminology libraries on health -Descriptors on Health Science (DeCS) from the Regional Medical Library at the Pan-American Organization for Health (OPAS/Bireme) and Medical Subject Headings (MSH) from the National Library of Medicine.Key terms were defined based on their concepts, which would make possible searches easy where a search using a descriptor did not provide satisfactory results.
A bibliographic study was done on the aforementioned databases and sources through cross-referencing the descriptors and the terms mentioned in Chart 1 between the months of October and December 2013.We also applied, as a search strategy, the use of the descriptor tuberculosis and the boolean operation "AND" with the other selected issues to be covered by the study.The aforementioned was combined in itself with the use of the operation "OR".The recuperation of the material produced in the Brazilian and Spanish contexts was done through the use of the words "Brasil" and "Espanha" in the search fields relative to the country that the research was affiliated to.This was also the case for the title and/or a summary of the material.
The inclusion criteria of identified articles through searches through the presence of theoretical concepts and/or evidence in relation to the evaluation of PCTs in the context of the health system.This was also the case for indicators that were duplications and irrelevant material was excluded from this study.
Once we had all of the documents we critically read through all of the material in order to produce an excellent final product.Data of interest was described and grouped in thematic categories to facilitate the presentation and discussion of the results.The identification and categorization of the utilized indicators in these materials followed the ideas of Waldman 16 and the Interagency Network for Information in Health (RIPSA) 17 , that defines indicators as a type of quantitative or qualitative measurement.It organizes and captures relevant information from elements that make up an objective that is evaluated.The indicators were categorized as epidemiological (which expresses the magnitude of the disease) and operational (which measures activities carried out and the quantity/quality of actions taken based on a proposal) 17 .
Figure 1 shows the quantity of information obtained and selected in accordance with the specified criteria.The result was 17 documents which included articles, monographies, theses and protocols.

results
Charts 2 and 3 show the principal data related to the material obtained, according to the year, source, document typology, and other considerations related to the study and main objective (when pertinent).It is based in a Spanish and Brazilian context respectively.
In a general sense the Spanish production is concentrated on a period before the production with the majority of the selected articles published before 2010.The search for Brazilian studies resulted in obtaining recently published articles -which meant that the final document was made up of many articles published in the last five years.In relation to the type of material identified, half of the documents selected came from different types of publications as well as incorporating original research results.For example there were review/critical articles and narrative reviews (n = 5), theses and monographies (n = 3) and protocols (n = 1).All of the above * Used for searches on the Medine, CINAHL, and SCOPUS databases.
Chart 1. Descriptors and proposed terms for the literature research and respective definitions.The process whose finality is the systematic and objective determination of relevance, effectiveness and the impact of public policies, programs and health projects.The objective of the evaluation is to improve the programs and projects with the view to directing where human and financial resources should be used.Studies whose aims center on evaluating the effectiveness of programs including an evaluation cost/efficiency and the scope or impact of met objectives.

Sub
Research whose objective is to evaluate quality and effectiveness in health care and which covered the end results.
Research that covers epidemiological, sociological and economic knowledge and other analytical sciences in the study of health services.Research into health care generally concerns the relationship with necessity, demand, offer, utilization and results.The objective of the research is an evaluation, specially covering structure, processes and results.
Norms, criteria, standards, and other measures, quantitative and qualitative directives used to determine quality in health care.

Tuberculosis Evaluation
Evaluation, Results, Closure, Process Health services, Health system Indicators for quality was an important source of knowledge for this present study.
In relation to choosing which original articles to be a part of this study we used a number of strategies.The most relevant was to use those that had the information on epidemiological investigations.This was the case in the majority of the original articles chosen (6 out of 9 publications from both countries).The majority of the articles with these elements were Brazilian (4 Brazilian articles in comparison to 1 Spanish article).
Throughout the research we used indicators and parameters to measure the performance.We used a number of different evaluative indicators that related to the following areas: structure, processes and results [18][19][20][21] .We also included categories such as: cover, quality, satisfaction and accessibility 18 all of which forms part of the process.We sought to define efficiency and effectiveness in the relevant contexts 18,22 .
The structural evaluation was defined based on the material and resources that were analyzed which were both material and human 18,21 .Also the services and activities that came from the above were also evaluated.The evaluation of the process involved investigations into the activities of the programs, their coverage and their quality 18,21 .We also took into account the satisfaction ratings of the patients and accessibility for the services providers to the patients.Finally an evaluation of the results focused on the set targets for the services and activities 18,21 .
Efficiency was measured looking at the relationship between the resources used and whether the objectives were met based on the acts that were done 18,22,23 .The economic evaluation was done analyzing the costs and benefits based on the use of the given resources 18,22 .Effectiveness was measured based on looking at the results for the target population for a particular program/ intervention and the effect and impact of that program on the community 18,22,23 .
The dimensions and indicators from evaluative research references was also taken into account [24][25][26] .The evaluation on what had been implemented used a logical model.It was based on creating a description of the components, inputs, activities, products, results and impacts expected for the PCT.The theoretical evaluative model looked at whether the resources and services used allowed for expected results to be achieved.We also wanted to see whether the actual results obtained were different to or close to what had be proposed.We also took into account the organizational and political context of the study.This included: political and financial autonomy, inter Material selected after reading the summary and applying the inclusion criteria sectoral actions, social vulnerabilities and management.In relation to implemented programs we took into account: service coverage, access, laboratorial diagnosis and support, pharmaceutical treatment and assistance, integrality and epidemiological vigilance.Related to the aforementioned we also looked at their effects (clinical end results and user satisfaction rating).Each dimension is subdivided into categories.The categories have their own criteria/indicators that make up the matrix used for the evaluations.They assess the degree to which the PCT was implemented into the health system.The continuation model for attention and the indicators used to measure them were identified in one of the Spanish documents.What was considered continuity of action was "a rational association between proposed clinical interventions, changes that were successfully achieved in the health of individuals and subsequent modifications in the utilization of health services" 22 .The definition still includes continuity of atten-Chart 2. Summary of material selected and produced in the Spanish context on the performance evaluation of programs to control tuberculosis, 2002 -2013.references Alonso JLP, García-Martos P, Casanova PM et al. 27 SEPAR 28 Rodrigo TS, Cayla JA 29 Rodrigo TS 23 Chaulk PC, Kazandin VA, Vellejo Gutierrez P 22 Luna-Sanchez A, Romero BR, Exposito-Garcia S et al. 30 Villalbi JR, Casasa C, Bartoll X et al. 31 Villalbi JR, Tresseras R 18

Discussion
In the area of TB there are difficulties in outlining the scope of studies about the performance of specific program actions or about the services that form part of the program.This is also the case for PCT which practices their own activities.The evidence showed that in situations where evaluations need to be carried out on services which have involved many different organizations and health professional tasked at dealing with TB, it is extremely difficult to establish measurements that take into account the responsibilities of all those involved in the process 18,22,31 .Reductions made in the amount of material obtained for this study (which was necessary) did not result in chances wider discussions and dealing with critical issues.However this act made it difficult to systematize the evidence built amidst the variety of publications that were available.The scarcity of published works in the area of PCT evaluation was noted in the papers that we used for both counties 22,23,27,29,33 .The theme of evaluation, in spite of being based on current established theories, is still an area that has been frequently studied in-depth in relation to TB. Kritski et al. 35 showed that, in Brazil, there was an increase in the number of study results on TB in indexed periodicals which have used scientific rigor from the year 2000.However they were based on studies related to basic sciences.In Spain, Ramos et al. 36 showed that studies on TB that exist are mainly clinical studies or commissioned by politicians.Doctors were the the main authors of these medical papers between 1997 and 2006 and hospitals were the main institutions that were responsible for their publications.The findings from Brazilian papers in this area which are recent and which were used in this paper can be related to the prioritization for the implementation of PCT actions in primary care in the country.Research publishers also support work carried out in this area.
The task of evaluating the performance of control programs requires the use of a well-defined terms of reference which allows for the selection  18,20,22,24,25,26, and analysis of pertinent indicators.This in turn allows for the study to be done and for the results to be clearly understood 37 .However in the area of evaluations, new definitions are constantly created in spite of the fact that many ideas and existing methods are used in specific ways.The upshot is a plethora of terminologies 38,39 and methodologies 40 that cover epidemiology to classical theories for evaluative research.Their evaluation quantitative and qualitative research.Fernandes et al. 5 , noted that in spite of the non-consensual nature that prevails in the field of evaluation, this very fact allows for researchers to be creative.It also allows for the identification of risks to be resolved and the publication of results and findings.
The methods and indicators identified in this review confirmed the existence of two strands.One limits the normative evaluation base with specific indicators and are limited to components in the control program.The other is close to the evaluative research with indicators that seek to relate context differences in which the components in the program relate to each other, including subjective aspects 20,24,26 .
The relevance of the use of epidemiological indicators is discussed in the identified material 32,34 , since the magnitude of the disease can influence the recognition of the problem by the health service.Indicators that made references to AIDs and the related infection HIV were also identified for this paper 32,33 .Its use was justified based on the known links between the two chronic illnesses, which makes care given more complex and can influence the performance of control programs for both diseases.
Limitations in the use of secondary data and the random nature of the choices for cuts in the points used in the classification criteria for the districts was mentioned 32 .What was also mentioned was the lack of information on the performance of the National Program for the Control of Tuberculosis in Brazil and its districts 33 .Considering the above, the role of the epidemiological regulator and the evaluation of various health information systems that bring together information of interest for a better understanding of the PCT, were issues that were frequently tackled in identified studies 18,22,23,25,29,31 .
The indicators used for evaluation must be clearly defined and objectively measurable.The criteria should be based on scientific evidence so that the consent mechanisms are able to provide relevant and pertinent parameters.This is the prerogative of evaluations done in the area of service management 28,41 .Aside from this, the indica-tors ought to be flexible and should consider the context in which that are to be used.This is because sometimes it is necessary to assume that, in certain situations, the objectives are not realistic and one must reconsider intentions in order to obtain certain goals 28 .In a general way a reading of the Brazilian literature show major variations in the scope and propositions of the methodologies for evaluating the PCT.The identified operational indicators are aimed at detection quality, diagnosis, treatment and compatibility with a country with an epidemiological situation.In Spain, contact exams and multi-drug resistance emerged as relevant themes and were highlighted in the utilized indicators in publication that covered the state of the art.
In spite of this in both the Brazilian and Spanish productions there was an identification of a necessity for resources and activity integration.Intervention and coordination of the above in the network of health care systems is necessary for the attainment of proposed goals.In order to monitor and evaluate the program it is necessary to determine responsibilities, assign resources to be used and establish criteria.
Villalbi e Tresseras 18 highlight the complicated and confused present situation in evaluating health programs.They noted that the real components of these complex programs are not always explicit particularly throughout the time that they are operational.Gonçalves 21 , in the same vein noted that a program that represents all of programs cannot be evaluated based on just one indicator.More than one needs to be analyzed together.In order for an evaluation and its results for a program to be effective, it is often necessary to obtain indicators from other sources of information which are not always integrated in the management of intervention in themselves.
It is worth remembering that Brazil, organized as a republic with 27 federal states and a president, has a health system the responsibility of which is divided amongst three strands of government: union, states and municipalities.Oversight and control lies with the Federal Government.The PCTs are formally organized as part of the above.Local bodies delimit their set objectives and identify key resources and processes that are necessary in order to meet their goals in their particular situation 42 .
For SUS it is incumbent upon municipalities and the local health care network to take actions to control TB.This includes mapping the spread of the disease and equipping the health centers with appropriate resources 42 .The municipalities are organized based on their specific administrative, political and geographic characteristics.They are part of a wider national network each with their own style of governance and it is for this reason it is necessary to have an evaluation methodology that recognizes the above 21,32,33 .
Spain has a parliamentary government under a constitutional monarchy and is organized as a unitary and decentralized state divided up into autonomous communities (CCAA).The structure of the health system lies the Unitary State administration and is shared with the autonomous communities.These communities have their own independence to manage their own local health system 14 .TB in Spain is dealt with at Unitary State level by a working group (GT) made up of specialists from each CCAA.Each Autonomous Community develops their own plan of activities.They organize Regional Programs or Committees to control TB in the best way possible based on the particular circumstances of an area 14 .
In Brazilian and Spanish health systems there is a notion of embedding monitoring and evaluation tools for routine service delivery which infers a management based on results approach.In Spain where the public health service covers a whole array of services, an evaluation through the use of appropriate indicators allows for the performance of health care interventions to gain visibility and legitimacy and this benefits the health care professionals.The professional can assess his/her own professional practice in an agile way and the health care managers will be able to assess the coverage and effectiveness of their activities.Government officials will be able to know what has been done, for what reason and at what cost 31 .The identification of two protocols with directives for the evaluation of PCT among the identified material, shows the efforts that have been made to institutionalize evaluation processes and routines in the management of regional programs.This is also the case for article developed 18,31 with the purpose of debating the use of indicators in the practice of program and service management.This emphasizes the need for information systems which can validate data and this data can be compared with the CCAAs.
In Brazil, in spite of the Health Ministry's efforts there still exists a fragmentation of evaluative processes which stops their being utilized in coordinated actions.There also is a form of institutionalization focused on the creation of new bodies which makes the system even more bureaucratic than necessary 43 .However what is set to be introduced is an agenda for monitoring and evaluation (M&A) in the policies on Brazilian health care.Advances were made with a discussion panel on indicators and a process to reach agreements on goals and propositions 44- The National Program for the Control of Tuberculosis legislated for monitoring and evaluation activities since 2000, which is currently done by a team of professionals from the Health Ministry and by a group of external monitors made up of TB expert from all around the country.Their objective is to contribute in the form of policy and technical development to improve the state's and districts' capacity to control TB 42 .
In this study some relevant publications were not included due to: difficulties in obtaining some material where searches using different descriptors did not prove to be fruitful.The issue of knowing the correct descriptor to use in searches for bibliographic material is a major problem for complex themes.It also has the tendency to reveal sub-themes closely related in the area of health evaluation 4,5 .In order to improve the selection of relevant documents, the authors systemized search terms.Some articles were excluded from this body of work such as specific sub-themes concerning the control of TB (accessibility to treatment, late diagnosis) as they fell out of the scope of this study.
The area of program evaluation often does not follow conventions and therefore it is sometimes difficult to access information in this area.A large amount of evaluations for control programs is covered in technical reports whose publication and dissemination only occur in the specific health body.This is the same body that executes and coordinates control actions.There is not a tradition of systemizing the content of scientific articles.

Final Considerations
The results reflect the complexities involved in evaluating programs that consist of multiple activities and various parties.This highlights the need for a universal set of epidemiological and operational performance indicators for tuberculosis.There also needs to be the use of different concepts for interpreting data related to the performance of the PCTs.The study based on the given theme was made up by a number of different publications program actions and service or activity attributes in specific populations.Yet there was found less information in this area compared with TB related themes such as clinical or scientific studies.We did, however, identify important debates around the management of services based on the use of evaluative indicators in both countries.Emphasis was placed on evaluative research in the Brazilian context and the debates concerning the use of indicators for the public health system in Spain.
The possibility of the use of the findings from this study in daily practice for health management would constitute an important contribution to the available scientific literature.For example, this could be through the creation of indicators and goals and a monitoring/evaluating system for TB.Finally we suggest the development of comparative studies on similar universal access health systems as in Brazil and Spain in order to deepen our understanding in this area.

Collaborations
T Arakawa did the following: worked on the initial idea for the study, defined the methodology to be used, carried out the research, critically analyzed the material used in the study and participated in drafting this paper.GT Magnabosco e LM Lopes aided in researching for the appropriate literature phases.They also read and critically analyzed the selected material and assisted in the drafting of this paper.MA Arce-Arnaez e MA Ordobás-Gavín participated in the discussions on the selection of appropriate material and in the drafting of this paper.MP Serrano-Gallardo worked on defining the scope of this study, participated in selecting appropriate material and critically reviewed the material.AA Monroe e TCS Villa participated in interpreting the findings of the study and drafting this paper.All of the above individuals reviewed this paper and approved the final version.
infectious disease for human beings and other animals caused by the species Mycobacterium Tuberculosis.

Figure 1 .
Figure 1.Summary of the steps in searching and selecting material in accordance with the inclusion criteria for the study.

Chart 4 .
The main proposed indicators in the selected material.tuberculosis incidents categorized by clinical type, age and sex ( Summary of material selected and produced in the Brazilian context on the performance evaluation of programs to controltuberculosis, 2002 -2013.
tion as an indicator for the treatment of tuberculosis which should also take into account the participation of patients in the period during and after treatment.It should also take into account the integration between different public health bodies and the communities involved in these processes22.The main identified indicators were categorized as epidemiological or operational and are listed Chart 4. Chart 3.