Evaluation of data quality, timeliness and acceptability of the tuberculosis surveillance system in Brazil’s micro-regions

1 Programa de PósGraduação em Saúde Coletiva, Faculdade de Ciências da Saúde, Universidade de Brasília. Campus Universitário Darcy Ribeiro, Asa Norte. 70910900 Brasília DF Brasil. gabidrumm@gmail.com 2 Programa Nacional de Controle da Tuberculose, Secretaria de Vigilância em Saúde, Ministério da Saúde. Brasília DF Brasil. 3 Programa de Computação Científica, Fiocruz. Rio de Janeiro RJ Brasil. 4 Instituto de Pesquisa Econômica Aplicada. Brasília DF Brasil. Avaliação da qualidade dos dados, oportunidade e aceitabilidade da vigilância da tuberculose nas microrregiões do Brasil


Introduction
The Brazilian economic and political crisis could have an impact on the control of tuberculosis (TB).Significant negative outcomes of this crisis, such as food insecurity and unemployment, are determinants that have been evidenced and are associated to the escalation of this disease 1,2 .Furthermore, the limited resources imposed by the Constitutional Amendment nº 95/2016 3 could further reduce health actions and services related to the surveillance and treatment of TB.
In Brazil, the TB incidence rate in 2016 was 32.4/100,000 inhabitants and the mortality rate, in 2015, was 2.2/100,000 inhabitants.Both these indicators have seen a decline of 2% and 3% per year, respectively 4,5 .These results, however, are not sufficient to meet the goals of the End TB Strategy, approved by the World Health Assembly in 2014, whose aim is to ensure TB is eliminated and no longer considered a global public health issue.Its main goals are to bring incidence to levels lower than 10/100,000 inhabitants by 2035 and reduce deaths by 95% in comparison with 2015 death rates 5 .
To move forward with this objective, especially in an unfavorable economic scenario, strategic planning is essential in order to make programs more cost-effective.Furthermore, the fight against TB requires a wide-range of approaches not restricted to curative measures but which include audacious policies such as the social protection of individuals with the disease 6 .In this context, the Epidemiological Surveillance of TB can play a significant role.Some of the features of the Epidemiological Surveillance System in Brazil include case notification, contact investigation and case follow-up 7 .This system has seen some successes, as can be observed in the decrease in TB indicators in Brazil, and achieved some excellent results such as the improvement of TB detection in prisons 8 .Nevertheless, TB operational indicators are not homogeneous across geographic regions and population subgroups, reflecting variations in surveillance 9 .Even at national level, there are some important challenges related to Epidemiological Surveillance such as the low number of contact investigations and high levels of treatment non-adherence 5,[10][11][12] .
The analysis of Epidemiological Surveillance will steer recommendations and contribute to decision-making 13 .Evaluating TB, a chronic disease, is complex because of different social and epidemiological conditions and the need for joined up actions between the various spheres of the health system, because it requires continuous treatment for at least 6 months 14 .The Guidelines for Evaluating Surveillance Systems produced by the Center for Disease Control and Prevention (CDC) proposes an analysis of various attributes that could be useful in this context 15 .
Many studies in Brazil have used quantitative analysis to evaluate TB surveillance systems.However, in these studies, usually only one of the attributes proposed by the CDC (2001) 15 , data quality [16][17][18][19] was employed in calculations, they took place in small localities 12,[20][21][22] or in specific states 13,14,23 .The most extensive study analyzed the quality of TB surveillance in Brazilian municipalities between 2001 and 2003 10 .
Given the lack of wide-ranging and up-todate research, the aim of this paper is to evaluate the quality of the data, timeliness and acceptability of the tuberculosis surveillance system in the micro-regions of Brazil.

Study design
A hybrid study was conducted in two stages: (i) a qualitative analysis involving a semi-structured questionnaire in a consultation of TB surveillance specialists, for selecting the indicators and defining the parameters to evaluate the surveillance system; and (ii) a quantitative study with a cross-sectional ecological design where the selected indicators were calculated based on data about notified TB cases, available in the National Information System of Notifiable Diseases (SINAN) and diagnosed between 2012 and 2014 across all 558 Brazilian micro-regions.

First stage: indicators selection and parameter definition for evaluating the tuberculosis surveillance system
Specifically selected TB surveillance specialists, working in different regions of Brazil and with proven technical and scientific knowledge to contribute to this study, were consulted.They were recommended by National Tuberculosis Control and state program staff.Contact was made via telephone, email, or directly by researchers in the area.Some of the specialists were emailed a questionnaire about indicators related to the following attributes of the tuberculosis surveillance system: (i) data quality (completeness and consistency), timeliness and (iii) acceptability 15 .The questionnaire included closed questions, where respondents could select the most appropriate indicators for the TB surveillance system.In addition, specialists could also agree (or disagree) with the suggested cut off points or limits for evaluating the indicators.Some open questions were also included so that, if necessary, specialists could list alternative indicators, attributes and limits to those presented in the closed questions.A copy of the questionnaire is annexed.The format for evaluating attributes was as follows: Data quality: specialists were questioned about a list of 9 suggested indicators to estimate the completeness of the TB information system 7,12 .Furthermore, they were advised that all blank entries or those with the term 'unknown' , or any other term used to indicate lack of information, should be considered incomplete 20 .To assess data consistency, 5 scenarios were proposed, according to the Script for analyzing the tuberculosis database available in the National Information System of Notifiable Diseases -SINAN, the calculation of basic indicators 24 and the SINAN NET data dictionary v. 4.0 25 .An example of a suggested consistency scenario involves the percentage of recorded AIDS cases with events associated to TB and positive HIV tests.
ii) Timeliness: specialists were asked about the indicators described in Mandacarú 19 .Timeliness indicators proposed were: timeliness of notification, timeliness of investigation, timeliness of typing, timeliness of treatment and timeliness of case closure.
iii) Acceptability: the adherence of health professionals to the TB surveillance guidelines was considered to be the best way of evaluating the acceptability of the system 26 .Thus, the following indicators were proposed for evaluating acceptability: Braga's 10 detection and follow up quality indicators, complemented by Ministry of Health indicators 5 .In total, 7 indicators were suggested, including: 1) Percentage of new pulmonary TB cases confirmed by laboratorial tests that received directly observed therapy (DOT), 2) Percentage of new pulmonary TB cases where a sputum smear microscopy was conducted.
To reduce losses, a participation reminder was sent to specialists who, after 7 days, had not replied to the original email.The consultation period took place between October and December 2016.Six specialists working in the Northeast, Southeast and Center-West regions of the country answered the questionnaire.Another three researchers also made suggestions to the variables listed by the specialists in order to refine and conclude the selection.
14 indicators were selected from those classified as relevant for evaluating TB surveillance by at least two of the six specialists and by the three reviewers.These indicators and their form of calculation are shown in Chart 1.
The classification of continuous indicators was conducted in accordance to the parameters recommended by the literature 7,10,12,27,28 and, in their absence, according to the suggestions of the specialists.An indicator was classified as "invalid" in micro-regions if at least 15% of the records needed for calculation were incomplete or inconsistent.In the micro-regions with valid data, limits L1 and L2, shown in Chart 1, were used to classify the following performance groups: poor (indicator < L1%), regular (indicator > = L1% and <L2%) and optimal (indicator > = L2%).

Second stage: calculation of indicators
The ecological study used all 558 micro-regions as units of analysis, encompassing all 5,570 Brazilian municipalities.All notified TB cases were analyzed per Federation Unit (FU or states) and by notification micro-region.All cases with a change in diagnosis were excluded.Six micro-regions were excluded from the micro-region analysis but were considered in the FU calculations because they had five or less notifications during the period analyzed.Data used was extracted from SINAN in May 2016.The 2013 Digital Geographical Grid, produced by the Brazilian Institute of Geography and Statistics (IBGE), was used for the mapping exercise.
General completeness was calculated as the median of all completeness indicators.Considering that acceptability and timeliness can be evaluated only if the information system is reliable, indicators related to these attributes were calculated only in micro-regions with a median completeness above or equal to 70%.

Data analysis
The indicators selected were calculated for each micro-region of the country and also per FU or state.Total completeness and consistency per FU are shown in a contingency table.
The Burt Table was used to work out the relation between timeliness and acceptability indicators.It is a square table, where the ij box contains the number of micro-regions simultaneously classified in categories i and j, i, j = 1, ..., k, where k is the total of all study categories 29 .
A theme map was produced for acceptability and timeliness indicators.In these images, each micro-region was represented by a color in accordance to their performance in each specific indicator."Incomplete" describes micro-regions where the indicator could not be calculated due to low median information system completeness.
In order to describe the regions in terms of the general surveillance system, a hybrid algorithm was used for cluster analysis, combining both the hierarchical and k-means methods 30 .Given that k-means are reasonably sensitive to the initially defined cluster centers, the hybrid cluster used results produced by the hierarchical method as reference for the initial k-means arguments.The number of clusters was defined using Data analysis was conducted using the R software, 3.4.0version, and the following packages: GISTools to produce the theme maps, GDAtools to calculate the Burt table and factoextra to analyze the clusters.

Ethical considerations
The study involved consulting specialists on issues related to their professional practice whilst preserving their identity and included an analysis of secondary data accessible to the public.Therefore, the current study could not be registered or assessed by the Research Ethics Committee, in accordance with Resolution n. 510/2016, of the Brazilian National Health Council.Furthermore, secondary data contained individual information on TB cases, although individuals cannot be identified through the variables.

Results
In Brazil, between 2012 and 2014, there were 251,693 TB cases notified in 4,620 municipalities and 558 micro-regions, excluding the cases that underwent a change in diagnosis.There were 206,467 new cases, 20,211 re-entry cases following non-adherence, 17,335 recurrences and 6,202 transferals.In addition, there are 800 cases without information on entry type and 678 after death notifications.Furthermore, there were 210,358 pulmonary cases, 32,901 extra-pulmonary cases, 8,278 cases involving both pulmonary and extra-pulmonary TB simultaneously and 156 without information on the clinical form of the disease.All cases contained information on the municipality of notification.Therefore, there were no losses in the calculation of indicators per micro-region.
The completeness of treatment starting day, closure situation and consistency were optimal (> = 90%) in all FUs except Mato Grosso do Sul, where completeness of closure situations was 89.9%.By contrast, completeness of contacts investigated, and of 2nd and 6th month microscopy were poor (< 70%) or regular (> = 70% and < 90%) in practically all Brazilian states.Completeness of contacts investigated was poor in eight states and Paraíba (36%) particularly stands out.Microscopy completeness in the 2nd month was poor in five states in the Northeast: Rio Grande do Norte, Paraíba, Pernambuco, Alagoas and Bahia.In fifteen states 6th month microscopy completeness was poor, in particular in Paraíba and Pernambuco, with completeness rates lower than 50% (Table 1).
It is important to highlight that the state of Acre had one of the best levels of completeness in terms of contacts investigated and 2nd and 6th month microscopy.The South was the only region where 6th month microscopy completeness was above 70% in all states (Table 1).
Among micro-regions with over 5 notifications (n = 552), the median completeness was optimal or regular in 85.7% (n = 473) and consistency was optimal in 98.0% (n = 541).Figures 1 and 2 show the distribution of acceptability and timeliness indicators in micro-regions where the median completeness was either regular or optimal.Micro-regions where median completeness was poor were concentrated in a band that started in the south of the state of Piauí, passing through the state of Bahia, the north and south of the state of Minas Gerais and part of the state of Rio de Janeiro.Areas with poor median completeness were also found in the states of Amazonas, Pará, Rio Grande do Sul, Rio Grande do Norte and Paraíba (Figures 1 and 2).
The timeliness of notification was poor in the micro-regions that cross Brazil vertically from the south of Mato Grosso, past the states of Tocantins, Maranhão and Piauí.Invalid timeliness of typing were concentrated in the micro-regions of São Paulo.Timeliness of treatment and closure were poor in most Brazilian micro-regions.However, in these maps, São Paulo stands out for presenting predominantly regular timeliness of treatment and invalid results for timeliness of closure (Figure 1).
Micro-regions with invalid contacts investigated were found across Brazil, except for São Paulo and Acre.Optimal microscopy of new cases was found mainly in the North region, São Paulo, Espírito Santo, the center of the State of Paraná and in the center-west of Goiás and Tocantins.Mato Grosso do Sul, Goiás, the south of Mato Grosso and Pará presented high treatment non-adherence rates.With the exception of the South region and São Paulo, all states had poor or regular results in terms of the percentage of cases tested for HIV (Figure 2).
The cluster analysis did not include the indicator "percentage of contacts investigated" due to the high percentage of micro-regions with invalid results.Timeliness of typing and of closure were also excluded because of invalid results in the state of São Paulo.The Burt table was developed taking into account the same attributes.
In the Burt table, it is possible to observe that most Brazilian micro-regions with regular or optimal median completeness and over 5 notifications during the period studied (n = 473) had regular timeliness of notification (n = 303), poor timeliness of treatment (n = 381), optimal or regular proportions of cases tested for microscopy (n = 473), poor proportions of treatment adherence (n = 234) and of cases tested for HIV (n = 234).Among the micro-regions with optimal proportions of people tested for HIV (n = 129), only 18.6% (n = 24) showed poor timeliness of notification and 37.2% (n = 48) presented poor performance in terms of adherence to treatment.By contrast, among micro-regions with poor proportions of microscopy (n = 57), 38.6% (n = 22) had poor timeliness of notification and 52.6% (n = 30) presented high proportions of treatment non-adherence.Three groups of micro-regions had similar surveillance systems.The first cluster (n = 109) was made up of micro-regions with the worst mean timeliness of treatment and notification, 24.9% and 62.8%, respectively.This cluster had the second worst mean rate of individuals tested for HIV (60.2%), approximately five percentage points above the same rate in Cluster 2.
The second cluster (n = 143) presented a mean timeliness of notification, mean percentage of individuals tested for microscopy and mean percentage of individuals who adhered to treatment of around 80%.However, the mean timeliness of treatment was 49.7%, the second worst performance for this indicator, after Cluster 1.
Cluster 3 (n = 221) had a mean percentage of cases tested for microscopy and mean percentage of adherence to treatment equal to approximately 91%.The mean percentage of individuals tested for HIV and the mean timeliness of notification for this Cluster were 81.8% and 81.2%, respectively.The mean timeliness of treatment for this group, however, was 61.0%.
Cluster 1 micro-regions were mainly located in the center-north of the country.This band encompasses the south of the state of Goiás, passing through Mato Grosso, Tocantins, Maranhão, reaching Rio Grande do Norte, Paraíba, Alagoas, Sergipe and part of the state of Bahia.Cluster 2 micro-regions are mainly found in the states of Amazonas, Pará, Mato Grosso, Goiás, Minas Gerais, Rio de Janeiro and Rio Grande do Sul.Finally, Cluster 3 micro-regions are found in the South of Brazil, in São Paulo, Mato Grosso do Sul, Espírito Santo, part of Tocantins and part of the North region, in particular Acre and Rondônia.Clusters 1 and 3 showed no overlapping when a dispersion graph of the main components was used to compare them.There was some overlapping between Clusters 1 and 2 and slightly greater overlapping between Clusters 2 and 3.

Discussion
The consistency of the tuberculosis information system was optimal in almost all Brazilian micro-regions between 2012 and 2014.However, the completeness of contacts investigated and 2nd and 6th months microscopy needs to improve.Timeliness of notification was regular and timeliness of treatment poor in most micro-regions.Furthermore, the percentage of cases that fully adhered to treatment and cases that were tested for HIV was poor in approximately 40% of units analyzed.A low performance Cluster was identified relating to the following indicators: timeliness of notification, timeliness of treatment and percentage of cases tested for HIV in the center-north region of Brazil.
It is important to highlight that other methods of analyzing clusters presented convergence issues.This is because the performance of the micro-regions was similar in terms of timeliness and acceptability, making it difficult to identify The results of this study were consistent with those observed in other studies.The percentage of inconsistency of new TB cases in Rio de Janeiro between 2001 and 2006 was equal or less than 3% 12 .Furthermore, a study conducted in the Brazilian urban centers with the greatest TB load revealed that between 2001 and 2006, microscopy variables in the 2 nd , 4 th and 6 th months had the lowest proportions of entries in the TB information system 31 .In 2008, the states of Acre, Roraima, São Paulo and Mato Grosso do Sul presented good results in terms of linkage and the analysis of record duplication 16 .These findings, though for other indicators, corroborate the good quality found in this study of the Acre and Roraima information systems.
In some parts of the country, the low quality of the information system restricts TB epidemiological analysis.This may be partly connected to health professionals' lack of understanding of the importance of the records that feed the system.It is important that professionals understand that records are strategic tools to combat TB, and not PNCT bureaucratic requirements 32 .A change in paradigm would allow local programs to routinely follow PNCT recommended practice, such as data linkage and completeness and consistency analysis.
From 2005 to 2008, approximately 55% of TB cases were notified on the same day of the diagnosis 19 .2012 to 2014 findings suggest an improvement in the timeliness of notification in the country.The timeliness of treatment in Brazil was approximately 70% between 2005 and 2008 19 .When comparing poor micro-region performance in the years from 2012 to 2014, there is no evidence of a gain in the timeliness of treatment over time.
Difficulties in improving the timeliness of treatment may be related to the poor skills of care teams.In João Pessoa, one of the weaknesses in controlling tuberculosis in 2007 was the lack of  training vacancies and the irregularity of courses provided by the state 33 .An evaluation study conducted in five municipalities in 2011 also found that professional training for TB surveillance was inadequate 34 .
From 2001 to 2005 there was a reduction in TB treatment non-adherence, followed by its stabilization at around 10.0% between 2006 and 2009 35 .The relationship between patients and health professionals is important for appropriate treatment closure.Studies showed that a high turnover of primary care staff hinders the establishment of these relationships 33,36 .High staff turnover is linked to low salaries and inadequate working conditions 37 .Therefore, an improvement in the TB surveillance system is associated to the professional recognition of staff.
External factors affecting the surveillance system such as the costs of tuberculosis and loss of income due to lower productivity are associated to low levels of treatment adherence 38 .Thus, in socially disadvantaged regions, surveillance systems should provide care at times that do not clash with times when individuals have to work 35 .In addition, basic food baskets, free transport and other social protection services for TB patients should be provided in partnership with local governments.
In Brazil, the percentage of cases tested for HIV has risen from 25.8% in 2001 to 55.8% in 2010 35 .Although these results are an improvement over time, they are still inadequate vis-à-vis Ministry of Health's targets 28 and corroborate the findings in this study.
Primary basic care structure is extremely important for the improvement of the TB surveillance system.The percentage of cases tested for HIV in Brazilian capitals in 2013 was higher than for those cases treated at the primary care level outside state capitals 22 .Furthermore, the increase in the coverage of Family Health Teams in Curitiba between 2000 and 2009 was linked to a reduction in the percentage of TB treatment non-adherence 39 .
Nevertheless, primary health care is experiencing difficulties in incorporating tuberculosis control responsibilities.In João Pessoa, in 2009, the percentage of suspected TB cases and high levels of test referrals were found to be insatisfactory 40 .In São Carlos, in 2009, a large number of health community agents had basic knowledge of TB control measures, but believed that the disease is transmitted through household utensils 41 .
Between 2001 and 2003, the states that presented the worst surveillance in terms of the quality of information, detection and follow up of TB cases were Pernambuco, Ceará and Bahia, in the Northeast, and Amazonas, Pará and Amapá in the North 10 .In this study, the Northeast states are mainly found in Cluster 1, where TB surveillance needs to be prioritized, and the states of the North are found mainly in Cluster 2, where TB is only given average priority.These results reflect the deep geographic inequalities of Brazil, as well as the need for greater investment in the more disadvantaged areas, where there may be a need for differentiated surveillance measures.
Although there is no universal consensus on the best method for evaluating TB surveillance systems, the CDC guidelines are well-established in terms of evaluating surveillance systems in general 42 .In order to adapt them to the Brazilian context, only satisfactory completeness indicators were employed in regions where there were no serious completeness issues.Given this evaluation involved the analysis of secondary data, some CDC (2001) attributes 15 , such as sensitivity and predictive value positive could not be assessed.
Nevertheless, the use of secondary data and CDC guidelines ensured the evaluation was comprehensive, both in geographic terms and in terms of the attributes studied.The use of CDC's quantitative attributes could be reproduced by local surveillance systems, given their low-cost and the fact that this evaluation involves a continuous monitoring method.
Timeliness of typing and of closure indicators were not used for cluster analysis due to the low level of completeness in São Paulo.Given that the completeness of other variables was considered satisfactory in this state, which has its own register for tuberculosis cases, the TBWEB 14 , it is possible that there may have been a failure in data transmission between the state and the national systems.Thus, a revision of data transmission between the TBWEB and SINAN is recommended.
Another limitation of this study was the fact that the cluster analysis did not include the percentage of cases followed up with directly observed therapy (DOT) and the percentage of contacts investigated among those identified.Although there are Ministry of Health guidelines regarding the definition of "conducted DOT" 25 , in practice, there are different understandings of this concept in the various micro-regions.The meaning of "conducted DOT" can change, for example, according to who supervises the treatment (health professional, relative, etc.) and the entry date of this variable vis-à-vis the treatment starting date.Furthermore, contacts investigated could not be estimated due to the low rates of data completeness.The distribution of guidebooks on the importance of these indicators is recommended.It should include detailed explanations on the meaning of these variables and how they should be entered.
It is important to consider as a limitation of this study the fact that consistency analysis was based on only one scenario.Some of the indicators chosen by the specialists were not assessed due to the small number of cases in the indicator interest group in the micro-regions.It was not possible to calculate consistency in the following scenarios: 1) exclusive extrapulmonary forms with negative or no microscopy and 2) less than five years with results of first microscopy(negative or not conducted).The inclusion of these scenarios in studies on TB surveillance systems is recommended in more extensive units of analysis, such as Units of the Federation (states) or macro-regions.
Although micro-regions restrict the implementation of suggestions and comparability, this approach enabled the dissemination of data with lower internal heterogeneity and higher external heterogeneity in comparison with analyses by states.Furthermore, the analysis level chosen resulted in the exclusion of only 1% (n = 6) of micro-regions, due to the small number of notified cases during the three-year period studied.
An analysis by municipalities could have resulted in the exclusion of a higher percentage of analysis units.
Finally, given the importance of TB surveillance indicators, we recommend the establishment of national and regional targets.In particular for the Tripartite Resolutions and the National Plan for Combatting Tuberculosis.This will enable the follow up and self-assessment of local managers and the attainment of greater clarity of outcomes.Although some indicators are part of national pacts 28 , their inclusion in the Tripartite Resolution could have a positive impact on the funding of the TB surveillance system 43 .
In conclusion, this study identified priority areas for improving the tuberculosis surveillance system, particularly in the states of Mato Grosso and Rio Grande do Norte.Furthermore, it revealed system weaknesses, not only in these regions, but across the country, such as low timeliness of treatment and inadequate entry of contacts investigated.The findings of this work can contribute to the decision-making of managers so as to improve the tuberculosis surveillance system in Brazil.The study stresses that the system's improvement requires an increase in the training and the recognition of health professionals, and the strengthening of primary health care, indirectly resulting in an increase in the social protection of people diagnosed with this disease.

Collaborations
GDM Silva contributed to the conception and draft of this study, statistical analysis, results interpretation and the writing of the text.P Bartholomay contributed to the conception and draft of this study, results interpretation and critical review of the text.OG Cruz and LP Garcia participated in the conception and draft of the study, statistical analysis, result interpretation and critical review of the article.All the authors revised the final version of the text and are responsible for its accuracy and integrity.

Figure 1 .
Figure 1.Distribution of timeliness indicators of the tuberculosis surveillance system in Brazilian micro-regions, 2012-2014*.Source: produced for this study with data from the National Information System of Notifiable Diseases (SINAN).

Figure 2 .
Figure 2. Distribution of acceptability indicators of the tuberculosis surveillance system in Brazilian microregion, 2012-2014*.Source: produced for this study with data from the National Information System of Notifiable Diseases (SINAN).

Figure 3 .
Figure 3. Cluster of Brazilian microregion according to timeliness and acceptability indicators of the tuberculosis surveillance system, 2012-2014*.Source: produced for this study with data from the National Information System of Notifiable Diseases (SINAN).

Chart 1 .
Indicators chosen by specialists to analyze the tuberculosis epidemiological surveillance system in Brazilian micro-regions between 2012 and 2014, per attribute.

Table 1 .
General completeness and consistency indicators in the tuberculosis surveillance system per Federation Unit (FU), Brazil 2012 to 2014*.
Source: produced for this study with data from the National Information System of Notifiable Diseases (SINAN).