Evaluation of coverage and completeness of Information System variables on public health budgets

ABSTRACT

This article evaluates the quality of the Information Systems Brazilian Finances (Finbra) and Public Health Budget Information System (Siops), in terms of coverage and completeness for the municipalities of Brazil, between 2004 and 2012, considering some fiscal and investment variables in health. Finbra and Siops presented excellent coverage and the completeness ranged from excellent to regular. The Siops performed better for both dimensions. The worst performing variable was the Primary Care Expenditure, both for the coverage and completeness dimensions and for most of the years of the series. It can be concluded that both Information Systems present satisfactory quality, which should stimulate managers to use that information to aid decision making. Besides the respect for the pillar of transparency in the assurance of the exercise of citizenship rights.

KEYWORDS
Information Systems; Budgets; Indicators; Database

RESUMO

Este artigo avalia a qualidade dos Sistemas de Informação Finanças do Brasil (Finbra) e Sistema de Informação de Orçamento Público em Saúde (Siops), nas dimensões de cobertura e completitude para os municípios do Brasil, entre 2004 e 2012, considerando algumas variáveis fiscais e de investimento em saúde. O Finbra e o Siops apresentaram excelente cobertura; já a completitude variou de excelente a regular. O Siops teve melhor desempenho para as duas dimensões. A variável de pior desempenho foi a Despesa Atenção Básica, tanto para a dimensão de cobertura como para completitude e para a maioria dos anos da série. Conclui-se que os dois Sistemas de Informação apresentaram qualidade satisfatória, o que deve estimular os gestores a utilizarem essas informações para a tomada de decisão. Além do respeito ao pilar da transparência na garantia do exercício dos direitos dos cidadãos.

PALAVRAS-CHAVE
Sistemas de Informação; Orçamentos; Indicadores; Base de dados

Introduction

In Brazil, as from the Federal Constitution (FC) of 1988, there was an intense process of administrative and tax decentralization for the municipalities, which started having the status of federal entities, receiving assignments that increased their needs for financial and technical resources to plan and carry out the public services under their responsibility¹.

This new context has contributed to the institutionalization of public planning at the local level and stimulated discussion about the balance of public finances, which culminated in the approval of Complementary Law (CL) nº 101/2000, the Fiscal Responsibility Act (FRA), whose objective is to substantiate responsibility on fiscal management based on four pillars: planning, transparency, control and accountability².

With regard to planning, the FC of 1988 provides for the need to integrate with the budget, through the elaboration of the multiannual plan, budgetary directives and annual budgets (articles 165, I, II and III), which are basic instruments for this purpose. The pillar of transparency is based on encouraging the participation of society, conducting public hearings and wide dissemination, including electronic means of public access, of all acts and facts related to the collection of revenues and realization of expenditure by public entities. Regarding the control pillar, the FRA established the mechanism of fiscal management councils, made up of representatives of all the powers and spheres of government, Prosecution Office and representative technical entities of the society. Finally, the responsibility pillar, which imposes sanctions on the public manager in case of non-compliance with the rules established in the FRA².

With regard to health policy, the FC of 1988 instituted the Unified Health System (SUS), whose principles of decentralization, comprehensive care and community participation have led municipalities to improve public management in order to comply with the regulatory legal provisions of the system and the yearnings of society.

This environment has imposed the need to deploy and improve Information Systems (IS), which had budgetary data. Examples include Brazilian Finances (Finbra) and Public Health Budget Information System (Siops).

Finbra is the responsibility of the National Treasury Secretariat (NTS); and, since 1989, presents consolidated data on the annual budgetary execution (revenue and expenditure) of Brazilian municipalities. Siops is the responsibility of the Ministry of Health and provides health budget and expenditure data, allowing the monitoring of many indicators. Both IS are in the public domain and serve to comply with the pillars of the FRA, allowing managers and citizens access to information for planning and control of the public budget in a transparent and responsible manner.

For Almeida³⁽¹⁶⁷⁾, databases and IS are “tools for obtaining indicators that provide financial transfers [...] and, are frequently mentioned as important elements in the decision-making process”, which gives them great importance and valorization, and justifies the search for improvement.

In many countries, there is an important volume of funds invested in order to guarantee the quality of information. An example that demonstrates the importance given to the quality of the information is the fact that the International Monetary Fund (IMF) makes available, on its website, the initiatives of 18 countries to improve economic statistics, in addition to suggesting a model for improvement and evaluation of information: Data Quality Assessment Framework (DQAF).

There is, however, little consensus regarding the term quality of information, since it has a multidimensional and multifaceted character^4,5. By recognizing this aspect, in an approximation of the information quality concept, Paim et al.⁵ define that quality information is that apt/convenient for use, according to the need of the user.

In an integrative review, Lima et al.⁶ evaluated the quality of the information of some Brazilian IS, listing nine dimensions of quality: accessibility, methodological clarity, reliability, consistency, non-duplicity, opportunity, validity, coverage and completeness. Taking the last two dimensions, English⁷ defines coverage as the degree to which the events of the universe (scope) for which it was developed are recorded in the IS. Completeness is defined by Campbell et al.^{((4) )}as the degree to which the records of an SI have non-null values. These definitions allow us to affirm that such dimensions of quality aim at the availability of the data for broad access.

Correia et al.⁸, in a systematic review on the completeness dimension of IS, evidenced that the most evaluated systems were those of epidemiological rationality. With respect to the coverage dimension, the same can be observed. However, it is known that health information should cover not only aspects related to the health/disease process, but those of an administrative/managerial nature are also essential to the decision-making process⁹.

Some research groups have evaluated the quality of Finbra and Siops: Lima et al.¹⁰ investigated the reliability of Siops, comparing its records with those of the National Health Fund and NTS, using data from Finbra. Gonçalves et al.¹¹ evaluated the reliability of the municipal data declared in the Siops with the records of the Audit Court of the State of Pernambuco (TCE-PE), from 2000 to 2005; Medeiros¹² evaluated some dimensions of information quality of Finbra and Siops for the municipalities of Brazil, covering the years 2004 and 2009; and Medeiros et al.¹³ analyzed the dimensions of accessibility, opportunity and methodological clarity of the two IS.

The availability of data detailed on municipal revenues and expenditures in Finbra and Siops makes these systems the main sources for studies or estimates of this nature, a fundamental aspect in view of the importance of the municipal entity in public expenditure.

The scientific production evaluating the quality of these IS, besides being scarce, has prioritized the dimension of reliability. There are no studies of national scope that address the municipal budget and integrate the Finbra and Siops databases, analyzing the dimensions of coverage and completeness. This statement points to the need for investigations of this nature because of their potential for improving and enhancing the quality of information in these national databases.

To this end, this paper evaluates the quality of Finbra and Siops’ IS, in the dimensions coverage and completeness for the municipalities of Brazil, between 2004 and 2012, considering some fiscal and health investment variables.

Material and methods

This is an evaluative study, with descriptive design and quantitative approach, made from the Finbra and Siops databases. These databases are available, respectively, on the websites: http://www.tesouro.fazenda.gov.br/pt_PT/contas-anuais and http://portalsaude.saude.gov.br/index.php/o-ministerio/principal/siops.

The period analyzed refers to the years between 2004 and 2012. The initial milestone of this series (2004) is justified by the consolidation of the IS tackled as a tool for control and transparency in that year. Finbra, in 2004, completed 15 years of operation; and Siops, 4 years. The year 2012 was defined by ensuring that all data have been processed, once, in 2013, Finbra underwent a change in the way the data were available; therefore, the homogeneity of the information made available on this basis could be ensured by 2012.

All municipalities in Brazil were investigated. According to data from the Brazilian Institute of Geography and Statistics (IBGE), municipalities totaled 5,565 in the period. However, Brasília (DF) and Fernando de Noronha (PE) were excluded due to the differentiated character that these localities assume in the Brazilian federation. According to article 75, of the State Constitution, 1988, Fernando de Noronha is a State District of Pernambuco, and its budgetary information are provided in that state. In the case of Brasília, which is the only municipality in the Federal District and capital of the Country, its accountability is tied to the budget of the Union.

The quality of these databases was evaluated based on the coverage and completeness dimensions, considering the variables described in chart 1, which were selected for their relevance from the point of view of fiscal management and as expressions of the collection capacity and investment in in the municipalities of Brazil.

Chart 1
Description of Finbra and Siops variables, evaluated in coverage and completeness dimensions. Brazil, 2018

Information System	Variables	Relevance
Finbra	Revenue Budget;	Demonstrates the capacity of the municipality to finance municipal public policies;
	Personnel/Net Current Revenue;	Expresses the limits/possibilities of personnel expenses and ascertains compliance with the legal limit by the municipality in accordance with CL 101/2000, the FRA;
	Health Expenditure;	Demonstrates municipal investment in the health sector;
	Primary Care Expenditure;	Demonstrates municipal investment in Primary Care;
Siops	Participation of the tax revenue in the total revenue of the Municipality (indicator 1.1);	Demonstrates the collection capacity of municipalities;
	Participation of intergovernmental transfers in the total revenue of the Municipality (indicator 1.2);	Expresses the dependence of municipalities regarding transfers from other spheres of government;
	Participation of personnel expenditures in total health expenditure (indicator 2.2).	Shows the commitment of health expenses in relation to expenses with personnel of the sector.

Source: Own elaboration.

The coverage evaluation was carried out from the importation of variables into the ArcGIS 10.1 software with the linkage purpose of the Finbra and Siops databases with the IBGE municipal base, which served as the gold standard. The linkage result for each indicator provided the evaluation of the coverage, identifying the municipalities that did not provide data for the variables selected in the years of the series. For the completeness evaluation, the relative frequency (in percentage) of information absence (empty or zero field) was calculated. The percentages of coverage and completeness were classified according to the score elaborated by Romero and Cunha¹⁴, which is the most frequent in studies that evaluate completeness⁸.

The Romero and Cunha¹⁴ score define the following degrees of evaluation in relation to the lack of data: excellent (less than 5%), good (5% to 10%), regular (10% to 20%), poor 50%) and very bad (50% or more).

It is worth pointing that this article integrates part of the results of the survey ‘Municipalities of Brazil: expenditure on health personnel, relation with the forms of hiring and impact on the indicator of hospitalization for conditions sensitive to primary care’, supported by the notice of the Research Excellence Program and the Foundation for Science and Technology Support of the State of Pernambuco (Proep/ Facepe 19/2015).

Results and discussion

Coverage

The linkage process between the databases is illustrated in figure 1: of the 5,563 existing municipalities in the period, 47 did not have data in Finbra. In Siops, there were no losses in the period, which, in the end, resulted in 5,516 municipalities with correspondence in the two IS in the historical series.

Figure 1
Linkage between the databases of IBGE, Finbra and Siops for certain variables, between the years of 2004 and 2012. Brazil, 2018

Throughout this process, some differences in the two IS can be emphasized. In Finbra, in the studied period, the data are available in .mdb files (database files), which are generated annually and downloaded from the NTS website, and can be executed on the machine itself by navigating through commands, which quickly generate different reports related to expenditures and revenues, which bring a set of specific variables. Another feature of Finbra is the fact that, every year, changes are made in its usability interface. The aim is to improve the system, but these modifications become a complicating factor in the manipulation of the bases, due to the change in the names of the links that select the reports, as well as their location on the screen, requiring attention, since, the change in the paths to each year, implies different ways to access the same variable of a particular report.

At Siops, the work was made easier by the fact that the system has an intuitive, easy-toaccess online tabulator. All variables can be tabulated for the entire historical series and imported into electronic spreadsheets.

Another difference in relation to IS is the use of different codes to identify each municipality on the bases: the Finbra dismantles the code of the Federative Unit and the municipal code into two columns; Siops presents a sixdigit code for each municipality. In the case of the IBGE, gold-standard basis, each municipality is identified with a corresponding code, composed of seven digits. Although these forms of codification correspond to each other, these differences hamper the linkage between the bases of the systems, and it is necessary to standardize the codes previously, for the composition of the database and later analysis.

The use of municipal codes in the linkage process excluded problems such as that described by Medeiros¹², who, in a same year, identified municipalities in the base of Finbra and IBGE with the same code, but with different names. According to the author, these nominal differences can be attributed to changes in the name of the municipality in the years analyzed, without these changes having been introduced simultaneously in Finbra and IBGE.

According to figure 2A, it can be observed that Finbra had coverage of 99.16%, corresponding to 4,516 municipalities with data available in the system; and Siops had 100% coverage. According to the Romero and Cunha¹⁴ score, the coverage of the two IS is classified as excellent, since, the losses of municipalities represented less than 5%.

Figure 2A
Frequency for obtaining data records of the municipalities of Brazil at the IBGE, Finbra and Siops databases, between 2004 and 2012. Brazil, 2018

In the studied period, the IS had different rules and criteria for reporting the data by the municipalities. For Finbra’s feeding system, the municipalities fill out forms with data of declaratory nature extracted from their balance sheets and send them to Caixa Econômica Federal (CEF), which, after consolidation, sends them to NTS. Failure to submit reports to Finbra implies legal noncompliance and may result in penalties or sanctions, as defined in the FRA¹⁵. In the case of Siops, during the study period, the non-feeding of the database did not imply sanctions, although it is an IS that generates data to compose the management reports of the municipalities, fundamental for changes in the habilitation in the funding modalities. In this way, it was expected that Finbra would present greater coverage in relation to Siops, due to its mandatory character.

By analyzing each selected variable on a case-by-case basis, table 1 shows that all variables of both systems presented excellent coverage, with less than 1% of losses. However, it is worth mentioning that variables of the Finbra had inferior performance compared to the variables of the Siops.

Table 1
Description of the coverage of selected Finbra and Siops variables between 2004 and 2012, for the municipalities of Brazil. Brazil, 2018

Indicator		Nº of municipalities with information	Nº of losses	% of municipalities with information
Finbra	Revenue Budget	5,563	0	100.00
	Personnel/Net Current Revenue	5,560	3	99.95
	Health Expenditure	5,561	2	99.96
	Primary Care Expenditure	5,518	45	99.19
Siops	Participation of the tax revenue in the total revenue of the Municipality	5,563	0	100.00
	Participation of intergovernmental transfers in the total revenue of the Municipality	5,563	0	100.00
	Participation of personnel expenditures in total health expenditure	5,563	0	100.00

Source: Own elaboration.

Among Finbra’s variables, the variable Primary Care Expenditures had the worst result, with coverage of 99.19%, corresponding to 45 municipalities without information, reducing the number of municipalities analyzed in this study. This finding is noteworthy, since, in the period from 2004 to 2012, the resource for primary care used to be passed on to the municipalities fund to fund, in a specific block, that is, it guaranteed the linkage of the resource and, therefore, it was expected that this data was readily available in Finbra.

Completeness

Table 2 shows the incompleteness of each variable selected in the databases, describing the number of data omissions year by year. In Finbra, the variable that showed the best performance was the Revenue Budget, followed by the variables Personnel/Net Current Revenue, Health Expenditure and Primary Care Expenditures. Finbra’s completeness ranged from very good to regular. Still according to table 2, it can be observed that the three Siops indicators obtained similar behavior in the years of the series, with a percentage of losses always lower than 5%, which characterizes excellent completeness.

Table 2
Description of the incompleteness of each variable selected from Finbra and Siops, between 2004 and 2012, for the municipalities of Brazil. Brazil, 2018

Variable	System	2004		2005				2006*		2007*
			%	n		%		n	%	n	%
Revenue Budget	Finbra	235	4.23	317		5.70		26	0.47	10	0.18
Personnel/Net Current Revenue	Finbra	319	5.74	319		5.74		24	0.43	38	0.68
Health Expenditure	Finbra	490	8.97	341		6.24		58	1.06	58	1.06
Primary Care Expenditure	Finbra	1,025	18.58	924		16.75		524	9.50	524	9.50
Participation of the tax revenue in the total revenue of the Municipality	Siops	154	2.77	16		0.29		17	0.31	26	0.47
Participation of intergovernmental transfers in the total revenue of the Municipality	Siops	154	2.77	15		0.27		17	0.31	26	0.47
Participation of personnel expenditures in total health expenditure	Siops	155	2.79	15		0.27		17	0.31	26	0.47
Variable	Sistem	2008		2009		2010		2011		2012
		n	%	n	%	n	%	n	%	n	%
Revenue Budget	Finbra	80	1.44	42	0.76	67	1.20	178	3.20	387	6.96
Personnel/Net Current Revenue	Finbra	82	1.47	49	0.88	72	1.29	183	3.29	395	7.10
Health Expenditure	Finbra	98	1.79	68	1.25	105	1.92	218	3.99	432	7.91
Primary Care Expenditure	Finbra	501	9.08	460	8.34	470	8.52	496	8.99	729	13.21
Participation of the tax revenue in the total revenue of the Municipality	Siops	55	0.99	15	0.27	21	0.38	40	0.72	121	2.18
Participation of intergovernmental transfers in the total revenue of the Municipality	Siops	52	0.93	15	0.27	20	0.36	38	0.68	118	2.12
Participation of personnel expenditures in total health expenditure	Siops	52	0.93	15	0.27	20	0.36	38	0.68	118	2.12

Source: Own elaboration.

In a more detailed way, figure 2B shows the behavior of the Finbra’s variables in the studied period, noting that the behavior of the variables is quite similar. The variables Revenue Budget, Personnel/Net Current Revenue and Health Expenditure presented an excellent score between the years 2006 and 2011. Meanwhile, Primary Care Expenditure, a variable with the worst performance in the series, presented regular completeness with losses between 10% and 20% for the first four and for the last year studied. The years 2004, 2005 and 2012 were the ones with the highest occurrence of absence of data.

Figure 2B
Percentage of losses of Finbra’s selected variables, between 2004 and 2012, for the municipalities of Brazil. Brazil, 2018

According to figure 2C, it can be observed that the variables of the Siops had the highest levels of data omission in the years 2004, 2008 and 2012, coinciding in large part with the behavior of variables of the Finbra.

Figure 2C
Percentage of losses of the selected Siops’ variables, between the years 2004 to 2012, for the municipalities of Brazil. Brazil, 2018

There is no doubt that Finbra and Siops have introduced advances in the process of standardization, collection, treatment and dissemination of budget data and health financing³. However, the interface of each system differs completely, reflecting the lack of interoperability between them. This finding implies less speed to match the data, that is, there is a greater expenditure of time to match/ pair up the data. Far from being a problem exclusively to budgetary IS, issues involving the lack of integration between systems have been observed in other studies^16-18.

Specific features of the different reports generated by Finbra and the annual changes in its interface demand certain accounting knowledge of the user, an aspect highlighted in Medeiros et al.¹³, which mention that there is proportionality of the level of accounting knowledge of the user with the exploitation potential of Finbra. In this regard, Schettini¹⁹ points out that the handling of Finbra presupposes that the researcher has the capacity to overcome problems derived, on the one hand, from the growing detail of the accounting information in its various reports and, on the other, from the conceptual changes of public accounting.

As for the annual modifications in the usability interface of Finbra, a study¹³ warns that manipulation of data, especially when involving the use of more than one year, is a rather complex task due to their differences. However, these changes did not make the data collection process of this system unfeasible.

Finbra was expected to be better evaluated in relation to Siops, due to the obligatory character of that system. However, Finbra has a much more critical method of criticism, becasue, when the information reaches the System of Collection of Accounting Data of the Institutions of the Federation (SISTN), the CEF technicians check what was published in the municipal balance sheet. Additionally, when the data are sent from the CEF to the NTS, there is a new conference between the transfer information of the Municipal Participation Fund (MPF) declared by the municipality and what, in fact, was passed on²⁰. Submitted to these criticisms, the municipalities in which there are inconsistencies in the records have their reports eliminated from Finbra, resulting in the impediment of access to the data of revenues and expenditures, because in these cases, municipal data are blocked, which justifies the less coverage and completeness of this system²¹.

The Primary Care Expenditures performed worse at Finbra, both in terms of coverage and completeness. Although the resource for primary care is, during the years studied, transferred to the municipalities in a specific block and fund to fund, it is important to understand that, according to the functional classification of expenditure, primary care expenditure is a subfunction of the expenditure function with health²². That is, the primary care data is self-contained in health expenditure, which requires a certain disaggregation of the information on the expenditures incurred in the municipalities. In this sense, it is relevant the availability of qualified personnel in the municipalities for the development of monitoring tools and accountability of the expense with this level of disaggregation.

It should be emphasized that, of the 45 municipalities with no data for Primary Care Expenditure, at the Finbra, 37 are small, with less than 20 thousand inhabitants. Research indicates that the population size is related to the qualification of the human resources of municipal management. The Profile Research of the Municipalities of Brazil²³ demonstrates that, in smaller municipalities, the number of mayors with incomplete elementary school is higher. This same finding is evidenced by Bernd et al.²⁴, who, when analyzing the qualification of the servers of the municipalities of Paraná, have demonstrated that servers with a fundamental level of education in small municipalities prevail.

Rezende et al.²⁵ state that small-sized administrations do not directly follow their economic and financial indicators, as required by legal provisions such as the FRA, Constitutional Amendment nº 29/2000 and CL nº 141/2012. Many municipalities hire private organizations to provide specialized public finance services to city halls; and municipal managers only follow the administrative situation through reports issued by these companies that, in fact, monitor municipal public finances and public accounting. This practice is justified, according to the public agents themselves, by the complexity of the rules and by the absence of municipal officials qualified for such tasks²⁶. In the completed reports from the Office of the Comptroller General it’s said that, of the 200 municipalities surveyed, only 7 showed no evidences of any administrative problem²⁷.

The two IS were well evaluated in the completeness dimension, ranging from excellent (<5% of losses) to regular (between 10% and 20% of losses). There are no other studies that evaluate these IS in this dimension. Several studies have been carried out, verifying the completeness of epidemiologically based IS, such as the Mortality Information System (SIM), the Information System on Live Births (Sinasc) and the Notifiable Diseases Information System (Sinan)^6,8,14,27-32. It is known how vast the arsenal of available variables is in each of these systems, as well as the possibilities of approach with respect to method, locations and periods. Considering the approach of the completeness dimension, an approximation to these studies allows us to affirm that, even with a distinct content to the epidemiological, Finbra and Siops are very well evaluated in relation to these dimensions, when compared to another IS.

It is noteworthy that both the Finbra and Siops variables present the highest levels of data omission in the years 2004, 2008 and 2012, being these election years. In an empirical study carried out in the state of Bahia³³, it was evidenced that the electoral year and the alternation of power were factors that have helped to increase the probability of rejection of municipal accounts by the TCE-BA. This finding is supported by the Theory of Political Cycles, which assumes that election years determine aspects of the behavior of public agents in terms of accountability^33,34.

Conclusions

The research data show that both Finbra and Siops presented, between 2004 and 2012, excellent coverage and completeness, ranging from excellent to regular, and Siops presented better performance for the two dimensions evaluated, which may be related to two questions. The first one refers to the fact that critique system of Siops, at the time, is less rigid than Finbra’s, in which the data go through different stages of conference, before being made available for public access. The other issue concerns the use of Siops as a reference for the monitoring, inspection and control of the application of resources linked to actions and public health services, in determination of the impositions of Constitutional Amendment nº 29/2000, which leads municipalities to prioritize and enhance Siops’ filling.

The worst-performing variable was the Primary Care Expenditure, both for the coverage dimension and for the completeness dimension. For most of the years of the series, the completeness of this variable was regular, which may be related to the availability of qualified personnel in the municipalities for the development of tools for monitoring and accountability of the expenditure with this level of disaggregation, mainly, in the smallsized municipalities.

Another relevant finding was the fact that the years of lower coverage, for both IS, were 2004, 2008 and 2012, years of municipal elections, which can be explained by the influence of electoral years on the disclosure of budget data, with the Theory of Political Cycles being the support for this assertion.

The findings demonstrate the quality of the data disseminated in these IS, which should stimulate the development of skills in managers to use these data in decision making regarding the collection capacity, funding and fiscal management of municipalities. For the academic area, these findings should stimulate the production of nationally based studies from the data available in these IS.

Monitoring the transfers and behavior of the subnational bodies in relation to public spending is essential for the analysis of the decentralization process of the SUS’s policies, as well as of other policies, besides being one of the pillars of the public administration. In this sense, the availability of quality information for free access to citizens is aligned with transparency in public administration, one of the prerequisites for democratic management, enabling decision-making processes and ensuring the exercise of citizenship rights.

• *
  Orcid (Open Researcher and Contributor ID).
• Financial support: Foundation for Science and Technology Support of the State of Pernambuco - Facep-PE (Edital Proep/Facepe nº 19/2015)

Referências

• ¹
  Abrucio FL. Desafios contemporâneos para a reforma da administração pública brasileira. In: Peters BG, Pierre J. Administração pública: coletânea. São Paulo: Unesp; 2010. p. 537-548.
• ²
  Cruz CF, Macedo MAS, Sauerbronn FF. Responsabilidade fiscal de grandes municípios brasileiros: uma análise de suas características. Rev. adm. pública. 2013; 47(6):1375-1399.
• ³
  Almeida MF. Debate sobre o artigo de Hillegonda Maria Dutilh Novaes. Cad. Saúde Pública. 2004; 20:167-168.
• ⁴
  Campbell SE, Campbell MK, Grimshaw JM, et al. A systematic review of discharge coding accuracy. J. public health med. 2001; 23(3):2015-211.
• ⁵
  Paim I, Nehmy RMQ, Guimarães CG. Problematização do conceito "Qualidade" da Informação. Perspect. ciênc. inf. 1996; 1(1):111-119.
• ⁶
  Lima CRA, Schramm JMA, Coeli CM, et al. Revisão das dimensões de qualidade dos dados e métodos aplicados na avaliação dos sistemas de informação em saúde. Cad. Saúde Pública. 2009; 25(10):2095-2109.
• ⁷
  English LP. Improving data warehouse and business information quality. New York: John Wiley & Sons; 1999.
• ⁸
  Correia LOS, Padilha BM, Vasconcelos SML. Métodos para avaliar a completitude dos dados dos sistemas de informação em saúde do Brasil: uma revisão sistemática. Ciênc. Saúde Colet. 2014; 19(11):4467-4478.
• ⁹
  Carvalho ALB. Informação em saúde como ferramenta estratégica para a qualificação da gestão e o fortalecimento do controle social no SUS. Tempus (Brasília). 2009; 3(3):16-30.
• ¹⁰
  Lima CRA, Carvalho MS, Schramm JMA. Financiamento público em saúde e confiabilidade dos bancos de dados nacionais: um estudo dos anos de 2001 e 2002. Cad. Saúde Pública. 2006; 22(9):1855-1864.
• ¹¹
  Gonçalves RF, Bezerra AFB, Espírito Santo ACG, et al. Confiabilidade dos dados relativos ao cumprimento da Emenda Constitucional nº 29 declarados ao Sistema de Informações sobre Orçamentos Públicos em Saúde pelos municípios de Pernambuco, Brasil. Cad. Saúde Pública. 2009; 25(12):2612-2620.
• ¹²
  Medeiros KR. Lei de Responsabilidade Fiscal e despesas com pessoal da saúde: um estudo dos municípios brasileiros [tese]. Recife: Centro de Pesquisa Aggeu Magalhães, Fundação Osvaldo Cruz; 2011. 185 p.
• ¹³
  Medeiros KR, Albuquerque PC, Diniz GTN, et al. Bases de dados orçamentários e qualidade da informação: uma avaliação do Finanças do Brasil (Finbra) e do Sistema de Informações sobre Orçamentos Públicos em Saúde (Siops). Rev. adm. pública. 2014; 48(5):1113-1133.
• ¹⁴
  Romero DE, Cunha CB. Avaliação da qualidade das variáveis sócio-econômicas e demográficas dos óbitos de crianças menores de um ano registrados no Sistema de Informações sobre Mortalidade do Brasil (1996/2001). Cad. Saúde Pública. 2006; 22(3):673-684.
• ¹⁵
  Menezes RT. Efeitos da LRF sobre as categorias e funções de despesas dos municípios brasileiros (1998 a 2004) [dissertação]. São Paulo: Universidade de São Paulo; 2006. 128 p.
• ¹⁶
  Morais RM, Costa AL. Um modelo para avaliação de sistemas de informação do SUS de abrangência nacional: o processo de seleção e estruturação de indicadores. Rev. adm. pública. 2014; 48(3):767-793.
• ¹⁷
  Souza RC, Freire SM, Almeida RT. Sistema de informação para integrar os dados da assistência oncológica ambulatorial do Sistema Único de Saúde. Cad. Saúde Pública. 2010; 26(6):1131-1140.
• ¹⁸
  Damé PKV, Pedroso MRO, Marinho CL, et al. Sistema de Vigilância Alimentar e Nutricional (Sisvan) em crianças do Rio Grande do Sul, Brasil: cobertura, estado nutricional e confiabilidade dos dados. Cad. Saúde Pública. 2011; 27(11):2155-2165.
• ¹⁹
  Schettini BP. Análises da dinâmica orçamentária dos municípios brasileiros: uma aplicação da metodologia VAR com dados empilhados. Econ.apl. 2012; 16(2):291-310.
• ²⁰
  Faveret ACSC. Prontuários de bases de dados: informação sistematizada para as contas de saúde do Brasil. Brasília, DF: IPEA, 2009.
• ²¹
  Pinheiro TC. Hierarquia urbana e situação fiscal dos municípios brasileiros. Rev. ass. bra. de est. reg. e urb. 2009; 3(1):107-127.
• ²²
  Brasil. Ministério da Fazenda. Secretaria do Tesouro Nacional. Manual Técnico de Orçamento. Brasília, DF; 2018.
• ²³
  Instituto Brasileiro de Geografia e Estatística. Perfil dos municípios brasileiros: pesquisa de informações básicas municipais 2017. Rio de Janeiro: IBGE; 2018.
• ²⁴
  Bernd DC, Machado RF, Clemente A. Porte municipal e capital humano nas administrações municipais paranaenses. In: Anais do 17º Seminários em Administração; 2014; São Paulo: ALAC, 2014. p. 1-15.
• ²⁵
  Rezende JB, Pereira JR. Gestão pública em municípios de pequeno porte: velhas questões, leis modernas e práticas patrimonialistas. In: Anais do 4° Congresso Internacional Governo, Gestão e Profissionalização em Âmbito Local Frente aos Grandes Desafios de Nosso Tempo; Belo Horizonte: Fundação João Pinheiro; 2014. p. 1-20.
• ²⁶
  Brasil. Controladoria Geral da União. Controladoria aponta irregularidades em mais de 300 municípios [internet]. [acesso em 2018 abr 15]. Disponível em http://www.presidencia.gov.br/cgu/
  » http://www.presidencia.gov.br/cgu/
• ²⁷
  Glatt R. Análise da qualidade da base de dados de AIDS do Sistema de Informação de Agravos de Notificação (Sinan) [dissertação]. Rio de Janeiro: Fundação Oswaldo Cruz, Escola Nacional de Saúde Pública; 2005.
• ²⁸
  Melo GBT, Valongueiro S. Incompletude dos registros de óbitos por causas externas no Sistema de Informações sobre Mortalidade em Pernambuco, Brasil, 2000-2002 e 2008-2010. Epidemiol. Serv. Saúde. 2015; 24(4):651-660.
• ²⁹
  Bonilha EA, Vico ESR, Freitas M, et al. Cobertura, completude e confiabilidade das informações do Sistema de Informações sobre Nascidos Vivos de maternidades da rede pública no município de São Paulo, 2011. Epidemiol. Serv. Saúde. 2018; 27(1):1-10.
• ³⁰
  Barbosa DA, Barbosa AMF. Avaliação da completitude e consistência do banco de dados das hepatites virais no Estado de Pernambuco, Brasil, no período de 2007 a 2010. Epidemiol. Serv. Saúde. 2013; 22(1):49-58.
• ³¹
  Barbuscia DM, Rodrigues-Júniro AL. Completude da informação nas Declarações de Nascido Vivo e nas Declarações de Óbito, neonatal precoce e fetal, da região de Ribeirão Preto, São Paulo, Brasil, 2000-2007. Cad. Saúde Pública. 2011; 27(6):1192-1200.
• ³²
  Neri SCC. A qualidade dos dados dos sistemas de informação em saúde aplicados na atenção à saúde materno-infantil [tese]. Salvador: Universidade Federal da Bahia, Instituto de Saúde Coletiva; 2016. 182 p.
• ³³
  Pereira JRT. Gestão e controle de recursos públicos - um estudo sobre a rejeição de prestação de contas nos governos municipais do estado da Bahia [dissertação]. Salvador: Universidade Federal da Bahia, Faculdade de Ciências Contábeis; 2010. 114 p.
• ³⁴
  Pereira JRT, Cordeiro-Filho JB. Rejeições de Prestação de Contas de Governos Municipais: o que está acontecendo? Contab. Gest. e Gov. 2012; 15(1):33-43.

Publication Dates

• Publication in this collection
  05 Aug 2019
• Date of issue
  Apr-Jun 2019

History

• Received
  24 July 2018
• Accepted
  17 Feb 2019