Acessibilidade / Reportar erro

Alternative funding sources’ impact on efficiency and productivity of subnational entities in Brazil after the Fiscal Responsibility Law

Abstract

We use panel data envelopment analysis on a sample that aims to model the process of social welfare generation by federation units. The first innovation is the use of input sources associated with traditional revenues, such as transfers and taxes, as well as alternative sources of funding via national credit and paradiplomacy. As a result of this process, the State provides goods and services associated with socioeconomic variables such as, health, violence, demographics, study, and basic infrastructure. Only seven units, Bahia, Ceará, Maranhão, Minas Gerais, Pará, Paraná, and São Paulo, were entirely efficient in the decade studied (2004 to 2013). The incorporation of credit sources as inputs significantly influences efficiency gains and changes to national ranking. The level of efficiency of some states might be compromised by excessive debt. In terms of public policy, the use of efficiency as a meritocracy metric in tax reform is suggested.

Keywords:
efficiency and productivity; credit and paradiplomacy; federative subnational entities; social welfare; debt.

Resumo

Este artigo faz uso da técnica de análise envoltória de dados em painel, para uma amostra balanceada com dados anuais de 2004 a 2013, visando modelar o processo de geração de bem-estar social por parte das unidades da federação. Uma primeira inovação está no uso de fontes de insumos associados às receitas tradicionais, como transferências e tributos, e de fontes alternativas de financiamento, via crédito nacional e via paradiplomacia. Como resultado desse processo, o Estado proporciona produtos e serviços associados a variáveis de natureza socioeconômica, de saúde, de violência, de demografia, de estudo e de infraestrutura básica. Apenas sete unidades foram plenamente eficientes na década em questão: Bahia, Ceará, Maranhão, Minas Gerais, Pará, Paraná e São Paulo. A incorporação das fontes de crédito com insumos aumenta a eficiência dos estados, alterando o ranking nacional. O nível de eficiência de alguns estados pode vir a ser comprometido pelo endividamento excessivo. Em termos de política pública, sugere-se o uso da eficiência como métrica de meritocracia na reforma tributária.

Palavras-chave:
eficiência e produtividade; crédito e paradiplomacia; entes federativos subnacionais; bem-estar social; endividamento.

Rsumen

Este artículo utiliza envolvente de datos panel de la técnica de análisis de una muestra equilibrada de los datos anuales de 2004 a 2013, con el objetivo de modelar el proceso de generación de bienestar social por las unidades de la federación. La primera innovación es el uso de fuentes de entradas asociadas con recetas tradicionales, como las transferencias y los impuestos, así como fuentes alternativas de financiación a través de crédito nacional y por medio de la paradiplomacia. Como resultado de este proceso, el Estado proporciona bienes y servicios asociados a variables de carácter socioeconómico, la salud, la violencia, la demografía, el estudio y la infraestructura básica. Sólo siete unidades eran totalmente eficiente en la década en cuestión, Bahia, Ceará, Maranhão, Minas Gerais, Pará, Paraná y São Paulo. La incorporación de fuentes de crédito con entradas influye significativamente en la eficiencia y en el ranking nacional. El nivel de la eficiencia de algunos estados podría verse comprometido por la deuda excesiva. En términos de políticas públicas, se sugiere el uso de la eficiencia como métrica meritocracia en la reforma fiscal.

Palabras clave:
eficiencia y productividad; crédito y paradiplomacia; las entidades federativas subnacionales; bienestar social; deuda.

1 Introduction

The concept of efficiency is recurrent in international theoretical and empirical literature comparing countries, regions, states or municipalities on the allocation of resources and generation of social welfare. In this literature, it is common to observe the measurement of efficiency at the macro level — limited to modelling broad governmental items, such as health, education, income — as well as at the micro level, which is more specific and focused in certain issues, such as fighting dengue virus.

The value added in modelling the efficiency of public management lies in the attempt to measure the level of satisfaction of the population regarding the ongoing management. There are also international institutions, such as the International Monetary Fund (IMF) and the World Bank, which publicly express concern about national or subnational managements characterized as inefficient. However, despite the relevance, it seems there is not an extensive literature applied to Brazil, possibly due to the complexity of measuring efficiency (see Tanzi (1974TANZI, Vito. Redistribution of income through the budget in Latin America. Banca Nazionale del Lavoro Quarterly Review, v. 27, n. 108, p. 65-87, 1974.) for a deeper understanding of limitations in conducting efficiency exercises).

In a didactic way, this complexity in measurement is due to the extraction of a wide and representative set of inputs and products and service, since that many variables are latent or not observable in Brazil and because measurement must allow ex post comparison between the objects of study. In the case of public management and social welfare, there is a greater complexity: How can one define a vector of weights for public services that is compatible or equivalent to the latent vector of social priorities? In short, what is the optimal form for products and services to be offered by the state to society?

In this broad context of discussion about efficiency of the public institutions, this article aims to provide the most comprehensive and recent study on the efficiency of the 27 Brazilian states (including the federal district), after the implementation of the Fiscal Responsibility Law (Supplementary Law No. 101 of May 4, 2000).

The study adopts a nonparametric linear mathematical programming technique, called data envelopment analysis (DEA), for a balanced panel with annual data from 2004 to 2013. The study does not use the expenditures by budget items, because these tend to present a homogeneous pattern as the states tend to respect the Fiscal Responsibility Law (FRL), observing the limits and keeping to the base percentage of total revenues in most of the obligatory items. This article makes use of actual monetary values of revenues as input and of classic indicators as a proxy of the service provided by the state, following previous contributions, such as Afonso, Schuknecht and Tanzi (2005AFONSO, António; SCHUKNECHT, Ludger; TANZI, Vito. Public sector efficiency: an international comparison. Public Choice, v. 123, n. 3, p. 321-347, 2005.) and Maciel, Piza and Penoff (2009MACIEL, Vladimir; PIZA, Caio; PENOFF, Roberto. Desigualdades regionais e bem-estar no brasil: quão eficiente tem sido a atividade tributária dos estados para a sociedade? Planejamento e Políticas Públicas, n. 33, p. 291-318, 2009.).

More specifically, in terms of inputs from the process of social welfare generation, the study used five revenue variables, including traditional revenues associated with taxes and transfers, as well as revenues obtained from credit operations contracted with national institutions and via paradiplomacy. As for the social welfare generated, the study uses 10 observable and measurable variables related to health, violence, demography, education, socioeconomic conditions and basic infrastructure.

The first contribution of this article is the use of the broadest and most representative set of variables considered relevant and intuitive, either as inputs or as outputs, taking into account the limitation of data for a broad panel for all Brazilian states). In addition, although the set of variables used here is broader and more representative than those used in previous analogous studies, this study contributes when it compares such data with evidence obtained for ex ante periods of the FRL.

Unlike all other correlated studies, this article considers the relevance of the proactive effort of each Brazilian state to seek alternative funding sources. In addition to the traditional revenues, the states use credits offered by national banks or negotiated between the state and international financial and development agencies.

This main innovation of the article enables the discussion on the need, relevance and efficiency of alternative sources of revenue for Brazilian states. It also allows inferring ex post impacts associated with debt services contracted in local and foreign currency. Moreover, it is possible to deal with the intertemporal issue of the consolidated net debt stock, which today prevents states from honoring their commitments, as well as being part of the conditions in the relationship between the federal and state government, in the renegotiations of the debts of the states.

Methodologically, the study uses the DEA, as suggested by Charnes, Cooper and Rhodes (1978CHARNES, Abraham; COOPER, William; RHODES, Edwardo. Measuring the efficiency of decision making units. European Journal of Operational Research, v. 2, n. 6, p. 429-444, 1978.), but incorporates an indispensable extension: the Malmquist indexes, which allow inferring about the change of productivity of each ‘productive unit’ over time. This model, described in Fare and partners (1994), makes it possible to measure changes over time in terms of technology and in terms of total factor productivity.

Finally, the main purposes of this study are: (i) to infer about the efficiency levels of each state per year; (ii) aggregate this result into a single average and propose a ranking of the entire decade, comparing the results of technical efficiency (total, pure and in scale) with and without the incorporation of credit revenues as inputs; (iii) aggregate via Malmquist index and infer about gains and losses of different types of productivity; (iv) identify fully efficient states that can serve as reference for the inefficient; and (v) infer about patterns of the use of inputs and social welfare services generation. Specifically, in terms of public policies, the purposes are: (i) to analyze whether efficiency may be compromised in the future due to the debts contracted; and (ii) motivate the discussion of meritocracy through efficiency in controversial issues of interests of policy makers and society, such as tax reform.

The article is structured in five sections: following this introduction, the next section presents a brief empiric literature review. The third and four sections present the methodology and the discussion of results, respectively. The fifth and last section presents the conclusion.

2. Empirical literature review

Non-parametric linear mathematical programming techniques consist of tools of wide application on empirical studies in social sciences, with the purpose to measure efficiency and productivity of organizations that can be characterized as units with comparable production process. Gattoufi and partners (2004GATTOUFI, Said; ORAL, Muhittin; REISMAN, Arnold. Data envelopment analysis literature: a bibliography update (1951-2001). Socio-Economic Planning Sciences, v. 38, n. 2, p. 159-229, 2004.) list more than 1,800 published examples using such techniques. It is possible to list their application on a variety of cases from the Brazilian financial market, as presented by Matos and partners (2015MATOS, Paulo; PADILHA, Guilherme; BENEGAS, Mauricio. On the management efficiency of Brazilian stock mutual funds. Operational Research, v. 1, n. 1, p. 1-35, 2015.), to specific areas such as the efficiency in the public system of renal transplant, as presented by Costa and partners (2014COSTA, Cassia; BALBINOTTO NETO, Giácomo; SAMPAIO, Luciano. Eficiência dos estados brasileiros e do Distrito Federal no sistema público de transplante renal: uma análise usando método DEA (análise envoltória de dados) e índice de Malmquist. Cadernos de Saúde Pública, v. 30, n. 8, p. 1667-1679, 2014.).

It is possible to highlight international contributions similar to the object of study proposed here. In Tanzi and Schuknecht (1997TANZI, Vito; SCHUKNECHT, Ludger. Reconsidering the fiscal role of government: the international perspective. The American Economic Review, v. 87, n. 2, p. 164-168, 1997., 2000), the empirical application analyzes 18 developed and industrialized countries, combining public expenditures that should be able to provide a given set of socioeconomic indicators. In Gupta and Verhoeven (2001GUPTA, Sanjeev; VERHOEVEN, Marijn. The efficiency of government expenditure: experiences from Africa. Journal of Policy Modeling, v. 23, n. 4, p. 433-467, 2001.), non-parametric mathematical frameworks are used to model the efficiency of 37 African governments in education and health. Clements (2002CLEMENTS, Benedict. How efficient is education spending in Europe? European Review of Economics and Finance, v. 1, n. 1, p. 3-26, 2002.) measures the efficiency of education in the European Union. Afonso, Schuknecht and Tanzi (2005AFONSO, António; SCHUKNECHT, Ludger; TANZI, Vito. Public sector efficiency: an international comparison. Public Choice, v. 123, n. 3, p. 321-347, 2005.) analyze the efficiency in the public expenditure of countries of the Organization for Economic Cooperation and Development (OECD).

Studies closely related to the empirical exercise presented in this article include the work developed by Machado Júnior, Irffi and Benegas (2011MACHADO JUNIOR, Sáris; IRFFI, Guilherme; BENEGAS, Mauricio. Análise da eficiência técnica dos gastos com educação, saúde e assistência social dos municípios cearenses. Planejamento e Políticas Públicas, n. 36, p. 87-113, 2011.), focusing on the municipalities of Ceará; the work by Schull, Feitósa and Hein (2014SCHULL, Adiulli; FEITÓSA, Camila; HEIN, André. Análise da eficiência dos gastos em segurança pública nos estados brasileiros através da análise envoltória de dados (DEA). Revista Capital Científico - Eletrônica, v. 12, n. 3, p. 1-15, 2014.) about public safety in Brazil; and the work by Benegas and Da Silva (2014)BENEGAS, Mauricio; DA SILVA, Francisco. Estimação da eficiência técnica do SUS nos estados brasileiros na presença de insumos não discricionários. Revista Brasileira de Economia, v. 68, n. 2, p. 171-196, 2014. applying the techniques to measure the efficiency of the national health system (SUS) in the Brazilian states. Finally, the most aligned empirical exercise to this study consists of Maciel, Piza and Penoff (2009MACIEL, Vladimir; PIZA, Caio; PENOFF, Roberto. Desigualdades regionais e bem-estar no brasil: quão eficiente tem sido a atividade tributária dos estados para a sociedade? Planejamento e Políticas Públicas, n. 33, p. 291-318, 2009.), who use data envelopment analysis (DEA) to discuss the efficiencies of Brazilian states from 1990 to 2000, but from a much more restricted set of variables than that proposed in this study.

3. Methodology

This article considers the 27 Brazilian states (including as “state” the federal district) as welfare producers, which are comparable to each other when it comes to inputs and services generated. It is assumed that the states are operating in an optimal scale, suggesting, in this case, the adoption of technical assumption of constant return of scale (CRS).

It is also assumed that the process of welfare generation from a variety of revenues as inputs can be modeled through DEA, a technique that aims, in the simplest way, at establishing what could be the limit of relative efficiency. The use of DEA is adequate, even if there are bad outputs (undesirable services), because these outputs are translated through metrics, which can be normalized and transformed into desirable services, only for the purposes of DEA estimation. As an example, the poverty indicator given by the percentage of poor people is undesirable - in the sense of the greater the number, the worse is poverty. However, it is possible to work using the technique with the percentage of non-poor.

This same normalization is used for all the bad outputs used in this work, so it is possible to use the characteristics of the DEA vis-a-vis the Directional Distance Function (DDF). That is, each state generates only desirable public services and products, based on tax revenues, transfers, or other regular revenues and revenues from national and international credit.

This normalization procedure simplifies the implementation of the concepts associated to the Malmquist index, useful for understanding the evolution of states over time. Formally, the outputs are defined by and the inputs by . Thus, in a Brazilian state indexed by , the representative vector of this modeling will be given by and the set of production possibilities is represented by

(1)

According to Park and Weber (2006PARK, Kang; WEBER, William. A note on efficiency and productivity growth in the Korean banking industry, 1992-2002. Journal of Banking & Finance, v. 30, n. 8, p. 2371-2386, 2006.), it is assumed that, given a feasible vector of services generated from a vector of input, any proportional reduction of both vectors consists of a feasible production set. In this case, the limit of production for a Brazilian state i, modeled as a Decision Making Unit (DMU), can be given by

(2)

In this relationship (2), is a constant of intensity, used for linear combinations of all the 27 states. The problem of maximizing the efficiency of the state subject to the constraints that all efficiencies oscillate between 0 and 1 can be rewritten in an equivalent way, in terms of envelopment, as follows:

(3)

In this enveloping problem that uses fewer constraints than the original form of the problem, λ is a vector Nx1 composed of the intensity constants λi , while the value of θ endogenously measures the inefficiency of the analyzed state, so that a null value would imply a state at its efficiency limit. This problem is solved N times, one for each state. After solving this problem T times, one for each panel period, Fare and partners (1994) specify an index that measures the change in Malmquist productivity between the periods t + 1, which is given by the following relationship, which consists of a geometric mean of the distance functions in t + 1:

(4)

In case of assuming a value greater than 1, there is an increase in productivity over time, and the opposite is true: a value less than 1 implies a fall in productivity. Variations of this Malmquist index, as well as the derivation of the optimization problem, can be seen in Coelli (2008COELLI, Tim. A guide to Deap version 2.1: a data envelopment analysis (computer) program. CEPA Working Paper 96/08, 2008.).

4. Empirical exercise

4.1 Database

4.1.1 Inputs of Brazilian states

The ideal database for an empirical study with the purpose of measuring the efficiency of the social welfare generation process based on the inputs available to Brazilian states should be as broad as possible in the temporal dimension, considering the quantity of inputs and services taken as outputs. The main limitation is the frequency of key economic and social variables, as well as the variables related to public finance of the Brazilian states, which are only available annually.

Regarding the inputs, it is observed that the main revenue sources are taxes and transfers. There are also other revenues such as those gained over assets, socioeconomic contribution, as well as contributions from the economic sectors. Aiming at establishing a more parsimonious framework, the first inputs were grouped as tax revenues, transfer revenues and other revenues.

The main theoretical innovation of this study is the inclusion of items that capture the effort of each state to find other funding sources, except those already listed, associated to credit operations. First, the internal credit contract operations, mostly contracted in local currency, usually obtained from Banco do Brasil, the National Social Development Bank (BNDES), Banco do Nordeste and other public banks. Second, by exploring the paradiplomatic ability of states, there are contractual foreign credit operations, all expressed in foreign currencies (US Dollar, Euro, Special Drawing Right or Japanese Yen), linked to foreign development agencies.

The credit activity of subnational entities does not consist of a recent phenomenon, as observed in Maia and Saraiva (2012MAIA, José; SARAIVA, José. A paradiplomacia financeira no Brasil da República Velha, 1890-1930. Revista Brasileira de Política Internacional, v. 55, n. 1, p. 106-134, 2012.), who presented and detailed operations with international organizations developed during the time known as República Velha in Brazil, i.e., more than one century ago. However, in recent years there has been a more frequent experience of obtaining alternative sources from national public and private banks, as well as from paradiplomatic experience, both in developed countries and in emerging economies, which has attracted the attention of scientific literature.

The practice of obtaining national credit is more common and well known, often associated with specific social programs or linked to Public-Private Partnerships (PPP). Paradiplomacy, on the other hand, seems to be less common, due to (among other things) the confusion of the use of this term. Considering the diversity of opinions and the different use of terms aimed at capturing the meaning of the paradiplomatic phenomenon, the word “paradiplomacy” was not the first to be used to designate the international activities of subnational entities. As emphasized in Matos and Bessa (2014MATOS, Paulo; BESSA, Ricardo. Determinantes da captação de recursos junto a órgãos internacionais pelo estado do Ceará. Revista de Economia do Nordeste, v. 45, n. 1, p. 6-18, 2014.) the term “microdiplomacy” was used first, in Paquin (2004PAQUIN, Stéphane. La paradiplomatie identitaire: le Québec, la Catalogne et la Flandre en relations internationales. Politique et Sociétés, Quebec, v. 23, n. 2-3, p. 203-237, 2004.) and Farias (2000FARIAS, Deborah. Federalismo e relações internacionais. Dissertação (mestrado em relações internacionais) - Departamento de Relações Internacionais, Universidade de Brasília, Brasília, 2000.). This expression was brought to the academic debate by Soldatos (1990SOLDATOS, Panayotis. An explanatory framework for the study of federated states as foreign-policy actors. In: MICHELMANN, Hans J.; SOLDATOS, Panayotis (Ed.). Federalism and international relations: the role of subnational Units. Oxford: Clarendon Press, 1990. p. 34-53.) with the intention of designating the diplomatic activity developed between non-central political entities located in different states.

As a practical and recent example, the state of Ceará, whose paradiplomacy is the object of study in the work of Matos and Bessa (2014MATOS, Paulo; BESSA, Ricardo. Determinantes da captação de recursos junto a órgãos internacionais pelo estado do Ceará. Revista de Economia do Nordeste, v. 45, n. 1, p. 6-18, 2014.), contracted at the end of 2013 a credit operation with the World Bank of US $ 350 million, having a focus on results, in order to finance the Project to Support Economic Growth in Reducing Inequalities and Environmental Sustainability of Ceará — Program for Results (PforR Ceará).

All five series of revenues were collected from the National Treasury Secretariat (STN). The amounts were originally expressed in the national currency (Real R$), with the exception of the series regarding international credit, which required conversion into the local currency. . The conversion is based on the official exchange rate, provided by the Central Bank of Brazil, quoted on the day the transaction was approved. All of these series originally nominal were brought to the future value, based on the month of December 2013 (the last of the sample), applying the Brazilian consumer price Index IPCA. Finally, in order to avoid distortions associated with the different size of the states, all revenues are calculated in per capita terms, based on the annual population provided by the Brazilian Institute of Geography and Statistics (IBGE).

The real and per capita series of these five revenue variables are the process inputs, whose average values per state in each region are reported in table 1.

The values presented in table 1 suggests that the items ‘other revenues’ and ‘credit’ are not negligible compared to traditional revenues. As for the proportion of each item in the total revenue, the states in the South and Southeast showed more than 60% of the total revenue coming from tax collection, while in the North, with the exception of the state of Amazonas, this participation ranged between 18% and 45%. Except in the case of Rio de Janeiro, the share of ‘other revenues’ in the states’ total revenue ranged from 14% to 28%.

Although it is not considered as an item to form the total revenues according to the public accounting that guides the preparation of the balance sheet of Brazilian states, the relationship between national credit and total revenue portrays an interesting picture. The states in the North region stood out with the highest average percentages, 5%, while in the other regions this level did not exceed 3%, except in Goiás (state in Central-West region). Regarding the relative volume associated with paradiplomatic activity of the states, the majority ranged between 0% and 2%. It is important to highlight the states of Ceará and Tocantins, which presented volumes over 4% of their total revenues related with paradiplomatic activity.

Table 1
Annual average amount of the considered inputs (period: 2004-13)

4.1.2 Bad outputs of Brazilian states

The choice of measurable variables during the period 2004-13 for all states implies a dilemma, since many variables that are a priori intuitive and interesting, do not satisfy this criterion because they are not accessible. Thus, the variables associated with environmental, institutional, cultural, citizenship, communication and urbanism/transport issues are not considered. Observing different sources of data, the representative base of bad outputs is composed of social, safety, health and unemployment variables that can be considered of extreme relevance, whose average values are shown in table 2.

The first indicator is the percentage of people living in poverty, characterized as a condition of insufficient income to allow survival under decent conditions. As reported in other studies, it is possible to observe that the North and Northeast regions presented a higher percentage of people in poverty when compared to other regions. The state with the lowest poverty rate in the North and Northeast regions presented almost twice the number of individuals in conditions unsuitable for survival than in the state with the highest poverty rate in the other regions.

The degree of income inequality measured by the Gini Index assumes values between 0 (perfect equality in income distribution) and 1 (maximum inequality). Gini index presented little variation among the states and indicated a high concentration of income in the whole country in the period analyzed. A commonly reported pattern in the literature is that poorer regions or states are also more unequal, which was also observed in this study, except for the Distrito Federal, which had the lowest poverty indicator and the highest inequality indicator.

As for unemployment, there is heterogeneity even within the regions themselves. In the case of the Northeast, the state of Piauí, for instance, presented 5.94% unemployment rate whereas the neighbor state of Pernambuco showed 11.33%. In the South, the unemployment rate was more homogeneous with less than 6.10%.

Table 2
Annual average value of the bad outputs generated (period: 2004-13)

Regarding health, the choice of using the indicator ‘child mortality per 1,000 live births’ provided by the Brazilian Institute of Geography and Statistics (IBGE) is based on Afonso, Schuknecht and Tanzi (2005AFONSO, António; SCHUKNECHT, Ludger; TANZI, Vito. Public sector efficiency: an international comparison. Public Choice, v. 123, n. 3, p. 321-347, 2005.), who suggest that this information is minimally representative of the complex panorama of health as a whole. While the disparity in average terms was three times when it comes to income, or even four times in the case of credit, in health — based on child mortality per 1,000 live births — it was observed that mortality ranged from 22 to almost 35 deaths per 1,000 live births in the Northeast, while in the South this figure was 13 at most. In average terms, there was a ratio of child mortality approximately twice as high in the Northeast as in the South.

In terms of security, by the asymmetry of standards and information among states’ public safety secretariats, the information homogeneously collected and made available in Datasus on deaths from external causes per thousand inhabitants is a standard proxy for capturing the wide and diffuse sense of safety in society (Jorge et al., 2007JORGE, Maria; KOIZUMI, Maria; TONO, Vanessa. Causas externas: o que são, como afetam o setor saúde, sua medida e alguns subsídios para a sua prevenção. Revista Saúde, v. 1, n. 1, p. 37-47, 2007.).

Here, there is a very different pattern from the other indicators analyzed. The Northeast and North no longer appeared as the regions with the worst indicators. The Central-West region presented the worst indicators, followed by states from the Southeast region. The figures close to one death per thousand inhabitants observed in the average in Mato Grosso and Espírito Santo were almost double the figures observed in states of the North and Northeast regions.

4.1.3 Good outputs of Brazilian states

Table 3 shows the average value of good outputs provided by the state such as relevant variables on macroeconomics, demographics, human capital and infrastructure.

Regarding education, the inequality was constant among the regions. Regions North and Northeast presented greater amplitude and lower averages when it comes to the total of years of schooling. The state with the lowest average years of schooling in the population was Alagoas, with only 4.96 years of schooling, almost half of the 9.46 average observed in Distrito Federal.

Table 3
Average value of good outputs generated (period: 2004-2013)

The variable ‘annual income per capita’ presented strong inequality, evidence quite common in the literature. The highest average amounts of income were in Distrito Federal, with over R$ 21,000, followed by São Paulo and Santa Catarina, with amounts close to R$ 14,000. At the other end, with lower average incomes, were the states of Maranhão and Alagoas in the Northeast, with values close to R$ 5,500. This context suggests a reflection: if human and financial capital are fundamental to income convergence and subsequent reduction of inequality between the rich and the poor, the strong heterogeneity of all variables, such as credit and education, when comparing Northeast and North versus South and Southeast, suggests that these capital vectors can be increasing divergences, rather than reducing them.

Regarding the demographic variable ‘life expectancy at birth’, it reflects in a single metric the influences of several others variables of social and economic (among others) dimensions. This variable is an unusual indicator of social welfare, but relevant in this analysis. Again, the regional disparity was evidenced, as shown in table 3. It was clear that, considering the North and Northeast regions, only Rio Grande do Norte had expectations comparable to the expectations of the states in the other regions. In terms of the average, the expectation in these two most unassisted regions was approximately 70.5 years, much lower than the average in the South, for example, 75.5 years old.

Taking into account the realities of the societies in each state in terms of social infrastructure (housing, represented by the indicator ‘number of private households per capita’ or living conditions, expressed by the indicator ‘percentage of households with adequate sanitation’), the disparity among the regions was clear. For instance, the state with fewer private households per capita in the Central-West, South and Southeast regions was Mato Grosso, with 0.324. This number was equivalent to that registered in Rondônia and Sergipe, states that presented the highest number of private households per capita in the other regions.

This situation was exacerbated by the fact that these houses in the North, Northeast and also Central-West regions, with exceptions of Distrito Federal, Roraima and Sergipe, were characterized by a lack of adequate sanitation in very high percentages. While in São Paulo and the Distrito Federal, more than 90% of the homes had adequate sanitation, in Tocantins and Alagoas, just over 30% had this basic condition of decent housing. On average, in the most deprived areas, less than 55% of the homes had sewage, whereas in the South and Southeast the average values exceeded 80%.

4.3 Results of efficiency disaggregated over time

In this subsection, the results are presented in order to meet the first main objective of this work: to infer about the efficiency levels of each state per year. Starting in the North region (graph 1), Acre, Rondônia and Tocantins presented seemingly common cyclic behaviors whose values were among the lowest observed for this region. Amazonas was another state with high levels of relative inefficiency, the only one in the region not to achieve full efficiency in any of the 10 years analyzed. The average efficiency of these states ranged from 76.46% to 79.08%.

Graph 1
Development of technical efficiency of states in Brazil’s North region (period: 2004-13)

The Northeast region had the most technically 100% efficient states in each of the 10 analyzed years: Bahia, Ceará and Maranhão. States such as Alagoas, Paraíba and Piauí had correlated oscillations characterized by inefficiencies higher than 10% between 2007 and 2010 and, consequently, full and robust efficiency in the years following (2011-2013). With a very idiosyncratic pattern, the state of Pernambuco had an average efficiency of 92.49%, higher than Rio Grande do Norte and Sergipe, whose inefficiencies above 25% in 2008 led to average efficiencies of around 80%, as shown in graph 2.

According to graph 3, for the Central-West region the analysis of efficiency in the early years of the sample suggests high levels of inefficiency, ranging from 13% to 38%, for all states, except for Goiás that presented a more homogeneous and higher efficiency (ranging from 95% to 100%) over the years. The average efficiency of Goiás was 98.61%, while in Distrito Federal it was 87.77%, and in the other states, it was lower than 80%.

Graph 2
Development of technical efficiency of states in Brazil’s Northeast region (period: 2004-13)

Graph 3
Development of technical efficiency of states in Brazil’s Central-West region (period: 2004-13)

In the Southeast, except for the state of Espírito Santo that presented a behavior characterized by strong inefficiency as of 2005, the other states were 100% efficient. In the case of São Paulo and Minas Gerais, 100% efficient at all times, and in the case of Rio de Janeiro during the latter years, as shown in graph 4.

Graph 4
Development of technical efficiency of states in Brazil’s Southeast region (period: 2004-13)

The South region was considerably more efficient in aggregated terms when compared to others. In addition, the region was more homogeneous not only over time but also between the states. Graph 5shows that the only state 100% efficient was Paraná. However, Rio Grande do Sul and Santa Catarina presented average inefficiency close to 1%, which put this region as almost 100% efficient both in the dimension of time and cross-section.

Graph 5
Development of technical efficiency of states in Brazil’s South region (period: 2004-13)

4.3 Results of efficiency aggregated over time

Table 4 shows the average total efficiency in the second column. The following columns show the decompositions into pure technical efficiency and scale efficiency.

Regarding the second objective of this article, in order to provide better visualization and stratification, the states were segmented and characterized in groups according to their efficiency. The states were categorized according to their efficiency throughout the decade (seven in total): the least inefficient (efficiency superior to 90% on average); the moderately efficient (efficiency ranges from 80% to 90%); and the very inefficient (efficiency level is less than 80%).

Compared to the results obtained for the decade from 1991 to 2000 reported by Maciel, Piza and Penoff (2009MACIEL, Vladimir; PIZA, Caio; PENOFF, Roberto. Desigualdades regionais e bem-estar no brasil: quão eficiente tem sido a atividade tributária dos estados para a sociedade? Planejamento e Políticas Públicas, n. 33, p. 291-318, 2009.), when there were six states with full efficiency in 1991 and five in 2000, seven states with full efficiency were observed throughout the period from 2004 to 2013. There was no state shown to be efficient in both periods compared here. The efficiencies oscillate between very similar amplitudes, with the most inefficient reaching maximum inefficiency of approximately 25%.

The elaborated ranking also allows comparing the efficiency of the states over these two decades, and it is possible to identify a considerable change before and after the Fiscal Responsibility Law. It is important to emphasize that these differences are also due to the sets of inputs and outputs used in both studies, which are very different. In Maciel, Piza and Penoff (2009MACIEL, Vladimir; PIZA, Caio; PENOFF, Roberto. Desigualdades regionais e bem-estar no brasil: quão eficiente tem sido a atividade tributária dos estados para a sociedade? Planejamento e Políticas Públicas, n. 33, p. 291-318, 2009.), two inputs and two outputs were used, whereas in this work there were five inputs and 10 outputs.

In this set of variables, one of the innovations proposed is the incorporation of revenues associated with the proactivity of state governments in seeking external funding through credit operations contracted with national or foreign financial agencies. In order to measure whether the effect of this additional source as input in the DEA was a differential for the results, the differentials in ranking and aggregate technical efficiency are shown in columns 6 and 7 of table 4. The table considers the reported result (using credit as input) and the unreported result that is based only on the three traditional inputs in terms of revenue sources.

Initially, there was no reduction of efficiency when considering the revenues of national or international credit operations in any of the states, with an average efficiency gain of 3.61%. Clearly, the most efficient states with credit remain the most efficient states without credit as input. In the results considering credit, only Bahia and Minas Gerais came to full efficiency, making up the group previously formed by only five states when credit was not considered.

Changes in the ranking when considering credits were more intense for the less efficient states. In this new context of analysis, Bahia, Minas Gerais, Rio Grande do Sul, Alagoas, Sergipe, Acre and Rondônia got a better position in the ranking. States with downward arrows in red, such as Rio de Janeiro and Piauí, despite the efficiency gains, had — in relative terms — worsened in the ranking that takes into account the input ‘credit’ in the data envelopment analysis.

Regarding the third objective of this article, columns 8 to 12 of table 3 show the average productivity indexes over the decade 2004-13 (value higher than 1 suggests productivity gain). It is possible to show that, unlike the results for the decade from 1991 to 2000 (Maciel, Piza and Penoff, 2009MACIEL, Vladimir; PIZA, Caio; PENOFF, Roberto. Desigualdades regionais e bem-estar no brasil: quão eficiente tem sido a atividade tributária dos estados para a sociedade? Planejamento e Políticas Públicas, n. 33, p. 291-318, 2009.), in which productivity was gained for all states, this article identified that states such as Santa Catarina, Goiás, Pernambuco, Amazonas and Espírito Santo presented loss of productivity in terms of total technical efficiency, even in small percentages, ranging from 0.1% in Pernambuco to 1.4% in Espírito Santo, considering the whole decade 2004-13.

The fully efficient states did not show an evolution in productivity and the others had a gain ranging from 0.3% in Piauí and Rio de Janeiro to 4.3% in Acre. These values, except for the states characterized as very inefficient, are all associated with efficiency gain via scale and not via pure technical efficiency.

Although it is important to note that five states presented loss of productivity, even if small, in terms of technical efficiency, it is more important to pay attention in the average loss of total productivity of the factors in approximately 7%, and the average productivity loss associated with the technology of almost 8%. Those are high values for a decade, especially when it is perceived that they were generalized, since only Acre and Mato Grosso have gained total productivity of the factors and no state has gained in technology.

Table 4
Aggregate results over time (period: 2004-13)

Extreme examples were Maranhão where it was observed a total productivity loss of almost 18% and Rio Grande do Norte, which lost 16.5% in terms of technology.

Taking into account the fourth main objective of the article, table 4identifies the fully efficient states considered the benchmarks for the inefficient ones, and the peer states able to serve as a punctual example. In order to summarize this result for the decade, these results were obtained from a single stage DEA having as inputs and outputs the average values from 2004 to 2013.

Considering the fifth main objective, what can be learned from the most efficient states in Brazil in the decade of 2004 to 2013? Is there something to learn from states classified as very inefficient during that decade? What are the patterns?

A first lesson learned is on the intense use of inputs as something quite punitive in the efficiency, as it could be observed in Distrito Federal, despite the good relative performance in terms of social welfare. In a similar trend of thought, with the exception of the state of Pará and Amazonas, the others in the North region may have their inefficiency associated with the high levels of real transfer per capita, to the point that Rondônia presented a level of about 10 times the one observed in Rio de Janeiro.

Focusing on the good examples, what has possibly led the state of Maranhão to a maximum level of efficiency, despite the well-known low social indicators in comparison to other states in the country, was the very low use of inputs, mainly in the item of taxes and other sources of revenue. The revenues of the other very efficient states of the Northeast, Bahia and Ceará, as well as Pará, can be synthesized by the average use of inputs and average generation of outputs (good or bad), i.e., a fairly balanced recipe.

Taking into account the figures presented in the research, the states of Paraná and Minas Gerais used minimally comparable revenues in order to provide better indicators of social and economic variables than most of the efficient states, but at the cost of a more intense use of revenues. Finally, the incomparable state of São Paulo made intense use of resources obtained through taxes and it was not one of the most favored with resources coming from federal government transfers. With this balance, the state generated excellent levels of good outputs and high levels of bad outputs, with the exception of unemployment.

A secondary result is the correlation between state efficiencies and the levels of each input used or output generated. According to the data reported in table 5, the only exceptions in terms of the expected results were poverty as a bad output and the years of study as good output. In terms of inputs, higher levels of technical efficiency in the years 2004 to 2013 were linearly associated with the less intense use of revenues obtained through taxes, transfers and other sources, in addition to sources through national credit and via paradiplomacy.

Table 5
Correlation between efficiency and inputs and outputs (period: 2004-13)

4.4 Discussion on the role of external funding sources

A final result, in view of the first specific objective, is the sustainability of efficiency levels given the levels of indebtedness. The Fiscal Responsibility Law (Complementary Law No. 101, of May 4, 2000) aims at establishing nationwide guidelines regarding public expenditure of each state. The law imposes budgetary restrictions that preserve the fiscal situation of the states, according to their annual balance sheets, maintaining the financial health of the states over time. This law establishes lower and higher limits to the most relevant budget items, as well as limiting states’ indebtedness.

In this context, an analysis of the debt service reported in the annual balance sheets of the states (available from the Secretariat of National Treasury), as well as an analysis of the consolidated net debt (CND) in relation to the total revenue levels, may be useful in order to indicate whether the states were or were not promoting efficiency in the analyzed decade at the cost of excessive indebtedness, which would compromise the continuity of this desired efficiency.

Table 6 reports some relevant figures. Observing the correlation between average total technical efficiency in the decade and each of the items listed, it is possible to identify that the services of internal and external debt correlated positively, but with low values (0.094 and 0.054 respectively). This suggests that the debt services during the analyzed decade did not seem to compromise efficiency so much — evidence corroborated by the order of magnitude, lower than the level of 6% for most states, with concerning exceptions in the states of Alagoas, Goiás and Mato Grosso.

The correlation in the cross section between efficiency and the CND/Total revenue ratio, in the value of 0.460, may be pointing out that some states were quite dependent on indebtedness to be able to show — at least temporarily — their efficiency in the provision of social welfare, based on the output variables used here.

Table 6
Relationship between efficiency and debt (period: 2004-13)

This may be the case for São Paulo, Minas Gerais, Rio Grande do Sul, Rio de Janeiro, and Alagoas, whose ratios of average real consolidated net debt to average real total revenue in December 2013 exceeded the threshold of 100%. Even more concerning is the case of Mato Grosso do Sul, whose ratio is 84%, without the desirable counterpart of high efficiency, with the inefficiency of 21.46% being one of the highest in Brazil. Graph 6reports the dispersion between the series in the cross section of efficiency and CND/Total revenue ratio.

Graph 6
Cross section dispersion of Brazilian states on efficiency and CND/Total revenue ratio (period: 2004-13)

Corroborating the evidence associated with the correlation of 0.460, a comfortable scenario was observed for the states that managed to use the resources obtained from indebtedness in order to promote efficiency, while others in the lower right quadrant associated high levels of indebtedness with low efficiency. It should be noted that this indebtedness measured by the CND/Total revenue ratio was not solely due to the credit operations contracted in the intersection analyzed here, from 2004 to 2013, and there were remnants of older operations.

In any case, considering the entire history of credit operations of the states, it has been shown that the debt service payment did not yet seem to be compromising efficiency, while the debt stock had already indicated that it was high enough to jeopardize current and future efficiency of some of the states.

4.5 Discussion on tax collection, efficiency and tax reform

Regarding the second specific objective of this article, there is an ongoing discussion started perhaps around 2003 about tax reform. Common with most of the various proposals to change Brazilian tax legislation is the unification of the Tax on Circulation of Goods and Services (ICMS) for all states and the unification of taxes such as the Contribution to the Financing of Social Security (Cofins) and the Social Integration Program (PIS).

Other elements of this discussion around the reform are certainly secondary, since this revenue is an important part of the financial budget of all states, being the highest tax collected nationwide. The key question is how to compensate for the states impaired by a change in the tax law, and the poorer states, which adopt policies of tax reduction as their main tool to promote local development. There are numerous compensatory ideas, such as the creation of a new tax on services, or the creation of a Regional Development Fund that would allow states to continue with budgetary and financial resources to support investments.

The clear impression on the discussion around the measures after tax reform is to compensate each of the impaired states, so that they return to a level close to previous revenue status, before the reform, as if the current situation was fair and efficient management was being employed.

The present article allows to point out a scenario of imbalances of efficiency and possible sources of mismanagement, as well as it shows states that serve as examples for others, an instrument that should be fundamental in trying to conciliate the interests of states. More precisely, tax reform cannot ignore the conclusions drawn from the analysis of linear input-orientated programming reported, and it is necessary to promote this debate by observing inputs in terms of sources of revenue.

As discussed previously in subsection 4.1 on the patterns of inputs of Brazilian states, it is clear that states in the South and Southeast regions had more than 60% of their total revenue from tax collection, while in the North (except for the state of Amazonas) tax revenues oscillated between 18% and 45%. In the North region the percentage was 31% on average, half of the average 62% obtained for the South and Southeast. The Northeast and Central-West regions showed average percentages of 41% and 57%, respectively.

However, the relationship between national credit and total revenue suggests an interesting scenario, where the states in the North stood out with the highest percentages, with national credit volume averaging 5% of total revenue, while in other regions this level did not exceeded 3% (except the state of Goiás, in the Central-West region). As for the relative volume associated with the states’ paradiplomatic activity, the majority presented figures between 0% and 2%. It is worth highlighting the states Ceará and Tocantins, whose volumes contracted with foreign institutions exceeded 4% of their total revenues.

With such a disparity in the revenue framework, the fiscal challenge for the states is observed not only by the adoption of exaggerated practices in the competition among each other in order to attract private investments, disregarding the National Council of Finance Policy (Confaz). This challenge is also observed in the efforts of each state government to obtain debt relief (partial or total cancellation) from the federal government, as well as the efforts to at least maintain the level of tax collection through the state tax ICMS or compensation.

In addition to the fact that these measures seem to be unfruitful in fighting Brazilian social inequality and actually jeopardize the country’s fiscal sustainability — if the federal government does not have primary revenue to contribute to such a Regional Development Fund, the government may offer debt issues as it has done with public banks. It is estimated an amount of approximately R$300 billion over 20 years — these measures simply ignore any meritocracy in the management of states. This may give signs that the mismanagement of public resources and consequent indebtedness, even at unsustainable levels as is the case of the five states whose ratio of consolidated net debt/total revenue exceeded 100%, are not considered by the federal government in the future as an instrument of negotiation, punishment or incentive.

In sum, perhaps the most concerning issue regarding inequality is not income, credit, or human capital, all of which are already observed in Brazil and commonly mentioned by policy makers. The most concerning issue is the per capita tax revenue and the consequent heterogeneity of states management efficiency. Even more concerning, these serious heterogeneities of revenue and efficiency do not seem to worry the federal government at all. More complex still: the summary of these discrepancies translated into the social welfare provided to citizens in each state is probably not being analyzed and used for some purpose, since there are no reliable historical series with consolidated methodology on the satisfaction of the society with certain services provided by the state.

5. Conclusion

The main concern of this study is the measurement of the relative efficiencies of the Brazilian states, analyzing the evolution over the decade after the Fiscal Responsibility Law came into action, and proposing comparisons over time and between states. However, in terms of the implementation of public policies, the aim is to promote the discussion in the context of tax reform and debt renegotiation. By showing a scenario of efficiency imbalance and possible sources of mismanagement and by identifying states that serve as examples for others, the evidence reported here can serve as a fundamental tool in attempting to conciliate the interests of states.

point is that despite the relevance that society and international institutions attribute to the efficiency in resource management and the consequent generation of social welfare by public institutions, the clear impression is that the discussion on measures after the tax reform analyzed here is focused on compensating each impaired state. The states then return to a level close to the previous revenue status, ex ante reform, as if the current situation was fair and efficient management was being employed.

It is suggested that the tax reform should not ignore the conclusions reported here and other evidence reported in the literature on the efficiency of the public institutions in Brazil. The political management has so far ignored any meritocracy regarding the states management, indicating that the poor administration of public resources and the consequent indebtedness, even at unsustainable levels, are not considered by the federal government in the future as an instrument of negotiation, punishment or incentive.

Finally, this issue gains crucial relevance because of the evidence that some states are strictly dependent of indebtedness so that the level of efficiency in the provision of social welfare can be temporarily maintained, which is an unsustainable scenario. In this sense, refusing to include meritocracy associated to the efficiency can jeopardize the quality of the reform. This can discourage the states that respected the FRL and committed to the sustainable and efficient use of both traditional resources and those resulted from states proactivity in searching other funding sources.

Referências

  • AFONSO, António; SCHUKNECHT, Ludger; TANZI, Vito. Public sector efficiency: an international comparison. Public Choice, v. 123, n. 3, p. 321-347, 2005.
  • BANKER, Rajiv; CHARNES, Abraham; COOPER, William. Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science, v. 30, n. 9, p. 1078-1092, 1984.
  • BENEGAS, Mauricio; DA SILVA, Francisco. Estimação da eficiência técnica do SUS nos estados brasileiros na presença de insumos não discricionários. Revista Brasileira de Economia, v. 68, n. 2, p. 171-196, 2014.
  • CHARNES, Abraham; COOPER, William; RHODES, Edwardo. Measuring the efficiency of decision making units. European Journal of Operational Research, v. 2, n. 6, p. 429-444, 1978.
  • CLEMENTS, Benedict. How efficient is education spending in Europe? European Review of Economics and Finance, v. 1, n. 1, p. 3-26, 2002.
  • COELLI, Tim. A guide to Deap version 2.1: a data envelopment analysis (computer) program. CEPA Working Paper 96/08, 2008.
  • COSTA, Cassia; BALBINOTTO NETO, Giácomo; SAMPAIO, Luciano. Eficiência dos estados brasileiros e do Distrito Federal no sistema público de transplante renal: uma análise usando método DEA (análise envoltória de dados) e índice de Malmquist. Cadernos de Saúde Pública, v. 30, n. 8, p. 1667-1679, 2014.
  • FARE, Rolf; GROSSKOPF, Shawna; LOVELL, Knox. Production frontiers Cambridge: University Press, 1994.
  • FARIAS, Deborah. Federalismo e relações internacionais. Dissertação (mestrado em relações internacionais) - Departamento de Relações Internacionais, Universidade de Brasília, Brasília, 2000.
  • GATTOUFI, Said; ORAL, Muhittin; REISMAN, Arnold. Data envelopment analysis literature: a bibliography update (1951-2001). Socio-Economic Planning Sciences, v. 38, n. 2, p. 159-229, 2004.
  • GUPTA, Sanjeev; VERHOEVEN, Marijn. The efficiency of government expenditure: experiences from Africa. Journal of Policy Modeling, v. 23, n. 4, p. 433-467, 2001.
  • JORGE, Maria; KOIZUMI, Maria; TONO, Vanessa. Causas externas: o que são, como afetam o setor saúde, sua medida e alguns subsídios para a sua prevenção. Revista Saúde, v. 1, n. 1, p. 37-47, 2007.
  • MACHADO JUNIOR, Sáris; IRFFI, Guilherme; BENEGAS, Mauricio. Análise da eficiência técnica dos gastos com educação, saúde e assistência social dos municípios cearenses. Planejamento e Políticas Públicas, n. 36, p. 87-113, 2011.
  • MACIEL, Vladimir; PIZA, Caio; PENOFF, Roberto. Desigualdades regionais e bem-estar no brasil: quão eficiente tem sido a atividade tributária dos estados para a sociedade? Planejamento e Políticas Públicas, n. 33, p. 291-318, 2009.
  • MAIA, José. A paradiplomacia financeira dos estados brasileiros: evolução, fatores determinantes, impactos e perspectivas. Tese (doutorado) - Programa de Doutorado em Relações Internacionais, Universidade de Brasília, Brasília, 2012.
  • MAIA, José; SARAIVA, José. A paradiplomacia financeira no Brasil da República Velha, 1890-1930. Revista Brasileira de Política Internacional, v. 55, n. 1, p. 106-134, 2012.
  • MATOS, Paulo; BESSA, Ricardo. Determinantes da captação de recursos junto a órgãos internacionais pelo estado do Ceará. Revista de Economia do Nordeste, v. 45, n. 1, p. 6-18, 2014.
  • MATOS, Paulo; PADILHA, Guilherme; BENEGAS, Mauricio. On the management efficiency of Brazilian stock mutual funds. Operational Research, v. 1, n. 1, p. 1-35, 2015.
  • PAQUIN, Stéphane. La paradiplomatie identitaire: le Québec, la Catalogne et la Flandre en relations internationales. Politique et Sociétés, Quebec, v. 23, n. 2-3, p. 203-237, 2004.
  • PARK, Kang; WEBER, William. A note on efficiency and productivity growth in the Korean banking industry, 1992-2002. Journal of Banking & Finance, v. 30, n. 8, p. 2371-2386, 2006.
  • SCHULL, Adiulli; FEITÓSA, Camila; HEIN, André. Análise da eficiência dos gastos em segurança pública nos estados brasileiros através da análise envoltória de dados (DEA). Revista Capital Científico - Eletrônica, v. 12, n. 3, p. 1-15, 2014.
  • SOLDATOS, Panayotis. An explanatory framework for the study of federated states as foreign-policy actors. In: MICHELMANN, Hans J.; SOLDATOS, Panayotis (Ed.). Federalism and international relations: the role of subnational Units. Oxford: Clarendon Press, 1990. p. 34-53.
  • TANZI, Vito. Redistribution of income through the budget in Latin America. Banca Nazionale del Lavoro Quarterly Review, v. 27, n. 108, p. 65-87, 1974.
  • TANZI, Vito; SCHUKNECHT, Ludger. Public spending in the 20th century, a global perspective. Cambridge: Cambridge University Press, 2000.
  • TANZI, Vito; SCHUKNECHT, Ludger. Reconsidering the fiscal role of government: the international perspective. The American Economic Review, v. 87, n. 2, p. 164-168, 1997.
  • The author thanks CNPq (Scholarship PQ) and the National Treasure Secretariat. This article was awarded at the XX Prêmio STN de Monografias in 2015.

Publication Dates

  • Publication in this collection
    Jul-Aug 2017

History

  • Received
    19 Apr 2016
  • Accepted
    20 Sept 2016
Fundação Getulio Vargas Fundaçãoo Getulio Vargas, Rua Jornalista Orlando Dantas, 30, CEP: 22231-010 / Rio de Janeiro-RJ Brasil, Tel.: +55 (21) 3083-2731 - Rio de Janeiro - RJ - Brazil
E-mail: rap@fgv.br