The Evolution of the Benchmarking Methodology Data Envelopment Analysis - DEA in the Cost Regulation of the Brazilian Electric Power Transmission Sector: a critical look at the renewal of concessions

A Evolução da Metodologia de Benchmarking Data Envelopment Analysis – DEA na Regulação de Custos do Setor de Transmissão de Energia Elétrica Brasileiro: um olhar crítico na renovação das concessões

Abstract:

In 2012, the Federal Government announced a reduction of around 18% of the electric energy tariff going to the consumer. It was up to the National Electric Energy Agency (ANEEL) to create a model that led to the reduction projected by the authorities. To reach this level, the Ministry of Mines and Energy proposed that the electric power transmission concessions be renewed in advance, according to new rules (MP 579/2012 and Law 12.783/2013). An important part of the new rules was the calculation of efficient operational cost through Data Envelopment Analysis - DEA, a methodology already used by ANEEL in two tariff reviews. The objective of this article was to study the evolution of the efficient operational cost calculation models adopted at these times. This analysis showed that ANEEL still needs to promote improvements in the modeling to better represent the operational reality of the concessionaires and eradicate the need for adjustments that reduce the reliability and transparency of the process.

Keywords:
Tariff regulation; Benchmarking; DEA; Efficient costs; ANEEL; Power transmission

Resumo:

Em 2012 o Governo Federal anunciou uma redução em torno de 18% da tarifa de energia elétrica partindo para o consumidor. Coube à Agência Nacional de Energia Elétrica (ANEEL) criar um modelo que levasse à redução projetada pelas autoridades. Para atingir esse patamar o Ministério das Minas e Energia propôs que as concessões de transmissão de energia elétrica fossem renovadas antecipadamente, segundo novas regras (MP 579/2012 e Lei 12.783/2013). Parte importante das novas regras foi o cálculo do custo operacional eficiente por meio de Data Envelopment Analysis – DEA, metodologia já utilizada pela ANEEL em duas revisões tarifárias. Este trabalho teve como objetivo estudar a evolução dos modelos de cálculo do custo operacional eficiente adotados nessas ocasiões. Tal análise evidenciou que a ANEEL ainda precisa promover melhorias na modelagem para melhor representar a realidade operacional das concessionárias e erradicar a necessidade de ajustes que diminuem a confiabilidade e transparência do processo.

Palavras-chave:
Regulação tarifária; Benchmarking; DEA; Custos eficientes; ANEEL; Transmissão de energia

1 Introduction

The Brazilian electricity sector is divided into generation, transmission and distribution of energy. Transmission is the connection between generation and distribution of energy to consumers. Presently, Brazil has eight large electric power transmission companies, and with an estimated growth in energy demand around 4.5% p.a., new investments in transmission lines are required. According to Acende Brasil Institute (IAB, 2015), in the last 10 years the transmission network has been expanded by 45%, but the Ten-Year Energy Expansion Plan (2013-2023) also foresees that the Basic Network will have an extension of more than 182,000 km. Today, it is around 125,000 km, with a 56% expansion in their processing capacity.

Economic regulation has the objective of replicating the effects of competition to the monopolist company in a way similar to what happens in the competitive markets (Beesley & Littlechild, 1989). The National Electricity Agency (ANEEL) is an autarchy linked to the Ministry of Mines and Energy and, according to information gathered on its website, its mission is to provide favorable conditions for the electric energy market to develop with a balance between agents and for the benefit of society. Seeking tariff moderation and the economic and financial balance of the companies, ANEEL conducts periodic tariff reviews every four years, at which time the tariff or revenue can be changed upwards or downwards according to the performance conditions of the concessionaires (Savoia & Moreira, 2012).

When the time comes for the tariff review, the annual revenue allowed for the companies is defined and, in most cases, this occurs with some change in the calculation methodology or in the modeling items. This way, the regulation allows the transfer of efficiency gains from the concessionaires to society, following its duty to offer the best service at the lowest price (NT # 182/2007 - SRE/ANEEL) (ANEEL, 2007b).

Two components are essential for the calculation of the concessionaries’ tariff of transmission: the operational costs, represented by the maintenance and operation of the assets of the companies necessary for the execution of the service, and the remuneration of the assets necessary to provide the service with the required quality, ensuring the sustainability of the business. After the tariff repositioning, the annual revenue of the concessionaires is recalculated, and it can also be considered as the required revenue compatible with covering the efficient operational costs and with an adequate return on invested capital (NT 116/2007 - SRE/ANEEL) (ANEEL, 2007a). This model was in force until the introduction of MP (A Provisional Measure is a legal act in Brazil through which the President of Brazil can enact laws without approval by the National Congress) 579/2012 (Brasil, 2012), transformed into Law 12.783/2013 (Brasil, 2013), which established the “operation and maintenance regime”, through which the assets are no longer remunerated, being incorporated into the Government through indemnification, and the company has as revenue the cost of efficient operation and maintenance of its concessions.

With regard to the operation and maintenance costs, the regulator has been using the benchmarking methodology called Data Envelopment Analysis (DEA) to calculate the efficient operational costs of electricity transmission companies. This methodology was implemented for the first time in 2007 and was also used in 2010, and had a major impact on the companies' revenues with the anticipation of the renewal of the current concessions in 2012.

On September 11, 2012, industry agents took notice of the Government's proposal for early renewal of energy generation and transmission concessions, causing real panic on the São Paulo Stock Exchange - BM & FBOVESPA. In two days (10 to 12/9) the companies in the sector had their market value reduced by R$28 billion, affecting also the actions of companies in other regulated sectors such as ports, roads and sanitation with a decrease of 15%, 7% and 11% respectively.

Due to the importance of the sector for the economy of a country and the economic impact of this last adopted model, studying the different models developed by the Brazilian regulator from 2007 to date is the first step to understand the complexity of the regulation of the revenues of the concessionaires of transmission of Brazilian electric energy. This historical and critical analysis of the models used is the main objective of this article, in which the models used in each tariff review and in 2012 will be discussed in detail, observing that here the methodology for the review of operational costs and not the methodology for calculating the remuneration of assets will be analyzed.

As to its structure, the article is subdivided into the following parts: Introduction, a brief contextualization of the research topic; Theoretical Framework, where the literature review is made for theoretical background on the subjects studied; Methodology, which refers to the methodological procedures adopted in the research; Evolution of the Benchmarking Process in the Regulation of the Brazilian Electric Power Transmission Sector; Comments on Respect of the Current Model, a critical analysis of the model adopted in the renewal of the concessions; and Final Considerations, with specific comments on the research carried out.

2 Theoretical background

2.1 Tariff regulation of concessionaires of public services

The regulation has the objective of guaranteeing stability and coherence between different agents of the same economy, given a common geographical condition, as well as promoting the economic sustainability of the sectors. Train (1991) considers that the great challenge of the regulatory activity is to induce companies operating in non-competitive markets to act in a manner compatible with social objectives. According to him, the regulator agent must establish methods and incentives so that, in the search for profits, the firm also achieves social results.

Thus, the central issue of the economic regulation is the development of mechanisms capable of induction to achieve optimal results. For this, the first step in defining the regulatory framework is the characterization of the desired result. The second step is the construction of a regulatory mechanism capable of inducing the firm to act in order to produce the desired result. In this respect, effective regulation establishes a situation in which the desired social result is also what generates the greatest profit for the company, so that the firm chooses it naturally.

The task of the regulatory agent is quite complex, since public service companies have more information about production technology and demand structure, but the regulator cannot rely on the disclosure of such data (Berg & Tschirhart, 1998). Thus, the development of optimal regulatory procedures involves incentive mechanisms for regulators to accurately report cost and demand information so that the regulator can determine optimal prices and results.

Jamison & Berg (2012) mention four basic regulatory mechanisms: rate of return, price cap, revenue cap, and yardstick regulation. However, they recognize that in practice, yardstick regulation or benchmarking has been used in price or revenue cap regulation and sometimes in rate of return regulation, which restricts the regulatory mechanisms proposed to three distinct modalities. Berg (1997) points out that the main property of rate-of-return regulation is that it allows the regulator to limit the level of profit that can be achieved by the regulated firm, setting the maximum return on investment (in real terms), and leaving the company a degree of autonomy to conduct business. The price cap regulation, also called RPI-X, where RPI is the English acronym for retail price index, and X is the productivity factor. In such modality, the price is established by the regulator indexing the previous price according to inflation minus a productivity adjustment obtained over the period since the last tariff adjustment.

The regulation called revenue cap, which refers to the imposition of a limit on the operator's income, is similar to the price cap regulation, but the inflation rate and the X-factor apply to revenue rather than prices. In this case, the regulator determines the amount of revenue the firm needs to cover its operational costs, depreciation and cost of capital (Jamison, 2007).

Jamasb & Pollitt (2001) identified the predominant use of Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis (SFA) methodologies in regulatory benchmarking, methodologies developed by Charnes, Cooper and Rhodes (1978), and by Aigner, Lovell & Schmidt (1977) and Meeusen & Van den Broeck (1977). According to Bogetoft and Otto (2011), DEA is advantageous in having a very flexible structure, while SFA has the advantage of separating noise from inefficiency.

Regulatory practice is recent among the functions performed by the Brazilian government. It was from the National Privatization Program - PND in 1995, with the end of the monopoly of state companies in certain sectors that it has begun in Brazil the emergence of regulatory agencies. The Public Services Concessions Law, Law No. 8,987 also of 1995 regulated what is necessary for the emergence of regulatory mechanisms in Brazil. In 1966, ANEEL was created through Law 9,427/1996 and Decree No. 2,335/1997 to regulate the Brazilian electricity sector. ANEEL, together with other research and development agencies in the electricity sector, has helped to ensure the growth and stability of the sector in the coming years, with the regulatory model being established as a fundamental instrument for the development of the sector (Doile, 2012).

2.2 DEA as benchmarking tool

Charnes, Cooper & Rhodes, in 1978, constructed a mathematical programming methodology for the empirical evaluation of the relative technical efficiency of Decision Making Units (DMUs), these being public or private institutions or even organizational units of the same company. This evaluation is based on the comparison of the observed quantities of resources used (inputs) and products or services provided among a similar group of DMUs. The methodology was named by the authors as Data Envelopment Analysis (DEA), given the surface that it produces when points (information) are connected by constructing an empirical boundary.

DEA is a non-parametric mathematical programming tool that aims to evaluate the technical efficiency of decision-making units without requiring knowledge about the prices or weights of the inputs used and the products achieved by them. The developed model was called CCR, in honor of its authors, but also called CRS due to the boundary defined by it, that of constant returns to scale (Constant Returns to Scale).

In 1984, Banker, Charnes, and Cooper extended this model to evaluate units acting on different types of scale returns: Increasing Returns to Scale (IRS), Decreasing Returns to Scale (DRS) and Constant (Constant Returns to Scale - CRS). This was then titled BCC model, also named after the authors, but also called VRS because of the boundary defined by it, Variable Returns to Scale (Banker et al., 1984).

It is important to observe the assumption that in a DEA model the analyzed units / companies show the same production technology, that is, they use the same inputs/resources in the production of the same products / services. One of the advantages of DEA is its ability to take into account multiple inputs and outputs, without the need for the numerator and denominator to be in a common metric basis for the comparison between the different DMUs (Pessanha et al., 2004).

Macedo et al. (2006) point out that one of the main benefits of using the DEA methodology lies in the fact that it is possible to indicate which points of the inefficient units must be improved in order to become efficient. Linked to this, the methodology also points out which competitor or comparative company should be used as a benchmark to outline new strategies that should lead inefficient firms to maximum efficiency.

The two following DEA models show the efficiency analysis of the DMU j imposing different returns to scale. Θ is the efficiency score of the DMU 0, the decision-making unit under analysis, obtained by solving the models (1) or (2), yrj and xij are variables that reflect the quantity of product r (r = 1, ..., s) produced by the inputs i (i = 1, ..., m) by the DMU and λj are weights of the observations used as benchmarking for the DMU under analysis.

Model 1 is known as CCR (Charnes et al., 1978), which imposes constant returns of scale (CRS) on production technology, while Model 2 imposes variable returns of scale (VRS) (Banker et al., 1984). Both here shown in their so-called enveloping form and oriented to the minimization of inputs in Table 1.

Table 1
Return Models in DEA.

Constant Returns to Scale - CRS

Variable Returns to Scale- VRS

$\begin{array}{l}Min{\theta }_0\\ s.t.\\ {\displaystyle \sum _{j=1}^n{{{\displaystyle \lambda }}_{rj}}_{}}{{\displaystyle y}}_{rj}\ge {{\displaystyle y}}_{r0},\text{r=1}\mathrm{,..,}\text{s}\\ {\displaystyle \sum _{j=1}^n{{{\displaystyle \lambda }}_{rj}}_{}}{{\displaystyle x}}_{rj}\le {{\displaystyle \theta x}}_{r0},\text{i=1}\mathrm{,..,}\text{m}\\ {{\displaystyle \lambda }}_{rj}\ge 0\end{array}$(1)

$\begin{array}{l}Min{\theta }_0\\ s.t.\\ {\displaystyle \sum _{j=1}^n{{{\displaystyle \lambda }}_{rj}}_{}}{{\displaystyle y}}_{rj}\ge {{\displaystyle y}}_{r0},\text{r=1}\mathrm{,..,}\text{s}\\ {\displaystyle \sum _{j=1}^n{{{\displaystyle \lambda }}_{rj}}_{}}{{\displaystyle x}}_{rj}\le {{\displaystyle \theta x}}_{r0},\text{i=1}\mathrm{,..,}\text{m}\\ {\displaystyle \sum _{j=1}^n{{{\displaystyle \lambda }}_{rj}}_{}}=1\\ {{\displaystyle \lambda }}_{rj}\ge 0\end{array}$(2)

The DEA methodology has a history of use by the Brazilian regulator in the transmission and distribution of energy sectors. In the transmission the regulator has used DEA to calculate the efficient operational cost of the transmission companies since 2007, and this methodology has participated in 2 review cycles (1CRTP - NT No. 182/2007 - SRE/ANEEL (ANEEL, 2007b) and 2CRTP - NT nº 274/2009 - SRE/ANEEL (ANEEL, 2009a) and the renewal of concessions ((NT 383/2012 (ANEEL, 2012) and Law 12.783/2012 (Brasil, 2013)).

Previous studies have employed DEA to evaluate efficiency and productivity change of energy distribution and transmission companies (Arocena, 2008; Bagdadioglu et al., 1996; Berg et al., 2005; Hjalmarsson & Veiderpass, 1992; Kumbhakar & Hjalmarsson, 1998; Pacudan & de Guzman, 2002). Lins et al. (2007) presented a case study of Brazilian energy distribution companies where two different DEA models were proposed, one that reflected important criteria for regulatory agencies and another that reflected the criteria of companies. Agrell et al. (2005) present a multi-period and multi-product arrangement for energy distribution companies built on cost information from a productivity analysis model and an agency theory model. Zhou et al. (2008) present the results of a research study involving DEA in the energy and environment sector, while Wang et al. (2007) propose a DEA model for regulation in Hong Kong.

Recent studies, such as Altoé et al. (2017) point out that the Brazilian regulator has failed to build a suitable model for the distribution sector. In this paper, the authors show that the constructed model has only two variables that are statistically significant for the calculation of the technical efficiency, the size of the firms and the revenue of the previous year. Lopes et al. (2016) discuss the energy distribution, pointing out failures in the use of the DEA methodology employed in the regulatory model of the Fourth Cycle of Periodic Tariff Review. Throughout the discussions, the authors make three suggestions for improvement for the model: removal of variables with zero information, inclusion of environmental variables that correct the model and removal of weight restrictions. Azevedo et al. (2015) discuss the 2011 model adopted by ANEEL again in the distribution sector, where the regulator used the average of the results of the DEA and COLS (Corrected Ordinary Least Squares) methodologies to calculate the efficiency score of the companies, which, according to the authors, was not a correct model, since the results of the application of the COLS methodology were very different from the results obtained with DEA, this in function of the model that used of COLS to have great deficiencies for the calculation of the efficiency scores. Finally, Xavier et al. (2015) also discuss the use of the DEA methodology in the distribution model, pointing mainly to the unequal way in which companies are compared, given the environment and the quality of service delivery so heterogeneous in the Brazilian market.

3 Methodology

This study includes documentary research on models created by the Brazilian regulator for the electric power transmission sector to identify the changes and failures that they have suffered over the years through revisions and renewals of concessions.

The research follows a qualitative approach through an exploratory perspective to identify the models used by the National Electric Energy Agency in the tariff regulation of the Brazilian electric power transmission sector. Documentary research was carried out by means of the analysis of the official technical notes of the regulatory agency, obtained in its website, to describe and critically analyze the evolution of the regulatory model of the Brazilian electric power transmission. The selected technical notes were those that had relevance for the analysis and, in this way, only those that dealt with processes of revision and renewal of contracts. Finally, the technical notes contemplated were those published between 2007 and 2013, regarding the calculation of the operational costs of the transmission companies. Therefore, it is considered that this research deals with an empirical phenomenon, socially located, and interactive (Kirk & Miller, 1988).

The main focus of the investigation was the technical note that based the calculation of the efficient operational cost of electric power transmission companies when anticipating the renewal of the concessions in 2012/2013, since it is the most recent application and therefore deserves further study. Meetings with companies and specialized consultancies were carried out with the intention of, through the technique of brainstorming, to seek information that could help in the understanding of the proposal for later critical analysis. Simulations were carried out in order to understand the impact of the weight restrictions imposed by the regulator in its last application of DEA. The simulations were performed using the software PIM-DEA (deasoftware.co.uk) in its version 3.2. Criticism and suggestions to the model proposed and implemented in 2012/2013 are carried out.

4 The evolution of the application of DEA in the regulation of costs of the Brazilian electric power transmission sector

4.1 First periodic tariff review (2007)

The model used panel data from the electric power transmission concessionaires corresponding to the years 2003 to 2005 and was in force during the 2005-2008 cycle (NT No. 182/2007 - SRE/ANEEL) (ANEEL, 2007b). The methodology chosen by the regulator to measure efficient operational costs was DEA, considering non-decreasing returns to scale. According to the regulator, the imposition of non-decreasing returns to scale implies that the scale of larger firms cannot be reduced in order to be compared with smaller firms; on the other hand, smaller firms can be compared with larger firms. No. 182/2007 - SRE/ANEEL (ANEEL, 2007b). In the process of choosing the variables, the Brazilian regulator chose to use TOTEX (total cost) as input variable. Table 2 presents the product variables used (NT No. 182/2007 - SRE/ANEEL) (ANEEL, 2007b).

Table 2
Product variables used in the first periodic tariff review of the Brazilian electricity transmission concessionaires.

Product to be represented	Product variable
Transmission Lines	Network Length (km)
Maneuver Modules	Sum of Modules: EL, CT and IB
Equipment Modules	Number of Transformers
	Transformation Capacity (MVA)

• Source: NT nº 182/2007 - SRE/ANEEL (ANEEL, 2007b).

After the model was built and executed, a normalization by the median was performed, in which it was determined that the efficiency parameters estimated would be normalized so that the lowest value was 80% and the highest 100% (NT No. 182/2007 - SRE/ANEEL) (ANEEL, 2007b). Table 3 presents the result of this process.

Table 3
Efficiency scores obtained and normalization by the median.

Estimated Parameters
Companies	2005 %	2004 %	2003 %	Median	Normalized Median
COPEL	93	81	100	93	100
CEMIG	69	91	89	89	99
CEEE	70	75	85	75	94
CHESF	75	73	91	75	93
ELETROSUL	73	71	81	73	93
FURNAS	55	65	71	65	90
CTEEP	43	43	50	43	82
ELETRONORTE	34	36	41	36	80

• Source: NT n° 182/2007 – SRE/ANEEL (ANEEL, 2007b).

4.2 Second periodic tariff review (2009)

The second use of DEA as a benchmarking methodology for Brazilian electric power transmission companies occurred in 2010, with its retroactive effect to 2009, and application for the 2009-2012 cycle. In this review the major change in the model was the use of DEA in two stages. In the second stage, environmental variables are used to adjust the scores obtained in the first stage, considering the peculiarities of the region where each concessionaire is inserted, theoretically allowing a better comparison between companies. (NT No. 274/2009 - SRE/ANEEL) (ANEEL 2009a). The variables used in the second stage were: average labor remuneration, measured according to the location where the concessionaire operates; the voltage level of the transmission lines (the regulator considers that transmission lines with different voltages have different costs and generally the higher the voltage, the higher the maintenance costs); and dispersion of networks, as previously mentioned (NT No. 396/2009 - SRE/ANEEL) (ANEEL, 2009b). Table 4 presents the result of the efficiency of the two stages in the period.

Table 4
Company efficiency scores in 2009.

Companies	DEA score 1st stage %	Final result DEA 2nd stage %
CTEEP	100	100
COPEL	100	95
CEEE	73	73
ELETROSUL	53	68
CEMIG	70	66
FURNAS	51	64
ELETRONORTE	22	55
CHESF	55	44

• Source: NT No. 396/2009 - SRE/ANEEL (ANEEL, 2009b).

4.3 Early renewal of concessions (2012)

On September 11, 2012, the Ministry of Mines and Energy (MME) proposed the anticipation of the renewal of all electric power generation and transmission concessions that would expire by 2015. It was incumbent upon the Superintendence of Economic Regulation of ANEEL - SRE/ANEEL to propose a model for the calculation of the new Allowed Annual Revenues (RAP), which substantially changed the rules that had been in force until then. At this time, 2012, companies, through a provisional measure ((MP 579/2012 (Brasil, 2012), transformed into Law 12,783/2013 (Brasil, 2013)), had the option to choose between renewing their concession, according to new rules, or delivering their concession at the end of the period of contract for new bidding.

The model, described in Technical Note 383/2012 - SRE/ANEEL (2012), and which, in a non-ANEEL compliance process, was not sanctioned by the Board of Directors of the Institution and did not go through a public hearing, proposed that the companies' revenues were formed by their efficient operational cost, calculated by DEA, with no further fully depreciated asset remuneration. As for the efficient operational cost calculation model, the main changes were related to the abandonment of the DEA methodology in two stages and the environmental variables and the sum of the salary difference to the “Personal” account in OPEX (NT nº 383/2012 - SRE/ANEEL) (ANEEL, 2012). The input variable continued to be the OPEX, but the output variables increased in quantity due to the breakdown of the variable Network Length (Km) into tension levels, as shown in Table 5.

Table 5
Representative modular units of inputs and outputs.

Inputs	OPEX	Personnel Accounts, Materials, Third Party Services, Insurance, Taxes and Others
Products	Transmission lines	Network Length (Km) - 600 to 765kV
		Length of Network (Km) - 440 to 525kV
		Network Length (Km) - 325kV
		Network Length (Km) - 230kV
		Network Length (Km) - 138kV
		Network Length (Km) - 69 to 88kV
	Maneuver Modules	Sum of Modules: EL, CT and IB
	Equipment Modules	Number of Transformers
		Transformation Capacity (MVA)

• Source: NT nº 383/2012 - SRE/ANEEL (ANEEL, 2012).

For the first time, weight restrictions were introduced in the model, as shown in Table 6.

Table 6
List of weight restrictions attributed to products related to Km of transmission line.

$1.05\le \frac{u_{\left(600-765\right)}}{u_{\left(230\right)}}\le 1.57$	u(600-765)>=1.05 u(230) u(600-765) <=1.57 u(230)	(1)
$1.01\le \frac{u_{\left(440-525\right)}}{u_{\left(230\right)}}\le 1.51$	u(440-525)>=1.01 u(230) u(440-525)<=1.51 u(230)	(2)
$0.98\le \frac{u_{\left(345\right)}}{u_{\left(230\right)}}\le 1.38$	u(345)>=0.92 u(230) u(345)<=1.38 u(230)	(3)
$0.68\le \frac{u_{\left(138\right)}}{u_{\left(230\right)}}\le 1.02$	u(138)>=0.68 u(230) u(138)<=1.02 u(230)	(4)
$0.59\le \frac{u_{\left(69-88\right)}}{u_{\left(230\right)}}\le 0.88$	u(69-88)>=0.59 u(230) u(69-88)<=0.88 u(230)	(5)

• Source: survey data, 2017.

Where u (69-88) refers to the weight assigned to the network extension variable at voltages from 69 to 88kV, the other weights u (230), u (345), u (440-525) and u (600 to 765) the same way. The value in parentheses refers to the transmission line voltage level. This type of restriction is known in the literature as Assurance Region Type I (Thompson et al., 1986).

With this measure the regulator tried to prevent the weights of the variables related to the kilometers of line in each tension from assuming zero value, at the same time that the higher tensions and, consequently, of higher maintenance cost, had weights bigger than the lower tensions (NT No. 383/2012 - SRE/ANEEL (ANEEL, 2012). The average result of this model was an efficiency score of 51% (Table 7).

Table 7
Efficiency scores obtained by the DEA IRS model.

Company	Efficiency score (%)	Company	Efficiency score (%)	Company	Efficiency score (%)
CEEE	58.3	COPEL	45.5	ELETROSUL	46.9
CEMIG	61.5	CTEEP	96.0	FURNAS	39.3
CHESF	36.7	ELETRONORTE	26.7

• Source: NT nº 383/2012 - SRE/ANEEL (ANEEL, 2012).

In addition, the regulator, considering that the higher the ratio between the variable portion (PV) and total revenue (RT) the greater the system unavailability and the lower the quality of the transmitted energy, uses this result as a way of adjusting the efficiency scores obtained by DEA, in the perspective of quality of service provided. In order to do so, it calculated for each company the ratio between the Variable Portion and its Total Revenue.

According to the Instituto Acende Brasil (IAB, 2015), the transmission concession agreements provide for a Variable Installment that is deducted from the RAP (Allowed Annual Revenue) as a function of the level of unavailability of the transmission facilities. For this calculation, however, it decides to divide the companies into groups according to their PV/RT results and that the 49%, which would be necessary to raise the average to 100%, would be attributed only to the company that had the best value of it. The remaining companies would receive 39%, 29%, 19% and 10%, respectively. The final result of this quality adjustment is summarized in Table 8.

Table 8
Efficiency scores adjusted for quality.

Company	Efficiency achieved by DEA (%)	Quality adjustment (%)	Final efficiency score (%)
CEEE	58.3	19	77.30
CEMIG	61.5	29	90.50
CHESF	36.7	19	55.70
COPEL	45.5	39	84.50
CTEEP	96.0	39	135.0
ELETRONORTE	26.7	19	45.70
ELETROSUL	46.9	49	95.90
FURNAS	39.3	10	49.30

• Source: NT nº 383/2012 - SRE/ANEEL (ANEEL, 2012).

5 Discussions regarding the current model (LAI 12.783/2013) (Brasil, 2013)

In this item the data analysis and the discussion of the current model for the calculation of the efficient operational cost of the Brazilian electric power transmission companies are carried out.

5.1 Analysis of the data

Table 9 presents the descriptive statistics of the variables used by ANEEL.

Table 9
Descriptive Statistics of Data Used by ANEEL in 2012.

Variables		Average	Standard deviation	Minimum	Maximum
Operational cost (R$)		440,323,129.0	315,156,665.1	100,991,372.7	1,185,290,053.8
MVA Power		31,007	25,058	7,160	86,948
Nº of Modules		1,056.4	646.9	324	2,435
Nº of Transformers		279.9	167.0	80	601
Network Extension	69 to 88kV	246.1	350.4	0	1,099
	138kV	1,925.4	2,933.0	0	9,423
	230kV	4,294.5	3,904.3	757	13,472
	440 to 525kV	3,138.6	2,113.2	0	6,305
	345kV	1,133.9	2,062.7	0	6,221
	600 to 765kV	759.2	2,005.7	0	5,922

• Source: NT 383/2012-DRE/ANEEL (ANEEL, 2012).

A high standard deviation is observed in the variable related to the operational cost. This high value for the standard deviation is due to the choice of the regulatory agency for the use, in the same model, companies of different scales. It is also observed the occurrence of null minimum values for the variables network extension (km) from 600 to 765kV, 440 to 525kV, 138kV, 345kV indicating that the benchmarking performed by the regulator compares companies that do not operate at some voltage levels with the others, which damages the fundamental principle of the use of DEA, that of homogeneity. Table 10 shows the voltage levels of each company.

Table 10
Participation (%) of the Voltage (kV) in the Company's total (data for 2011).

Volts (kV)	Extension of transmission lines (km)
	CEEE	CEMIG	CHESF	COPEL	CTEEP	Eletro- norte	Eletro- Sul	Furnas	Total
69-88	227 3.94%	0	325 1.67%	13 0.68%	1,099 5.81%	248 2.71%	24 0.24%	0	1,936 2.13%
138	760 13.20%	0	572 2.93%	138 7.25%	9,423 49.84%	366 4.00%	1,828 18.37%	2,149 10.36%	15,236 16.78%
230	4,772 82.85%	768 15.56%	13,472 69.13%	1,596 83.82%	1,364 7.21%	5,295 57.94%	5,135 51.61%	1,883 9.09%	34,285 37.76%
345	0	1,967 40.11%	0	0	715 3.78%	0	0	6,221 29.99%	8,903 9.80%
440-525	0	2,169 44.23%	5,120 26.27%	157. 8.25%	6,305 33.35%	3,230 35.34%	2,962 29.77%	4,572 22.04%	24,515 26.99%
600-765	0	0	0	0	0	0	0	5,922 28.54%	5,922 6.52%
Total	5,759	4,904	19,489	1,904	18,906	9,139	9,949	20,747	90,797

Note that Furnas is the only company operating the 765 to 600kV network, which would make it incomparable with the others, considering the methodology adopted. Due to this specificity, the company should be treated separately by means of an international benchmarking that compares it with companies of the same size and level of tension. In addition, it is considered that there are differences in cost of the different voltages, which must be taken into account.

Regarding the variable Network Length (km), the operation and maintenance of networks of different voltage levels can be very different in frequency of activities, number of teams and costs with materials. Thus, networks were considered, disaggregated by voltage level [...] (NT 383/2012 - SRE/ANEEL of 10/24/2012, p. 8, item 48), as shown in Figure 1.

Figure 1
Relationship between operational cost and product variables.

Note that the operational cost composes the ordinate of each chart, while the different products used by ANEEL make up the abscissa. The size of the circles represents the year, and the most recent year, 2011, is given by the larger circle. In the figure, it is possible to observe a great operational diversity among the companies that work in the sector.

5.2 The choice of variables

5.2.1 Adjustment of operational cost per region average salary

In order to calculate the efficiency scores, the operational cost used was adjusted to reflect the wage differences between the regions in which the companies operate. For this adjustment, data from the Annual Social Information Report - RAIS, from 2007 to 2009 were used. The results showed an average salary of R $ 3,834.80, with almost all companies very close to this value. The only company that presented above-average salary was CTEEP, R $ 4,584.95. For this reason, its personnel expenses were adjusted downwards by a 19.59% reduction (Table 11), positively impacting its efficiency score and causing this company to define the efficiency frontier against which all other companies would be evaluated.

Table 11
Percentage adjustment of operational costs based on the region's average salary.

Company	Percentage of adjustment in personnel expenses %
CEEE	2.13
CEMIG	2.14
CHESF	5.51
COPEL	2,13
CTEEP	-19.59
Eletronorte	3.24
Eletrosul	2.13
Furnas	2.27

• Source: NT nº 383/2012 - SRE/ANEEL (ANEEL, 2012).

CTEEP, which in the 2nd cycle has an average remuneration of R $ 2,464.69 (average 2002-2008), in the renewal of the concessions has the average salary high to R $ 4,584.96 (2007-2009 average). This difference is due to the fact that ANEEL used the PNAD/IBGE database (2002-2008) in the 2nd cycle, and RAIS data (2007-2009) in the renewal of the concessions.

5.2.2. Absence of the reactive compensation variable

The Allowed Annual Revenue calculated in NT 383/2012 (ANEEL, 2012) takes into account in the calculation of DEA efficiency the following product variables: MVA power, number of modules, number of transformers, and kilometers of transmission network disaggregated by voltage level. It occurs that the reactive control (CR), which is considered by ANEEL as one of the basic functions of the transmission, is no longer considered as a product of the companies in the DEA model. The absence of this variable in the model makes it considered as inefficiency costs that the companies incurred with the maintenance and operation of these reactives. Table 12 shows the participation of companies in the reactive control of the country, while Table13 presents the efficiency scores, if this variable were considered in the model.

Table 12
Capacitive MVAR data of energy transmission companies - year 2011.

Company	MVAR Capacitive	%/Total
CEEE	598.2	1.7
CEMIG	1,008.3	2.8
CHESF	4,369.6	12.1
COPEL	311	0.9
CTEEP	3,597.3	10.0
Eletronorte	5,861.06	16.2
Eletrosul	684.4	1.9
Furnas	19,720.8	54.6
Total	36,150.66	100.0

Table 13
Transmission companies’ efficiency scores including capacitive MVAR - year 2011.

Company	DEA score with capacitive MVAR %	DEA NT Score 383/2012 %	Difference %
CEEE	58.3	58.3	0
CEMIG	61.6	61.6	0
CHESF	36.7	36.7	0
COPEL	45.5	45.5	0
CTEEP	96.0	96.0	0
ELETRONORTE	51.0	26.7	24.3
ELETROSUL	39.3	47.0	-7.7
FURNAS	74.3	39.3	35.0

It is observed that FURNAS accounts for 54.6% of the reactive control of the National Interconnected System (SIN), followed by ELETRONORTE (16.2%), CHESF (12.1%) and CTEEP (10%).

5.3 Weight restrictions

The granting power, when seeking to implement restrictions that would prevent the weights of the variables related to the kilometers of line in each tension assumed zero value, at the same time that the higher tensions and, consequently, of higher maintenance cost, had larger weights (NT No. 383/2012 - SRE/ANEEL) constructed a model that did not reach its objectives. Many of the variables continued with their weights equal to zero, as shown in Table 14.

Table 14
Optimum weights of the product variables obtained in the DEA CRS model with weight restrictions.

Company	Scores %	Weights assigned to the Network Product in the DEA Model
		MVA Power	Xfmr QTty	Modules QTty	600-765 KV	440-525 KV	345 KV	230 KV	138 KV	69-88 KV
CEEE 11	58.30	0	0	5.59E-04	0	0	0	0	0	0
CEMIG11	61.60	0	0	0	1.10E-04	1.10E-04	1.50E-04	1.09E-04	7.38E-05	6.40E-05
CHESF11	36.70	0	0	0	2.00E-05	1.93E-05	1.75E-05	1.91E-05	1.30E-05	1.13E-05
COPEL11	45.50	4.19E-05	0	0	0	0	0	0	0	0
CTEEP11	96.00	0	0	3.94E-04	0	0	0	0	0	0
ELETRONORTE11	26.7	0	0	0	3.13E-05	3.01E-05	2.74E-05	2.98E-05	2.03E-05	1.76E-05
ELETROSUL11	39.30	0	0	0	4.30E-05	4.23E-05	3.85E-05	4.18E-05	2.85E-05	2.47E-05
FURNAS11	39.30	4.52E-06	0	0	0	0	0	0	0	0

The most serious case can be observed in the company CTEEP, which defines the efficiency frontier and, consequently, the efficiency scores of all other companies, with only one variable: the number of modules. In relation to the operational cost of FURNAS, it appears in the model with weight restrictions, being explained only by its MVA Power. Again, this happens by error in restrictions modeling. Certainly, this was not the intention of the regulator, as can be observed by his statement given in NT # 383/2012 - SRE/ANEEL.

The impact of these constraints on company efficiency scores was not small. Figure2 illustrates the distance between the scores obtained by an unrestricted model and a model with the restrictions elaborated by ANEEL.

Figure 2
Comparison of the model with or without weight restrictions.

In order to avoid finding null weights in the optimal solution of a DEA model, an approach frequently used in the literature (Thanassoulis, 2001) is the use of additional constraints that impose that the optimal weights of the linear programming model assume a minimum value for all variables. The non-Archimedean number (0.000001 or 1X10-6) is the most used. In order to meet the two goals stated by ANEEL in NT No. 383/2012-SRE/ANEEL (p. 8), an alternative would be to add a restriction for each variable by limiting the weights of the variables MVA power, number of modules, number of transformers and extension of network, disaggregated by voltage level, to assume a minimum value of 1x10-6. The results of this model do not present null values ​​for the optimal weights, consider all the variables and generate results for the efficiency scores of the companies superior to those found by ANEEL, reducing the need for adjustments, always very controversial, such as the quality adjustment. The efficiency scores found by this model for 2011 are: ELETRONORTE (48.94%), FURNAS (74.90%), CHESF (67.60%), CEEE (77.40%), COPEL (70.17%), CEMIG (100%), CTEEP (95.84%) and ELETROSUL (65.53%).

5.4 The quality adjustment

Another questionable point is the financial bonus established after the calculation of the efficiency scores by means of DEA, and that was attributed to quality. According to NT 383/2012 (2012, p. 12), ANEEL makes this adjustment with the objective of increasing the average efficiency from 51% to 100%, and later defines that this adjustment will only be achieved by the company that presents better transmission service quality. The regulator decides to establish different adjustment percentages for the different quality values, measured by dividing the variable portion by the transmitter's revenue (PV/RT) and dividing the companies into 5 groups, as described in the previous item. Table 15 shows how much, in percentage, was received by each company as a form of “premium for quality”, as well as the value that was used of PV/RT of each company.

Table 15
Quality Adjustment.

Company	Efficiency %	Average PV/RT %	Group	Difference PV/RT %	Quality Adjustment %	Final Efficiency %
ELETROSUL	46.90	0.05	1	-	49.00	95.90
CTEEP	96.00	-0.08	2	0.13	39.00	135.00
COPEL	45.50	-0.11	2	0.16	39.00	84.50
CEMIG	61.50	-0.32	3	0.37	29.00	90.50
CEEE	58.30	-0.53	4	0.58	19.00	77.30
ELETRONORTE	26.70	-0.65	4	0.70	19.00	45.70
CHESF	36.70	-0.78	4	0.83	19.00	55.70
FURNAS	39.30	-1.31	5	1.36	10.00	49.30

It is observed that ELETROSUL, with a PV/RT of 0.05%, has added to its efficiency score that is 46.9% a percentage of 49%, reaching almost 100% of the operational cost realized in 2011. It is interesting to note the discrepancies between the differences between the PV/RT values (%) and the differences between the percentages defined as quality premium. It is questioned here whether it can be affirmed that more than 40% of a company's operational cost comes from costs of transmission quality, that is, it is directly related to quality, as is the case of ELETROSUL.

6 Final considerations

This article aimed to describe the methodology adopted by the National Electric Energy Agency for the calculation of the efficient operational costs of the Brazilian electric power transmission companies since 2007. The main focus was placed on the current model on which some criticisms were made, such as: error in the construction of the model with weight restrictions; 'ad hoc' adjustment given by the quality of the power transmission and, in a way, unjustifiable to the consumer; not use of an important variable and that is part of the transmission service as the reactive capacity; and the non-consideration of the environmental variable net dispersion, considered important in previous reviews.

It is clear that ANEEL still has a long way to go until the optimum model of operational cost regulation is achieved. From the moment the regulator uses discretionary adjustments like what was done in NT 383/2012, it is clear that the regulator is still not managing to correctly capture the differences between the companies in the sector and, as a result, creates artificial mechanisms to correct the results.

Data Envelopment Analysis has proven to be a widely used methodology in regulating naturally monopolized markets around the world. By building a correct model, the Brazilian electricity sector will be regulated by the premise of efficiency. With this, the concessionaires will be encouraged to be more efficient, bringing benefits to the final consumer who will receive a quality service with the lowest prices possible, but guaranteeing the sustainability of the sector.

It is the responsibility of the Agency and the concessionaires to better study the existing methodologies and, together with Data Envelopment Analysis (DEA) experts, develop a model that meets the needs of tariff modality and that provides conditions for business survival and investments in the sector. For this, a study should be made in order to analyze all variables of the internal and external environment of the companies that affect their operational or total costs, considering all of them in their respective disaggregations and amplitudes. After identifying these variables, they must be implemented to the model, respecting the variations between the environmental conditions of each company. Recalling that ad hoc adjustments are unnecessary and questionable when the DEA model, constructed with all the necessary variables, effectively shows the correct scores of companies.

• Financial support: CNPq.
• How to cite: Lopes, A. L. M., Mesquita, R. B., Lambertucci, M., & Lara, J. E. (2020). The Evolution of the Benchmarking Methodology Data Envelopment Analysis - DEA in the Cost Regulation of the Brazilian Electric Power Transmission Sector: a critical look at the renewal of concessions. Gestão & Produção, 27(1), e3940. https://doi.org/10.1590/0104-530X3940-20

References

• Agência Nacional de Energia Elétrica – ANEEL. (2007a). Nota Técnica 116/2007, de 11 de maio de 2007. Primeira Revisão Tarifária Periódica da Concessionária de Transmissão de Energia Elétrica – Furnas Centrais Elétricas S.A Brasília: ANEEL. Retrieved in 2015, October 28, from http://www.aneel.gov.br
  » http://www.aneel.gov.br
• Agência Nacional de Energia Elétrica – ANEEL. (2007b). Nota Técnica 182/2007, de 22 de junho de 2007. Benchmarking dos Custos Operacionais das Concessionárias de Transmissão de Energia Elétrica Brasília: ANEEL. Retrieved in 2015, October 28, from http://www.aneel.gov.br
  » http://www.aneel.gov.br
• Agência Nacional de Energia Elétrica – ANEEL. (2009a). Nota Técnica 274/2009, de 05 de agosto de 2009. Benchmarking dos Custos Operacionais das Concessionárias de Transmissão de Energia Elétrica Brasília: ANEEL. Retrieved in 2015, October 28, from http://www.aneel.gov.br
  » http://www.aneel.gov.br
• Agência Nacional de Energia Elétrica – ANEEL. (2009b). Nota Técnica 396/2009, de 01 de dezembro de 2009. Benchmarking dos Custos Operacionais das Concessionárias de Transmissão de Energia Elétrica Brasília: ANEEL. Retrieved in 2015, October 28, from http://www.aneel.gov.br
  » http://www.aneel.gov.br
• Agência Nacional de Energia Elétrica – ANEEL. (2012). Nota Técnica 383/2012, de 24 de outubro de 2012. Contribuição ao processo de fixação da receita inicial de transmissão, de responsabilidade do Ministério de Minas e Energia – MME, Art. 13 da Medida Provisória no 579, de 11 de setembro de 2012 Brasília: ANEEL. Retrieved in 2015, October 28, from http://www.aneel.gov.br
  » http://www.aneel.gov.br
• Agrell, P. J., Bogetoft, P., & Tind, J. (2005). DEA and dynamic yardstick competition in scandinavian electricity distribution. Journal of Productivity Analysis, 23(2), 173-201. http://dx.doi.org/10.1007/s11123-005-1327-6
  » http://dx.doi.org/10.1007/s11123-005-1327-6
• Aigner, D., Lovell, C. A. K., & Schmidt, P. (1977). Formulation and estimation of stochastic frontier production function models. Journal of Econometrics, 6(1), 21-37. http://dx.doi.org/10.1016/0304-4076(77)90052-5
  » http://dx.doi.org/10.1016/0304-4076(77)90052-5
• Altoé, A. V., Costa, N., Jr., Lopes, A. L. M., Veloso, T. R. M., & Saurin, V. (2017). Technical efficiency and financial performance in the Brazilian distribution service operators. Socio-Economic Planning Sciences, 1, 11-25. http://dx.doi.org/10.1016/j.seps.2017.01.003
  » http://dx.doi.org/10.1016/j.seps.2017.01.003
• Arocena, P. (2008). Cost and quality gains from diversification and vertical integration in the electricity industry: a DEA approach. Energy Economics, 30(1), 39-58. http://dx.doi.org/10.1016/j.eneco.2006.09.001
  » http://dx.doi.org/10.1016/j.eneco.2006.09.001
• Azevedo, M., Lopes, A. L. M., & Pinho, G. (2015). Statistical evaluation of Data Envelopment Analysis versus COLS Cobb-Douglas benchmarking models for the 2011 Brazilian tariff revision. Socio-Economic Planning Sciences, 49, 47-60. http://dx.doi.org/10.1016/j.seps.2014.11.001
  » http://dx.doi.org/10.1016/j.seps.2014.11.001
• Bagdadioglu, N., Waddams Price, C. M., & Weyman-Jones, T. G. (1996). Efficiency and ownership in electricity distribution: a non-parametric model of the Turkish experience. Energy Economics, 18(1-2), 1-23. http://dx.doi.org/10.1016/0140-9883(95)00042-9
  » http://dx.doi.org/10.1016/0140-9883(95)00042-9
• Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Models for the Estimation of Technical and Scale Inefficiencies in Data Envelopment Analysis. Management Science, 30(9), 1078-1092. http://dx.doi.org/10.1287/mnsc.30.9.1078
  » http://dx.doi.org/10.1287/mnsc.30.9.1078
• Beesley, M. E., & Littlechild, S. C. (1989). The Regulation of Privatized Monopolies in the United Kingdom. The Rand Journal of Economics, 20(3), 454-472. http://dx.doi.org/10.2307/2555582
  » http://dx.doi.org/10.2307/2555582
• Berg, S. V. (1997). Introduction to the fundamentals of incentive regulation. In M. Arblaster & M. Jamison. Infrastructure Regulation and Market Reform: principles and practice Melbourne, Australia: Australian Competition & Consumer Commission.
• Berg, S. V., & Tschirhart, J. (1998). Natural monopoly regulation: principles and practice. Cambridge: Cambridge University Press.
• Berg, S., Lin, C., & Tsaplin, V. (2005). Regulation of state-owned and privatized utilities: ukraine electricity distribution company performance. Journal of Regulatory Economics, 28(3), 259-287. http://dx.doi.org/10.1007/s11149-005-3957-z
  » http://dx.doi.org/10.1007/s11149-005-3957-z
• Bogetoft, P., & Otto, L. (2011). Benchmarking with DEA, SFA, and R. New York: Springer. http://dx.doi.org/10.1007/978-1-4419-7961-2
  » http://dx.doi.org/10.1007/978-1-4419-7961-2
• Brasil. (2012, 12 de setembtro). Medida Provisória nº 579, de 11 de setembro de 2012. Dispõe sobre as concessões de geração, transmissão e distribuição de energia elétrica, sobre a redução dos encargos setoriais, sobre a modicidade tarifária, e dá outras providências Brasília, DF: Diário Oficial da República Federativa do Brasil. Retrieved in 2015, October 28, from http://www.planalto.gov.br
  » http://www.planalto.gov.br
• Brasil. (2013, 3 de abril). Lei nº 12.793, de 11 de janeiro de 2013. Dispõe sobre o Fundo de Desenvolvimento do Centro-Oeste - FDCO; altera a Lei nº 12.712, de 30 de agosto de 2012, para autorizar a União a conceder subvenção econômica às instituições financeiras oficiais federais, sob a forma de equalização de taxa de juros nas operações de crédito para investimentos no âmbito do FDCO; altera as Leis nº 7.827, de 27 de setembro de 1989, e nº 10.177, de 12 de janeiro de 2001, que tratam das operações com recursos dos Fundos Constitucionais de Financiamento do Norte, do Nordeste e do Centro-Oeste; constitui fonte adicional de recursos para ampliação de limites operacionais da Caixa Econômica Federal e do Banco do Brasil S.A.; altera a Medida Provisória nº 2.199-14, de 24 de agosto de 2001, e a Lei nº 11.196, de 21 de novembro de 2005, para estender à Região Centro-Oeste incentivos fiscais vigentes em benefício das Regiões Norte e Nordeste; e dá outras providências Brasília, DF: Diário Oficial da República Federativa do Brasil. Retrieved in 2015, October 28, from http://www.planalto.gov.br
  » http://www.planalto.gov.br
• Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(6), 429-444. http://dx.doi.org/10.1016/0377-2217(78)90138-8
  » http://dx.doi.org/10.1016/0377-2217(78)90138-8
• Doile, G. (2012). Regulação do setor elétrico: histórico, agência reguladora, atualidades e perspectivas futuras. In E. Nery. Mercados e regulação de energia elétrica (pp. 469-481). Rio de Janeiro: Interciência.
• Hjalmarsson, L., & Veiderpass, A. (1992). Efficiency and ownership in Swedish electricity retail distribution. Journal of Productivity Analysis, 3(1-2), 7-23. http://dx.doi.org/10.1007/BF00158766
  » http://dx.doi.org/10.1007/BF00158766
• Instituto Acende Brasil – IAB. (2015). Transmissão: o elo integrador São Paulo: IAB.
• Jamasb, T., & Pollitt, M. (2001). Benchmarking and regulation: international electricity experience. Utilities Policy, 9(3), 107-130. http://dx.doi.org/10.1016/S0957-1787(01)00010-8
  » http://dx.doi.org/10.1016/S0957-1787(01)00010-8
• Jamison, M. A. (2007). Regulation: rate of return. In B. L. Capehart. Encyclopedia of energy engineering and technology New York: CRC Press. http://dx.doi.org/10.1201/9780849338960.ch145
  » http://dx.doi.org/10.1201/9780849338960.ch145
• Jamison, M. A., & Berg, S. (2012). Overview of Utility Regulation. In Public Utility Research Center of Florida University – PURC. Utility regulatory fundamentals: a reference handbook from PURC Training (pp. 7-23). Gainesville, FL: PURC.
• Kirk, J., & Miller, M. L. (1988). Reliability and validity in qualitative research. London: Sage.
• Kumbhakar, S. C., & Hjalmarsson, L. (1998). Relative performance of public and private ownership under yardstick competition: electricity retail distribution. European Economic Review, 42(1), 97-122. http://dx.doi.org/10.1016/S0014-2921(96)00052-9
  » http://dx.doi.org/10.1016/S0014-2921(96)00052-9
• Lins, M. P. E., Sollero, M. K. V., Calôba, G. M., & Silva, A. C. M. (2007). Integrating the regulatory and utility firm perspectives, when measuring the efficiency of electricity distribution. European Journal of Operational Research, 181(3), 1413-1424. http://dx.doi.org/10.1016/j.ejor.2005.10.072
  » http://dx.doi.org/10.1016/j.ejor.2005.10.072
• Lopes, A. L. M., Vilela, B., Costa, M., & Lanzer, E. (2016). Critical evaluation of the performance assessment model of Brazilian electricity distribution companies. Revista Gestão & Tecnologia, 16(3), 5-30. http://dx.doi.org/10.20397/2177-6652/2016.v16i3.1091
  » http://dx.doi.org/10.20397/2177-6652/2016.v16i3.1091
• Macedo, M. A. S., Santos, R. M., & Silva, F. F. (2006). Desempenho Organizacional no Setor Bancário Brasileiro: uma aplicação da Análise Envoltória de Dados. Revista de Administração da Mackenzie, 7(1), 11-44.
• Meeusen, W., & Van den Broeck, J. (1977). Efficiency estimation from Cobb-Douglas production functions with composite error. International Economic Review, 18(2), 435-444. http://dx.doi.org/10.2307/2525757
  » http://dx.doi.org/10.2307/2525757
• Pacudan, R., & de Guzman, E. (2002). Impact of energy efficiency policy to productive efficiency of electricity distribution industry in the Philippines. Energy Economics, 24(1), 41-54. http://dx.doi.org/10.1016/S0140-9883(01)00085-8
  » http://dx.doi.org/10.1016/S0140-9883(01)00085-8
• Pessanha, J. F. M., Souza, R. C., & Laurencel, L. C. (2004). Usando DEA na avaliação da eficiência operacional das distribuidoras do setor elétrico brasileiro. In Anais do Congreso Latino-Ibero-americano de Investigación de Operaciones y Sistemas La Havana: Universidade de la Habana.
• Savoia, R., & Moreira, F. (2012). Métodos de precificação. Tarifação & tributação. In: Nery, E. Mercados e regulação de energia elétrica Rio de Janeiro: Interciência, p. 659-688.
• Thanassoulis, E. (2001). Introduction to the theory and application of data envelopment analysis Dordrecht: Kluwer Academic Publishers. http://dx.doi.org/10.1007/978-1-4615-1407-7
  » http://dx.doi.org/10.1007/978-1-4615-1407-7
• Thompson, R. G., Singleton, F. D., Jr, Thrall, R. M., & Smith, B. A. (1986). Comparative site evaluations for locating a high-energy physics lab in Texas. Interfaces, 16(6), 35-49. http://dx.doi.org/10.1287/inte.16.6.35
  » http://dx.doi.org/10.1287/inte.16.6.35
• Train, K. E. (1991). Optimal regulation: the economic theory of natural monopoly. Boston: MIT Press.
• Wang, J. H., Ngan, H. W., Engriwan, W., & Lo, K. L. (2007). Performance based regulation of electricity supply industry in Hong Kong: an empirical efficiency analysis approach. Energy Policy, 35(1), 609-615. http://dx.doi.org/10.1016/j.enpol.2006.01.011
  » http://dx.doi.org/10.1016/j.enpol.2006.01.011
• Xavier, S., Lima, J., Lima, L., & Lopes, A. L. M. (2015). How Efficient are the Brazilian Electricity Distribution Companies? Journal of Control, Automation and Electrical Systems, 26(3), 283-296.
• Zhou, P., Ang, B. W., & Poh, K. L. (2008). A survey of data envelopment analysis in energy and environmental studies. European Journal of Operational Research, 189(1), 1-18. http://dx.doi.org/10.1016/j.ejor.2007.04.042
  » http://dx.doi.org/10.1016/j.ejor.2007.04.042

Publication Dates

• Publication in this collection
  07 Feb 2020
• Date of issue
  2020

History

• Received
  05 May 2017
• Accepted
  23 Apr 2018