Efficiency and Effectiveness Analysis of Public Transport of Brazilian Cities

Abstract

The public sector is responsible for the implementation of transport public policies that maximize the competitiveness of urban space and minimize the negative effects of its growth. For that, the proposal of actions must be guided by good performance indices. Taking the Competitiveness Indicators of Campinas (ICC) as a reference, this paper introduces three main contributions: improve the ICC public transport performance indicator; evaluate the performance of Brazilian municipalities over six years and analyses the efficiency vs. effectiveness strategy for each city. The Data Envelopment Analysis (DEA) results show the conflict among different municipalities strategies, while Curitiba and Betim emphasize the effectiveness of the service delivered São Bernardo e Salvador strategy is centered on efficiency.

Key words:
Urban Public Transportation; Data Envelopment Analysis; Government and Public; Policies

Introduction

This article evaluates urban public transport of twenty one Brazilian largest cities through Data Envelopment Analysis Method (DEA) from 2005 to 2010 for three scenarios: infrastructure efficiency, service effectiveness, and efficiency versus effectiveness. Infrastructure Efficiency Indicator (IEI) is a measure of operational excellence in the resource utilization while Effectiveness Indicator (EI) refers to the use of outputs to achieve the passenger interest (Chu at al. 1992). From the relationship of the IEI and EI indicators is possible to compare the strategic position of cities regarding to the productivity and level of service delivered.

The urban population, embedded in social groups usually far from food and raw material sources, jobs and leisure, needs to move in a complex public transport system. The local government acts fixing the ticket price, maintaining lines and frequency that are not justified economically (Pina and Torres, 2001). On the other hand the efficiency with which the service is delivered is required. In general, efficiency and effectiveness are negatively correlated and a balance should be pursued (Chu et al, 1992). The scarcity of resources forces the public sector to strive to balance the benefits and consequences for the population of this activity. But, how to measure the productivity and service level in urban public transport? To formulate and evaluate strategic policies, the Campinas municipality created the ICC project with the participation of 50 municipalities with more than 300,000 inhabitants (ICC, 2011). Two indicators of ICC are dedicated to evaluate the urban transport: % Daily Transported Passenger/Population and % Gratuity/ Transported Passenger. Although the ICC constitutes a work of large importance, a multiple input and multiple output evaluation of the urban transport of could contribute to better public actions. DEA is the most used tool for comparing the performance of organizations with multiple variables and has been utilized to evaluate various contexts. Some applications of DEA in the area of logistics include Karlaftis (2004); Karlaftis and Tsamboulas (2012); Ozbek et al. (2009); Novaes (2001); and Tamagawa, et. al (2010), among others. Its application to public road transport is reviewed by Jarboui et al. (2012).

The next section reviews the DEA concept; section two presents the methodology; section three evaluates urban public transport in 21 Brazilian cities and compares the strategy of each one regarding to the balance between efficiency and effectiveness; and the last section presents the final considerations and conclusions.

1. DEA Model

1.1. Efficiency Frontier

The classical work of Farrell (1957) highlighted the importance of measuring production efficiency for both economic theorists (to evaluate the relative efficiency between industries) and economic policy making (to formulate economic policy for a specific industry). He noted that generating a satisfactory measure of efficiency for multiple input systems was a challenge. A mayor problem with using a single performance indicator is that it the indicator tends to ignore interactions or tradeoffs among various separated measures (Cook and Zhu, 2008). Twenty years after the Farrell seminal work, Charnes et al. (1978) introduced a methodology for evaluating the relative efficiency of production units with multiple inputs and multiple outputs (Cook and Seiford, 2009). The methodology, later called Data Envelopment Analysis, allows identifying, within a set of comparable decision making units (DMUs), those exhibiting best practices. The Figure 1 shows two representations: the absolute efficiency function as an isoquant curve; and in the absence of formal production standards, best practice function, formed by a piecewise linear approximation (Cook and Zhu, 2008).

Figure 1
The concept of efficient frontier adapted of Cook and Zhu, 2008

Efficiency means that the organization uses its resources productively and cost effectively, produces more with fewer resources, or even rationalizes its inputs. Also the methodology enables to measure the level of efficiency of non-frontier units. In the Figure 1, the Ox and Oy units are dominated by the other non-dominated units that define the best practice frontier. One unit (Ox or Oy) must improve its performance, seeking a new position situated on the efficient frontier, decreasing the input maintaining the same output or increasing the output considering the same input. That is, an inefficient DMU should undertake actions that make it efficient by applying efforts to reach any point on the border efficiency line. This work takes the Foroughi (2011) for selection of better decision making unit problem through a mixed integer linear model.

1.2. The Super Efficiency DEA Model

The super-efficiency data envelopment analysis (DEA) model enables one to differentiate the efficient decision-making units (DMUs) with efficiency scores equal to one of the basic DEA models. To reach this discriminatory power, Foroughi (2011) proposed a mixed integer linear model for selecting the best decision making units as:

ν and u are the set of all accepted weights selected as: urk > ε >0; vik > ε >0, and ε > 0 is a non-Archimedean infinitesimal number determined as Hadad et al. (2013): (Rj-= maxj (Xij) for i =1, 2, ... , m Rj+ = maxj (Yrj) for r = 1, 2...s) with upon bounds common to all the inputs and outputs are defined as: Vi ≥ {(1/(m + s) Ri-} i= 1 ,2,.m Ur ≥ {1/(m + s) Ri+} r = 1 ,2, .s.

2. Methodology

For implementing DEA, Golany and Roll (1989) proposed a model with three phases: the DMUs to be included in the study; election of the relevant and appropriate variables (inputs and outputs); and the application of DEA model. Dyson et al. (2001) and Ozbek (2009) extended the previous work to include six phases. This work is based on the Ozbek methodology.

3. Analysis of public transportation of Brazilian cities

The Campinas Competitiveness Indicator Report (ICC, 2011), considering statistical data from 2005 to 2010 for 50 Brazilian municipalities, with more than 300 thousand inhabitants is the reference for this study. The data were classified into nine themes. This study considers transport and transit data.

3.1. Objective of the study

The purpose is to benchmark Brazilian cities using secondary data on three performance measures: infrastructure efficiency, service effectiveness vs. efficiency score. Efficiency is an indicator of operational excellence, thus of service provider interest. Service effectiveness is a measure of the user satisfaction with the service delivered, thus of passenger interest. The last one allows to identify the city strategy balance regarding both criteria.

3.2. DMUs definitions and selection

The cities (DMUs) were selected from the ICC document taking into account the availability of data. As the topic under analysis is public road transport operating in the urban environment, the DMUs are in similar environments and have similar objectives. The requirement of similarity of technology was considered to have been met, since few cities have train or subway transportation and these data were not considered. The data were collected and analyzed from 2005 to 2010.

3.3. Definitions and selection of the variables

The selected variables were: Municipality inhabitants, Number of buses, Average daily passengers, Average gratuity.

3.4. Model selection and formalization

The DEA model can be input-oriented if the desire is to minimize the resources used in the operation without reducing the level of output, or output-oriented if the desire is to maximize the output without reducing the level of input. This study used the CCR (or CRS)) output-oriented models.

3.5. Validation

DEA measures the relative efficiency among comparable units, hence it is sensitive to extreme values, thus caution must be taken for inclusion a DMU in the study, (Dyson et al., 2001). To overcome the above difficulty the development of the model is an iterative process where the results must be validated at each iteration until to reach a reliable model.

3.6. Scenarios and results

The ICC report ranks cities, according to the index defined as the % passengers/population and assumes that the higher index more attractive will be the public transport. It ranks Porto Alegre 1st, São Paulo 2nd and Vila Velha is the last among the 21 cities considered, last column of Table 1.

Table 1
Ranking comparison

City	Average Efficiency	Average Effective	Ranking Efficiency	Ranking Effective	Ranking Foroughi	Ranking ICC
Anapólis	51	77	17	11	8	17
Belo Horizonte	68	88	9	6	7	6
Betim	38	98	20	2	2	20
Blumenau	67	71	10	14	11	9
Campinas	73	83	7	8	9	5
Campo Grande	62	74	12	13	13	13
Caruaru	48	78	18	10	10	19
Curitiba	60	100	14	1	1	7
Fortaleza	70	66	8	17	17	11
Guarulhos	46	86	19	7	6	18
Juiz de Fora	93	74	5	12	14	3
Mogi das Cruzes	65	61	11	20	20	16
Porto Alegre	100	81	1	9	11	1
Salvador	93	61	4	19	18	4
Santos	77	63	6	18	19	10
Serra	60	92	15	5	5	12
Sorocaba	58	70	16	15	15	2
São B. do Campo	100	42	1	21	21	8
São Paulo	93	98	3	3	3	15
Uberlândia	60	70	13	16	16	14
Vila Velha	38	92	21	4	4	21

3.6.1. Scenario 1 -Infrastructure efficiency

The inputs were considered as Municipality inhabitants (to introduce one characteristic of the environment), and the number of urban buses. The output were the average daily passengers (average daily paying passengers+ average gratuity) carried.

The number of urban buses is the data largely considered in the literature (Jarbouir, 2012). The results, considering as the objective the maximization of the average daily passengers carried, show Porto Alegre and São Bernardo do Campo as the cities having the highest infrastructure efficiency over the six years analyzed, Table 1. This measure evaluates the efficiency of resource use and represents the point of view of the service provider, then does not indicate that the municipalities have "perfect equilibrium" in their transportation systems. Rather, the municipalities are benchmarked with respect to the use of their infrastructure.

3.6.2. Scenario 2 -Service effectiveness

The inputs are population and the average daily passengers carried and the output public transportation fleet. This scenario seeks to distinguish the "comfort" offered by public transportation system to passengers through the number of bus fleet (Chu at al. 1992). Greater public fleet more will be the passenger comfort. The results present Curitiba and Betim as 1th and 2th respectively, while the ICC index ranks Curitiba as 7th and Betim as 20th, last column of Table 1. Local strategy affects significantly the index. An action of Betim created the Regional Commissions of Transport and Traffic (CRTT), which is composed of community members, helps shape public policies regarding public transportation and certainly contributed to increase municipality index. In 1974, Curitiba implemented a high capacity bus system, BRT (Bus Rapid Transit), with 20 miles of exclusive bus way for urban transport. Significant improvements in infrastructure, vehicles and operational measures contributed to the higher quality of service, as demonstrated by the leading position of Curitiba throughout the years analyzed.

For Campinas to improve its service effectiveness given its number of inhabitants, it should increase the fleet by 34%, 21%, 14% and 18% in the years 2005, 2006, 2007 and 2008, respectively, and in 2009 and 2010 by 20% and 18% while decreased the transported passengers by 17% and 22%, respectively to reach the maximum rank of effectiveness as shown in the table 2.

Table 2
Efficiency analysis - Campinas

Years	Average passengers carried			Fleet			Referential
	Actual	Efficiency Frontier	PI(%)*	Actual	Efficiency Frontier	PI(%)*
2005	368,557.00	368,577.00	0.00	836.00	1,124.52	34.51	Curitiba e Guarulhos
2006	456,351.00	456,351.00	0.00	1,185.00	1,434.31	21.04	Curitiba e Betim
2007	553,217.00	553,217.00	0.00	1,229.00	1,411.38	14.84	Curitiba e São Paulo
2008	595,352.00	461,970.06	-22.40	1,233.00	1,450.79	17.66	Curitiba
2009	596,488.00	494,792.43	-17.05	1,220.00	1,468.01	20.33	Serra
2010	611,794.00	472,695.36	-22.74	1,256.00	1,492.02	18.79	Curitiba

• ^*
  Potential of improvement

3.6.3. Scenario 3 -The most efficient unit

This scenario, taking the same inputs and outputs variables of scenario 1 and scenario 2, identifies the most efficient by Foroughi (2011) proposal. In scenario 2, the cities of Curitiba and Betim had the highest service effectiveness. The results of Foroughi's extreme efficiency for the years 2006, 2007, 2008, and 2010 show Curitiba with the highest rank. In 2009, São Paulo was the most efficient.

The Cooper et al. (2007) method also points Curitiba as the most efficient, which is consistent with the measure of extreme efficiency by Foroughi (2011). Finally, was calculated the average efficiency ranking for the DMUs. Figure 2 shows the DMUs ranking, and again, Curitiba is the most efficient, followed by Betim, São Paulo and Vila Velha. Campinas is ranked ninth. These results confirm the applicability of DEA for this study. The indexes, Efficiency, Effectiveness, Foroughi and ICC are shown in Table 1.

Figure 2
Performance of the seven cities

3.6.4. Efficiency vs. effectiveness analysis

Following Mouzas (2006) efficiency and effectiveness are the central terms used in measuring and evaluating the performance of organizations. The author defines efficiency as a necessary condition that reflects the company's operating margins and effectiveness as the company's ability to reach their objectives regarding the service level. Efficiency and effectiveness of public transport are essential to people's daily lives and to guide public policy.

Transportation planning should prioritize the needs of citizens, having as a main actor and the ultimate goal, although and the public policy should involve a combination of both efficiency that represents the service provider interest and effectiveness that represents the passenger interest. Following this nomenclature efficiency represents the ability of the service provider to attain the passenger with the minimum level of fleets then concerned with minimum cost or maximum revenue. The effectiveness measures the ability of the public transport system to attain the passenger with the maximum comfort or fewer passengers per fleet. Then efficiency and effectiveness are conflicting and mutually exclusive events and the public sector is responsible for promoting policies that balance both interests.

The accomplishment of a balanced equilibrium of a level of efficiency and effectiveness confronts companies as the 'issue of relativity' and can be defined as the ratio, (Mouzas, 2006):

If the performance is equal to 1 indicates the ability of a system to achieve the sense of balance of profitability and service level for the passenger; if the performance is less than 1 emphasizes a public policy focused on the passenger, and if it is greater than 1 the emphasis is on the revenue of the service provider.

From Figure 2 the public policy of Curitiba focus is on the effectiveness while São Bernardo do Campo emphasizes the efficiency. It shows that a city can be more efficient in utilizing the resource, but not effective in meeting the user requirement or can be effective, but not efficient.

Figure 2 describes the distribution of seven cities around the equilibrium line. Salvador and São Bernardo do Campo are above the equilibrium line, meaning that the efficiency is preferred over the effectiveness. Blumenau, Campinas and São Paulo are near the equilibrium line and while Curitiba and Betim focus on the effectiveness of the customer service. These analyses are important to identify the relative position of each municipality strategy and in order to guide new policies.

Conclusions

The results shown that contribution of the DEA is quite significant to the urban transport area that has three main stakeholders: the service provider, the public sector and the passenger. It provides clear results while avoiding the evaluator's interference through the assignment of weights in their own interest. The modeling process is neither complex nor laborious with the availability of several software tools. Also, the DEA results obtained with the raw data extracted directly from the database showed little difference when compared with the normalized data. On the other side, using different input and output variables significantly influences the results. This study considered three scenarios, each with a set of inputs and outputs. Each scenario yields a different ranking for the municipalities. For example, Porto Alegre and São Bernardo have the highest infrastructure efficiency but low service effectiveness. In contrast, Curitiba and Betim have high service effectiveness, but low infrastructure efficiency. Public administrators are thus mediators of the interests of the population of the municipality and the interests of carriers that aim to achieve efficiency and competitiveness.

Urban public transport plan aims to improve quality of life in cities. In addition, a policy aimed at urban public transport planning should establish regulations to protect the rights of users and ensure that the quality of the public transport is maintained.

Future studies should consider the standardization of data collection for the 50 cities considered in ICC report. Another point to include would be the places offered and places occupied instead of the number of passengers carried by a public transport fleet, and a deep analysis of a city transportation system, considering individual bus lines, should be interesting.

References

• Charnes, A., Cooper, W. W. & Rhodes, E. (1978) Measuring the efficiency of decision making units, European Journal of Operational Research, 2, 429-444.
• Chu, X., Fielding, G.J. & Lamar, B. W. (1992) Measuring transit performance using data envelopment analysis, Transportation Research Part A: Policy and Practice, 26A(3), 223-230.
• Cook, W. D. & Seiford, L. M. (2009) Data envelopment analysis (DEA), Thirty years on Invited Review, European Journal of Operational Research, 192, 1-17.
• Cook, W. D. & Zhu, J. (2008) Data Envelopment Analysis – Modeling Operational Processes and Measuring Productivity, York University, Canada.
• Cooper, W.W., Seiford, L.M. & Tone, K. (2007) Data Envelopment Analysis – A Comprehensive Text with Models, Applications, References and DEA-Solver Software, Springer.
• Dyson, R. G., Allen, R., Camanho, A. S., Podinovski, V. V., Sarrico, C. S. & Shale, E. A. (2001) Pitfalls and protocols in DEA, European Journal of Operational Research, 132, . 245-259.
• Farrell, M.J. (1957) The Measurement of productive Efficiency, Journal of the Royal Statistical Society, Series A (general ), 120 (3), 253-290.
• Foroughi, A. A. (2011) A new mixed integer model for selecting the best decision making units in data envelopment analysis, Computers & Industrial Engineering, 60, 550-554.
• Golany, B. & Roll, Y. (1989) An application procedure for DEA, OMEGA International Journal of Management Science, 17 (3), 237-250.
• Hadad, Y., Keren, B. & Hanani, M.Z. (2013) Hybrid methods for ranking DMUs that combine performance and improvement trend over successive periods, International Journal of Logistics and Management, 16 (3), 269-287.
• ICC (2011) Available at http://www.campinas.sp.gov.br/ arquivos/finances/indicadores_ competitividade.pdf, Accessed on 20 /03/2012.
  » http://www.campinas.sp.gov.br/ arquivos/finances/indicadores_ competitividade.pdf
• Jarboui, S., Forget, P. & Boujelbene, Y. (2012) Public road transport efficiency: a literature review via the classification scheme, Public Transp 4:101–128.
• Karlaftis, M. (2004) A DEA approach for evaluating the efficiency and effectiveness of urban transit systems, European Journal of Operational Research, 152, 354-364.
• Karlaftis. M.G. & Tsamboulas, D. (2012) Efficiency measurement in public transport: Are findings specification sensitive? Transportation Research Part A, 46, 392-402.
• Mouzas, S. (2006) Efficiency Versus Effectiveness in Business Network, Journal of Business Research, 59 (10/11), 1124-32.
• Novaes, A. G. (2001) Rapid-Transit Efficiency Analysis with the Assurance-Region DEA Method, Pesquisa Operacional, 21, 179-197.
• Ozbek, M. E., Garza, J. M. E. & Triantis, K. (2009) Data Envelopment Analysis as a Decision-Making Tool for Transportation Professional, Journal of Transportation Engineering, 135 (11), 822-831.
• Pina, V.; Torres, L: (2001) Analysis of the efficiency of the local government service delivery. An application to urban public transport, Transportation Research Part A, (35), , 929-934.
• Tamagawa, D., Taniguchi, E. & Yamada, T. (2010) Evaluating city logistics measures using a multi-agent model, Procedia Social and Behavioral Sciences, 2, 6002-6012.

Publication Dates

• Publication in this collection
  Jul-Sep 2015

History

• Received
  11 June 2014
• Reviewed
  17 Nov 2015
• Accepted
  20 Apr 2015