Multicriteria approach to data envelopment analysis

Multicriteria approach to data envelopment analysis

Hélcio Vieira Junior

Instituto Tecnológico de Aeronáutica (ITA), São José dos Campos - SP, [e-mail]; [e-mail]

ABSTRACT

With the aim of making Data Envelopment Analysis (DEA) more acceptable to the managers' community, the Weights Restrictions approaches were born. They allow DEA to not dispose of any data and permit the Decision Maker (DM) to have some management over the method. The purpose of this paper is to suggest a Weights Restrictions DEA model that incorporates the DM preference. In order to perform that, we employed the MACBETH methodology as a tool to find out the bounds of the weights to be used in a Weights Restrictions approach named Virtual Weights Restrictions. Our proposal achieved an outcome that has an expressive correlation with three widely used decision-aids methodologies: the ELECTRE III, the SMART and the PROMETHEE I and II. In addition, our approach was able to join the most significant outcomes of all the above three Multicriteria decision-aids methodologies in one unique outcome.

Keywords: data envelopment analysis (DEA); decision support systems; environmental studies.

RESUMO

Com o objetivo de fazer a Análise Envoltória de Dados (DEA) mais aceitável pela comunidade gerencial, as abordagens de Restrição aos Pesos foram criadas. Estas abordagens fazem com que a DEA não descarte nenhum dado e permitem que o Decisor (DM) tenha alguma gerência sobre o método. O objetivo deste artigo é sugerir um modelo de restrição aos pesos que incorpore as preferências do DM. Para realizar isto, nós empregamos a metodologia MACBETH como ferramenta para descobrir os limites dos pesos a serem utilizados na abordagem de restrição aos pesos chamada "Restrição aos Pesos Virtuais". Nossa proposta alcançou um resultado que apresenta uma correlação expressiva com três metodologias de apoio à decisão amplamente utilizadas: o ELECTRE III, o SMART e o PROMETHEE I e II. Adicionalmente, nossa abordagem foi capaz de reunir os resultados mais significativos de todas estas três metodologias de apoio à decisão em um único resultado.

Palavras-chave: análise envoltória de dados (DEA); sistemas de apoio à decisão; estudos ambientais.

1. Introduction

Although being the goal of the original Data Envelopment Analysis (DEA) proposal, the total freedom of variation of weights has been criticized by several researchers. This complete weight flexibility leads to the following problems: most of the time, the unrestricted DEA program solution attributes very low values to some inputs and/or outputs, disregarding, this way, these data; in addition, this freedom of the weights may contradict prior available information of the Decision Maker (DM).

A lot of research has been done to find an efficient way to oppose these problems: Thompson et al. (1990) proposed the use of Assurance Regions (AR). Shang & Sueyoshi (1995) suggested the use of the Analytic Hierarchy Process (AHP) (Saaty, 1980) to derive relative weights among the outputs for each input; the bounds of a specific output in the Virtual Weights Restrictions DEA model (Wong & Beasley, 1990) will be the maximum and minimum weights reached by this output in the different inputs. Sinuany-Stern et al. (2000) ran "DEA for every pair of units, two units at a time, ignoring the others; from the results of the first stage, they create a pairwise comparison matrix to which they apply a single level AHP, which provides a full ranking scale of all the units". Liu (2003) used a different approach integrating the AHP and DEA; he got two weights: one from AHP and one from classical DEA, and them he made a new weight by mixing these two; this new weight is used to rank the DMUs. Takamura & Tone (2003) proposed to combine the Assurance Regions and AHP; they used AHP to derive the weights of the criteria for each DM and then used as AR's bounds the maximum and minimum rates between the different weights proposed by each DM. Other authors proposed an approach that seems to be similar to ours (Soares de Mello et al., 2002). As they did not name the Weights Restrictions approach they used, it is very difficult to make any comparison between our work and theirs.

The main methodological difference of these previous works in relation to ours is that we utilize MACBETH (Bana e Costa et al., 2003) as a MCDA (Multi-Criteria Decision Analysis) tool to find out the weights' bounds to be applied in the Virtual Weights Restrictions DEA model. While Shang & Sueyoshi (1995) do not work with only one input, Sinuany-Stern et al. (2000) do not incorporate the DM preferences, and Takamura & Tone (2003) do not work with only one DM, our approach does not have any of these inconveniences. In addition, our algorithm was able to join the most significant outcomes of three widely used decision-aid methodologies (the ELECTRE III, the SMART and the PROMETHEE I and II) in one unique outcome, fact that was not registered previously.

The paper is organized as follows: In the second Section, we will review the DEA formulation. In third Section, the Virtual Weights Restrictions approach will be presented. The MACBETH methodology will be detailed in the fourth Section. The fifth Section will describe our proposal by utilizing a numerical example. The sixth Section will analyze the results of the suggested methodology and the seventh Section comprehends our conclusions.

2. Data Envelopment Analysis

The most basic DEA model is the CCR, which was proposed by Charnes, Cooper & Rhodes (1978). The basic idea of the CCR model is the following: the efficiency of an observed DMU (Decision Making Unit), which is the organization to be evaluated, can be measured by the ratio output per input, i.e., how well a DMU can convert its inputs into its outputs. As we usually work in situations where we face multiples inputs and outputs, we are going to form a unique virtual output and a unique virtual input, for the observed DMU o, by the yet unknown weights υ_i and u_r:

where x_io is the data of the input i on the DMU o, y_ro is the data of the output r on the DMU o, υi is the weight of the input i and u_r is the weight of the output r.

By using linear programming (LP), we can find the weights by maximizing the ratio:

The variable θ_o will be the efficiency of the DMU o. By acting this way, we will have the maximum possible efficiency for the observed DMU.

If we add a constraint limiting the efficiency to be less or equal to 1 for every DMU, we will have the formulation of the CCR model:

In order to transform the above fractional program into a linear program, we will force the virtual input of the observed DMU to be equal to 1, obtaining, this way, the final formulation of the CCR model:

For an extended discussion of the CCR and several other DEA models, readers are referred to Copper et al. (2000).

3. Virtual Weights Restrictions Approach

The use of weights restrictions was first proposed by Thompson et al. (1986). Additional approaches for restricting weights were proposed by several researchers. The main issue with these methods (Simple Absolute Weights Restrictions or the Assurance Regions) is that they restrict the weights directly. As the weights are dependent of the units of measurement, it leads to a dimensional problem. In addition, Allen et al. (1997) presented that it is very difficult to get reliable rates of substitution (Assurance Regions) or to find out clear bounds directly on individual weights (Simple Absolute Weights Restrictions). With the purpose of avoiding these problems, Wong & Beasley (1990) proposed the use of virtual weights.

The virtual weights are based on the use of proportions, and then, are dimensionless. By definition, the virtual output is the weighted sum of all outputs, therefore: ( u_ry_ro/ Virtual Output). It is easy to see that the ratio u_ry_ro/Virtual Output represents the proportion of the total output for the observed DMU devoted to output measure r. By setting lower and upper limits [ LOr, UOr](0 < LO_r<UO_r< 1 ) for each output r, we can restrict the importance this output has in the virtual output:

Proceeding the same way as described above in relation to the inputs, we will have:

where:0< LI_i< UI_i<1i.

Wong and Beasley proposed three different approaches for the utilization of the virtual weights:

a) Restrict only the observed DMU;

b) Restrict all the DMUs; and

c) Restrict the observed DMU and an "average" DMU.

In this work, we are going to use the approach (a), which will be denoted as Virtual Weights Restrictions approach.

Additional information about Virtual Weights Restrictions feasibility can be found in Estellita Lins et al. (2007).

4. The MACBETH Methodology

The MACBETH (Measuring Attractiveness by a Categorical Based Evaluation Technique) (Bana e Costa et al., 2003) was proposed in the early 1990's. It is a multicriteria decision analysis approach that by qualitative judgments about differences of value can quantify the relative attractiveness of options. According to its authors, the MACBETH is a humanistic, interactive and constructive methodology that adopts the additive value aggregation model with the objective of obtaining a cardinal scale of attractiveness of options.

Let A={a₁, a₂,...,a_n} be a finite set of options. We say that the option a_i is more attractive than a_j (a_iPa_j) if it is possible to associate to these options numbers v(a_i) and v(a_j) such that v(a_i)>v(a_j). Similarly, we say that the option a_i is as attractive as a_j (a_iIa_j) if v(a_i)=v(a_j).

The methodology starts by asking the DM to verbally judge the difference of attractiveness between each pair of options (a_i, a_j) ∈ A by choosing one of the semantic categories listed in Table 1.

For example, if the DM says that a_i is more attractive than and the difference of attractiveness is weak, then (a_i, a_j) ∈ C₂.

After obtaining the data, MACBETH will propose, if possible, a scale that satisfies:

With the purpose of obtaining such scale, MACBETH will solve the following LP:

The MACBETH implementation, which is called M-MACBETH, offers as outcomes the MACBETH basic scale and the MACBETH transformed scale. The latter is a positive affine function of the former. If it is not possible to construct a scale satisfying (7) and (8), the software M-MACBETH will point out the inconsistencies of the judgment matrix and will suggest some options to solve these contradictions.

We elected MACBETH as our MCDA methodology for two reasons: its friendliness in obtaining the weights' bounds and because of its very good mathematical basis.

5. Our proposal: a numerical example

The scope of our proposal is to use MACBETH to obtain the bounds of the weights to be applied in the Virtual Weights Restrictions DEA model. In this section, we are going to utilize the data set that was evaluated in the paper of Hokkanen & Salminen (1997). Table 2 has the criteria and the normalized data used to choose the location of a solid waste management system in Oulu, Finland. These data were normalized by dividing each alternative data in one criterion by the criterion mean. In their paper, Hokkanen and Salminen interviewed 113 DMs and they had as answers the following weights for the various criteria: Cost: 0.2700; Global Effects: 0.0160; Health Effects: 0.0960; Acidificative Releases: 0.0470; Surface Water Dispersed Releases: 0.0900; Technical Reliability: 0.2600; Employees: 0.0500; and Resource Recovery: 0.1400. For an extended discussion about the definition of the criteria, the acquirement of the data and the achievement of the weights, readers are referred to the original paper (Hokkanen & Salminen, 1997). We considered the minimizing criteria as inputs and the maximizing criteria as outputs.

We started by constructing a judgment matrix in the MACBETH that produces as result the weights found by Hokkanen and Salminen. In order to perform that, we set, in the software M-MACBETH, the limit values of the inputs as shown in Figure 1.

Figure 2 illustrates the inputs judgment matrix that was able to produce the desired weights. With the intention of normalizing the outcomes, we generated a MACBETH transformed scale by defining the current scale to have the sum of scores 100. The result, shown in Figure 3, displays the new scale and the bounds that these weights can reach without disagreeing to the judgment matrix. For example: the new value of the criterion Cost is 52.02 and it can vary from 41.43 to 99.87. The same practice was done in relation to the outputs.

In real world applications, we would construct a matrix by questioning the DMs about their judgments, and the result of this process would be the weights. As a matter of fact, what we did in the previous steps was to simulate the interviewing procedure by finding out a judgment matrix that was able to translate the implicit value function of the DMs into the resultant weights of the Hokkanen and Salminen's work, i.e., instead of constructing a matrix by interviewing the DMs and then have the weights as outcomes, we started with the weights and then constructed a matrix that resulted in these weights.

The MACBETH outcomes (bounds divided by 100) for all the inputs and outputs criteria are listed in Table 3. We normalized the MACBETH results (fixed the sum in 100 and then divided the results by 100) in order to allow the outcomes to be used as lower and upper bounds in the Virtual Weights Restrictions approach and to keep the decimal place precision in four digits (the MACBETH standard is two digits).

Observe that the upper bound of the criterion Surface is greater than the lower bound of the criterion Health. This fact also occurs with the following pairs of criteria: {(Surface, Acidificative), (Acidificative, Global), (Feasibility, Employee), (Employee, Recovery)}. This situation can lead to a DEA solution that contradicts the MACBETH rank of preference. In order to avoid this, we are going to add some constraints to the Virtual Weights Restrictions DEA model:

The next step was to run the Virtual Weights Restrictions DEA model augmented by restrictions (10). We named this formulation as Multicriteria DEA model:

where LO_k and UO_k are, respectively, the lower and upper limits of output k; LI_k and UI_k are, respectively, the lower and upper limits of input k; I^* and O^* are the sets of inputs and outputs, respectively, in which the lower and upper bounds do not obey the MACBETH preference rank.

The results of the Multicriteria DEA model are displayed in Table 4.

6. Multicriteria DEA Model Results' Analysis

Salminen et al. (1998) studied three widely used decision-aids Multicriteria models: the ELECTRE III, the SMART and the PROMETHEE I and II. In Table 4 of their paper, they ranked, with the above three models, the same data we used in our numerical example. As our Multicriteria DEA model indicated three DMUs as efficient, we are going to compare them to the three first DMUs ranked by the work of Salminen et al. (1998). This comparison is shown in Table 5.

Observe that the first DMU ranked by the ELECTRE III and the top two DMUs ranked by the SMART and by the PROMETHEE I and II are the ones our methodology points as efficient. Another aspect that deserves attention is that the bottom four DMUs ranked by all decision-aid Multicriteria methodologies and by the Multicriteria DEA model are exactly the same.

Table 6 shows the Pearson and the Kendall's Tau-b correlations existent among our model and the three decision-aids Multicriteria ones. The Multicriteria DEA model correlation with all the three models was significant at the 0.01 level (2-tailed) and had 22 valid cases.

We made another numerical study with the data presented by Olson (2001). In his work he used the SMART, the SMARTER (centroid approach), the PROMETHEE I (type 6 criterion - Gaussian distribution based on the standard deviation) and the PROMETHEE II to analyze the data of various seasons of American major league professional baseball. The resultant rank of Olson's paper was compared with the outcomes of the Multicriteria DEA model. We obtained total agreement in all the eight positions of these ranks.

Our numerical studies showed that the DEA characteristic of choosing the best weight for each input/output emulates, in some way, the different preference modeling structure of several Multicriteria models.

As seen in the last two sections, the main advantage of our proposal is:

• The DM does not have to establish weights directly (job that is known to be very difficult) in order to apply a Multicriteria method;

• The DM does not have to blindly accept the outcomes of DEA "black box".

In other words, the DM can have the best of two worlds: DEA's needless of setting up weights and MACBETH first steps' ease of use (the achievement of a pre-cardinal scale).

7. Conclusions

We proposed the utilization of the MACBETH methodology to define the bounds of the weights for the Multicriteria DEA model. The outcomes of our proposal were compared with traditional Multicriteria decision-aids methodologies. This comparison demonstrated evidences of significant correlation among our methodology and the traditional ones. Besides, the Multicriteria DEA model was able to join the most significant outcomes of three traditional Multicriteria decision-aid methodologies in one unique outcome, as seen in last section. This result elects our algorithm as a good approach to generate a wide overview of Multicriteria decision-aid methodologies results.

In our numerical example, we generated a judgment matrix which had as outcomes the weights found in the Hokkanen and Salminen's paper. In real world applications, we usually do not have these weights clearly defined by the DM(s). The obtainment of these weights has demonstrated to be a thorny issue. This fact justifies the use of MACBETH, since this methodology has proven to be an excellent tool for acquiring the weights.

The traditional DEA can be used if the DM is not concerned with the "black box" mathematic behind DEA or if he/she does not want to express his/her prior information about the problem. Unfortunately, the incorporation of the DM's preference into DEA models seems more acceptable by the managers' community. We provided one methodology that does it.

On the other hand, using Multicriteria methodologies can be seen as very difficult by some DM. Our proposal used only the initial steps of a well-known and respected Multicriteria model (MACBETH), facilitating, this way, the DM job.

For future works, we should test the Multicriteria DEA model in further real world problems and compare its results with different decision-aid Multicriteria methodologies.

Recebido em 10/2006; aceito em 03/2008 após 1 revisão

Received October 2006; accepted March 2008 after one revision

Appendices

• (1) Allen, R.; Athanassopoulos, A.D.; Dyson, R.G. & Thanassoulis, E. (1997). Weights Restrictions and Value Judgments in Data Envelopment Analysis: Evolution, Development and Future Directions. Annals of Operations Research, 73, 13-34.
• (2) Bana e Costa, C.A.; De Corte, J.-M. & Vansnick, J.C. (2003). MACBETH LSE OR Working Paper 03.56. London School of Economics, UK.
• (3) Charnes, A.; Cooper, W.W. & Rhodes, E. (1978). Measuring the Efficiency of Decision Making Units. European Journal of Operational Research, 2, 429-444.
• (4) Estellita Lins, M.P.; Moreira da Silva, A.C. & Lovell, C.A.K. (2007). Avoiding infeasibility in DEA models with weight restrictions. European Journal of Operational Research, 181, 956-966.
• (5) Hokkanen, J. & Salminen, P. (1997). Choosing a Solid Waste Management System Using Multicriteria Decision Analysis. European Journal of Operational Research, 98, 19-36.
• (6) Liu, C.C. (2003). Simulating Weights Restrictions in Data Envelopment Analysis by the Subjective and Objective Integrated Approach. Web Journal of Chinese Management Review, 6(1), 68-78.
• (7) Olson, D.L. (2001). Comparison of Three Multicriteria Methods to Predict Known Outcomes. European Journal of Operational Research, 130, 576-587.
• (8) Saaty, T.L. (1980). The Analytic Hierarchy Process McGraw-Hill.
• (9) Salminen, P.; Hokkanen, J. & Lahdelma, R. (1998). Comparing Multicriteria Methods in the Context of Environmental Problems. European Journal of Operational Research, 104, 485-496.
• (10) Shang, J. & Sueyoshi, T. (1995). A Unified Framework for the Selection of a Flexible Manufacturing System. European Journal of Operational Research, 85, 297-315.
• (11) Sinuany-Stern, Z.; Mehrez, A. & Hada, Y. (2000). An AHP/DEA Methodology for Ranking Decision Making Units. International Transactions in Operations Research, 7, 109-124.
• (12) Soares de Mello, J.C.C.B.; Lins, M.P.E.; Soares de Mello, M.H.C.S. & Gomes, E.G. (2002). Evaluating the performance of calculus using operational research tools. European Journal of Engineering Education, 27(2), 209-218.
• (13) Takamura, Y. & Tone, K. (2003). A Comparative Site Evaluation Study for Relocating Japanese Government Agencies Out of Tokyo. Socio-Economic Planning Sciences, 37, 85-102.
• (14) Thompson, R.G.; Langemeier, L.N.; Lee, C.T. & Thrall, R.M. (1990). The Role of Multiplier Bounds in Efficiency Analysis with Application to Kansas Farming. Journal of Econometrics, 46, 93-108.
• (15) Thompson, R.G.; Singleton, F.D.; Thrall, R.M. & Smith, B.A. (1986). Comparative Site Evaluations for Locating a High-Energy Physics Lab in Texas. Interfaces, 16, 35-49.
• (16) Wong, Y.-H.B. & Beasley, J.E. (1990). Restricting Weight Flexibility in Data Envelopment Analysis. Journal of the Operational Research Society, 41(9), 829-835.

Publication Dates

• Publication in this collection
  14 Nov 2008
• Date of issue
  Aug 2008

History

• Received
  Oct 2006
• Accepted
  Mar 2008