Goal programming associated with the non-archimedean infinitesimal: a case study applied in the agricultural sector

Abstract

Paper aims

This work presents a multi-objective method based on goal programming associated with non-Archimedean infinitesimal (NAI) (Improved Weighted Goal Programming method, input-oriented IWGP-MCDEA-BCC).

Originality

The MCDEA is applied for the first time in a large agricultural company (over 11 hectares). A new method is proposed which consists of an improvement on input-oriented WGP-MCDEA-BCC approaches.

Research method

The performance of the proposed method was compared to the classic Data Envelopment Analysis and the Weighted sum Goal Programming methods. The case study comprises an agricultural company located in the São Francisco Valley (Brazil).

Main findings

The proposed method can help decision makers to improve efficiency in the production of different types of fruits.

Implications for theory and practice

The proposed model is capable of overcoming the deficiencies associated with classical DEA and allows the company to identify effective ways to increase productivity by reducing input costs.

Keywords
Data envelopment analysis; Multiple criteria data envelopment analysis; Variable return to scale; São Francisco Valley

1. Introduction

In recent years, Brazil has stood out in the ranking of the world's major mango producers. In 2019, 67,328 hectares were cultivated in the country, reaching a production of 1,414,338 tonnes. Productivity reached 21,007 tonnes per hectare, with an estimated production value of US$ 1,639.250 (Instituto Brasileiro de Geografia e Estatística, 2019). Although cultivated throughout the country, mango production is concentrated in the irrigated fruit pole of the São Francisco Valley where 85% of the mangoes exported to the European Union and the United States are grown. The export of mangoes from São Francisco Valley was worth about US$ 227 million in 2019 (Instituto Brasileiro de Geografia e Estatística, 2019).

The main problems in the mango market involve planning, control, production scheduling and export logistics. Multiple Criteria Data Envelopment Analysis (MCDEA) can contribute to addressing these challenges as it is a decision-making tool designed to support decision makers in relation to conflicting alternatives (Adler & Yazhemsky, 2010). The aim of DEA is to identify a set of efficient and inefficient units (Charnes et al., 1978). MCDEA methods determine the best alternative from a set of available alternatives or from a group of optimal alternatives. However, in such problems, the solution depends on the preferences of the decision maker (Ghasemi et al., 2014).

The methods associated with MultiCriteria Decision Analysis (MCDA) or Multicriteria Decision Making (MDM) are classified into three major approaches, namely: i) Single Synthesis Criterion (e.g., Multi-Attribute Utility Theory, MAUT) (Nara et al., 2019); ii) Out ranking (most applications involve ELECTRE, Elimination Et Choix Traduisant la Realité, or PROMETHEE, Preference Ranking Organisation Method for Enrichment Evaluation, family methods) (Marttunen et al., 2017) and iii) Iterative methods based on Multiple Objective Linear Programming (MOLP) (Rubem et al. 2017; Almeida et al., 2015). Iterative methods based on MOLP permit a less subjective participation of the decision maker mainly due to the determination of input and output weights, which does not exclude the possibility of analyzing and revising these weights using the knowledge of the production manager (Almeida et al., 2015; Nara et al., 2019; Marttunen et al., 2017). On the other hand, subjective aspects inherent to the decision maker are strongly present in methods based on the Single Synthesis Criterion and Out Ranking (Ângulo-Meza et al., 2019; Silva et al., 2019) which in some cases justifies the choice of methods based on Multiple Criteria Data Envelopment Analysis (MCDEA) (Rubem et al. 2017; Hatami-Marbini & Toloo, 2017).

Although DEA is a powerful management tool, the low level of discrimination among the Decision Making Units (DMUs) is a limitation (shortcoming) which has already been mentioned in other works (Pereira & Mello, 2015). This shortcoming occurs when the number of DMUs is insufficient in relation to the total number of inputs and outputs, thus not following the Golden Rule (Banker et al., 1989). This rule establishes that the number of DMUs should be the maximum between triple the total number of variables (inputs and outputs) and the product between the number of inputs and outputs (Ângulo-Meza et al., 2019; Iqbal & Sial, 2018; Silva et al., 2017a).

The low level of discrimination is often referred to as the “curse of dimensionality” (Daraio & Simar, 2007). The lack of discriminatory power can limit the conclusions and decisions from the analyzed DMUs and the application of other techniques (Analytic Hierarchy Process – AHP; Analytic Network Process – ANP; Multi-Attribute Value Theory – MAVT; Technique for Order Preference by Similarity to Ideal Solution – TOPSIS; ELECTRE; Multi-Attribute Utility Theory - MAUT and PROMETHEE) in these units (Ghasemi et al., 2019; Marttunen et al., 2017).

Some subsequent improvements to the MCDEA model have been proposed by Bal & Örkcü (2007), Bal et al. (2010), Ghasemi et al. (2014), Rubem et al. (2017), Hatami-Marbini & Toloo (2017) and Silva et al. (2019) with the purpose of simultaneously optimizing the objective functions of the Li & Reeves model (1999). Bal & Örkcü (2007) developed the GPMCDEA (Goal Programming MCDEA) model. Bal et al. (10) proposed models based on weighted goal programming (GPDEA models, Goal Programming DEA), which consider constant return and scale variables.

Ghasemi et al. (2014) proposed the BiO-MCDEA (Bi-Objective MCDEA) model and presented a critical analysis of GPDEA models related to obtaining null multipliers for all DMUs. However, in certain applications (Rubem et al., 2017; Silva et al., 2019; Ghasemi et al., 2019) the BiO-MCDEA model did not solve the discrimination problems involving the DMUs and the unrealistic weight distribution of the inputs and outputs. It is important to emphasize that the failure associated with obtaining unrealistic weights leads to an inadequate result regarding the effect of inputs and outputs in each DMU, which reduces the low level of discrimination of these units.

Hatami-Marbini & Toloo (2017) present a critical analysis identifying three failures in the BiO-MCDEA model and propose the following approaches: the lower limit Non-Archimedean Infinitesimal $\left(\text{NAI},\epsilon \right)$ model for the variables (inputs and outputs); extended model (Extended-MCDEA) based on the BiO-MCDEA-CCR model; and a BCC-DEA minisum model.

Rubem et al. (2017) and Rubem (2016) extended the work of Ghasemi et al. (2014) and proposed the models WGP-MCDEA-CCR and WGP-MCDEA-BCC oriented to input and output. Silva et al. (2019) present a new model from MCDEA - CCR, based on super-efficiency, and compare the performance with the models of Rubem et al. (2017), Hatami-Marbini & Toloo (2017), and Ghasemi et al. (2014). Their model shows a better discrimination of DMUs and a weight dispersion statistically equal to that obtained by other MCDEA models.

General applications of DEA are presented in the works of Emrouznejad & Yang (2018); Aldamak & Zolfaghari (2017); Tan et al. (2019); Li et al. (2019). Iqbal & Sial (2018), Raheli et al. (2017) and Aydin & Unakitan (2018) analyzed technical efficiency (applications of DEA) in the production of grains and fruit. Few local works (e.g., Passos et al., 2020; Silva et al., 2017b.; Silva & Sampaio, 2002) are specifically related to the evaluation of the technical efficiency of fruit producers in the São Francisco Valley region.

Works involving applications of MCDEA to date have been associated with rice production, management of animal feed nutrients, analysis of water quality, ports and dairy cattle (Krcmar & Van Kooter, 2008; Andrade et al., 2019). There are no works involving applications of MCDEA in fruit production in the Sao Francisco Valley, Brazil.

When the Golden Rule is not followed, as usually happens in real cases, the low level of discrimination of DMUs and the distribution of unrealistic weights remain the main problems in the application of classic DEA models (Hatami-Marbini & Toloo, 2017). Rubem et al. (2017) addressed some WGP-MCDEA models in an attempt to deal with these two problems, but the application of these models in real cases, such as in fruit production, continued to obtain an unrealistic weight distribution.

Table 1 presents related works involving DEA applications, highlighting, in each case, the type of problem addressed (A: the low level of discrimination of the DMUs and B: the unrealistic weight distribution) and drawbacks which make the application of the method in the analyzed problem (agricultural sector) unfeasible. Some of the proposed approaches do not satisfy the golden rule (Banker et al., 1989) and therefore are unable to effectively address the low level of discrimination of DMUs. None of the methods proposed in the literature were able to simultaneously minimize (or mitigate) the two DEA failures (A and B) in problems related to Variable Return to Scale (BCC), which makes it difficult to apply the method in real cases. This work provides contributions related to the application of the inverted frontier/DEA, correcting the problem of the unrealistic distribution of input and output weights in the classical DEA results which suggest the equality of efficiency between DMUs.

Table 1
DEA applications and shortcomings.

Authors	Method/approach	Type of problem addressed:	Drawbacks and feasibility of application (agricultural sector)
		A: low level of discrimination among DMUs
		B: unrealistic weight distribution
Ueda & Hoshiai (1997), Adler & Golany (2001)	PCA (Principal component analysis) – dimensionality reduction (number of inputs and outputs of the problem).	A	The application of the PCA can lead to a reduced number of inputs and outputs without complying with the Golden Rule (Banker et al., 1989)
Simar & Wilson (2001)	Selection of inputs and outputs using bootstrap (one of the most commonly used resampling methods).	A	The Spearman's Correlation Test is most appropriate for the selection of inputs and outputs (Silva et al., 2017a).
Pastor et al. (2002)	Radial DEA models.	A	Does not follow the Golden Rule (Banker et al., 1989)
Fanchon (2003)	Variable selection method for measuring efficiency over time.	A	Does not follow the Golden Rule (Banker et al., 1989)
Ruggiero (2005)	Selection of variables based on regression analysis.	A	This method does not follow the Golden Rule (Banker et al., 1989) and it is suitable for production processes with few inputs and outputs.
Jenkins & Anderson (2003)	Selecting variables based on partial covariance	A	Apply only to problems involving Constant Return to Scale (CCR) (input-oriented)
Wagner & Shimshak (2007)	Progressive or “STEPWISE” Selection Process.	A	Identification of only one model (“core” model) which relates a single input and output variable to efficiency through a mechanistic model
Bal & Örkcü (2007)	(Goal Programming - GPMCDEA) – lexicographic-based approach.	B	Null multipliers for all DMUs (Ghasemi et al., 2014). The failure related to the unrealistic weight distribution has not been mitigated
Bal et al. (2010)	GPDEA models (Goal Programming DEA). GPDEA-CCR-input e GPDEA-BCC-input. Weighted sum-based approach.	A and B (*)	Null multipliers for all DMUs and solutions associated with a single objective/criterion (Ghasemi et al., 2014).
Ghasemi et al. (2014)	Modelo bi-objetivo MCDEA (BiO-MCDEA) para o CCR – input. Weighted sum-based approach.	A and B	Apply only to problems involving Constant Return to Scale (CCR) (input-oriented).
Rubem et al. (2017)	Modelo WGP-MCDEA-GP para o CCR-Input, CCR-output, BCC-input e BCC-output. Weighted sum-based approach.	A	Did not solve the failure related to obtaining unrealistic weights.
Hatami-Marbini & Toloo (2017)	Extended model (Extended-MCDEA). BiO-MCDEA-CCR model and input-oriented BCC-DEA minisome. Optimal lower limit for input and output weights (non-Archimedian infinitesimal -ε).	A and B	Apply only to problems involving Constant Return to Scale (CCR) (input-oriented)
Silva et al. (2019)	New MCDEA – CCR model, based on super-efficiency.	A and B	Apply only to problems involving Constant Return to Scale (CCR) (input-oriented)

• ^*
  The two failures related to the DEA (low level of discrimination and unrealistic weight distribution) have not been mitigated.

On the other hand, in real applications, decision makers (DMs) are usually not interested in simply classifying DMUs into efficient and inefficient. They often want to rank all units according to their technical efficiency. Aldamak & Zolfaghari (2017) presented a review about ranking methods of DMUs involving DEA and categorized the following categories of approaches: cross efficiency, super efficiency, benchmarking, statistical techniques, inefficient DMUs, MultiCriteria Decision Analysis (MCDA), inefficient frontier, virtual DMUs, DM interference, and fuzzy-based methods. Most of these approaches are also customized for application in problems involving Constant Return to Scale (CCR).

This work presents an innovative approach that involves the integration of the WGP-MCDEA-BCC model oriented to input (Rubem, 2016 and Rubem et al., 2017) with the maximum value model for $\text{NAI}$ (Amin & Toloo, 2007; Hatami-Marbini & Toloo, 2017). A multi-objective model based on goal programming associated with NAI is proposed, which comprises an improvement on the input-oriented WGP-MCDEA-BCC model (Improved Weighted Goal Programming method, input-oriented IWGP-MCDEA-BCC). The proposed model enables the use of the lower limit NAI for the inputs and outputs together with a Variable Return to Scale (BCC) approach which assumes variable returns to scale, unlike the traditional model limited to applications involving Constant Return to Scale (CCR).

The main contributions of this work are:

1. 1

   The MCDEA is applied for the first time in a large agricultural company (over 11 hectares);

2. 2

   Proposal of a new method (IWGP-MCDEA-BCC) which consists of an improvement on the input-oriented WGP-MCDEA-BCC approaches aimed at goal programming in agricultural production. This new method is capable of overcoming the shortcomings associated with classical DEA which comprise the low level of discrimination of DMUs and an unrealistic weight distribution (Ghasemi et al., 2014; Pereira & Mello, 2015; Banker et al., 1989; Ângulo-Meza et al., 2019; Iqbal & Sial, 2018; Silva et al., 2017a; Rubem et al. (2017); Hatami-Marbini & Toloo, 2017).

The paper is structured as follows: section two presents the classic DEA and the WGP-MCDEA model oriented to input. Section three presents the proposed method and section four presents the results and discussions.

2. DEA and MCDEA models

2.1. Classic DEA models

Data Envelopment Analysis (DEA) was introduced over 40 years ago by Charnes et al. (1978) and consists of a method based on linear programming which can be used to evaluate the efficiency of a set of productive units called DMUs (Decision Making Units).

There are two classic models in DEA: CCR (Constant Return to Scale, proposed by Charnes, Cooper and Rhodes) and BCC (Variable Return to Scale, proposed by Banker, Charnes and Cooper) (Charnes et al., 1978; Banker et al., 1984).

The application of the classic DEA models involves the use of the Golden Rule proposed by Banker et al. (1989). The input-oriented BCC multiplier model (Banker et al., 1984) comprises Equations 1-4. This model is an extension of the CCR model since the hypothesis of variable returns to scale is assumed and the axiom of proportionality between inputs and outputs is replaced by the axiom of convexity.

$Max{E}_0={{\displaystyle \sum }}_{j=1}^s{u}_j{y}_{j0}+u_{*}$ (1)

$s.t.{{\displaystyle \sum }}_{i=1}^r{v}_i{x}_{i0}=1$ (2)

${{\displaystyle \sum }}_{j=1}^s{u}_j{y}_{jk}-{{\displaystyle \sum }}_{i=1}^r{v}_i{x}_{ik}+u_{*}\le 0,\forall k$ (3)

$u_j,v_i\ge \mathrm{0,}\forall j,i$ (4)

Each DMU_k$\left(k=\mathrm{1,}\dots ,n\right)$ consumes r inputs $x_{ik}\left(i=\mathrm{1,}\dots ,r\right)$ for the production of $s$ outputs $y_{jk}\left(j=\mathrm{1,}\dots ,s\right)$. $y_{jk}\text{and}{x}_{ik}$ are the production of output $j$ and the consumption of input $i$ related to the DMU_k respectively. In turn, $u_j\text{and}{v}_i$ are the decision variables of the problem that weigh the outputs $j$ and $i$, respectively. $E_0$ is the efficiency of the DMU₀ (a given DMU). The axiom of convexity is accomplished by incorporating an additional decision variable $\left(u_{\text{*}}\in ℝ\right)$ in the input orientation. This new variable is considered a scale factor (Rubem et al., 2017).

The model (Equations 1-4) is oriented to input and aims to minimize the use of inputs, without changing the level of production. A DMU is efficient when its efficiency is equal to the unit $\left(E_0=1\right)$ which means that the constraint related to that DMU is active (without clearance). The insertion of the scale factor $\left(u_{*}\right)$ implies that the DMU will be efficient when $E_0=1$ and inefficient when $E_0<1$.

The Inverted Frontier is one of the initial techniques that is associated with classic DEA methods (CCR and BCC) to solve the problem of unrealistic weight distribution of inputs and outputs. For a DMU to be highly efficient, it must have a high degree of efficiency in relation to the optimistic frontier (classic DEA model) and a low degree of efficiency in relation to the inverted frontier. In this way, all variables are analyzed, without attributing any subjective weight to any criterion (Entani et al., 2002). The concept of the inverted frontier (Yamada et al., 1994; Entani et al., 2002) comprises a pessimistic assessment of the analyzed DMUS, represented as inefficiency. In this case, each input is exchanged for its respective output in the classic DEA model (equations 1 - 4) (Shen et al., 2016; Cao et al., 2016).

A composite efficiency index is used for the DMU efficiency ranking (Mello et al., 2008) (Equation 10). This index is the average between efficiency in relation to the standard frontier (classic DEA) and inefficiency in relation to the inverted frontier (Eq 5). The ranking of the DMUs is performed based on the normalization of the composite efficiency indexes, considering the highest value.

$\left(Compositeefficiency=\frac{standardefficiency+\left(1-invertedfrontier\right)}{2}\right)$ (5)

If the golden rule is not satisfied, the two approaches (CCR and BCC) may have the following shortcomings:

1. 1

   Inadequate discrimination of efficient DMUs, i.e. the solutions obtained identify many efficient DMUs (Silva et al., 2017a).

2. 2

   Unrealistic weight distribution, i.e. some DMUs can be classified as efficient because they have very high weights in a single output and/or input. These high weights are inconsistent and therefore undesirable (Ghasemi et al., 2014; Hatami-Marbini & Toloo, 2017).

2.2. MCDEA models

Li & Reeves (1999) developed the MCDEA model using a multi-objective linear programming approach. The authors included two additional objective functions in the input-oriented CCR model and these are considered separately, not in order of priority. The works of Bal & Örkcü (2007), Bal et al. (2010), Ghasemi et al. (2014), Hatami-Marbini & Toloo (2017), Rubem et al. (2017) and Silva et al. (2019) have proposed a simultaneous optimization of the objective functions.

Hatami-Marbini & Toloo (2017) presented a model (Extended BiO-MCDEA) in order to determine and assign an optimal lower limit for the input and output variables. The maximum value of the non-Archimedean infinitesimal (ε) for the BCC model is calculated from Equations 6-10 (Amin & Toloo, 2007):

${\epsilon }^{*}=\max \epsilon$ (6)

$s.t.{{\displaystyle \sum }}_{i=1}^m{v}_i{x}_{ij}\le \mathrm{1,}\text{j}=\mathrm{1,}\dots ,\text{n}$ (7)

$${{\displaystyle \sum }}_{r=1}^s{u}_r{y}_{rj}-{{\displaystyle \sum }}_{i=1}^m{v}_i{x}_{ij}+u_0\le \mathrm{0,}\text{j}=\mathrm{1,}\dots ,\text{n}$$ (8)

$\epsilon -u_r\le \mathrm{0,}r=\mathrm{1,}\dots ,s$ (9)

$\epsilon -v_i\le \mathrm{0,}i=\mathrm{1,}\dots ,m$ (10)

The maximum discriminatory power occurs if se $\epsilon ={\text{ε}}^{\text{*}}$. $u_0$is a signal free scaling factor.

Rubem et al. (2017) and Rubem (2016) proposed the WGP-MCDEA model based on Goal Programming (GP). The input-oriented WGP-MCDEA-BCC comprises the following model:

$Mina=\left\{{\delta }_1{d}_1^{+}+{\delta }_2{d}_2^{+}+{\delta }_3{d}_3^{+}\right\}$ (11)

$s.t.{{\displaystyle \sum }}_{i=1}^r{v}_i{x}_{i0}=1$ (12)

${{\displaystyle \sum }}_{j=1}^s{u}_j{y}_{jk}-{{\displaystyle \sum }}_{i=1}^r{v}_i{x}_{ik}+d_k+u_0\le \mathrm{0,}\forall k$ (13)

$M-d_k\ge \mathrm{0,}\forall k$ (14)

$d_0+d_1^{-}+d_1^{+}\le g_1$ (15)

$M+d_2^{-}+d_2^{+}\le g_2,\forall k$ (16)

${{\displaystyle \sum }}_{k=1}^n{d}_k+d_3^{-}+d_3^{+}\le g_3,\forall k$ (17)

$u_j,v_i\ge \mathrm{0,}\forall j,i$ (18)

The objective function (Equation 11) is called the realization function and it minimizes the weighted sum of undesirable deviations ($d_1^{+},d_2^{+},d_3^{+}$), which can vary depending on the type of initial goals established. ${\delta }_1,{\delta }_2,{\delta }_3$ are weights of the realization function which allow a flexible order of preference among the three objectives. $d_0$ is the only efficiency deviation limited to the range $\left[\mathrm{0,}1\right]$. $M$ is the maximum deviation for a given $DM{U}_K\left(k=\mathrm{1,}\dots ,n\right)$. The aspiration levels $\left(g_1,g_2\text{and}{g}_3\right)$ can be adjusted by the decision maker (manager of the production process). These must not be exceeded to achieve the objective (Equation 11) (Caballero et al., 1997).

The resolution of multi-objective programming problems aims to obtain the set of non-dominated solutions (Clímaco et al., 2008). In this work, in order to avoid ambiguity, the term “efficient” is used related to the specific situations within the scope of the DEA, while in the resolution of MCDEA models, Pareto's optimal solutions are the non-dominated solutions (Rubem et al., 2017; Rubem, 2016).

3. Proposed method and case study

The proposed method (input-oriented IWGP-MCDEA-BCC) consists of the integration of WGP-MCDEA-BCC (Equations 11-18) and the lower limit model of optimal weights (Equations 6-10) for the inputs and outputs.

The proposed method (Equations 19-27) is applied in situations of limited production (subject to limits on inputs and outputs) since there is no guarantee of reliability of the indexes of the non-dominated solutions (efficiencies) in the case of free production (Podinovski & Bouzdine-Chameeva, 2013).

$Mina=\left\{{\delta }_1{d}_1^{+}+{\delta }_2{d}_2^{+}+{\delta }_3{d}_3^{+}\right\}$ (19)

$s.t.{{\displaystyle \sum }}_{i=1}^r{v}_i{x}_{i0}=1$ (20)

${{\displaystyle \sum }}_{j=1}^s{u}_j{y}_{jk}-{{\displaystyle \sum }}_{i=1}^r{v}_i{x}_{ik}+d_k+u_0\le \mathrm{0,}\forall k$ (21)

$M-d_k\ge \mathrm{0,}\forall k$ (22)

$d_0+d_1^{-}+d_1^{+}\le g_1$ (23)

$M+d_2^{-}+d_2^{+}\le g_2,\forall k$ (24)

${{\displaystyle \sum }}_{k=1}^n{d}_k+d_3^{-}+d_3^{+}\le g_3,\forall k$ (25)

$u_j,v_i\ge \epsilon ,\forall j,i$ (26)

${\delta }_1,{\delta }_2,{\delta }_3,d_k,d_1^{-},d_1^{+},d_2^{-},d_2^{+},d_3^{-},d_3^{+}\ge \mathrm{0,}\forall k$ (27)

Constraints 23, 24 and 25 represent the objective functions, namely, classic DEA, minimax and minisum, respectively, of the MCDEA model proposed by Li & Reeves (1999). The minimax (24) is more restrictive than the minisum (25) and both (24 and 25) tend to generate less efficient DMUs than constraint 23. Therefore, the input-oriented IWGP-MCDEA-BCC method increases the discriminatory power in relation to the classic DEA-BCC model.

The results obtained using the input-oriented WGP-MCDEA model (Equations 11-18) and the input-oriented IWGP-MCDEA method (Equations 19-27) were compared with the result obtained using the classic input-oriented DEA-BCC model (Equations 1-4) with and without the maximum value for $\text{NAI}\left(\epsilon \right)$ (Amin & Toloo, 2007; Hatami-Marbini & Toloo, 2017). In the latter case, the non-negativity condition of the multipliers (Equation 4) is changed to $u_j\ge \epsilon \left(j=\mathrm{1,.}..,s\right)\text{and}{v}_i\ge \epsilon \left(i=\mathrm{1,.}..,r\right)$ to improve the analysis of technical efficiency of DMUs.

The input-oriented IWGP-MCDEA-BCC method (Equations 19-27) was applied to evaluate farms which cultivate a certain variety of mango in order to reduce the costs of their main inputs without changing the outputs. The BCC model was chosen since this case study comprises a real situation and the production units of different scales (different fields) were compared.

The weights of the realization function were relaxed and considered equal to the unit $\left({\delta }_1={\delta }_2={\delta }_3=1\right)$ (Bal et al., 2010). This condition is consistent with the original concept of Li & Reeves (1999) for the MCDEA model since it does not establish an order of preference between the three objectives, even if indirect, after being transformed into goals.

The procedure/method is shown in Figure 1. The data collected (DMUs, inputs and outputs) refer to the year 2018 and were obtained directly from the administration of the exporting company, which has seven mango production farms producing the Kent, Keitt, Palmer and Tommy varieties of the fruit.

Figure 1
Flow of the methodological procedure.

DMUs represent the seven producing farms in the months from January to December 2018 (12 months). 39 DMUs were considered according to the Golden Rule and the Tommy mango variety was chosen as it accounted for the highest production in 2018. The set of variables (inputs and outputs) was chosen from the analysis of the correlation between the available variables (Gontijo et al., 2018), also following the Golden Rule.

The inputs considered were: 1) total cost of production for each DMU $\left(x_1,\text{US\$}\right)$; 2) total planted area of each DMU $\left(x_2,\text{hectares}\right)$; 3) labor costs ($x_3,\text{US\$}/\text{Kg}/\text{field}$); 4) cost of agricultural chemicals ($x_4,\text{US\$}/\text{Kg}/\text{field}$); 5) cost of fertilizers ($x_5,\text{US\$}/\text{Kg}/\text{field}$); 6) mechanization expenses ($x_6,\text{US\$}/\text{Kg}/\text{field}$); 7) electricity consumption ($x_7,\text{US\$}/\text{Kg}/\text{field}$) and 8) water consumption for irrigation ($x_8,\text{US\$}/\text{Kg}/\text{field}$).

The outputs considered were: 1) total production ($y_1,\text{Kg}$ $/$ field); 2) 1st quality production ($y_2,\text{Kg}/$ field); 3) 2nd quality production ($y_3,\text{Kg}/$field) and 4) 3rd quality production ($y_4,\text{Kg}/$ field).

The inputs and outputs were normalized by dividing each measured value by its respective maximum value (Bal et al., 2010; Rubem et al., 2017; Rubem, 2016; Hatami-Marbini & Toloo, 2017). The Spearman's Correlation Test $\left(r_s\right)$ and p-value were used to verify an existing correlation between the results of the efficiencies and the non-dominated solutions obtained by the models (DEA-BCC, WGP-MCDEA-BCC, IWGP-MCDEA-BCC) (Silva et al., 2017a). The Spearman´s rank correlation test $\left(r_s\right)$ provides a non-parametric correlation coefficient in the range of $-1$ (strong negative correlation) to $+1$ (strong positive correlation) ($r_s=0$ implies that there is no linear dependence between variables). If the p-value is lower than a given risk value $\left(\alpha \right)$, the result of the applied correlation test is valid. Otherwise, there is not enough information to complete the correlation test (Cakmakci, 2009). The Spearman’s Correlation Test coefficient is given by:

$r_s=1-\frac{6{{\displaystyle \sum }}_i^n{\Delta }_i^2}{(n^3-n)}$ (28)

Where $n$ is the number of DMUs and ${\Delta }_i$ is the difference between the results obtained using the models (DEA-BCC, WGP-MCDEA-BCC, IWGP-MCDEA-BCC).

The dispersion of the weights of the inputs and outputs was analyzed through the coefficient of variation $\left(CV\right)$ (Bal et al., 2010; Ghasemi et al., 2014; Hatami-Marbini & Toloo, 2017). The $CV$ is a metric that provides the relative variability of data and does not depend on the unit of measurement adopted (Koopmans et al., 1964) (Equations 29-31) (Bal et al., 2010; Hatami-Marbini & Toloo, 2017).

${\mu }_j=\frac{{{\displaystyle \sum }}_{r=1}^s{u}_{rj}+{{\displaystyle \sum }}_{i=1}^m{v}_{ij}}{m+s}$ (29)

${\sigma }_j=\frac{{({{\displaystyle \sum }}_{r=1}^s{u}_{rj}-\mu )}^2+{({{\displaystyle \sum }}_{i=1}^m{v}_{ij}-\mu )}^2}{m+s}$ (30)

$CV=\frac{\sigma }{\left|\mu \right|}$ (31)

$u_r$ and $v_i$ are the decision variables that represent the multipliers (weights) assigned to output $r$ and input $i$, respectively. Equation 31 shows the variability in relation to the average of the weights obtained. The higher the $CV$ value, the greater the dispersion in the analyzed variable (input or output). The CV provides a useful alternative for sensitivity analysis as it is able to compare variations between different data sets with different means (Hatami-Marbini & Toloo, 2017).

The computational tools and algorithms used in this work comprise: i) Minitab^® to perform Spearman´s Correlation Test, p-value and the Coefficient of Variation; ii) Ms-Excel^TM using Visual Basic Applications (VBA) and the iii) simplex algorithm to implement and solve the optimization models. ^{Appendix B} presents screens with results obtained using Minitab (Spearman´s Correlation Test) and Ms-Excel (Visual Basic). The algorithms were implemented using basic hardware resources (Intel Core i5-8265U, 8 Gb of RAM and 256 GB).

4. Results and discussion

The input-oriented DEA-BCC model (Equations 1-4) was applied to evaluate the efficiency of DMUs (Table 2) and 15 DMUs were considered efficient (Table 3). Among these efficient DMUs, 11 were considered to be falsely efficient in calculating the inverted frontier (underlined in Table 3). These DMUs consumed more inputs and produced fewer mangoes as a final product and therefore presented the worst production management practices. DMUs 2, 5, 10 and 16 had the best rankings based on normalized composite efficiency. The $\text{NAI}\left(\epsilon \right)$ was equal to 0.142 and 11 efficient DMUs were obtained by including the non-negativity condition of the multipliers ($u_j\ge \epsilon j=\mathrm{1,}...,se{v}_i\ge \epsilon i=\mathrm{1,.}..,r$).

Table 2
Input and output data (mango exporting company - Tommy variety).

	INPUT								OUTPUT
DMU	x1	x2	x3	x4	x5	x6	x7	x8	y1	y2	y3	y4
DMU 1	31,508.68	3.25	0.11	0.03	0.04	0.03	0.02	0.01	95,660	85,842.14	5,335.18	714.49
DMU2	32,615.32	3.63	0.08	0.02	0.03	0.03	0.02	0.01	125,909	110,816.06	6,302.76	743.71
DMU 3	28,965.25	3.41	0.11	0.03	0.04	0.03	0.02	0.01	84,386	73,986.13	4,831.72	664.59
DMU 4	32,907.16	3.75	0.09	0.03	0.03	0.03	0.02	0.01	114,434	100,109.53	8,252.58	643.18
DMU 5	32,708.99	3.48	0.09	0.02	0.03	0.03	0.02	0.01	125,392	109,454.33	7,522.38	1,456.76
DMU 6	31,403.82	3.26	0.10	0.02	0.05	0.03	0.02	0.01	99,506	86,498.95	6,549.34	510.79
DMU 7	30,776.63	3.01	0.11	0.03	0.05	0.03	0.02	0.01	91,327	78,747.79	8,275.02	1,371.44
DMU 8	30,517.56	3.28	0.10	0.03	0.04	0.03	0.02	0.01	96,832	83,324.95	8,872.38	1,314.25
DMU 9	35,956.88	3.76	0.15	0.04	0.07	0.04	0.03	0.01	75,661	65,018.32	4,086.03	373.64
DMU 10	40,405.29	3.60	0.09	0.03	0.03	0.03	0.01	0.01	154,461	132,286.36	11,472.59	1,459.22
DMU 11	32,873.46	3.51	0.12	0.03	0.04	0.04	0.02	0.01	89,937	76,995.84	8,339.36	1,309.45
DMU 12	35,027.83	3.10	0.12	0.03	0.03	0.04	0.02	0.01	106,700	91,319.16	9,638.81	1,300.07
DMU 13	31,227.29	3.34	0.11	0.03	0.04	0.03	0.02	0.01	91,002	77,540.54	8,964.70	794.99
DMU 14	31,793.01	3.24	0.11	0.03	0.05	0.03	0.02	0.01	90,644	77,187.02	8,662.29	2,333.55
DMU 15	29,142.23	3.40	0.12	0.03	0.06	0.03	0.02	0.01	80,139	68,204.91	5,620.37	917.31
DMU 16	45,533.56	3.89	0.09	0.03	0.03	0.02	0.01	0.01	168,442	142,210.43	14,079.39	2,187.20
DMU 17	9,499.83	1.21	0.09	0.03	0.03	0.02	0.01	0.01	32,660	27,517.97	2,623.29	844.72
DMU 18	34,823.29	3.13	0.14	0.03	0.05	0.05	0.02	0.01	84,196	70,269.73	7,367.63	2,005.84
DMU 19	30,641.85	2.91	0.13	0.03	0.04	0.03	0.02	0.01	82,074	68,357.07	8,945.65	624.48
DMU 20	37,418.78	3.73	0.12	0.03	0.04	0.03	0.02	0.01	104,780	86,913.34	10,526.55	1,708.38
DMU 21	31,735.15	3.25	0.14	0.04	0.04	0.03	0.02	0.01	80,625	66,726.24	8,073.59	2,107.48
DMU 22	12,181.84	1.17	0.11	0.02	0.02	0.04	0.01	0.01	40,346	33,157.74	3,319.69	613.81
DMU 23	34,158.69	3.72	0.11	0.03	0.05	0.03	0.02	0.01	103,382	84,707.59	10,050.04	1,223.82
DMU 24	41,682.72	3.78	0.14	0.02	0.04	0.04	0.01	0.01	116,847	95,738.33	10,751.11	1,534.97
DMU 25	30,386.55	3.65	0.09	0.03	0.05	0.03	0.02	0.01	96,749	78,985.92	9,382.29	1,182.17
DMU 26	52,985.56	4.04	0.14	0.03	0.04	0.04	0.01	0.01	149,643	122,073.50	15,538.53	5,670.55
DMU 27	33,827.88	3.51	0.11	0.03	0.05	0.04	0.02	0.01	98,827	80,570.32	9,062.28	1,107.01
DMU 28	31,390.46	3.80	0.10	0.02	0.02	0.23	0.01	0.01	110,624	89,843.067	11,359.31	3,613.37
DMU 29	12,060.89	1.00	0.14	0.04	0.05	0.04	0.01	0.01	30,151	24,483.79	2,782.80	574.68
DMU 30	15,030.35	1.71	0.09	0.02	0.02	0.03	0.02	0.01	55,919	45,191.02	8,201.73	326.95
DMU 31	34,330.07	3.50	0.12	0.03	0.05	0.03	0.01	0.01	94,086	76,033.61	12,372.37	410.29
DMU 32	33,789.23	2.96	0.15	0.04	0.03	0.04	0.02	0.01	73,876	59,555.13	8,731.56	1,663.37
DMU 33	12,886.15	1.30	0.08	0.02	0.02	0.04	0.01	0.01	50,994	41,063.26	6,723.64	313.61
DMU 34	32,333.67	3.16	0.14	0.04	0.05	0.04	0.02	0.02	80,431	64,753.66	8,792.67	1,911.20
DMU 35	41,627.79	3.82	0.14	0.04	0.03	0.04	0.02	0.01	99,849	80,302.13	12,196.01	1,656.20
DMU 36	28,118.11	3.30	0.09	0.02	0.03	0.03	0.01	0.01	108,350	87,065.86	13,011.72	2,517.50
DMU 37	27,018.15	3.13	0.09	0.03	0.04	0.03	0.01	0.01	92,867	74,234.02	10,621.58	2,233.07
DMU 38	29,482.63	3.56	0.09	0.02	0.04	0.03	0.02	0.01	104,049	82,639.81	12,055.49	1,188.30
DMU 39	24,992.83	3.40	0.08	0.03	0.04	0.02	0.02	0.01	95,015	75,419.98	12,777.21	1,043.74

Table 3
Efficiency, non-dominated solutions, main input and output.

DMU	Input-oriented DEA-BCC model		Inverted Frontier		Normalized Composite Efficiency		Input-oriented DEA-BCC-NAI model	Input-oriented WGP-MCDEA- BCC model	Input-oriented IWGP-MCDEA- BCC method	Total cost per hectare (US$)	First quality mango production (%)
DMU 1	0.839	0.72		0.77		0.737		0.769	0.734	9,690.51	90
DMU2	1.000	0.64		0.94		0.911		0.531	0.905	8,980.80	88
DMU 3	0.794	0.83		0.67		0.699		0.724	0.692	8,490.29	88
DMU 4	0.937	0.66		0.88		0.840		0.588	0.834	8,771.20	87
DMU 5	1.000	0.57		0.99		0.938		0.913	0.938	9,394.80	87
DMU 6	0.919	0.71		0.83		0.773		0.802	0.761	9,628.64	87
DMU 7	0.865	0.72		0.79		0.771		0.801	0.755	10,220.09	86
DMU 8	0.892	0.70		0.82		0.832		0.851	0.811	9,299.85	86
DMU 9	0.643	1.00		0.44		0.356		0.618	0.356	9,558.60	86
DMU 10	1.000	0.55		1.00		1.000		1.000	0.953	11,218.52	86
DMU 11	0.737	0.80		0.65		0.682		0.624	0.682	9,361.34	86
DMU 12	0.923	0.69		0.85		0.820		0.794	0.808	11,294.10	86
DMU 13	0.799	0.76		0.72		0.734		0.763	0.725	9,345.18	85
DMU 14	0.899	0.75		0.79		0.771		0.750	0.753	9,808.14	85
DMU 15	0.755	0.88		0.61		0.655		0.442	0.651	8,567.30	85
DMU 16	1.000	0.57		0.98		1.000		0.780	1.000	11,699.89	84
DMU 17	1.000	1.00		0.69		1.000		0.646	0.849	7,847.48	84
DMU 18	0.738	0.90		0.58		0.566		0.643	0.566	11,120.52	83
DMU 19	0.896	0.82		0.74		0.788		0.732	0.747	10,525.00	83
DMU 20	0.756	0.78		0.68		0.686		0.685	0.686	10,027.22	83
DMU 21	0.832	0.87		0.67		0.683		0.645	0.683	9,764.66	83
DMU 22	1.000	0.94		0.73		1.000		0.789	0.876	10,411.82	82
DMU 23	0.823	0.76		0.73		0.761		0.644	0.753	9,182.44	82
DMU 24	0.901	0.78		0.78		0.727		0.572	0.727	11,027.17	82
DMU 25	0.888	0.80		0.75		0.789		0.608	0.755	8,325.08	82
DMU 26	1.000	0.77		0.85		1.000		0.408	0.862	13,115.24	82
DMU 27	0.799	0.76		0.72		0.713		0.703	0.713	9,637.57	82
DMU 28	1.000	0.74		0.87		1.000		0.555	0.959	8,260.65	81
DMU 29	1.000	1.00		0.69		0.843		0.578	0.645	12,060.89	81
DMU 30	1.000	1.00		0.69		1.000		0.949	0.949	8,789.68	81
DMU 31	1.000	0.90		0.76		1.000		0.704	0.778	9,808.59	81
DMU 32	0.809	1.00		0.56		0.644		0.650	0.644	11,415.28	81
DMU 33	1.000	1.00		0.69		1.000		1.000	1.000	9,912.42	81
DMU 34	0.745	0.95		0.55		0.583		0.467	0.583	10,232.17	81
DMU 35	0.868	0.92		0.65		0.673		0.658	0.673	10,897.33	80
DMU 36	1.000	0.67		0.92		1.000		0.833	0.991	8,520.64	80
DMU 37	0.954	0.75		0.83		0.888		0.536	0.859	8,632.00	80
DMU 38	1.000	0.75		0.86		0.927		0.802	0.891	8,281.64	79
DMU 39	1.000	0.79		0.83		1.000		0.731	0.896	7,350.83	79

The comparison between the results of the applied methods related to the main input and output data identifies DMUs that serve as a reference for the others. According to Table 3, among the DMUs belonging to the set of non-dominated solutions (DMU 2, 5, 10, 16, 28, 30, 33 and 36), DMU 28 has the lowest total cost per hectare (US$ 8,260.65) and DMU 2 produced the highest percentage of first quality mangos (88%).

DMU 2 is the mango farm with the best cost-benefit ratio for the Tommy variety and can be used as a reference for the production manager to expand the company’s marketing goals. Process improvement can still be supported through the use of strategic planning techniques such as SWOT analysis which allows the identification of strengths, weaknesses, opportunities, and threats related to business competition (Borgheipour et al., 2018; Zare et al., 2015).

Most DMU efficiencies have been reduced by using the input-oriented DEA-BCC-NAI model instead of the input-oriented DEA-BCC (Figure 2). This shows that the absence of a priori information on weights (NAI) results in identifying wrongly efficient farms whose production costs have not actually decreased.

Figure 2
Comparison of results in terms of efficiency and non-dominated solution.

Figure 2 shows that the proposed method (IWGP-MCDEA-BCC) (in comparison with WGP-MCDEA-BCC model) is capable of obtaining more consistent weights for the units (DMUs 2, 5, 10, 16, 28, 30, 33 and 36) which did in fact minimize their production costs and reached the goal established by the production manager (80% of 1st quality mango production) (Table 3). For example, according to the WGP-MCDEA-BCC model, DMU 2 would not be considered as belonging to the set of non-dominated solutions (efficiency equal to 0.53). This would lead to decision-making related to the scheduling and production planning focused on DMU 33 as the main reference.

1 (^{Appendix A}) (input-oriented DEA-BCC model, Equations 1-4) presents several zero weights which show an unrealistic weight distribution mainly for the multipliers $v_7\text{e}{u}_1$. This means that the electricity consumption cost (input) and total production (output) have been omitted from the efficiency analysis.

2 (Inverted Frontier model) presents a significant amount of null weights for most multipliers, which implies excluding many inputs and outputs for the efficiency analysis. Therefore, the method proposed by Yamada et al. (1994) and Entani et al. (2002) was not able to solve the problem of the distribution of unrealistic weights of the inputs and outputs of the DEA method (BCC-input). On the other hand, the Inverted Frontier was still applied together with the classic DEA since it is able to recognize benchmark units which represent potential management references (Mota & Meza, 2020).

Hatami-Marbini & Toloo (2017) and Amin & Toloo, (2007) show the importance of the inclusion of all input and output multipliers for efficiency analysis. Non-zero multipliers are obtained if the NAI $\left(\epsilon =0.142\right)$ is considered in the input-oriented DEA-BCC model (3). In turn, Table 3 shows a reduction from 15 to 11 efficient DMUs which improves the discriminatory power and the distribution of unrealistic weights.

4 and 5 show a reduction in $\text{CV}$ when comparing the application of input-oriented DEA-BCC and input-oriented DEA-BCC with $\text{NAI}\left(\epsilon \right)$, (mainly for DMU 24).

DMU 9 (Table 3) showed low efficiency and a non-dominated solution when applying the input-oriented DEA-BCC-NAI (0.356) and input-oriented IWGP-MCDEA-BCC (0.356) due to the higher costs associated with labor ($x_3$), fertilizers ($x_5$) and electricity ($x_7$), which provided zero CV (5 and 4). These input costs have to be minimized because DMU 9 is the farm that achieved the company's main goal (80% production of first quality mangoes) producing 86% of first quality mangoes.

The non-dominated solutions obtained from the input-oriented WGP-MCDEA- BCC model (Equations 11-18) comprised DMUs 10 and 33 (Table 3). When considering $\epsilon =0.142$ (IWGP-MCDEA-BCC method), it was verified that DMUs 16 and 33 were the best among the set of non-dominated solutions (DMUs 2, 5, 10, 16, 28, 30, 33 and 36, Table 3).

In problems involving multi-objective programming, there is usually more than one non-dominated solution and, therefore, they are not comparable to each other (Rubem, 2016). In this case, the decision maker can choose a DMU with the lowest total production cost and the highest first quality production in order to achieve the company's marketing objectives. The results of DMUs 2, 5, 28, 30 and 36 improved (Table 3) (all with efficiencies above 0.9) showing that the input-oriented IWGP - MCDEA - BCC method (Equations 19-27) finds the set of non-dominated solutions.

4 (input-oriented WGP-MCDEA-BCC model) shows an undesirable amount of zero weights omitted from the analysis of non-dominated solutions, mainly in the multipliers $v_1,v_{3}and{u}_2$ (associated with total cost of production, cost of labor and first quality production, respectively).

The fault of null multipliers (weights) (4) was overcome by applying the proposed method (input-oriented IWGP - MCDEA – BCC) (5).

Comparing the input-oriented WGP-MCDEA-BCC model and input-oriented IWGP-MCDEA-BCC method, there is a reduction in the coefficients of variation (5 and 4), mainly for DMUs 10, 16 and 33 which belong to the set of non-dominated solutions.

Table 4
Spearman rank correlation coefficients and p-value.

DMU	input-oriented DEA-BCC model	input-oriented DEA-BCC-NAI model	input-oriented WGP-MCDEA-BCC model
input-oriented DEA-BCC-NAI model	0.927
p-value	≤ 0.05
input-oriented WGP-MCDEA-BCC model	0.324	0.355
p-value	≤ 0.05	≤ 0.05
input-oriented IWGP-MCDEA-BCC method	0.825	0.908	0.454
p-value	≤ 0.05	≤ 0.05	≤ 0.05

Table 4 shows the correlation level between the Spearman rank correlation test and p-value from the results obtained using the different approaches/models.

Table 4 hows two strong positive correlations. The first (0.927) is related to the input-oriented DEA-BCC models with and without $\text{NAI}\left(\epsilon \right)$ and the second (0.908) is related to the input-oriented DEA-BCC model with $\text{NAI}\left(\epsilon \right)$ and input-oriented IWGP-MCDEA-BCC method. It shows that the discriminatory power and the distribution of unrealistic weights improve when NAI $\left(\epsilon \right)$ is considered in the modelling.

DMUs 5, 10, 16, 30, 33 and 36 produce a large percentage of first quality mangos (greater than 80%, goal set by the production manager, Table 2). DMU 33 is the only non-dominated and efficient solution according to all applied models. The total cost per hectare of this DMU is equal to US$ 9,916.99 and the percentage of first quality mango production is 81%.

DMU 2 presented the best performance (88%) among DMUs belonging to the set of non-dominated solutions, based on the main input and output (Table 3). This unit represents the management reference to be followed by other farms which do not belong to the set of non-dominated solutions (such as DMUs 38 and 39, which had the lowest performance, 79%).

The results show that the absence of a priori information about weights (NAI) can identify efficient DMUs (agricultural farms) that do not belong to the set of non-dominated solutions (Tables 3 and Figure 2) (DMUs 9, 29, 38 and 39). Proper recognition of efficient farms (identified using the IWGP method) contributes to supporting production scheduling and planning.

5. Conclusions

The lack of discrimination within the analysed DMUs and the unrealistic distribution of input and output weights are the main shortcomings associated with classic approaches involving DEA. Some methods have been developed to overcome these, such as the inverted frontier method and other models involving simultaneous multi-objective optimization. The inverted frontier method did not solve the failure of the unrealistic weight distribution of the inputs and outputs, as shown in this work, despite being used to identify DMUs (benchmarks) with the best and worst management practices.

This paper proposes a method (input-oriented IWGP-MCDEA-BCC) suitable for applications involving limited production. It comprises a combination of the input-oriented WGP-MCDEA-BCC model and the maximum NAI (ε) value model for inputs and outputs. The results show that the proposed method is capable of overcoming the shortcomings associated with classical DEA. The models with the highest positive correlation (i.e. greatest discriminatory power) were the input-oriented DEA-BCC models with $\text{NAI}\left(\epsilon \right)$ and input-oriented IWGP-MCDEA-BCC method. The main advantage of using the MCDEA approach is the correct identification of efficient DMUs belonging to the set of non-dominated solutions. This identification is useful to support production planning and scheduling, enabling the reduction of input costs and the recognition of reference units (DMUs).

The level of complexity associated with the proposed method does not make its application impracticable in a real situation considering that the analysis of the DMUs efficiency is carried out offline. This analysis would provide a ranking of units and other useful information for the decision maker. On the other hand, in addition to a preliminary analysis of the production process and its demands, the application of the proposed method (IWGP-MCDEA-BCC) requires the availability of production data (input and output variables) over at least six months to avoid obtaining biased information. Then the type of DEA-BCC model to be applied (input-oriented or output-oriented) must be defined.

The proposed method (input-oriented IWGP-MCDEA-BCC) can help decision makers to improve performance (efficiency) in the production of other varieties of fruit produced in the São Francisco Valley Region (coconut, guava, papaya, grape, passion fruit, pineapple and melon). It is capable of providing improvement indicators for the production manager based on DMUs belonging to non-dominated solutions that optimize production costs.

Appendix A Input and output weights and CV for the models.

Table A1
Input and output Weights and $CV$ (input-oriented DEA-BCC model).

input-oriented DEA-BCC model	Weights of inputs								Weights of outputs
DMU	v1	v2	v3	v4	v5	v6	v7	v8	u1	u2	u3	u4	$CV$
DMU 1	0.177	0.240	0.000	0.521	0.000	0.339	0.000	0.244	0.000	0.242	0.000	0.000	1.187
DMU2	0.000	0.108	0.000	1.056	0.000	0.569	0.000	0.087	0.000	0.067	0.000	0.000	2.070
DMU 3	0.828	0.000	0.000	0.439	0.000	0.219	0.000	0.164	0.000	0.513	0.000	0.000	1.520
DMU 4	0.200	0.000	0.759	0.000	0.473	0.314	0.000	0.056	0.000	0.135	0.000	0.000	1.503
DMU 5	0.195	0.198	0.343	0.576	0.000	0.247	0.000	0.149	0.000	0.196	0.000	0.216	0.966
DMU 6	0.044	0.151	0.000	0.842	0.000	0.276	0.000	0.431	0.000	0.088	0.000	0.000	1.681
DMU 7	0.076	0.435	0.000	0.000	0.000	0.526	0.000	0.616	0.000	0.000	0.000	0.445	1.425
DMU 8	0.630	0.239	0.000	0.098	0.000	0.331	0.000	0.431	0.000	0.365	0.000	0.521	1.066
DMU 9	0.000	0.221	0.000	0.441	0.000	0.298	0.000	0.222	0.000	0.085	0.000	0.000	1.441
DMU 10	0.000	1.122	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.996	0.000	0.000	2.340
DMU 11	0.840	0.100	0.000	0.000	0.068	0.232	0.000	0.280	0.000	0.553	0.000	0.344	1.330
DMU 12	0.000	0.000	0.000	1.193	0.063	0.000	0.000	0.653	0.000	0.000	0.000	0.122	2.199
DMU 13	0.000	0.320	0.629	0.000	0.000	0.319	0.000	0.164	0.000	0.197	0.000	0.000	1.480
DMU 14	0.997	0.000	0.000	0.000	0.000	0.200	0.000	0.541	0.000	0.331	0.000	0.658	1.483
DMU 15	0.711	0.242	0.000	0.000	0.000	0.349	0.000	0.296	0.000	0.670	0.000	0.000	1.422
DMU 16	0.000	1.015	0.000	0.000	0.000	0.047	0.000	0.000	0.000	0.894	0.000	0.000	2.275
DMU 17	0.000	0.516	0.000	0.000	0.000	0.746	0.000	0.720	0.000	0.000	0.000	0.000	1.838
DMU 18	0.847	0.469	0.000	0.071	0.000	0.000	0.000	0.031	0.000	0.946	0.000	0.459	1.508
DMU 19	0.000	0.481	0.000	0.000	0.000	0.606	0.000	0.696	0.000	0.144	0.000	0.000	1.674
DMU 20	0.000	0.240	0.867	0.000	0.000	0.163	0.000	0.000	0.000	0.138	0.026	0.139	1.882
DMU 21	0.746	0.041	0.000	0.000	0.000	0.304	0.000	0.537	0.000	0.000	0.000	0.687	1.535
DMU 22	0.000	0.288	0.000	0.718	0.000	0.477	0.000	0.379	0.000	0.000	0.000	0.000	1.603
DMU 23	0.000	0.278	0.000	0.554	0.000	0.375	0.000	0.278	0.000	0.107	0.000	0.000	1.441
DMU 24	0.000	0.000	0.000	1.535	0.000	0.000	0.102	0.105	0.000	0.000	0.411	0.000	2.469
DMU 25	0.174	0.000	0.728	0.629	0.000	0.101	0.000	0.000	0.000	0.000	0.000	0.163	1.716
DMU 26	0.959	0.000	0.000	0.000	0.000	0.000	0.000	0.096	0.000	0.000	0.000	1.000	2.211
DMU 27	0.018	0.026	0.467	0.790	0.000	0.000	0.000	0.190	0.000	0.000	0.049	0.190	1.710
DMU 28	0.000	0.069	0.000	1.218	0.000	0.620	0.000	0.000	0.000	0.000	0.000	0.000	2.377
DMU 29	0.000	2.030	0.000	0.163	0.000	0.206	0.000	0.275	0.000	0.000	0.000	0.000	2.591
DMU 30	0.340	0.000	0.000	0.000	0.864	0.623	0.000	0.371	0.000	0.000	0.109	0.000	1.534
DMU 31	0.000	0.000	0.000	1.046	0.000	0.000	0.000	1.266	0.000	0.000	0.000	0.000	2.348
DMU 32	0.000	0.192	0.000	0.000	1.241	0.000	0.000	0.629	0.000	0.000	0.000	0.542	1.812
DMU 33	0.000	0.516	0.000	0.000	0.000	0.746	0.000	0.720	0.000	0.000	0.000	0.000	1.838
DMU 34	0.000	0.664	0.000	0.000	0.000	0.163	0.624	0.000	0.000	0.000	0.384	0.194	1.491
DMU 35	0.000	0.455	0.000	0.000	1.355	0.000	0.000	0.000	0.000	0.000	0.776	0.000	2.022
DMU 36	0.000	0.600	0.000	0.881	0.000	0.000	0.000	0.000	0.000	0.000	0.779	0.000	1.837
DMU 37	0.000	0.494	1.033	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.719	1.911
DMU 38	0.000	0.000	0.000	1246	0.000	0.511	0.000	0.144	0.000	0.000	0.009	0.000	2.344
DMU 39	0.000	0.067	0.000	1.123	0.000	0.596	0.000	0.087	0.000	0.000	0.000	0.000	2.232

Table A2
Input and output Weights (inverted frontier model).

Inverted frontier model	Weights of inputs								Weights of outputs
DMU	v1	v2	v3	v4	v5	v6	v7	v8	u1	u2	u3	u4
DMU 1	0.549	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	2.912	0.000
DMU2	0.000	0.368	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	2.465	0.000
DMU 3	0.000	0.807	0.000	0.000	0.000	0.000	0.000	0.000	1.996	0.000	0.000	0.000
DMU 4	0.000	0,595	0.000	0.000	0.000	0.000	0.000	0.000	1.472	0.000	0.000	0.000
DMU 5	0.000	0.543	0.000	0.000	0.000	0.000	0.000	0.000	1.343	0.000	0.000	0.000
DMU 6	0.486	0,000	0.000	0.000	0.000	0.000	0.000	0.000	1.218	0.000	0.000	3.111
DMU 7	1.106	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.844	0.000	0.000	0.000
DMU 8	0,000	0.704	0.000	0.000	0.000	0.000	0.000	0.000	1.740	0.000	0.000	0.000
DMU 9	0.000	0.847	0.000	0.000	0.212	0.000	0.000	0.000	0.000	2.187	0.000	0.000
DMU 10	0.724	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.437	0.644	0.000	0.000
DMU 11	0,000	0.772	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.847	0.000	0.000
DMU 12	0.967	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.754	0.813	0.000	0.000
DMU 13	0.000	0.766	0.000	0,000	0.000	0.000	0.000	0.000	0.000	1.834	0.000	0.000
DMU 14	0.905	0.172	0.000	0.000	0,000	0.000	0.000	0.000	0.000	1.842	0.000	0.000
DMU 15	0.000	0.871	0.000	0,000	0,000	0.000	0.000	0.000	0.000	2.085	0.000	0.000
DMU 16	0.669	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.404	0.596	0.000	0.000
DMU 17	0.000	0.000	0,000	0.000	0.000	0.000	0.000	0.481	5.157	0.000	0.000	0.000
DMU 18	1.257	0,000	0,000	0,000	0.000	0.000	0.000	0.103	0.244	1.777	0.000	0.000
DMU 19	1.290	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	2.054	0.000	0.117
DMU 20	0.969	0.037	0.000	0.000	0.000	0,000	0.000	0.085	0.000	1.636	0.000	0.000
DMU 21	1.046	0.199	0.000	0.000	0.000	0,000	0.000	0.000	0.000	2.131	0.000	0.000
DMU 22	0.000	0.000	0.000	0,000	0.000	0.000	0.000	1.440	0.000	1.133	0.000	6.797
DMU 23	0.000	0.701	0,000	0.000	0.000	0.000	0.000	0.000	0.000	1.679	0.000	0.000
DMU 24	0.989	0.000	0,000	0.000	0.000	0.000	0.000	0.000	0.000	1.457	0.000	0.070
DMU 25	0,000	0.752	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.800	0.000	0.000
DMU 26	0,770	0.000	0,000	0.000	0.000	0.000	0.000	0.000	0.000	1.119	0.039	0.000
DMU 27	0.867	0.165	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.765	0.000	0.000
DMU 28	0.000	0.683	0.000	0.000	0.000	0.000	0.000	0.132	0.000	1.583	0.000	0.000
DMU 29	0.000	0.000	0,000	0.000	0.000	0.125	0.000	0.000	0.000	0.000	5.584	0.000
DMU 30	0.000	0.000	0.000	0.000	0.000	0.000	0.774	0.034	0.000	2.142	0.000	5.540
DMU 31	0.336	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	13.821
DMU 32	1.066	0.302	0.000	0.099	0.000	0.000	0.000	0.000	0.000	2.388	0.000	0.000
DMU 33	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.581	1.018	7.081
DMU 34	0.406	0.631	0.000	0.000	0.000	0.000	0.000	0.211	0.000	2.196	0.000	0.000
DMU 35	1.173	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.728	0.000	0.083
DMU 36	0.000	0.682	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.633	0.000	0.000
DMU 37	0.000	0.800	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.916	0.000	0.000
DMU 38	0.000	0.719	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.721	0.000	0.000
DMU 39	0.000	0.788	0.000	0.000	0.000	0.000	0.000	0.000	0.000	1.886	0.000	0.000

Table A3
Input and output Weights and $CV$ (input-oriented DEA-BCC-NAI model).

input-oriented DEA-BCC-NAI model	Weights of inputs								Weights of outputs
DMU	v1	v2	v3	v4	v5	v6	v7	v8	u1	u2	u3	u4	$CV$
DMU 1	0.471	0.142	0.142	0.142	0.142	0.222	0.142	0.149	0.142	0.142	0.142	0.142	0.539
DMU2	0.392	0.142	0.515	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.638
DMU 3	0.466	0.142	0.142	0.142	0.142	0.212	0.142	0.146	0.142	0.142	0.142	0.142	0.534
DMU 4	0.495	0.142	0.142	0.142	0.142	0.267	0.142	0.167	0.142	0.142	0.142	0.142	0.566
DMU 5	0.718	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.391	0.142	0.142	0.828
DMU 6	0.417	0.142	0.142	0.331	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.509
DMU 7	0.472	0.142	0.142	0.142	0.142	0.223	0.142	0.150	0.142	0.142	0.142	0.142	0.539
DMU 8	0.496	0.142	0.142	0.142	0.142	0.268	0.142	0.167	0.142	0.142	0.142	0.142	0.567
DMU 9	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.000
DMU 10	0.142	0.476	0.142	0.142	0.142	0.142	0.402	0.142	0.142	0.154	0.142	0.142	0.602
DMU 11	0.421	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.485
DMU 12	0.419	0.142	0.142	0.309	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.498
DMU 13	0.458	0.142	0.142	0.142	0.142	0.197	0.142	0.142	0.142	0.142	0.142	0.142	0.525
DMU 14	0.394	0.226	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.236	0.427
DMU 15	0.439	0.142	0.142	0.142	0.142	0.160	0.142	0.142	0.142	0.142	0.142	0.42	0.506
DMU 16	0.142	0.507	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.222	0.142	0.142	0.588
DMU 17	0.142	0.621	0.142	0.142	0.142	0.538	0.142	0.361	0.142	0.142	0.142	0.142	0.746
DMU 18	0.289	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.273
DMU 19	0.441	0.142	0.142	0.142	0.142	0.142	0.142	0.513	0.142	0.142	0.142	0.142	0.662
DMU 20	0.365	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.398
DMU 21	0.422	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.487
DMU 22	0.142	0.499	0.142	0.567	0.142	0.368	0.142	0.142	0.142	0.142	0.142	0.142	0.697
DMU 23	0.451	0.142	0.142	0.142	0.142	0.184	0.142	0.142	0.142	0.142	0.142	0.142	0.518
DMU 24	0.392	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.442
DMU 25	0.476	0.142	0.142	0.142	0.142	0.230	0.142	0.153	0.142	0.142	0.142	0.142	0.544
DMU 26	0.315	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.367	0.445
DMU 27	0.429	0.142	0.142	0.161	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.491
DMU 28	0.284	0.409	0.142	0.142	0.142	0.150	0.142	0.142	0.142	0.142	0.142	0.380	0.512
DMU 29	0.142	1.292	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.693	1.248
DMU 30	1.286	0.142	0.142	0.142	0.487	0.142	0.142	0.142	0.142	0.142	0.921	0.142	1.153
DMU 31	0.142	0.277	0.142	0.142	0.142	0.142	0.142	1.464	0.142	0.142	0.142	0.142	0.440
DMU 32	0.398	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.451
DMU 33	0.142	0.599	0.142	0.142	0.142	0.496	0.142	0.321	0.142	0.142	0.142	0.142	0.715
DMU 34	0.307	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.304
DMU 35	0.346	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.368
DMU 36	0.451	0.260	0.142	0.142	0.142	0.318	0.142	0.142	0.142	0.142	0.210	0.142	0.498
DMU 37	0.282	0.411	0.177	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.393	0.512
DMU 38	0.571	0.142	0.142	0.233	0.142	0.142	0.142	0.262	0.142	0.142	0.329	0.142	0.613
DMU 39	0.459	0.241	0.142	0.142	0.142	0.391	0.142	0.251	0.142	0.142	0.142	0.142	0.533

Table A4
Weights of inputs, outputs and $CV$ (input-oriented WGP – MCDEA-BCC model).

input-oriented WGP – MCDEA-BCC model	Weights of inputs									Weights of outputs
DMU	v1	v2	v3	v4	v5	v6	v7	v8	u1		u2	u3	u4	CV
DMU 1	0.000	0.250	0.000	0.000	0.192	0.496	0.000	0.692	0.000		0.000	0.000	0.000	1.723
DMU 2	0.000	0.671	0.000	0.000	0.000	0.029	0.522	0.198	0.000		0.000	0.760	0.000	1.612
DMU 3	0.000	0.601	0.000	0.000	0.000	0.434	0.000	0.349	0.346		0.000	0.000	0.000	1.539
DMU 4	0.000	0.630	0.000	0.000	0.000	0.028	0.490	0.186	0.000		0.000	0.714	0.000	1.612
DMU 5	0.000	0.297	0.000	0.000	0.228	0.589	0.000	0.821	0.000		0.000	0.000	0.000	1.723
DMU 6	0.000	0.261	0.000	0.000	0.201	0.518	0.000	0.722	0.000		0.000	0.000	0.000	1.723
DMU 7	0.000	0.260	0.000	0.000	0.200	0.517	0.000	0.721	0.000		0.000	0.000	0.000	1.723
DMU 8	0.000	0.276	0.000	0.000	0.213	0.549	0.000	0.766	0.000		0.000	0.000	0.000	1.723
DMU 9	0.000	0.303	0.000	0.000	0.000	0.443	0.000	0.505	0.000		0.000	0.000	0.000	1.858
DMU 10	1.240	0.049	0.000	0.000	0.000	0.016	0.000	0.000	1.091		0.000	0.000	0.000	2.267
DMU 11	0.000	0.000	0.000	0.630	0.000	0.039	0.000	0.735	0.000		0.000	0.000	0.000	2.268
DMU 12	0.000	0.237	0.000	0.000	0.296	0.654	0.000	0.443	0.000		0.000	0.000	0.000	1.641
DMU 13	0.000	0.248	0.000	0.000	0.191	0.493	0.000	0.687	0.000		0.000	0.000	0.000	1.723
DMU 14	0.000	0.244	0.000	0.000	0.188	0.484	0.000	0.675	0.000		0.000	0.000	0.000	1.723
DMU 15	0.000	0.000	0.000	0.000	1.108	0.000	0.000	0.180	0.000		0.081	0.000	0.000	2.787
DMU 16	0.600	0.000	0.000	0.000	0.000	0.417	0.000	0.913	0.000		0.000	0.000	0.000	1.927
DMU 17	0.000	0.000	0.000	0.652	0.000	0.040	0.000	0.761	0.000		0.000	0.000	0.000	2.268
DMU 18	0.000	0.185	0.000	0.462	0.000	0.307	0.000	0.243	0.000		0.000	0.000	0.000	1.603
DMU 19	0.000	0.000	0.000	0.853	0.000	0.331	0.076	0.137	0.000		0.000	0.025	0.000	2.118
DMU 20	0.000	0.248	0.000	0.091	0.000	0.434	0.156	0.407	0.000		0.000	0.000	0.000	1.482
DMU 21	0.000	0.000	0.000	0.651	0.000	0.040	0.000	0.760	0.000		0.000	0.000	0.000	2.268
DMU 22	0.000	0.440	0.000	0.000	0.000	0.556	0.000	0.638	0.000		0.132	0.000	0.000	1.674
DMU 23	0.000	0.605	0.000	0.000	0.000	0.027	0.470	0.178	0.000		0.000	0.685	0.000	1.612
DMU 24	0.440	0.000	0.000	0.000	0.000	0306	0.000	0.669	0.000		0.000	0.000	0.000	1.927
DMU 25	0.000	0.598	0.000	0.000	0.000	0.026	0.466	0.177	0.000		0.000	0.678	0.000	1.612
DMU 26	0.208	0.647	0.000	0.000	0.000	0.000	0.000	0.340	0.063		0.000	0.000	0.000	1.927
DMU 27	0.000	0.228	0.000	0.000	0.176	0.454	0.000	0.632	0.000		0.000	0.000	0.000	1.723
DMU 28	0.000	0.480	0.000	0.000	0.000	0.000	0.000	0.765	0.421		0.000	0.000	0.000	1.895
DMU 29	0.000	0.000	0.000	0.584	0.000	0.036	0.000	0.681	0.000		0.000	0.000	0.000	2.268
DMU 30	0.000	0.522	0.000	0.000	0.000	0.659	0.000	0.757	0.000		0.157	0.000	0.000	1.674
DMU 31	1.049	0.000	0.000	0.000	0.288	0.174	0.000	0.000	0.889		0.000	0.000	0.000	1.861
DMU 32	0.000	0.187	0.000	0.467	0.000	0.310	0.000	0.246	0.000		0.000	0.000	0.000	1.603
DMU 33	0.000	0.553	0.000	0.000	0.000	0.698	0.000	0.801	0.000		0.166	0.000	0.000	1.674
DMU 34	0.000	0.000	0.000	0.471	0.000	0.029	0.000	0.550	0.000		0.000	0.000	0.000	2.268
DMU 35	0.000	0.190	0.000	0.473	0.000	0.314	0.000	0.249	0.000		0.000	0.000	0.000	1.603
DMU 36	0.000	0.647	0.000	0.000	0.000	0.467	0.000	0.376	0.372		0.000	0.000	0.000	1.539
DMU 37	0.000	0.000	0.000	0.000	1.331	0.000	0.000	0.216	0.000		0.097	0.000	0.000	2.787
DMU 38	0.000	0.662	0.000	0.000	0.000	0.029	0.515	0.195	0.000		0.000	0.750	0.000	1.612
DMU 39	0.000	0.217	0.000	0.104	1.150	0.277	0.000	0.000	0.000		0.000	0.000	0.000	2.270

Table A5
Weights of inputs, outputs and $CV$ (input-oriented IWGP–MCDEA-BCC method).

input-oriented IWGP–MCDEA-BCC method	Weights of inputs								Weights of outputs
DMU	v1	v2	v3	v4	v5	v6	v7	v8	u1	u2	u3	u4	CV
DMU 1	0.572	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.265	0.142	0.142	0.668
DMU2	0.705	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.380	0.142	0.142	0.815
DMU 3	0.423	0.142	0.142	0.249	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.481
DMU 4	0.625	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.311	0.142	0.142	0.729
DMU 5	0.718	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.391	0.142	0.142	0.828
DMU 6	0.603	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.292	0.142	0.142	0.703
DMU 7	0.423	0.142	0.142	0.255	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.482
DMU 8	0.638	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.323	0.142	0.142	0.744
DMU 9	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.000
DMU 10	0.581	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.273	0.142	0.142	0.678
DMU 11	0.421	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.485
DMU 12	0.572	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.265	0.142	0.142	0.668
DMU 13	0.425	0.142	0.142	0.217	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.479
DMU 14	0.426	0.142	0.142	0.204	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.480
DMU 15	0.429	0.142	0.142	0.165	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.489
DMU 16	0.551	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.247	0.142	0.142	0.642
DMU 17	0.303	0.142	0.142	0.860	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.965
DMU 18	0.289	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.273
DMU 19	0.420	0.142	0.142	0.298	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.494
DMU 20	0.365	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.398
DMU 21	0.422	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.487
DMU 22	0.443	0.282	0.142	0.142	0.142	0.439	0.142	0.295	0.142	0.142	0.142	0.142	0.548
DMU 23	0.490	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.194	0.142	0.142	0.570
DMU 24	0.392	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.442
DMU 25	0.448	0.142	0.142	0.142	0.142	0.142	0.241	0.142	0.142	0.142	0.142	0.142	0.512
DMU 26	0.315	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.317
DMU 27	0.436	0.142	0.142	0.142	0.142	0.153	0.142	0.142	0.142	0.142	0.142	0.142	0.503
DMU 28	0.712	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.387	0.142	0.142	0.823
DMU 29	0.414	0.142	0.142	0.378	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.536
DMU 30	0.142	0.569	0.142	0.142	0.142	0.439	0.142	0.266	0.142	0.142	0.142	0.142	0.672
DMU 31	0.421	0.142	0.142	0.274	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.486
DMU 32	0.398	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.451
DMU 33	0.398	0.242	0.142	0.573	0.142	0.336	0.142	0.142	0.142	0.142	0.142	0.142	0.628
DMU 34	0.307	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.304
DMU 35	0.346	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0142	0.368
DMU 36	0.817	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.478	0.142	0.142	0.924
DMU 37	0.415	0.142	0.142	0.374	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.534
DMU 38	0.737	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.142	0.408	0.142	0.142	0.848
DMU 39	0.525	0.142	0.142	0.142	0.142	0.142	0.451	0.142	0.142	0.142	0.142	0.142	0.677

Appendix B Screens with results obtained using Minitab and Ms-Excel.

Figure B1
Minitab - Spearman's Correlation Test.

Figure B2
Visual Basic Application.

References

• Adler, N., & Golany, B. (2001). Evaluation of deregulated airline networks using data envelopment analysis combined with principal component analysis with an application to Western Europe. European Journal of Operational Research, 132(2), 260-273. http://dx.doi.org/10.1016/S0377-2217(00)00150-8
  » http://dx.doi.org/10.1016/S0377-2217(00)00150-8
• Adler, N., & Yazhemsky, E. (2010). Improving discrimination in data envelopment analysis: PCA-DEA or variable reduction. European Journal of Operational Research, 202(1), 273-284. http://dx.doi.org/10.1016/j.ejor.2009.03.050
  » http://dx.doi.org/10.1016/j.ejor.2009.03.050
• Aldamak, A., & Zolfaghari, S. (2017). Review of efficiency ranking methods in data envelopment analysis. Measurement, 106, 161-172. http://dx.doi.org/10.1016/j.measurement.2017.04.028
  » http://dx.doi.org/10.1016/j.measurement.2017.04.028
• Almeida, A. T., Ferreira, R. J. P., & Cavalcante, C. A. (2015). A review of the use of multicriteria and multi-objective models in maintenance and reliability. IMA Journal of Management Mathematics, 26(3), 249-271. http://dx.doi.org/10.1093/imaman/dpv010
  » http://dx.doi.org/10.1093/imaman/dpv010
• Amin, G. R., & Toloo, M. (2007). Finding the most efficient DMUs in DEA: an improved integrated model. Computers & Industrial Engineering, 52(1), 71-77. http://dx.doi.org/10.1016/j.cie.2006.10.003
  » http://dx.doi.org/10.1016/j.cie.2006.10.003
• Andrade, R. M. D., Lee, S., Lee, P. T. W., Kwon, O. K., & Chung, H. M. (2019). Port efficiency incorporating service measurement variables by the BiO-MCDEA: Brazilian case. Sustainability, 11(16), 4340. http://dx.doi.org/10.3390/su11164340
  » http://dx.doi.org/10.3390/su11164340
• Ângulo-Meza, L., González-Araya, M. G., Iriarte, A., Rebolledo-Leiva, R., & Mello, J. C. S. (2019). A multiobjective DEA model to assess the eco-efficiency of agricultural practices within the CF+DEA method. Computers and Electronics in Agriculture, 161, 151-161. http://dx.doi.org/10.1016/j.compag.2018.05.037
  » http://dx.doi.org/10.1016/j.compag.2018.05.037
• Aydin, B., & Unakitan, G. (2018). Efficiency analysis in agricultural enterprises in Turkey: case of Thrace Region. Custos e @gronegócio online, 4, 137-160. Retrieved in 28 August 2019, from http://www.custoseagronegocioonline.com.br/numero2v14/OK%208%20efficiency.pdf
  » http://www.custoseagronegocioonline.com.br/numero2v14/OK%208%20efficiency.pdf
• Bal, H., & Örkcü, H. H. (2007). A goal programming approach to weight dispersion in data envelopment analysis. Gazi University Journal of Science, 20, 117-125.
• Bal, H., Örkcü, H. H., & Celebioglu, S. (2010). Improving the discrimination power and weights dispersion in the data envelopment analysis. Computers & Operations Research, 37(1), 99-107. http://dx.doi.org/10.1016/j.cor.2009.03.028
  » http://dx.doi.org/10.1016/j.cor.2009.03.028
• Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Some models for estimating tech- nical and scale inefficiencies in data envelopment analysis. Management Science, 30(9), 1078-1092. http://dx.doi.org/10.1287/mnsc.30.9.1078
  » http://dx.doi.org/10.1287/mnsc.30.9.1078
• Banker, R. D., Charnes, A., Cooper, W. W., Swarts, J., & Thomas, D. A. (1989). An introduction to data envelopment analysis with some of its models and their uses. Research in Governmental and Non-Profit Accounting, 5, 125-163.
• Borgheipour, H., Moghaddas, Z., Abbasi, M., & Abbaszadeh Tehrani, N. (2018). Application of DEA technique in SWOT analysis of oily sludge management using fuzzy data. Global Journal of Environmental Science and Management, 4, 183-194.
• Caballero, R., Gómez, T., González, M., Muñoz, M. M., Rey, L., & Ruiz, F. (1997). Mathematical programming for economists Málaga: Servicio de Publicaciones y Divulgación Científica de la Universidad de Málaga.
• Cakmakci, M. (2009). Process improvement: performance analysis of the setup time reduction-SMED in the automobile industry. International Journal of Advanced Manufacturing Technology, 41(1-2), 168-179. http://dx.doi.org/10.1007/s00170-008-1434-4
  » http://dx.doi.org/10.1007/s00170-008-1434-4
• Cao, J., Chen, G., Khoveyni, M., Eslami, R., & Yang, G. (2016). Specification of a performance indicator using the evidential-reasoning approach. Knowledge-Based Systems, 92, 138-150. http://dx.doi.org/10.1016/j.knosys.2015.10.023
  » http://dx.doi.org/10.1016/j.knosys.2015.10.023
• Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision-making units. European Journal of Operational Research, 2(6), 429-444. http://dx.doi.org/10.1016/0377-2217(78)90138-8
  » http://dx.doi.org/10.1016/0377-2217(78)90138-8
• Clímaco, J. C. N., Mello, J. C. C. B. S., & Ângulo-Meza, L. (2008). Performance measurement: from DEA to MOLP. In F. Adam (Eds.), Encyclopedia of decision making and decision support technologies Hershey: Information Science Reference.
• Daraio, C., & Simar, L. (2007). Advanced robust and nonparametric methods in efficiency analysis: methodology and applications New York: Springer.
• Emrouznejad, A., & Yang, G. (2018). A survey and analysis of the first 40 years of scholarly literature in DEA: 1978 e 2016. Socio-Economic Planning Sciences, 61, 1-5. http://dx.doi.org/10.1016/j.seps.2017.01.008
  » http://dx.doi.org/10.1016/j.seps.2017.01.008
• Entani, T., Maeda, Y., & Tanaka, H. (2002). Dual models of interval DEA and its extensions to interval data. European Journal of Operational Research, 136(1), 32-45. http://dx.doi.org/10.1016/S0377-2217(01)00055-8
  » http://dx.doi.org/10.1016/S0377-2217(01)00055-8
• Fanchon, P. (2003). Variable selection for dynamic measures of efficiency in the computer industry. International Advances in Economic Research, 9(3), 175-188. http://dx.doi.org/10.1007/BF02295441
  » http://dx.doi.org/10.1007/BF02295441
• Ghasemi, M. R., Ignatius, J., & Emrouznejad, A. (2014). A bi-objective weighted model for improving the discrimination power in MCDEA. European Journal of Operational Research, 233(3), 640-650. http://dx.doi.org/10.1016/j.ejor.2013.08.041
  » http://dx.doi.org/10.1016/j.ejor.2013.08.041
• Ghasemi, M. R., Ignatius, J., & Rezaee, B. (2019). Improving discriminating power in data envelopment models based on deviation variables framework. European Journal of Operational Research, 278(2), 442-447. http://dx.doi.org/10.1016/j.ejor.2018.08.046
  » http://dx.doi.org/10.1016/j.ejor.2018.08.046
• Gontijo, T. S., Rodrigues, A. C., & Muylder, C. F. (2018). Incorporating managed preferences in the evaluation of public organizations efficiency: a DEA approach. Independent Journal of Management & Production, 9(4), 1108-1126. http://dx.doi.org/10.14807/ijmp.v9i4.698
  » http://dx.doi.org/10.14807/ijmp.v9i4.698
• Hatami-Marbini, A., & Toloo, M. (2017). An extended multiple criteria data envelopment analysis model. Expert Systems with Applications, 73, 201-219. http://dx.doi.org/10.1016/j.eswa.2016.12.030
  » http://dx.doi.org/10.1016/j.eswa.2016.12.030
• Instituto Brasileiro de Geografia e Estatística – IBGE. (2019). Levantamento sistemático da produção 2019. Rio de Janeiro. Retrieved in 28 August 2019, from https://sidra.ibge.gov.br/home/lspa/brasil
  » https://sidra.ibge.gov.br/home/lspa/brasil
• Iqbal, N., & Sial, M. (2018). Semi-parametric analysis of agricultural production under dichotomy of inputs. Agricultural Economics, 64, 378-388. https://doi.org/10.17221/327/2016-AGRICECON
  » https://doi.org/10.17221/327/2016-AGRICECON
• Jenkins, L., & Anderson, M. (2003). A multivariate statistical approach to reducing the number of variables in data envelopment analysis. European Journal of Operational Research, 147(1), 51-61. http://dx.doi.org/10.1016/S0377-2217(02)00243-6
  » http://dx.doi.org/10.1016/S0377-2217(02)00243-6
• Koopmans, L. H., Owen, D. B., & Rosenblatt, J. I. (1964). Confidence intervals for the coefficient of variation for the normal and log normal distributions. Biometrika, 51(1-2), 25-32. http://dx.doi.org/10.1093/biomet/51.1-2.25
  » http://dx.doi.org/10.1093/biomet/51.1-2.25
• Krcmar, E., & Van Kooter, G. C. (2008). Economic development prospects of forest-dependent communities: Analyzing trade-offs using a compromise-fuzzy programming framework. American Journal of Agricultural Economics, 4(4), 1103-1117. http://dx.doi.org/10.1111/j.1467-8276.2008.01149.x
  » http://dx.doi.org/10.1111/j.1467-8276.2008.01149.x
• Li, H., Xiong, J., Xie, J., Zhou, Z., & Zhang, J. (2019). A unified approach to efficiency decomposition for a two-stage network DEA model with application of performance evaluation in banks and sustainable product design. Sustainability, 11(16), 4401. http://dx.doi.org/10.3390/su11164401
  » http://dx.doi.org/10.3390/su11164401
• Li, X. B., & Reeves, G. R. (1999). A multiple criteria approach to data envelopment analysis. European Journal of Operational Research, 115(3), 507-517. http://dx.doi.org/10.1016/S0377-2217(98)00130-1
  » http://dx.doi.org/10.1016/S0377-2217(98)00130-1
• Marttunen, M., Lienert, J., & Belton, V. (2017). Structuring problems for Multi-Criteria Decision Analysis in practice: A literature review of method combinations. European Journal of Operational Research, 263(1), 1-17. http://dx.doi.org/10.1016/j.ejor.2017.04.041
  » http://dx.doi.org/10.1016/j.ejor.2017.04.041
• Mello, J. C. C. B. S., Gomes, E. G., Meza, L. A., & Leta, F. R. (2008). DEA advanced models for geometric evaluation of used lathes. WSEAS Transactions on Systems, 7(5), 500-520.
• Mota, T. R. A., & Meza, L. A. (2020). The use of DEA as a tool to evaluate public expenditure on education: an analysis of the cities of the state of Rio de Janeiro. Annals of the Brazilian Academy of Sciences, 92(2), e20190187. http://dx.doi.org/10.1590/0001-3765202020190187 PMid:32667507.
  » http://dx.doi.org/10.1590/0001-3765202020190187
• Nara, E. O. B., Sordi, D. C., Schaefer, J. L., Schreiber, J. N. C., Baierle, I. C., Sellitto, M. A., & Furtado, J. C. (2019). Prioritization of OHS key performance indicators that affecting business competitiveness: a demonstration based on MAUT and Neural Networks. Safety Science, 118, 826-834. http://dx.doi.org/10.1016/j.ssci.2019.06.017
  » http://dx.doi.org/10.1016/j.ssci.2019.06.017
• Passos, F. G., Fontes, C. H. O., & Do Nascimento, A. N. (2020). Efficiency evaluation of a mango exporter in the São Francisco Valley, Brazil: a model on data envelopment analysis. Custos e @gronegócio online, 16, 105-141. Retrieved in 28 August 2019, from http://www.custoseagronegocioonline.com.br/especialv16/OK%205%20manga%20english.pdf
  » http://www.custoseagronegocioonline.com.br/especialv16/OK%205%20manga%20english.pdf
• Pastor, J. T., Ruiz, J. L., & Sirvent, I. (2002). A statistical test for nested radial DEA models. Operations Research, 50(4), 728-735. http://dx.doi.org/10.1287/opre.50.4.728.2866
  » http://dx.doi.org/10.1287/opre.50.4.728.2866
• Pereira, E. R., & & Mello, J. C. C. B. S. (2015). Smoothed frontier to determine a single set of weights in CCR models. Production, 25(3), 585-597. http://dx.doi.org/10.1590/0103-6513.057710
  » http://dx.doi.org/10.1590/0103-6513.057710
• Podinovski, V. V., & Bouzdine-Chameeva, T. (2013). Weight restrictions and free production in data envelopment analysis. Operations Research, 61(2), 426-437. http://dx.doi.org/10.1287/opre.1120.1122
  » http://dx.doi.org/10.1287/opre.1120.1122
• Raheli, H., Rezaei, R. M., Jadidi, M. R., & Mobtaker, H. G. (2017). A two-stage DEA model to evaluate sustainability and energy efficiency of tomato production. Information Processing in Agriculture, 4(4), 342-350. http://dx.doi.org/10.1016/j.inpa.2017.02.004
  » http://dx.doi.org/10.1016/j.inpa.2017.02.004
• Rubem, A. P. S. (2016). Resolution of the Li and Reeves model using goal programming (PhD thesis). Graduate Program in Production Engineering, Universidade Federal Fluminense, Rio de Janeiro.
• Rubem, A. P. S., Mello, J. C. C. B. S., & Ângulo-Meza, L. (2017). A goal programming approach to solve the multiple criteria DEA model. European Journal of Operational Research, 260(1), 134-139. http://dx.doi.org/10.1016/j.ejor.2016.11.049
  » http://dx.doi.org/10.1016/j.ejor.2016.11.049
• Ruggiero, J. (2005). Impact assessment of input omission on DEA. International Journal of Information Technology & Decision Making, 4(3), 359-368. http://dx.doi.org/10.1142/S021962200500160X
  » http://dx.doi.org/10.1142/S021962200500160X
• Shen, W., Zhang, D., Liu, W., & Yang, G. (2016). Increasing discrimination of DEA evaluation by utilizing distances to anti-efficient frontiers. Computers & Operations Research, 75, 163-173. http://dx.doi.org/10.1016/j.cor.2016.05.017
  » http://dx.doi.org/10.1016/j.cor.2016.05.017
• Silva, A. F., Marins, F. A. S., & Dias, E. X. (2019). Improving the discrimination power with a new multi-criteria data envelopment model. Annals of Operations Research, 37, 1-33. http://dx.doi.org/10.1007/s10479-019-03446-1
  » http://dx.doi.org/10.1007/s10479-019-03446-1
• Silva, A. F., Marins, F. A. S., Tamura, P. M., & Dias, E. X. (2017a). Bi-Objective multiple criteria data envelopment analysis combined with the overall equipment effectiveness: an application in an automotive company. Journal of Cleaner Production, 157, 278-288. http://dx.doi.org/10.1016/j.jclepro.2017.04.147
  » http://dx.doi.org/10.1016/j.jclepro.2017.04.147
• Silva, J. S., Ferreira, M. O., & Lima, J. R. F. (2017b). Eficiência técnica dos produtores de manga do Vale do São Francisco. Revista de Economia e Agronegócio, 15(1), 28-49. http://dx.doi.org/10.25070/rea.v15i1.367
  » http://dx.doi.org/10.25070/rea.v15i1.367
• Silva, J. L. M., & Sampaio, Y. S. B. (2002). A eficiência técnica dos colonos nos perímetros irrigados em Petrolina, Juazeiro: uma análise de modelos de fronteiras de produção. Revista Economica do Nordeste, 33(2), 159-179. http://dx.doi.org/10.1590/S0103-20032004000400004
  » http://dx.doi.org/10.1590/S0103-20032004000400004
• Simar, L., & Wilson, P. W. (2001). Testing restrictions in nonparametric efficiency models. Communications in Statistics. Simulation and Computation, 30(1), 159-184. http://dx.doi.org/10.1081/SAC-100001865
  » http://dx.doi.org/10.1081/SAC-100001865
• Tan, X., Na, S., Guo, L., Chen, J., & Ruan, Z. (2019). External financing efficiency of rural revitalization listed companies in China: based on two-stage DEA and grey relational analysis. Sustainability, 11(16), 4413. http://dx.doi.org/10.3390/su11164413
  » http://dx.doi.org/10.3390/su11164413
• Ueda, T., & Hoshiai, Y. (1997). Application of principal component analysis for parsimonious summarization of DEA inputs and/or outputs. Journal of the Operations Research Society of Japan, 40(4), 466-478. http://dx.doi.org/10.15807/jorsj.40.466
  » http://dx.doi.org/10.15807/jorsj.40.466
• Wagner, J. M., & Shimshak, D. G. (2007). Stepwise selection of variables in data envelopment analysis: procedures and managerial perspectives. European Journal of Operational Research, 180(1), 57-67. http://dx.doi.org/10.1016/j.ejor.2006.02.048
  » http://dx.doi.org/10.1016/j.ejor.2006.02.048
• Yamada, Y., Matui, T., & Sugiyam, M. (1994). New analysis of efficiency based on DEA. Journal of the Operations Research Society of Japan, 37(2), 158-167. http://dx.doi.org/10.15807/jorsj.37.158
  » http://dx.doi.org/10.15807/jorsj.37.158
• Zare, K., Mehri-Tekmeh, J., & Karimi, S. (2015). A SWOT framework for analyzing the electricity supply chain using na integrated AHP methodology combined with fuzzy-Topsis. International Strategic Management Review, 3(1-2), 66-80. http://dx.doi.org/10.1016/j.ism.2015.07.001
  » http://dx.doi.org/10.1016/j.ism.2015.07.001

Publication Dates

• Publication in this collection
  08 Nov 2021
• Date of issue
  2021

History

• Received
  03 Aug 2021
• Accepted
  01 Oct 2021