A data envelopment analysis model for rank ordering suppliers in the oil industry

A data envelopment analysis model for rank ordering suppliers in the oil industry

Rafael Brandão Rocha^{I, *}; Maria Aparecida Cavalcanti Netto^II

^IExploração & Produção/UN-BC, PETROBRAS S.A., Macaé, RJ

^IICOPPE, Universidade Federal do Rio de Janeiro, Rio de Janeiro, RJ

^{Address to correspondence}

RESUMO

Os benefícios da integração entre empresas e fornecedores assumem um cunho cada vez mais estratégico nas empresas. Porém, avaliar quais fornecedores são merecedores de aprofundar parceria tornou-se extremamente complexo pela grande quantidade de variáveis no processo. Na indústria do petróleo não é diferente. A empresa brasileira PETROBRAS S.A. possui um sistema para classificar e premiar seus fornecedores baseado em painéis de consenso entre gerentes. Este artigo apresenta uma metodologia alternativa para classificação e premiação, que utiliza o método DEA em duas fases. Primeiramente, os fornecedores são selecionados de acordo com sua eficiência baseando-se em variáveis relativas a transações realizadas. Em seguida, os fornecedores são classificados de acordo com a opinião dos gerentes, utilizando o método DEA, com regiões de segurança e super-eficiência, como forma de agregação dos votos dos gerentes, definindo-se assim seus melhores fornecedores. O artigo apresenta também estudo de caso no segmento E&P da PETROBRAS.

Palavras-chave: avaliação de fornecedores, ordenação de votos, logística de suprimento.

ABSTRACT

The benefits of integration companies-suppliers top the strategic agendas of managers. Developing a system showing which suppliers merit continuing and deepening the partnership is difficult because of the large quantity of variables to be analyzed. The internationalized petroleum industry, requiring a large variety of materials, is no different. In this context, the Brazilian company PETROBRAS S.A. has a system to evaluate its suppliers based on a consensus panel formed by its managers. This paper shows a two phase methodology for classifying and awarding suppliers using the DEA model. Firstly, the suppliers are classified according to their efficiency based on commercial transactions realized. Secondly they are classified according to the opinions of the managers, using a DEA model for calculating votes, with the assurance regions and superefficiency defining the best suppliers. The paper presents a case study in the E&P segment of PETROBRAS and the results obtained with the methodology.

Keywords: suppliers performance evaluation, ranked voting system, logistics of supply.

1. Introduction

The question of classification and ordering of contestants is an elective process that has been well discussed in the academic world. In 1781, Borda proposed a method that takes his name and is still widely used, with innumerable variations, producing an index that aggregates the opinions of voters in a determined process of classification. Since then various authors have dedicated themselves to the theme with various methodologies for generating indices which permit the classification of competitors (Cook & Kress, 1990). These methodologies for the aggregation of votes in classification processes are based on the Borda method that defines, for each competitor, an index given by the sum of products of their various placings obtained in the selection process, with weightings established previously for each placing. Its main deficiency is the arbitrariness when establishing the weighting scheme.

From the 80s, the DEA model (Cooper, Seiford & Tone, 1999), Data Envelopment Analysis applied originally to evaluate the relative efficiency of independent units, yet homogenous, which use the same inputs to produce the same outputs, came to be used as an alternative method of classification because it did not require the previous establishment of weighting, with the aim of avoiding such arbitrariness (Cook & Kress, 1990; Green, Doyle & Cook, 1996; Hashimoto, 1995).

However, a serious inconvenience in its utilization as a method of classification is the possibility of having units tied with relative efficiency equal to 100%, that is, units at the frontier of relative efficiency. Various authors tackled this inconvenience, using various devices to break the tie, such as crossed evaluation (Estellita Lins, 2000; Green, Doyle & Cook, 1996), superefficiency (Anderson & Peterson, 1993) or assurance regions (Cooper, Seiford & Tone, 1999), among others.

This paper presents a methodology developed for the selection process and classification of competing suppliers in the Brazilian oil industry and the results of a case study conducted at the upstream segment of this industry. The proposed methodology is divided into two phases. In the first, to assemble m candidate suppliers we select n candidates considered competitors, by the application of the DEA model with constant returns on the scale. In the second phase, from an elective process, these competitors are classified by way of a DEA model with assurance regions and superefficiency to avoid ties between efficient suppliers (Hashimoto, 1997). These two phases are very common, in practical applications, in the classification processes of units when the universemof candidates is much greater than the necessary number n of competitors.

The paper deals, in the second section, with a description of the specific situation of the actual case study for which the methodology was developed, in the third section with a description of the methodology and in following, in the fourth section, the results of its application are presented and discussed.

2. The case of "The Best Suppliers of Materials to PETROBRAS Award – Campos Basin"

PETROBRAS, in its Exploration and Production segment – E&P, in the Campos Basin Business Unit UN-BC, has a Suppliers Management System – SMS, that registers in a systematic manner a report on events that happen during commercial transactions with each supplier.

This system has registrations for around two thousand suppliers, of these suppliers about one thousand seven hundred are considered active suppliers, that is with commercial transactions in the year. Around six hundred can be considered frequent (Petro&Química, 1999).

One of the functions of the SMS system is the control of the performance of the suppliers using three basic ways of marking. The first, called the Global Marking Item Competitiveness, shows the company’s volume of transactions with the supplier, using an algorithm based on two measurements: the total of items and the value supplied by the supplier in a determined period of time. The second basic marking, called Positive Events, shows the favorable events and punctuality that exceeded PETROBRAS’ expectations in relation to the commercial relationship with the suppliers and the third, Negative Events, shows the problems that occurred during this relationship.

Each year, as one of the actions of the system as a consequence of SMS, there is a selection and an award to the best suppliers of materials to PETROBRAS – Campos Basin. For this selection, the suppliers are classified into three distinct groups: Suppliers of Low Value Goods, Suppliers of Goods by Tender as Required by Law and Suppliers with Long Term Contracts. This division is necessary because of the large number of suppliers and of purchases made by this unit of PETROBRAS and because of their distinct characteristics.

The suppliers in the group "Suppliers of Low Value Goods" are characterized by a large volume of items purchased, the low value involved, the speed and lack of complexity in each transaction. Generally they are commodities and there is no major difficulty in their supply. The number of suppliers is large and generally they are small local companies.

The suppliers in the group "Suppliers of Goods by Tender as Required by Law" are characterized, in large part, by being big companies, by punctual and intermittent supply with high values involved and of the high complexity of the materials supplied and agreed time-scales. They are typically investment materials and capital goods, imported materials and of exclusive manufacture.

Finally, the suppliers in group "Suppliers with Long Term Contracts" are characterized by having firm contracts with the company for the systematic supply of materials whether complex or not, with a large number of transactions, or with special supply clauses. These suppliers establish consistent partnerships, by firm contracts with PETROBRAS, where all of the necessary elements for the completion of the contracts are specified.

To choose the suppliers who will compete for the annual prize, the five suppliers with the highest marks in each of the three groups, in the period analyzed, are pre-selected, based on the Global Marking Item Competitiveness. After which the suppliers are evaluated by a committee, made up of managers and supervisors from the purchasing area, by way of a consensus panel in which the scores from the Positive and Negative Events reached in the period are taken into account. This panel confirms the competitors for the award.

The competing suppliers are made known to the managers of all the units in PETROBRAS to check for any black marks which could eliminate them from the competition. After this phase, by way of another consensus panel formed by managers from the area, the six finalists, two from each group, are chosen and finally, from these six finalists, the best supplier from each group is chosen and they are declared the winners of the annual award.

3. The proposed methodology for choosing the best suppliers

The current methodology used to choose the best suppliers can lead to arbitrary results due to the various phases of the selection, and their methodologies based on consensus panels and their well known shortcomings. However the qualitative opinions of the managers involved in the process are extremely important in the selection and must be taken in to account. Based on this fact we developed a selection method split into two phases which seeks to reconcile the existing data from the SMS and the opinion of the managers within the system.

In the first phase we identify the suppliers considered competitors for the award, using quantitative data taken from the data generated by the SMS. After which, in a second phase, we choose the finalists and award winners, using as a base the votes of the managers for each of the competing suppliers. We describe these two phases next.

3.1 The first phase – Choosing the competitors

We utilize a DEA model with constant returns on the scale – CCR, output-oriented, as the proposed methodology to choose the fifteen suppliers, five from each purchasing group, which are passed to the managers and supervisors consensus panel to define the winners of the award, applied to each purchasing group individually. The DMUs considered in the methodology are those suppliers who scored points in the three basic evaluations, in ranking of the Global Evaluation System.

The first input considered in the formation of the model is the volume of transactions by value made in the period and in the group considered and that, in theory, any supplier in the group could supply. As this input is a constant value for all of the DMUs in the group, there is no problem in substituting it for the value of one. This change will not effect the results of the application of the DEA/CCR model. We call this input the Transactions Volume Input – TV.

The other input used was the value of the marks from the Negative Events – NE. We considered this input value as an undesirable output, or rather, the higher it is the more problems existed in the relationship between PETROBRAS and the supplier. The need to turn it into an input was apparent because of the impossibility of working with negative values as outputs, or with any coefficient derived from the value of NE, as it can adopt invalid values. The other problem with the value of NE is the dubious interpretation that can result from a low value for the indicator, which can be interpreted as a low report of negative events or that the supplier doesn’t have a significant volume of trade with PETROBRAS and because of this has a low negative marking.

We considered as outputs the points from the Positive Events – PE, Points for Items supplied – PI and Points for Value Supplied – PV. These last two outputs are parts taken from the Global Marking Item Competitiveness. This division was necessary due to the distinct characteristics of each supply group. While the number of transactions and consequently the number of items sold favors, of course, the purchases from the small value group, the value of the transactions, in the case of the suppliers that supply materials from the groups "suppliers of goods by tender as required by law" and "suppliers with long term contracts" come out with an advantage.

It is also worth emphasizing that the five best placed suppliers in each group are considered suitable for the second phase. If there are more than five suppliers with an efficiency equal to 100% (one hundred percent), all of them will go through to the second phase, as it could mean that they have marks outside of the normal that deserve more detailed analysis by the units managers.

3.2 Second Phase – Choosing the winners

With the methodology of the first phase of the process defined, we went on to develop a methodology for adding the managers and supervisors votes to choose the best suppliers in each group. This methodology for arranging the votes begins with each voter choosing a set of t competitors in order of preference, from n possible candidates. Then the final score for each candidate can be shown by:

Where s_j is the final score for each supplier, y_rj is the number of votes received by candidate j for the rank r and u_r is the arbitrary sequence of weighting given to each rank. Nevertheless, the arbitrary choice of weighting can lead to different final placings and logically to different awards. We can achieve a partial ordering using only the concept of stochastic dominance (Stein, Mizzi & Pfaffenberger, 1994), without specifying the weighting beforehand, considering them only decreasing and convex and adding two restrictions to the model established earlier:

However, for a complete ordering it is necessary to utilize a mathematical model based on the DEA model starting from the convexity restrictions (Hashimoto, 1997). The model considers DMUs the units being ordered, t outputs, which are the t possible placings for each candidate and the only input with the value of one. This model, known as the pure output model (Adolphson, Cornia & Walters, 1991) associated with the concept of assurance regions (Cooper, Seifor & Tone, 1999) which guarantees the decrease of the weightings, is presented by:

Even though this model allows the ranking of candidates without the arbitrary specification of weighting, it is still necessary to define the function d(r,e; ). One possible solution for this question was proposed by Cook & Kress in 1990, which is to maximize the value of e; , subject to the conditions that it is kept within the area of safety defined in 3, resulting in a new formula for the model:

However, this model is also arbitrary, because the choice of the function d(r,e; ) influences the result of the ranking, which brings back the original problem, according to Green, Doyle & Cook (1996). The authors warn that the use of the function d(r,e; ) = c coincides with the actual Borda method and they also recommend, for the intended ranking, the use of d(r,e; ) = 0, and the technique called cross evaluation.

In spite of their suggestion, the authors themselves confirm that this technique may not be precise in the ranking results when the universe of DMUs can change during the process, with the inclusion or exclusion of a DMU in the system being analyzed. The entry or exit of a recommended or non-recommended supplier for the award in question can occur in fact, which makes it impossible to use the cross evaluation technique in the methodology developed here.

Therefore we opt for only one arbitration in the construction of the model to be used in the ranking of the best suppliers: to establish that the weighting applied to the rankings have decreasing and convex values. This arbitration is already commonly adopted in ranking processes (Stein, Mizzi & Pfaffenberger, 1994), leavingonly the ties between efficient DMUs as an inconvenience still to be solved. We can break these ties through the use of the concept of superefficiency (Andersen & Petersen, 1993), in which there is no limit to the efficiency of the DMU which is being evaluated, leading to evaluation score greater than 100%.

Nevertheless, in order to use this approach, we remind that these scores are not proper distances or measures and may not represent cardinal values themselves. Therefore, we should explain the situation on which we are using it to break ties.

Hashimoto stated in 1997 that, for the dual problem of the model that follows (equations 5a-e), DEA/CCR with superefficiency scores greater than or equal to unity may be interpreted as the reciprocals of the maximum proportional decrease in votes preserving the top ranking. This seems to be one criterion to evaluate the tied candidates. Additionally, for the primal problem of the model, the author interpreted the ranked voting system using the DEA/AR with superefficiency scores as follows: "score less than unity is of course the index of how near the candidate being evaluated can come closer to the top when the top score is supposed to be unity. On the other hand, score greater than or equal unity is the index of how far the candidate being evaluated can have a lead on the candidate in the second place when the score of the second place is supposed to be unity. This interpretation seems to be more appropriate to an evaluation criterion in the ranked voting system."

A shortcoming of the superefficiency approach is that specialized DMUs are ranked too high because the maximum proportional decrease in the output vector preserving efficiency in some instances is determined in a subspace of the output space (Andersen & Petersen, 1993). This problem can be minimised under DEA/AR model with superefficiency "because the weights assigned for the outputs take some a priori conditions into consideration as the assurance region... This strengthens the appropriateness of the ranked voting system using DEA/AR with superefficiency." (Hashimoto, 1997).

Therefore the definitive model used for the ranking comes to be the model proposed by Hashimoto in 1997 and described by:

4. The application of the methodology

We apply the first phase of the methodology described in the previous topic, choosing the suppliers competing for the "Best Suppliers of Materials to PETROBRAS Award – Campos Basin", to the existing data from the SMS system of the Company where the case study occurred, and we select, from its registered suppliers, the suppliers to compete for the award. As an example, we show in Table 1 the relative data for each supplier in the small value purchasing group and their efficiency calculated by the proposed method. We are not presenting the results from other purchasing groups as they have been obtained in the same manner as the results shown. In this phase, we select six efficient suppliers, A1, A3, A6, A15, A33 and A52 as competitors for the second phase of the methodology proposed in the paper.

Having as the competing suppliers the six outstanding efficient DMUs in the earlier phase, we apply the proposed methodology to the second phase of the work. With the aim of preserving the original data and at the same time to introduce the presented model, the data relative to the votes of the managers are the results of a simulated voting process and don’t correspond to results of the real award for the year 2000. We show, therefore, in Table 2, the data relative to the marks gained by each supplier in the elective process and results from the application of the DEA/AR model with superefficiency. We also show the data relative to the ranking obtained by the suppliers by using the Borda method, the conventional DEA/CCR model and the DEA/CCR with superefficiency.

As we can see, the ranking according to the proposed model is as follows: A3, A1, A6, A33, A15 and A52 and in which the supplier A3obtained the best score, therefore was the winner of the award. Its worth pointing out that this order is the reverse of the first and second places that would have been obtained if the Borda method had been applied for the rankings. It is also interesting to note that the use of a DEA/CCR model with superefficiency would lead to very different results to those reached and the order of ranking would have been A33, A1, A3, A6, A15 and A52.

In Table 3 we show the weighting obtained for each of the rankings of each competitor.

For DMU A3, the winner, the model assigned maximum weighting to the first place and weighting close to zero for the rest, as this DMU is the best ranked in the first places. For the DMU A1 there is a clear distribution of weighting with the decrease factor practically constant at around 0.0357 (zero point zero-three-five-seven) as it is already well marked in the three first rankings. And for the other DMUs there is a practical equality between all of the assigned weightings, as all of them are well marked in the final rankings.

5. Conclusions

This paper sought to propose an alternative methodology for the definition of the "Best Suppliers of Materials to PETROBRAS Award – Campos Basin". This award, given annually, awards the suppliers that, beside showing the highest scores in the marking relative to large quantities and value of transactions accomplished in the year, show many occurrences that exceeded the expectations of PETROBRAS and few negative occurrences, that indicate relationship problems between the companies.

Besides the quantitative variables, the opinion of the managers and supervisors from the areas involved with the supply of materials, by way of the consensus panels, are also incorporated into the decision process. Therefore the routine of awards is divided into two phases. In the first, the suppliers competing for the award are chosen and in the second phase, the suppliers are classified in the order of preference of the managers and the first placed receives the title of award winner given by PETROBRAS.

We propose, in this paper, the substitution of the first phase of the award by a DEA/CCR model, which seeks to select the competitors in a less arbitrary manner. We also propose the use of a DEA/CCR model with assurance region, defined in a way to make the distribution of weighting given to the various placings decreasing and convex, and superefficiency, as a procedure to avoid possible ties between the efficient DMUs, for the addition of the votes of the managers in the second phase of the choice for the award, in a way that this phase also becomes less arbitrary and more transparent to the competing suppliers.

After the definition of the methodology, we select a purchase group, small value purchases, and apply the proposed procedure, with real data taken from the Suppliers Management System. We then select six suppliers in the first phase and we simulate the elective process, with fifteen voters, to apply the methodology to the second phase and choose the winning supplier of the annual award.

The results obtained differ from those obtained by the Borda method and the proposed methodology shows itself to be completely viable for the practical application for the award.

Acknowledgments

We acknowledge the Brazilian Oil & Gas Company PETROBRAS S.A, the CTPETRO Program and the CNPq – the Brazilian Agency for Development of Science and Technology – for partially sponsor this research.

References

Maria Aparecida Cavalcanti Netto

E-mails: [e-mail]

[e-mail]

Received September 2001;

accepted October 2002 after one revision.

• (1)Adolphson, D.L.; Cornia, G.C. & Walters, L.C. (1991).  A Unified Framework for Classifying DEA Models. In: Operational Research 90 [edited by H.E. Bradley], Pergamon Press, Oxford, 647-667.
• (2)Andersen, P. & Petersen, N.C. (1993).  A Procedure for Ranking Efficient Units in Data Envelopment Analysis. Management Science, 39(10), 1261-1264.
• (3)Cook, W.D. & Kress, M. (1990).  A Data Envelopment Model for Aggregating Preference Rankings. Management Science, 36(11), 1302-1310.
• (4)Cooper, W.W.; Seiford, L.M. & Tone, K. (2000).  Data Envelopment Analysis: a Comprehensive Text with Models, Applications, References, and DEA-Solver Software Kluwer Academic Publishers, Boston.
• (5)Estellita Lins, M.P. (2000). Análise Envoltória de Dados e Perspectivas de Integração no Ambiente de Apoio à Decisão [editado por M.P. Estellita Lins e L.A. Angulo-Meza], COPPE/UFRJ, Rio de Janeiro, 65-84.
• (6)Green, R.H.; Doyle, J.R. & Cook, W.D. (1996).  Preference Voting and Project Ranking using DEA and Cross-evaluation. European Journal of Operational Research, 90, 461-472.
• (7)Hashimoto, Akihiro. (1997).  A Ranked Voting System using a DEA/AR Exclusion Model: A note. European Journal of Operational Research, 97, 600-604.
• (8)PETRO&QUÍMICA. (1999).  Fornecedores da Bacia de Campos, Agosto/99, 51-56.
• (9)Stein, W.E.; Mizzi, P.J. & Pfaffenberger, R.C. (1994).  A Stochastic Dominance Analysis of Ranked Voting Systems with Scoring. European Journal of Operational Research, 74, 78-85.

^*

Corresponding author / autor para quem as correspondências devem ser encaminhadas

Publication Dates

• Publication in this collection
  28 Feb 2003
• Date of issue
  Dec 2002

History

• Accepted
  Oct 2002
• Received
  Sept 2001