THE RANK REVERSAL PROBLEM IN MULTI-CRITERIA DECISION MAKING: A LITERATURE REVIEW

Aires, Renan Felinto de Farias; Ferreira, Luciano

doi:10.1590/0101-7438.2018.038.02.0331

Acessibilidade / Reportar erro

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Articles • Pesqui. Oper. 38 (2) • May-Aug 2018 • https://doi.org/10.1590/0101-7438.2018.038.02.0331 copy

THE RANK REVERSAL PROBLEM IN MULTI-CRITERIA DECISION MAKING: A LITERATURE REVIEW

Authorship SCIMAGO INSTITUTIONS RANKINGS

ABSTRACT

Despite the importance of multicriteria decision-making (MCDM) techniques for constructing effective decision models, there are many criticisms due to the occurrence of a problem called rank reversal. Nevertheless, there is a lack of a systematic literature review on this important subject which involves different methods. This study reviews the pertinent literature on rank reversal, based on 130 related articles published from 1980 to 2015 in international journals, which were gathered and analyzed according to the following perspectives: multicriteria technique, year and journal in which the papers were published, co-authorship network, rank reversal types, and research goal. Thus our survey provides recommendations for future research, besides useful information and knowledge regarding rank reversal in the MCDM field.

Keywords:
rank reversal; ranking irregularities; multi-criteria decision making; decision theory; literature review

Year	N	%
1983-1985	4	3.1
1986-1988	7	5.4
1989-1991	12	9.2
1992-1994	12	9.2
1995-1997	22	16.9
1998-2000	9	6.9
2001-2003	5	3.8
2004-2006	16	12.3
2007-2009	14	10.8
2010-2012	10	7.7
2013-2015	17	14.6
Total	130	100

Journal title	No. of articles	Percentage (%)
European Journal of Operational Research	23	17.7
Journal of Multi-Criteria Decision Analysis	19	14.6
Omega	9	6.9
Management Science	9	6.9
Decision Sciences	7	5.4
Computers & Operations Research	7	5.4
Mathematical and Computer Modelling	6	4.6
Journal of the Operational Research Society	4	3.1
Decision Support Systems	3	2.3
Computers & Industrial Engineering	3	2.3
Expert Systems with Applications	2	1.5
International Journal of Multicriteria Decision Making	2	1.5
International Journal of Operational Research	2	1.5
Linear Algebra and its Applications	2	1.5
Mathematical Modelling	2	1.5
Advanced Materials Research	1	0.8
African Journal of Business Management	1	0.8
Applied Mathematics and Computation	1	0.8
Applied Mathematical Modelling	1	0.8
Applied Mathematics Research eXpress	1	0.8
Applied Soft Computing	1	0.8
Central European Journal of Operations Research	1	0.8
Cybernetics and Systems Analysis	1	0.8
Ecological Indicators	1	0.8
IEEE Transactions on Engineering	1	0.8
European Journal of Marketing	1	0.8
Facilities	1	0.8
Far East Journal of Mathematical Sciences	1	0.8
Fuzzy Sets and Systems	1	0.8
I. J. of Industrial Engineering: Applications and Practice	1	0.8
International Journal of Industrial and Systems Engineering	1	0.8
Interfaces	1	0.8
International Journal Math. Operational Research	1	0.8
International Journal of Management and Decision Making	1	0.8
International Transactions in Operational Research	1	0.8
Journal of Advanced Transportation	1	0.8
Journal of Automation and Information Sciences	1	0.8
Journal of Mathematical Psychology	1	0.8
Knowledge-Based Systems	1	0.8
Operations Research Letters	1	0.8
Quality Engineering	1	0.8
Systems Engineering	1	0.8
TOP	1	0.8
Communications in Statistics - Theory and Methods	1	0.8
Acta Polytechnica Hungarica	1	0.8
Total	130	100

Cluster	%	Papers
Survey	0.8	^{Maleki & Zahir (2013}63 MALEKI H & ZAHIR S. 2013. A Comprehensive Literature Review of the Rank Reversal Phenomenon in the Analytic Hierarchy Process. Journal of Multi-Criteria Decision Analysis , 20(3-4): 141-155.).
Application	9.23	^{Hämäläinen (1990}40 HÄMÄLÄINEN RP. 1990. A decision aid in the public debate on nuclear power. European Journal of Operational Research, 48(1): 66-76.), ^{Stewart (1992}113 STEWART T. 1992. A critical survey on the status of multiple criteria decision making theory and practice. Omega , 20(5-6): 569-586.), ^{Triantaphyllou & Mann (1995}121 TRIANTAPHYLLOU E & MANN SH. 1995. Using the Analytic Hierarchy Process For Decision Making in Engineering Applications: Some Challenges. Journal of Industrial Engineering: Applications and Practice, 2(1): 35-44.), ^{Davies (2001}20 DAVIES M. 2001. Adaptive AHP: a review of marketing applications with extensions. European Journal of Marketing, 35(7/8): 872-894.), ^{Sundarraj (2004}114 SUNDARRAJ RP. 2004. A Web-based AHP approach to standardize the process of managing service-contracts. Decision Support Systems, 37(3): 343-365.), ^{Wang, Parkan & Luo (2007}137 WANG Y-M, PARKAN C & LUO Y. 2007. A linear programming method for generating the most favorable weights from a pairwise comparison matrix. Computers & Operations Research, 35(12): 3918-3930.), ^{Troutt, Tadisina & Pendharkar (2009}125 TROUTT MD, TADISINA SK & PENDHARKAR PC. 2009. Value functions and trade-offs associated with the analytic hierarchy process composition law. International Journal of Mathematics in Operational Research, 1(1-2): 97-120.), ^{Bottero & Ferretti (2011}14 BOTTERO M & FERRETTI V. 2011. An analytic network process-based approach for location problems: the case of a new waste incinerator plant in the Province of Torino (Italy). Journal of Multi-Criteria Decision Analysis, 17(3-4): 63-84.), ^{Ishizaka & Labib (2011}46 ISHIZAKA A & LABIB A. 2011. Review of the main developments in the analytic hierarchy process. Expert Systems with Applications, 38(11): 14336-14345.), ^{Lima Junior et al. (2014}58 LIMA JUNIOR FR, OSIRO L & CARPINETTI LCR. 2014. A comparison between Fuzzy AHP and Fuzzy TOPSIS methods to supplier selection. Applied Soft Computing, 21: 194-209.), ^{Cinelli et al. (2014}19 CINELLI M, COLES SR & KIRWAN K. 2014. Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. Ecological Indicators, 4: 138-148.), ^{Bulut, Duru & Koçak (2015}18 BULUT E, DURU O & KOÇAK G. 2015. Rotational priority investigation in fuzzy analytic hierarchy process design: An empirical study on the marine engine selection problem. Applied Mathematical Modelling, 39(2): 913-923.).
Problem Solution	14.62	^{Belton & Gear (1983}10 BELTON V & GEAR T. 1983. On a short-coming of Saaty’s method of analytic hierarchies. Omega, 11(3): 228-230.), ^{Schoner & Wedley (1989}104 SCHONER B & WEDLEY WC. 1989. Ambiguous Criteria Weights in AHP: Consequences and Solutions. Decision Sciences, 20(3): 462-475.), ^{Lootsma (1993}60 LOOTSMA FA. 1993. Scale sensitivity in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 2(2): 87-110.), ^{Schoner et al. (1993}106 SCHONER B, WEDLEY WC & CHOO EU. 1993. A unified approach to AHP with linking pins. European Journal of Operational Research , 64(3): 384-392.), ^{Saaty (1994b}88 SAATY TL. 1994b. How to Make a Decision: The Analytic Hierarchy Process. Interfaces, 24(6): 19-43.), ^{Olson et al. (1995a}68 OLSON DL, FLIEDNER G & CURRIE K. 1995a. Comparison of the REMBRANDT system with Analytic hierarchy process. European Journal of Operational Research , 82(3): 522-539.), ^{Leung & Cao (2001}56 LEUNG LC & CAO D. 2001. On the efficacy of modeling multi-attribute decision problems using AHP and Sinarchy. European Journal of Operational Research, 132(1): 39-49.), ^{Kujawski(2005}53 KUJAWSKI E. 2005. A reference-dependent regret model for deterministic tradeoff studies. Systems Engineering, 8(2): 119-137.), ^{Totsenko (2006}117 TOTSENKO VG. 2006. On Problem of Reversal of Alternatives Ranks while Multicriteria Estimating. Journal of Automation and Information Sciences, 38(6): 1-11.), ^{Ramanathan (2006}76 RAMANATHAN R. 2006. Data envelopment analysis for weight derivation and aggregation in the Analytic hierarchy process. Computers & Operations Research , 33(5): 1289-1307.), ^{Ren et al. (2007}78 REN L, ZHANG Y, WANG Y & SUN Z. 2007. Comparative Analysis of a Novel M-TOPSIS Method and TOPSIS. Applied Mathematics Research eXpress.), ^{Jung et al. (2009}49 JUNG S-T, WOU Y-W, LI S-P & JULIAN P. 2009. A revisit to wash criteria in analytic Hierarchy Process. Far East Journal of Mathematical Sciences, 34(1): 31-36.), ^{Kong (2011}51 KONG F. 2011. Rank Reversal and Rank Preservation in TOPSIS. Advanced Materials Research, 204-210: 36-41.), ^{García-Cascales & Lamata (2012}36 GARCÍA-CASCALES MS & LAMATA MT. 2012. On rank reversal and TOPSIS method. Mathematical and Computer Modelling, 56: 123-132.), ^{Maleki & Saadat (2013}62 MALEKI H & SAADAT Y. 2013. Comparison of failure mode and effects analysis by using AHP vs. REMBRANDT system. International Journal of Industrial and Systems Engineering , 14(1): 5-19.), ^{Iç (2014}45 IÇ YT. 2014. A TOPSIS based design of experiment approach to assess company ranking. Applied Mathematics and Computation, 227: 630-647.), ^{Pankratova & Nedashkovskay (2014}71 PANKRATOVA ND & NEDASHKOVSKAY NI. 2014. Hybrid Method of Multicriteria Evaluation of Decision Alternatives. Cybernetics and Systems Analysis, 50(5): 701-711.), ^{Bairagi et al. (2015}5 BAIRAGI B, DEY B, SARKAR B & SANYAL SK. 2015. A De Novo multi-approaches multicriteria decision making technique with an application in performance evaluation of material handling device. Computers & Industrial Engineering, 87: 267-282.) and ^{Salabun (2015}97 SALABUN W. 2015. The Characteristic Objects Method: A New Distance-based Approach to Multicriteria Decision-making Problems. Journal of Multi-Criteria Decision Analysis , 22(1-2): 37-50.).
Simulation	13.85	^{Saaty & Vargas (1984a}92 SAATY TL & VARGAS LG. 1984a. Inconsistency and rank preservation. Journal of Mathematical Psychology, 28(2): 205-214.), ^{Triantaphyllou & Mann (1989}119 TRIANTAPHYLLOU E & MANN SH. 1989. An Examination of the Effectiveness of Multi-Dimentional Decision-Making Methods: A Decision-Making Paradox. International Journal of Decision Support Systems, 5: 303-312.), ^{Saaty & Vargas (1993}95 SAATY TL & VARGAS LG. 1993. Experiments on rank preservation and reversal in relative measurement. Mathematical and Computer Modelling, 17(4-5): 13-18.), ^{Saaty (1994a}87 SAATY TL. 1994a. Highlights and critical points inthe theory and application of the Analytic Hierarchy Process. European Journal of Operational Research , 74(3): 426-447.), ^{Olson et al. (1995b}69 OLSON DL, MOSHKOVICH HM, SCHELLENBERGER R & MECHITOV AI. 1995b. Consistency and Accuracy in Decision Aids: Experiments with Four Multiattribute Systems. Decision Sciences , 26(6): 723-747.), ^{Paulson & Zahir (1995}70 PAULSON D & ZAHIR S. 1995. Consequences of uncertainty in the analytic hierarchy process: A simulation approach. European Journal of Operational Research , 87(1): 45-56.), ^{Buede & Maxwell (1995}17 BUEDE DM & MAXWELL DT. 1995. Rank disagreement: A comparison of multi-criteria methodologies. Journal of Multi-Criteria Decision Analysis, 4(1): 1-21.), ^{Stam & Silva (1997}111 STAM A & SILVA APD. 1997. Stochastic Judgments in the AHP: The Measurement of Rank Reversal Probabilities. Decision Sciences, 28(3): 665-688.), ^{Triantaphyllou & Sánchez (1997}122 TRIANTAPHYLLOU E & SÁNCHEZ A. 1997. A Sensitivity Analysis Approach for Some Deterministic Multi-Criteria Decision-Making Methods. Decision Sciences, 28(1): 151-194.), ^{Zanakis et al. (1998}147 ZANAKIS SH, SOLOMON A, WISHART N & DUBLISH S. 1998. Multi-attribute decision making: A simulation comparison of select methods. European Journal of Operational Research , 107(3): 507-529.), ^{Srdjevic (2005}109 SRDJEVIC B. 2005. Combining different prioritization methods in the analytic hierarchy process synthesis. Computers & Operations Research, 32(7): 1897-1919.), ^{Ishizaka & Lusti (2006}47 ISHIZAKA A & LUSTI M. 2006. How to derive priorities in AHP: a comparative study. Central European Journal of Operations Research, 14(4): 387-400.), ^{Wang & Triantaphyllou (2008}133 WANG X & TRIANTAPHYLLOU E. 2008. Ranking irregularities when evaluating alternatives by using some ELECTRE methods. Omega , 36(1): 45-63.), ^{Wang & Luo (2009}135 WANG Y-M & LUO Y. 2009. On rank reversal in decision analysis. Mathematical and Computer Modelling, 49(5-6): 1221-1229.), ^{Verly & De Smet (2013}129 VERLY C & DE SMET Y. 2013. Some results about rank reversal instances in the PROMETHEE methods. International Journal of Multicriteria Decision Making , 3(4): 325-345.), ^{Soltanifar & Shahghobadi (2014}108 SOLTANIFAR M & SHAHGHOBADI S. 2014. Survey on rank preservation and rank reversal in data envelopment analysis. Knowledge-Based Systems, 60: 10-19.), ^{Dede et al. (2015}23 DEDE G, KAMALAKIS T & SPHICOPOULOS T. 2015. Convergence properties and practical estimation of the probability of rank reversal in pairwise comparisons for multicriteria decision making problems. European Journal of Operational Research, 241(2): 458-468.) and ^{Yaraghi et al. (2015}144 YARAGHI N, TABESH P, GUAN P & ZHUANG J. 2015. Comparison of AHP and Monte Carlo AHP Under Different Levels of Uncertainty. IEEE Transactions on Engineering Management, 62(1): 122-132.).
Problem Identification	61.54	^{Saaty & Vargas (1984b}93 SAATY TL & VARGAS LG. 1984b. The legitimacy of rank reversal. Omega , 12(5): 513-516.), ^{Belton & Gear (1985}11 BELTON V & GEAR T. 1985. The legitimacy of rank reversal: A comment. Omega, 13(3): 143-144.), ^{Saaty (1986}81 SAATY TL. 1986. Axiomatic Foundation of the Analytic Hierarchy Process. Management Science, 32(7): 841-855.), ^{Saaty (1987a}83 SAATY TL. 1987a. The analytic hierarchy process: what it is and how it is used. Mathematical Modelling, 9(3-5): 161-176.), ^{Saaty (1987b}84 SAATY TL. 1987b. Decision making, new information, ranking and structure. Mathematical Modelling, 8: 125-132.), ^{Saaty (1987c}85 SAATY TL. 1987c. Rank Generation, Preservation, and Reversal in the Analytic Hierarchy Decision Process. Decision Sciences, 18(2): 157-177.), ^{Harker & Vargas (1987}39 HARKER PT & VARGAS LG. 1987. The Theory of Ratio Scale Estimation: Saaty’s Analytic Hierarchy Process. Management Science, 33(11): 1383-1403.), ^{Saaty & Vargas (1987}94 SAATY TL & VARGAS LG. 1987. Uncertainty and rank order in the analytic hierarchy process. European Journal of Operational Research , 32(1): 107-117.), ^{Troutt (1988}123 TROUTT M. 1988. Rank reversal and the dependence of priorities on the underlying MAV function. Omega , 16(4): 365-367.), ^{Barzilai & Golany (1990}8 BARZILAI J & GOLANY B. 1990. Deriving weights from pairwise comparison matrices: The additive case. Operations Research Letters, 9(6): 407-410.), ^{Dyer (1990a}25 DYER JS. 1990a. A Clarification of Remarks on the Analytic Hierarchy Process. Management Science, 36(3): 274-275.), ^{Dyer (1990b}26 DYER JS. 1990b. Remarks on the Analytic Hierarchy Process. Management Science, 36(3): 249-258.), ^{Saaty (1990}86 SAATY TL. 1990. An Exposition of the AHP in Reply to the Paper “Remarks on the Analytic Hierarchy Process”. Management Sciences, 36(3): 259-268.), Harker & Vargas (1990), ^{Forman (1990}33 FORMAN EH. 1990. AHP Is Intended for More Than Expected Value Calculations. Decision Sciences, 21(3): 670-672.), ^{Troutt & Tadisina (1990}124 TROUTT M & TADISINA S. 1990. Corrigendum and further results: Rank reversal and the dependence of priorities on the underlying MAV function. Omega , 18(6): 655-656.), ^{Holder (1990}42 HOLDER RD. 1990. Some Comments on the Analytic Hierarchy Process. The Journal of the Operational Research Society, 41(11): 1073-1076.), ^{Gomes (1990}37 GOMES LFAM. 1990. Eliminating rank reversal in multicriteria analysis of urban transportation system alternatives. Journal of Advanced Transportation, 24(2): 181-184.), ^{Schoner et al. (1992}105 SCHONER B, WEDLEY WC & CHOO EU. 1992. A Rejoinder to Forman on AHP, with Emphasis on the Requirements of Composite Ratio Scales. Decision Sciences, 23(2): 509-517.), ^{Forman (1993}34 FORMAN EH. 1993. Facts and fictions about the analytic hierarchy process. Mathematical and Computer Modelling, 17(4-5): 19-26.), ^{Wedley et al. (1993}139 WEDLEY WC, SCHONER B & CHOO EU. 1993. Clustering, dependence and ratio scales in AHP: Rank reversals and incorrect priorities with a single criterion. Journal of Multi-Criteria Decision Analysis , 2(3): 145-158.), ^{Schenkerman (1994}102 SCHENKERMAN S. 1994. Avoiding rank reversal in AHP decision-support models. European Journal of Operational Research , 74(3): 407-419.), ^{Vargas (1994}127 VARGAS LG. 1994. Reply to Schenkerman’savoiding rank reversal in AHP decision support models. European Journal of Operational Research , 74(3): 420-425.), ^{Triantaphyllou & Mann (1994}120 TRIANTAPHYLLOU E & MANN SH. 1994. A computational evaluation of the original and revised Analytic hierarchy process. Computers & Industrial Engineering , 26(3): 609-618.), ^{Lai (1995}54 LAI S-K. 1995. A preference-based interpretation of AHP. Omega, 23(4): 453-462.), ^{Pérez (1995}72 PÉREZ J. 1995. Some Comments on Saaty’s AHP. Management Science , 41(6): 1091-1095.), ^{Dodd et al. (1995}24 DODD FJ, DONEGAN HA & MCMASTER TBM. 1995. Inverse inconsistency in analytic hierarchies. European Journal of Operational Research, 80(1): 86-93.), ^{Vinod Kumar & Ganesh (1996}132 VINOD KUMAR N & GANESH LS. 1996. An empirical analysis of the use of the Analytic Hierarchy Process for estimating membership values in a fuzzy set. Fuzzy Sets and Systems, 82(1): 1-16.), ^{De Keyser & Peeters (1996}22 DE KEYSER W & PEETERS P. 1996. A note on the use of PROMETHEE multicriteria methods. European Journal of Operational Research, 89(3): 457-461.), ^{Barzilai & Lootsma (1997}9 BARZILAI J & LOOTSMA FA. 1997. Power Relations and Group Aggregation in the Multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 6(3): 155-165.), ^{Salo & Hämäläinen (1997}99 SALO AA & HÄMÄLÄINEN RP. 1997. On the measurement of preferences in the analytic Hierarchy process. Journal of Multi-Criteria Decision Analysis , 6(6): 309-319.), ^{Belton & Gear (1997}12 BELTON V & GEAR T. 1997. On the meaning of relative importance. Journal of Multi-Criteria Decision Analysis, 6(6): 335-338.), ^{Lootsma & Barzilai (1997}61 LOOTSMA FA & BARZILAI J. 1997. Responseto the Comments by Larichev, Korhonen and Vargason “Power Relations and Group Aggregation in the Multiplicative AHP and SMART”. Journal of Multi-Criteria Decision Analysis , 6(3): 171-174.), ^{Barzilai (1997}7 BARZILAI J. 1997. Deriving Weights from Pairwise Comparison Matrices. The Journal of the Operational Research Society, 48(12): 1226-1232.), Vargas (1997), ^{Schoner et al. (1997}103 SCHONER B, CHOO EU & WEDLEY WC. 1997. A Comment on Rank Disagreement: A Comparison of Multi-Criteria Methodologies. Journal of Multi-Criteria Decision Analysis , 6(4): 197-200.), ^{Stam (1997}110 STAM A. 1997. Short Note on “On the Measurement of Preferences in the Analytic Hierarchy Process” by SALO AA & HÄMÄLÄINEN RP. Journal of Multi-Criteria Decision Analysis , 6(6): 338-339.), ^{Saaty (1997}89 SAATY TL. 1997. That is not the analytic Hierarchy process: what the AHP is and what it is not. Journal of Multi-Criteria Decision Analysis , 6(6): 324-335.), ^{Yang & Lee (1997}143 YANG J & LEE H. 1997. An AHP decision model for facility location selection. Facilities, 15(9-10): 241-254.), ^{Van Den Honert (1998}126 VAN DEN HONERT RC. 1998. Stochastic pairwise comparative judgements and direct ratings of alternatives in the REMBRANDT system. Journal of Multi-Criteria Decision Analysis , 7(2): 87-97.), ^{Farkas et al. (1999}29 FARKAS A, RÓZSA P & STUBNYA E. 1999. Transitive matrices and their applications. Linear Algebra and its Applications, 302-303: 423-433.), ^{Sarkis (1999}101 SARKIS J. 1999. A methodological framework for evaluating environmentally conscious manufacturing programs. Computers & Industrial Engineering , 36(4): 793-810.), ^{Zahir (1999}145 ZAHIR S. 1999. Geometry of decision making and the vector space formulation of the analytic hierarchy process. European Journal of Operational Research , 112(2): 373-396.), ^{Millet & Saaty (2000}64 MILLET I & SAATY TL. 2000. On the relativity of relative measures: accommodating both rank preservation and rank reversals in the AHP. European Journal of Operational Research , 121(1): 205-212.), ^{Sinuany-Stern et al. (2000}107 SINUANY-STERN Z, MEHREZ A & HADAD Y. 2000. An AHP/DEA methodology for ranking decision making units. International Transactions in Operational Research, 7(2): 109-124.), ^{Aguarón & Moreno-Jiménez (2000}2 AGUARÓN J & MORENO-JIMÉNEZ JM. 2000. Local stability intervals in the analytic hierarchy process. European Journal of Operational Research, 125(1): 113-132.), ^{Triantaphyllou (2001}118 TRIANTAPHYLLOU E. 2001. Two new cases of rank reversals when the AHP and some of its additive variants are used that do not occur with the multiplicative AHP. Journal of Multi-Criteria Decision Analysis , 10(1): 11-25.), ^{Finan & Hurley (2002}32 FINAN JS & HURLEY WJ. 2002. The analytic Hierarchy process: can wash criteria be ignored? Computers & Operations Research, 29(8): 1025-1030.), ^{Stam & Silva (2003}112 STAM A & SILVA APD. 2003. On multiplicative priority rating methods for the AHP. European Journal of Operational Research , 145(1): 92-108.), ^{Liberatore & Nydick (2004}57 LIBERATORE MJ & NYDICK RL. 2004. Wash criteria and the analytic hierarchy process. Computers & Operations Research, 31(6): 889-892), ^{Farkas et al. (2004}28 FARKAS A, GYÖRGY A & RÓZSA P. 2004. On the spectrum of pairwise comparison matrices. Linear Algebra and its Applications, 385: 443-462.), ^{Tavana (2004}115 TAVANA M. 2004. A subjective assessment of alternative mission architectures for the human exploration of Mars at NASA using multicriteria decision making. Computers & Operations Research, 31(7): 1147-1164.), ^{Leskinen & Kangas (2005}55 LESKINEN P & KANGAS J. 2005. Rank Reversals in Multi-Criteria Decision Analysis with Statistical Modelling of Ratio-Scale Pairwise Comparisons. The Journal of the Operational Research Society, 56(7): 855-861.), ^{Raharjo & Endah (2006}75 RAHARJO H & ENDAH D. 2006. Evaluating Relationship of Consistency Ratio and Number of Alternatives on Rank Reversal in the AHP. Quality Engineering, 18(1): 39-46.), ^{Pérez et al. (2006}73 PÉREZ J, JIMENO JL & MOKOTOFF E. 2006. Another potential shortcoming of AHP. Top, 14(1): 99-111.), ^{Saaty (2006}90 SAATY TL. 2006. Rank from comparisons and from ratings in the Analytic hierarchy/network processes. European Journal of Operational Research , 168(2): 557-570.), ^{Wang & Elhag (2006}134 WANG Y-M & ELHAG TMS. 2006. An approach to avoiding rank reversal in AHP. Decision Support Systems, 42(3): 1474-1480.), ^{Hochbaum & Levin (2006}41 HOCHBAUM DS & LEVIN A. 2006. Methodologies and Algorithms for Group-Rankings Decision. Management Science, 52(9): 1394-1408.), ^{Saaty & Vargas (2006}96 SAATY TL & VARGAS LG. 2006. The Analytic Hierarchy Process: wash criteria should not be ignored. International Journal of Management and Decision Making, 7(2): 180-188.), ^{Farkas (2007}27 FARKAS A. 2007. The analysis of the principal eigenvector of pairwise comparison matrices. Acta Polytechnica Hungarica, 4(2): 99-115.), ^{Bana e Costa & Vansnick (2008}6 BANA E COSTA CA & VANSNICK J-C. 2008. A critical analysis of the eigenvalue method used to derive priorities in AHP. European Journal of Operational Research, 187: 1422-1428.), ^{Lin et al. (2008}59 LIN J S-J, CHOU S-Y, CHOUHUANG WT & HSU CP. 2008. Note on “Wash criterion in analytic Hierarchy process”. European Journal of Operational Research, 185(1): 444-447.), ^{Saaty & Sagir (2009}91 SAATY TL & SAGIR M. 2009. An essay on rank preservation and reversal. Mathematical and Computer Modelling, 49: 1230-1243.), ^{Wijnmalen & Wedley (2009a}140 WIJNMALEN DJD & WEDLEY WC. 2009a. Correcting illegitimate rank reversals: proper adjustment of criteria weights prevent alleged AHP intransitivity. Journal of Multi-Criteria Decision Analysis , 15(5-6): 135-141.), ^{Wijnmalen & Wedley (2009b}141 WIJNMALEN DJD & WEDLEY WC. 2009b. Nondiscriminating criteria in the AHP: removal and rank reversal. Journal of Multi-Criteria Decision Analysis , 15(5-6): 143-149.), ^{Zahir (2009}146 ZAHIR S. 2009. Normalisation and rank reversals in the additive analytic hierarchy process: a new analysis. International Journal of Operational Research, 4(4): 446-467.), ^{Figueira & Roy (2009}31 FIGUEIRA JR & ROY B. 2009. A note on the paper, “Ranking irregularities when evaluating alternatives by using some ELECTRE methods”, by Wang and Triantaphyllou. Omega, 37(3): 731-733.), ^{Ramanathan & Ramanathan (2011}77 RAMANATHAN U & RAMANATHAN R. 2011. An investigation into rank reversal properties of the multiplicative AHP. International Journal of Operational Research, 11(1): 54-77.), ^{Jan et al. (2011}48 JAN KH, TUNG C-T & DENG P. 2011. Rank reversal problem related to wash criterion in Analytic hierarchy process (AHP). African Journal of Business Management, 5(20): 8301-8306.), ^{Mirhedayatian & Farzipoor Saen (2011}65 MIRHEDAYATIAN SM & FARZIPOOR SAEN R. 2011. A new approach for weight derivation using data envelopment analysis in the analytic hierarchy process. The Journal of the Operational Research Society , 62(8): 1585-1595.), ^{Salo & Punkka (2011}100 SALO A & PUNKKA A. 2011. Ranking Intervals and Dominance Relations for Ratio-Based Efficiency Analysis. Management Science, 57(1): 200-214.), ^{Hou (2012}43 HOU F. 2012. Rank Preserved Aggregation Rules and Application to Reliability Allocation. Communications in Statistics - Theory and Methods, 41(21): 3831-3845.), ^{Wang & Luo (2012}136 WANG Y-M & LUO Y. 2012. A note on “A new approach for weight derivation using data envelopment analysis in the analytic hierarchy process”. Mathematical and Computer Modelling, 56(3-4): 49-55.), ^{Vetschera & Almeida (2012}130 VETSCHERA R & ALMEIDA AT. 2012. A PROMETHEE-based approach to portfolio selection problems. Computers & Operations Research, 39(5): 1010-1020), ^{Rodríguez et al. (2013}79 RODRÍGUEZ A, ORTEGA F & CONCEPCIÓN R. 2013. A method for the selection of customized equipment suppliers. Expert Systems with Applications , 40(4): 1170-1176.), ^{Figueira et al. (2013}30 FIGUEIRA JR, GRECO S, ROY B & SLOWINSKI R. 2013. An Overview of ELECTRE Methods and their Recent Extensions. Journal of Multi-Criteria Decision Analysis, 20(1-2): 61-85.), ^{Brans (2015}15 BRANS J-P. 2015. The PROMETHEE adventure. International Journal of Multicriteria Decision Making, 5(4): 297-308.), ^{Morton (2015}66 MORTON A. 2015. Measurement issues in the evaluation of projects in a project portfolio. European Journal of Operational Research , 245(3): 789-796.), ^{Tomashevskii (2015}116 TOMASHEVSKII IL. 2015. Eigenvector ranking method as a measuring tool: Formulas for errors. European Journal of Operational Research , 240(3): 774-780.) and ^{Kulakowski (2015}52 KULAKOWSKI K. 2015. Notes on order preservation and consistency in AHP. European Journal of Operational Research, 245(1): 333-337.).

Approaches	Authors	Contribution
AHP	^{Belton & Gear (1983}10 BELTON V & GEAR T. 1983. On a short-coming of Saaty’s method of analytic hierarchies. Omega, 11(3): 228-230.)	BG normalization procedure.
	^{Saaty & Vargas (1984a}92 SAATY TL & VARGAS LG. 1984a. Inconsistency and rank preservation. Journal of Mathematical Psychology, 28(2): 205-214.)	Evaluation of the deriving ratio estimates regarding inconsistency and preservation ranking.
	^{Saaty & Vargas (1984b}93 SAATY TL & VARGAS LG. 1984b. The legitimacy of rank reversal. Omega , 12(5): 513-516.)	Demonstration of ranking inconsistencies in ^{Belton & Gear (1983}10 BELTON V & GEAR T. 1983. On a short-coming of Saaty’s method of analytic hierarchies. Omega, 11(3): 228-230.).
	^{Belton & Gear (1985}11 BELTON V & GEAR T. 1985. The legitimacy of rank reversal: A comment. Omega, 13(3): 143-144.)	Explanation on BG normalization procedure.
	^{Saaty (1986}81 SAATY TL. 1986. Axiomatic Foundation of the Analytic Hierarchy Process. Management Science, 32(7): 841-855.)	Structural dependence of the criteria may cause RRP.
	^{Saaty (1987a}83 SAATY TL. 1987a. The analytic hierarchy process: what it is and how it is used. Mathematical Modelling, 9(3-5): 161-176.)	Similar to ^{Saaty (1986}81 SAATY TL. 1986. Axiomatic Foundation of the Analytic Hierarchy Process. Management Science, 32(7): 841-855.).
	^{Saaty (1987b}84 SAATY TL. 1987b. Decision making, new information, ranking and structure. Mathematical Modelling, 8: 125-132.)	RRP may be caused if the importance of the criteria depends both on the number of alternatives and on the strength of their ranking.
	^{Saaty (1987c}85 SAATY TL. 1987c. Rank Generation, Preservation, and Reversal in the Analytic Hierarchy Decision Process. Decision Sciences, 18(2): 157-177.)	Similar to ^{Saaty (1986}81 SAATY TL. 1986. Axiomatic Foundation of the Analytic Hierarchy Process. Management Science, 32(7): 841-855.) and ^{Saaty (1987a}83 SAATY TL. 1987a. The analytic hierarchy process: what it is and how it is used. Mathematical Modelling, 9(3-5): 161-176.).
	^{Harker & Vargas (1987}39 HARKER PT & VARGAS LG. 1987. The Theory of Ratio Scale Estimation: Saaty’s Analytic Hierarchy Process. Management Science, 33(11): 1383-1403.)	RRP is caused by misuse of the method.
	^{Saaty & Vargas (1987}94 SAATY TL & VARGAS LG. 1987. Uncertainty and rank order in the analytic hierarchy process. European Journal of Operational Research , 32(1): 107-117.)	RRP is caused by uncertainty of the decision-maker.
	^{Troutt (1988}123 TROUTT M. 1988. Rank reversal and the dependence of priorities on the underlying MAV function. Omega , 16(4): 365-367.)	RRP is caused by aggregation rule.
	^{Triantaphyllou & Mann (1989}119 TRIANTAPHYLLOU E & MANN SH. 1989. An Examination of the Effectiveness of Multi-Dimentional Decision-Making Methods: A Decision-Making Paradox. International Journal of Decision Support Systems, 5: 303-312.)	Analysis of four methods regarding rank reversal rate.
	^{Schoner & Wedley (1989}104 SCHONER B & WEDLEY WC. 1989. Ambiguous Criteria Weights in AHP: Consequences and Solutions. Decision Sciences, 20(3): 462-475.)	RRP is caused by incorrect criteria assessments.
	^{Barzilai & Golany (1990}8 BARZILAI J & GOLANY B. 1990. Deriving weights from pairwise comparison matrices: The additive case. Operations Research Letters, 9(6): 407-410.)	Arithmetic mean is used to solve the RRP.
	^{Dyer (1990a}25 DYER JS. 1990a. A Clarification of Remarks on the Analytic Hierarchy Process. Management Science, 36(3): 274-275.)	Rankings are arbitrary in the face of inappropriate criteria weights.
	^{Dyer (1990b}26 DYER JS. 1990b. Remarks on the Analytic Hierarchy Process. Management Science, 36(3): 249-258.)	Similar to ^{Dyer (1990a}25 DYER JS. 1990a. A Clarification of Remarks on the Analytic Hierarchy Process. Management Science, 36(3): 274-275.).
	^{Saaty (1990}86 SAATY TL. 1990. An Exposition of the AHP in Reply to the Paper “Remarks on the Analytic Hierarchy Process”. Management Sciences, 36(3): 259-268.)	Similar to ^{Harker & Vargas (1987}39 HARKER PT & VARGAS LG. 1987. The Theory of Ratio Scale Estimation: Saaty’s Analytic Hierarchy Process. Management Science, 33(11): 1383-1403.).
	^{Harker & Vargas (1990}39 HARKER PT & VARGAS LG. 1987. The Theory of Ratio Scale Estimation: Saaty’s Analytic Hierarchy Process. Management Science, 33(11): 1383-1403.)	Similar to ^{Harker & Vargas (1987}39 HARKER PT & VARGAS LG. 1987. The Theory of Ratio Scale Estimation: Saaty’s Analytic Hierarchy Process. Management Science, 33(11): 1383-1403.) and ^{Saaty (1990}86 SAATY TL. 1990. An Exposition of the AHP in Reply to the Paper “Remarks on the Analytic Hierarchy Process”. Management Sciences, 36(3): 259-268.).
	^{Forman (1990}33 FORMAN EH. 1990. AHP Is Intended for More Than Expected Value Calculations. Decision Sciences, 21(3): 670-672.)	Referenced AHP does not provide convincing arguments for its use.
	^{Troutt & Tadisina (1990}124 TROUTT M & TADISINA S. 1990. Corrigendum and further results: Rank reversal and the dependence of priorities on the underlying MAV function. Omega , 18(6): 655-656.)	RRP was avoided through the use of Multiattribute Value (MAV).
	^{Holder (1990}42 HOLDER RD. 1990. Some Comments on the Analytic Hierarchy Process. The Journal of the Operational Research Society, 41(11): 1073-1076.)	Reported inconsistencies in the method.
	Hämäläinen (1990)	RRP is mentioned in a specific context of application context.
	^{Stewart (1992}113 STEWART T. 1992. A critical survey on the status of multiple criteria decision making theory and practice. Omega , 20(5-6): 569-586.)	RRP is mentioned.
	^{Schoner et al. (1992}105 SCHONER B, WEDLEY WC & CHOO EU. 1992. A Rejoinder to Forman on AHP, with Emphasis on the Requirements of Composite Ratio Scales. Decision Sciences, 23(2): 509-517.)	RRP may be caused by the process of normalizing local priorities.
	^{Forman (1993}34 FORMAN EH. 1993. Facts and fictions about the analytic hierarchy process. Mathematical and Computer Modelling, 17(4-5): 19-26.)	RRP may be caused by substitution effect.
	^{Lootsma (1993}60 LOOTSMA FA. 1993. Scale sensitivity in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 2(2): 87-110.)	Multiplicative AHP method.
	^{Schoner et al. (1993}106 SCHONER B, WEDLEY WC & CHOO EU. 1993. A unified approach to AHP with linking pins. European Journal of Operational Research , 64(3): 384-392.)	RRP is mentioned.
	^{Saaty & Vargas (1993}95 SAATY TL & VARGAS LG. 1993. Experiments on rank preservation and reversal in relative measurement. Mathematical and Computer Modelling, 17(4-5): 13-18.)	Evaluation of inconsistencies and ranking preservation.
	^{Wedley et al. (1993}139 WEDLEY WC, SCHONER B & CHOO EU. 1993. Clustering, dependence and ratio scales in AHP: Rank reversals and incorrect priorities with a single criterion. Journal of Multi-Criteria Decision Analysis , 2(3): 145-158.)	Axiom 3 causes the RRP.
	^{Saaty (1994a}87 SAATY TL. 1994a. Highlights and critical points inthe theory and application of the Analytic Hierarchy Process. European Journal of Operational Research , 74(3): 426-447.)	Analysis of ranking procedures with implications to RRP.
	^{Saaty (1994b}88 SAATY TL. 1994b. How to Make a Decision: The Analytic Hierarchy Process. Interfaces, 24(6): 19-43.)	Ideal mode of setting priorities may avoid RRP.
	^{Schenkerman (1994}102 SCHENKERMAN S. 1994. Avoiding rank reversal in AHP decision-support models. European Journal of Operational Research , 74(3): 407-419.)	Relative measurement is not a valid manner of measuring.
	^{Vargas (1994}127 VARGAS LG. 1994. Reply to Schenkerman’savoiding rank reversal in AHP decision support models. European Journal of Operational Research , 74(3): 420-425.)	RRP is an acceptable phenomenon.
	^{Triantaphyllou & Mann (1994}120 TRIANTAPHYLLOU E & MANN SH. 1994. A computational evaluation of the original and revised Analytic hierarchy process. Computers & Industrial Engineering , 26(3): 609-618.)	Occurrence of ranking irregularities in AHP or revised AHP.
	^{Lai (1995}54 LAI S-K. 1995. A preference-based interpretation of AHP. Omega, 23(4): 453-462.)	AHP’- new scaling technique.
	^{Olson et al. (1995b}69 OLSON DL, MOSHKOVICH HM, SCHELLENBERGER R & MECHITOV AI. 1995b. Consistency and Accuracy in Decision Aids: Experiments with Four Multiattribute Systems. Decision Sciences , 26(6): 723-747.)	Evaluation of four decision support systems regarding inconsistency and preservation ranking.
	^{Paulson & Zahir (1995}70 PAULSON D & ZAHIR S. 1995. Consequences of uncertainty in the analytic hierarchy process: A simulation approach. European Journal of Operational Research , 87(1): 45-56.)	RRP and its probability of occurrence over a wide range of uncertainties.
	^{Pérez (1995}72 PÉREZ J. 1995. Some Comments on Saaty’s AHP. Management Science , 41(6): 1091-1095.)	Relation between the criteria weights and the RRP.
	^{Dodd et al. (1995}24 DODD FJ, DONEGAN HA & MCMASTER TBM. 1995. Inverse inconsistency in analytic hierarchies. European Journal of Operational Research, 80(1): 86-93.)	MAHP method.
	^{Triantaphyllou & Mann (1995}121 TRIANTAPHYLLOU E & MANN SH. 1995. Using the Analytic Hierarchy Process For Decision Making in Engineering Applications: Some Challenges. Journal of Industrial Engineering: Applications and Practice, 2(1): 35-44.)	RRP is mentioned.
	^{Vinod Kumar & Ganesh (1996}132 VINOD KUMAR N & GANESH LS. 1996. An empirical analysis of the use of the Analytic Hierarchy Process for estimating membership values in a fuzzy set. Fuzzy Sets and Systems, 82(1): 1-16.)	Influence of the RRP for estimating membership values in a fuzzy set.
	^{Barzilai & Lootsma (1997}9 BARZILAI J & LOOTSMA FA. 1997. Power Relations and Group Aggregation in the Multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 6(3): 155-165.)	Similar to ^{Lootsma (1993}60 LOOTSMA FA. 1993. Scale sensitivity in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 2(2): 87-110.).
	^{Salo & Hämäläinen (1997}99 SALO AA & HÄMÄLÄINEN RP. 1997. On the measurement of preferences in the analytic Hierarchy process. Journal of Multi-Criteria Decision Analysis , 6(6): 309-319.)	Supermatrix technique does not eliminate the RRP.
	^{Belton & Gear (1997}12 BELTON V & GEAR T. 1997. On the meaning of relative importance. Journal of Multi-Criteria Decision Analysis, 6(6): 335-338.)	RRP is caused by the relative importance of the average performance on each criterion.
	^{Lootsma & Barzilai (1997}61 LOOTSMA FA & BARZILAI J. 1997. Responseto the Comments by Larichev, Korhonen and Vargason “Power Relations and Group Aggregation in the Multiplicative AHP and SMART”. Journal of Multi-Criteria Decision Analysis , 6(3): 171-174.)	Similar to ^{Lootsma (1993}60 LOOTSMA FA. 1993. Scale sensitivity in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 2(2): 87-110.) and ^{Barzilai & Lootsma (1997}9 BARZILAI J & LOOTSMA FA. 1997. Power Relations and Group Aggregation in the Multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 6(3): 155-165.).
	^{Barzilai (1997}7 BARZILAI J. 1997. Deriving Weights from Pairwise Comparison Matrices. The Journal of the Operational Research Society, 48(12): 1226-1232.)	Similar to ^{Lootsma (1993}60 LOOTSMA FA. 1993. Scale sensitivity in the multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 2(2): 87-110.), ^{Barzilai & Lootsma (1997}9 BARZILAI J & LOOTSMA FA. 1997. Power Relations and Group Aggregation in the Multiplicative AHP and SMART. Journal of Multi-Criteria Decision Analysis, 6(3): 155-165.) and ^{Lootsma & Barzilai (1997}61 LOOTSMA FA & BARZILAI J. 1997. Responseto the Comments by Larichev, Korhonen and Vargason “Power Relations and Group Aggregation in the Multiplicative AHP and SMART”. Journal of Multi-Criteria Decision Analysis , 6(3): 171-174.).
	Vargas (1997)	Multiplicative AHP is invalid.
	^{Schoner et al. (1997}103 SCHONER B, CHOO EU & WEDLEY WC. 1997. A Comment on Rank Disagreement: A Comparison of Multi-Criteria Methodologies. Journal of Multi-Criteria Decision Analysis , 6(4): 197-200.)	Describe the modified AHP techniques.
	^{Stam (1997}110 STAM A. 1997. Short Note on “On the Measurement of Preferences in the Analytic Hierarchy Process” by SALO AA & HÄMÄLÄINEN RP. Journal of Multi-Criteria Decision Analysis , 6(6): 338-339.)	Multiplicative AHP method may avoid RRP.
	^{Stam & Silva (1997}111 STAM A & SILVA APD. 1997. Stochastic Judgments in the AHP: The Measurement of Rank Reversal Probabilities. Decision Sciences, 28(3): 665-688.)	Evaluation of the RRP in a stochastic context.
	^{Saaty (1997}89 SAATY TL. 1997. That is not the analytic Hierarchy process: what the AHP is and what it is not. Journal of Multi-Criteria Decision Analysis , 6(6): 324-335.)	Similar to ^{Saaty (1994b}88 SAATY TL. 1994b. How to Make a Decision: The Analytic Hierarchy Process. Interfaces, 24(6): 19-43.).
	^{Yang & Lee (1997}143 YANG J & LEE H. 1997. An AHP decision model for facility location selection. Facilities, 15(9-10): 241-254.)	RRP is mentioned.
	^{Triantaphyllou & Sánchez (1997}122 TRIANTAPHYLLOU E & SÁNCHEZ A. 1997. A Sensitivity Analysis Approach for Some Deterministic Multi-Criteria Decision-Making Methods. Decision Sciences, 28(1): 151-194.)	Evaluation of three decision methods regarding to the RRP.
	^{Van Den Honert (1998}126 VAN DEN HONERT RC. 1998. Stochastic pairwise comparative judgements and direct ratings of alternatives in the REMBRANDT system. Journal of Multi-Criteria Decision Analysis , 7(2): 87-97.)	RRP is caused by uncertainty in the decision-maker’s preference judgments.
	^{Farkas et al. (1999}29 FARKAS A, RÓZSA P & STUBNYA E. 1999. Transitive matrices and their applications. Linear Algebra and its Applications, 302-303: 423-433.)	RRP is inherent in the AHP.
	^{Zahir (1999}145 ZAHIR S. 1999. Geometry of decision making and the vector space formulation of the analytic hierarchy process. European Journal of Operational Research , 112(2): 373-396.)	RRP is mentioned.
	^{Sarkis (1999}101 SARKIS J. 1999. A methodological framework for evaluating environmentally conscious manufacturing programs. Computers & Industrial Engineering , 36(4): 793-810.)	RRP is avoided through the use of ANP method.
	^{Millet & Saaty (2000}64 MILLET I & SAATY TL. 2000. On the relativity of relative measures: accommodating both rank preservation and rank reversals in the AHP. European Journal of Operational Research , 121(1): 205-212.)	Similar to ^{Saaty (1994b}88 SAATY TL. 1994b. How to Make a Decision: The Analytic Hierarchy Process. Interfaces, 24(6): 19-43.).
	^{Sinuany-Stern et al. (2000}107 SINUANY-STERN Z, MEHREZ A & HADAD Y. 2000. An AHP/DEA methodology for ranking decision making units. International Transactions in Operational Research, 7(2): 109-124.)	AHP/DEA method.
	^{Aguarón & Moreno-Jiménez (2000}2 AGUARÓN J & MORENO-JIMÉNEZ JM. 2000. Local stability intervals in the analytic hierarchy process. European Journal of Operational Research, 125(1): 113-132.)	Illustration of the RRP regarding to the alteration of the best alternative and transitivity property.
	^{Davies (2001}20 DAVIES M. 2001. Adaptive AHP: a review of marketing applications with extensions. European Journal of Marketing, 35(7/8): 872-894.)	RRP is mentioned.
	^{Leung & Cao (2001}56 LEUNG LC & CAO D. 2001. On the efficacy of modeling multi-attribute decision problems using AHP and Sinarchy. European Journal of Operational Research, 132(1): 39-49.)	RRP is avoided through the use of Sinarchy, a particular form of ANP method.
	^{Triantaphyllou (2001}118 TRIANTAPHYLLOU E. 2001. Two new cases of rank reversals when the AHP and some of its additive variants are used that do not occur with the multiplicative AHP. Journal of Multi-Criteria Decision Analysis , 10(1): 11-25.)	RRP is caused by normalization step and use of additive function.
	^{Finan & Hurley (2002}32 FINAN JS & HURLEY WJ. 2002. The analytic Hierarchy process: can wash criteria be ignored? Computers & Operations Research, 29(8): 1025-1030.)	RRP and non-discriminating criterion.
	^{Stam & Silva (2003}112 STAM A & SILVA APD. 2003. On multiplicative priority rating methods for the AHP. European Journal of Operational Research , 145(1): 92-108.)	Similar to ^{Stam (1997}110 STAM A. 1997. Short Note on “On the Measurement of Preferences in the Analytic Hierarchy Process” by SALO AA & HÄMÄLÄINEN RP. Journal of Multi-Criteria Decision Analysis , 6(6): 338-339.).
	^{Liberatore & Nydick (2004}57 LIBERATORE MJ & NYDICK RL. 2004. Wash criteria and the analytic hierarchy process. Computers & Operations Research, 31(6): 889-892)	Non-discriminating criterion a may not cause RRP.
	^{Farkas et al. (2004}28 FARKAS A, GYÖRGY A & RÓZSA P. 2004. On the spectrum of pairwise comparison matrices. Linear Algebra and its Applications, 385: 443-462.)	Illustration of the RRP.
	^{Tavana (2004}115 TAVANA M. 2004. A subjective assessment of alternative mission architectures for the human exploration of Mars at NASA using multicriteria decision making. Computers & Operations Research, 31(7): 1147-1164.)	RRP is mentioned.
	^{Sundarraj (2004}114 SUNDARRAJ RP. 2004. A Web-based AHP approach to standardize the process of managing service-contracts. Decision Support Systems, 37(3): 343-365.)	RRP is mentioned.
	^{Srdjevic (2005}109 SRDJEVIC B. 2005. Combining different prioritization methods in the analytic hierarchy process synthesis. Computers & Operations Research, 32(7): 1897-1919.)	Multicriteria Preference Synthesis Procedure (MPS).
	^{Leskinen & Kangas (2005}55 LESKINEN P & KANGAS J. 2005. Rank Reversals in Multi-Criteria Decision Analysis with Statistical Modelling of Ratio-Scale Pairwise Comparisons. The Journal of the Operational Research Society, 56(7): 855-861.)	RRP is avoided through the use of ratio-scale measurement.
	^{Kujawski (2005}53 KUJAWSKI E. 2005. A reference-dependent regret model for deterministic tradeoff studies. Systems Engineering, 8(2): 119-137.)	Reference-Dependent Regret Model (RDRM).
	^{Raharjo & Endah (2006}75 RAHARJO H & ENDAH D. 2006. Evaluating Relationship of Consistency Ratio and Number of Alternatives on Rank Reversal in the AHP. Quality Engineering, 18(1): 39-46.)	RRP is caused by consistency ratio and the number of alternatives.
	^{Pérez et al. (2006}73 PÉREZ J, JIMENO JL & MOKOTOFF E. 2006. Another potential shortcoming of AHP. Top, 14(1): 99-111.)	RRP may be caused by adding indifferent criteria.
	^{Saaty (2006}90 SAATY TL. 2006. Rank from comparisons and from ratings in the Analytic hierarchy/network processes. European Journal of Operational Research , 168(2): 557-570.)	Relative measurement, conditional independence or structural dependence plays an important role in influencing the RRP.
	^{Wang & Elhag (2006}134 WANG Y-M & ELHAG TMS. 2006. An approach to avoiding rank reversal in AHP. Decision Support Systems, 42(3): 1474-1480.)	RRP is caused by the change of local priorities before and after adding or removing alternatives.
	^{Totsenko (2006}117 TOTSENKO VG. 2006. On Problem of Reversal of Alternatives Ranks while Multicriteria Estimating. Journal of Automation and Information Sciences, 38(6): 1-11.)	Analytic Hierarchy Process Advanced (AHPA).
	^{Hochbaum & Levin (2006}41 HOCHBAUM DS & LEVIN A. 2006. Methodologies and Algorithms for Group-Rankings Decision. Management Science, 52(9): 1394-1408.)	RRP is mentioned in a group decision context.
	^{Saaty & Vargas (2006}96 SAATY TL & VARGAS LG. 2006. The Analytic Hierarchy Process: wash criteria should not be ignored. International Journal of Management and Decision Making, 7(2): 180-188.)	Similar to ^{Liberatore & Nydick (2004}57 LIBERATORE MJ & NYDICK RL. 2004. Wash criteria and the analytic hierarchy process. Computers & Operations Research, 31(6): 889-892).
	^{Ramanathan (2006}76 RAMANATHAN R. 2006. Data envelopment analysis for weight derivation and aggregation in the Analytic hierarchy process. Computers & Operations Research , 33(5): 1289-1307.)	AHP/DEA method.
	^{Ishizaka & Lusti (2006}47 ISHIZAKA A & LUSTI M. 2006. How to derive priorities in AHP: a comparative study. Central European Journal of Operations Research, 14(4): 387-400.)	Evaluation of four AHP ratio scaling methods: the right eigenvalue method, the left eigenvalue method, the geometric mean and the mean of normalized values.
	^{Farkas (2007}27 FARKAS A. 2007. The analysis of the principal eigenvector of pairwise comparison matrices. Acta Polytechnica Hungarica, 4(2): 99-115.)	Similar to ^{Farkas et al. (1999}29 FARKAS A, RÓZSA P & STUBNYA E. 1999. Transitive matrices and their applications. Linear Algebra and its Applications, 302-303: 423-433.).
	^{Wang, Parkan & Luo (2007}137 WANG Y-M, PARKAN C & LUO Y. 2007. A linear programming method for generating the most favorable weights from a pairwise comparison matrix. Computers & Operations Research, 35(12): 3918-3930.)	DEAHP suffers from RRP.
	^{Lin et al. (2008}59 LIN J S-J, CHOU S-Y, CHOUHUANG WT & HSU CP. 2008. Note on “Wash criterion in analytic Hierarchy process”. European Journal of Operational Research, 185(1): 444-447.)	RRP is not caused by non-discriminating criterion.
	^{Bana e Costa & Vansnick (2008}6 BANA E COSTA CA & VANSNICK J-C. 2008. A critical analysis of the eigenvalue method used to derive priorities in AHP. European Journal of Operational Research, 187: 1422-1428.)	RRP is mentioned.
	^{Saaty & Sagir (2009}91 SAATY TL & SAGIR M. 2009. An essay on rank preservation and reversal. Mathematical and Computer Modelling, 49: 1230-1243.)	Preserve rank in all situations may be wrong.
	^{Wijnmalen & Wedley (2009a}140 WIJNMALEN DJD & WEDLEY WC. 2009a. Correcting illegitimate rank reversals: proper adjustment of criteria weights prevent alleged AHP intransitivity. Journal of Multi-Criteria Decision Analysis , 15(5-6): 135-141.)	RRP is caused by the independence axiom.
	^{Wijnmalen & Wedley (2009b}141 WIJNMALEN DJD & WEDLEY WC. 2009b. Nondiscriminating criteria in the AHP: removal and rank reversal. Journal of Multi-Criteria Decision Analysis , 15(5-6): 143-149.)	Similar to ^{Liberatore & Nydick (2004}57 LIBERATORE MJ & NYDICK RL. 2004. Wash criteria and the analytic hierarchy process. Computers & Operations Research, 31(6): 889-892) and ^{Saaty & Vargas (2006}96 SAATY TL & VARGAS LG. 2006. The Analytic Hierarchy Process: wash criteria should not be ignored. International Journal of Management and Decision Making, 7(2): 180-188.).
	^{Zahir (2009}146 ZAHIR S. 2009. Normalisation and rank reversals in the additive analytic hierarchy process: a new analysis. International Journal of Operational Research, 4(4): 446-467.)	RRP is caused by the aggregation rule.
	^{Jung et al. (2009}49 JUNG S-T, WOU Y-W, LI S-P & JULIAN P. 2009. A revisit to wash criteria in analytic Hierarchy Process. Far East Journal of Mathematical Sciences, 34(1): 31-36.)	How avoid RRP from non-discriminating criterion criteria.
	^{Troutt, Tadisina & Pendharkar (2009}125 TROUTT MD, TADISINA SK & PENDHARKAR PC. 2009. Value functions and trade-offs associated with the analytic hierarchy process composition law. International Journal of Mathematics in Operational Research, 1(1-2): 97-120.)	The number of alternatives being compared in the AHP is the main cause of rank reversal.
	^{Ramanathan & Ramanathan (2011}77 RAMANATHAN U & RAMANATHAN R. 2011. An investigation into rank reversal properties of the multiplicative AHP. International Journal of Operational Research, 11(1): 54-77.)	Relation between the Multiplicative AHP and the RRP.
	^{Ishizaka & Labib (2011}46 ISHIZAKA A & LABIB A. 2011. Review of the main developments in the analytic hierarchy process. Expert Systems with Applications, 38(11): 14336-14345.)	RRP is mentioned.
	^{Jan et al. (2011}48 JAN KH, TUNG C-T & DENG P. 2011. Rank reversal problem related to wash criterion in Analytic hierarchy process (AHP). African Journal of Business Management, 5(20): 8301-8306.)	Relation between the non-discriminating criterion and the RRP.
	^{Mirhedayatian & Farzipoor Saen (2011}65 MIRHEDAYATIAN SM & FARZIPOOR SAEN R. 2011. A new approach for weight derivation using data envelopment analysis in the analytic hierarchy process. The Journal of the Operational Research Society , 62(8): 1585-1595.)	Revised DEAHP does not suffer from RRP.
	^{Wang & Luo (2012}136 WANG Y-M & LUO Y. 2012. A note on “A new approach for weight derivation using data envelopment analysis in the analytic hierarchy process”. Mathematical and Computer Modelling, 56(3-4): 49-55.)	Revised DEAHP suffers from RRP.
	^{Maleki & Zahir (2013}63 MALEKI H & ZAHIR S. 2013. A Comprehensive Literature Review of the Rank Reversal Phenomenon in the Analytic Hierarchy Process. Journal of Multi-Criteria Decision Analysis , 20(3-4): 141-155.)	RRP literature review on AHP method.
	^{Kulakowski (2015}52 KULAKOWSKI K. 2015. Notes on order preservation and consistency in AHP. European Journal of Operational Research, 245(1): 333-337.)	Order preservation and consistency in AHP method.
	^{Yaraghi et al. (2015}144 YARAGHI N, TABESH P, GUAN P & ZHUANG J. 2015. Comparison of AHP and Monte Carlo AHP Under Different Levels of Uncertainty. IEEE Transactions on Engineering Management, 62(1): 122-132.)	RRP is mentioned.
TOPSIS	^{Ren et al. (2007}78 REN L, ZHANG Y, WANG Y & SUN Z. 2007. Comparative Analysis of a Novel M-TOPSIS Method and TOPSIS. Applied Mathematics Research eXpress.)	M-TOPSIS method.
	^{Kong (2011}51 KONG F. 2011. Rank Reversal and Rank Preservation in TOPSIS. Advanced Materials Research, 204-210: 36-41.)	New normalization procedure, new PIS/NIS obtaining method.
	^{García-Cascales & Lamata (2012}36 GARCÍA-CASCALES MS & LAMATA MT. 2012. On rank reversal and TOPSIS method. Mathematical and Computer Modelling, 56: 123-132.)	Presentation of a new normalization and a new PIS/NIS calculation procedure for the TOPSIS.
ELECTRE	^{Wang & Triantaphyllou (2008}133 WANG X & TRIANTAPHYLLOU E. 2008. Ranking irregularities when evaluating alternatives by using some ELECTRE methods. Omega , 36(1): 45-63.)	Evaluation of ELECTRE II e ELECTRE III regarding to the RRP.
	^{Figueira & Roy (2009}31 FIGUEIRA JR & ROY B. 2009. A note on the paper, “Ranking irregularities when evaluating alternatives by using some ELECTRE methods”, by Wang and Triantaphyllou. Omega, 37(3): 731-733.)	Relation between the RRP and ELECTRE methods.
	^{Figueira et al. (2013}30 FIGUEIRA JR, GRECO S, ROY B & SLOWINSKI R. 2013. An Overview of ELECTRE Methods and their Recent Extensions. Journal of Multi-Criteria Decision Analysis, 20(1-2): 61-85.)	Similar to ^{Figueira & Roy (2009}31 FIGUEIRA JR & ROY B. 2009. A note on the paper, “Ranking irregularities when evaluating alternatives by using some ELECTRE methods”, by Wang and Triantaphyllou. Omega, 37(3): 731-733.).
PROMETHEE	^{De Keyser & Peeters (1996}22 DE KEYSER W & PEETERS P. 1996. A note on the use of PROMETHEE multicriteria methods. European Journal of Operational Research, 89(3): 457-461.)	RRP is mentioned.
	^{Vetschera & Almeida (2012}130 VETSCHERA R & ALMEIDA AT. 2012. A PROMETHEE-based approach to portfolio selection problems. Computers & Operations Research, 39(5): 1010-1020)	RRP is mentioned.
	^{Verly & De Smet (2013}129 VERLY C & DE SMET Y. 2013. Some results about rank reversal instances in the PROMETHEE methods. International Journal of Multicriteria Decision Making , 3(4): 325-345.)	Evaluation of PROMETHEE I and PROMETHEE II regarding to the RRP.
	^{Brans (2015}15 BRANS J-P. 2015. The PROMETHEE adventure. International Journal of Multicriteria Decision Making, 5(4): 297-308.)	RRP is mentioned.
OTHERS	^{Gomes (1990}37 GOMES LFAM. 1990. Eliminating rank reversal in multicriteria analysis of urban transportation system alternatives. Journal of Advanced Transportation, 24(2): 181-184.)	Relation between the RRP and TODIM method.
	^{Olson et al. (1995a}68 OLSON DL, FLIEDNER G & CURRIE K. 1995a. Comparison of the REMBRANDT system with Analytic hierarchy process. European Journal of Operational Research , 82(3): 522-539.)	RRP is avoided through the use of REMBRANDT method.
	^{Maleki & Saadat (2013}62 MALEKI H & SAADAT Y. 2013. Comparison of failure mode and effects analysis by using AHP vs. REMBRANDT system. International Journal of Industrial and Systems Engineering , 14(1): 5-19.)	Similar to ^{Olson et al. (1995a}68 OLSON DL, FLIEDNER G & CURRIE K. 1995a. Comparison of the REMBRANDT system with Analytic hierarchy process. European Journal of Operational Research , 82(3): 522-539.).
	^{Salo & Punkka (2011}100 SALO A & PUNKKA A. 2011. Ranking Intervals and Dominance Relations for Ratio-Based Efficiency Analysis. Management Science, 57(1): 200-214.)	RRP is mentioned in DEA.
	^{Soltanifar & Shahghobadi (2014}108 SOLTANIFAR M & SHAHGHOBADI S. 2014. Survey on rank preservation and rank reversal in data envelopment analysis. Knowledge-Based Systems, 60: 10-19.)	RRP in DEA methods.
	^{Rodríguez et al. (2013}79 RODRÍGUEZ A, ORTEGA F & CONCEPCIÓN R. 2013. A method for the selection of customized equipment suppliers. Expert Systems with Applications , 40(4): 1170-1176.)	RRP in FUZZY-AHP-TOPSIS.
	^{Lima Junior et al. (2014}58 LIMA JUNIOR FR, OSIRO L & CARPINETTI LCR. 2014. A comparison between Fuzzy AHP and Fuzzy TOPSIS methods to supplier selection. Applied Soft Computing, 21: 194-209.)	Exploration of the FUZZY-AHP and FUZZY-TOPSIS regarding to the RRP in a specific context.
	^{Bulut, Duru & Koçak (2015}18 BULUT E, DURU O & KOÇAK G. 2015. Rotational priority investigation in fuzzy analytic hierarchy process design: An empirical study on the marine engine selection problem. Applied Mathematical Modelling, 39(2): 913-923.)	RRP is mentioned in FUZZY-AHP.
	^{Bottero & Ferretti (2011}14 BOTTERO M & FERRETTI V. 2011. An analytic network process-based approach for location problems: the case of a new waste incinerator plant in the Province of Torino (Italy). Journal of Multi-Criteria Decision Analysis, 17(3-4): 63-84.)	Propose the ANP?BOCR method.
	^{Hou (2012}43 HOU F. 2012. Rank Preserved Aggregation Rules and Application to Reliability Allocation. Communications in Statistics - Theory and Methods, 41(21): 3831-3845.)	FRP-AHP method.
	^{Pankratova & Nedashkovskay (2014}71 PANKRATOVA ND & NEDASHKOVSKAY NI. 2014. Hybrid Method of Multicriteria Evaluation of Decision Alternatives. Cybernetics and Systems Analysis, 50(5): 701-711.)	Propose the MEDM method.
	^{Salabun (2015}97 SALABUN W. 2015. The Characteristic Objects Method: A New Distance-based Approach to Multicriteria Decision-making Problems. Journal of Multi-Criteria Decision Analysis , 22(1-2): 37-50.)	Propose the COMET method.
	^{Bairagi et al. (2015}5 BAIRAGI B, DEY B, SARKAR B & SANYAL SK. 2015. A De Novo multi-approaches multicriteria decision making technique with an application in performance evaluation of material handling device. Computers & Industrial Engineering, 87: 267-282.)	Propose the TPOP method.
	^{Iç (2014}45 IÇ YT. 2014. A TOPSIS based design of experiment approach to assess company ranking. Applied Mathematics and Computation, 227: 630-647.)	Propose the DOE-TOPSIS method.
	^{Morton (2015}66 MORTON A. 2015. Measurement issues in the evaluation of projects in a project portfolio. European Journal of Operational Research , 245(3): 789-796.)	Criticizes of the way of using value functions in the evaluation of portfolio project’s by showing it can lead to a rank reversal.
	^{Tomashevskii (2015}116 TOMASHEVSKII IL. 2015. Eigenvector ranking method as a measuring tool: Formulas for errors. European Journal of Operational Research , 240(3): 774-780.)	RRP is caused by the inaccuracy of the measurement scale and by the inconsistent judgements of experts.
	^{Dede et al. (2015}23 DEDE G, KAMALAKIS T & SPHICOPOULOS T. 2015. Convergence properties and practical estimation of the probability of rank reversal in pairwise comparisons for multicriteria decision making problems. European Journal of Operational Research, 241(2): 458-468.)	Evaluation of probability ocorrence of the RRP in the context of pairwise comparisons.
	^{Buede & Maxwell (1995}17 BUEDE DM & MAXWELL DT. 1995. Rank disagreement: A comparison of multi-criteria methodologies. Journal of Multi-Criteria Decision Analysis, 4(1): 1-21.)	Examination of the frequency and magnitude of the RRP in four different methods.
	^{Zanakis et al. (1998}147 ZANAKIS SH, SOLOMON A, WISHART N & DUBLISH S. 1998. Multi-attribute decision making: A simulation comparison of select methods. European Journal of Operational Research , 107(3): 507-529.)	Evaluation of eight methods regarding to the RRP.
	^{Wang & Luo (2009}135 WANG Y-M & LUO Y. 2009. On rank reversal in decision analysis. Mathematical and Computer Modelling, 49(5-6): 1221-1229.)	Demonstration of the RRP in four different methods.
	^{Cinelli et al. (2014}19 CINELLI M, COLES SR & KIRWAN K. 2014. Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. Ecological Indicators, 4: 138-148.)	Occurrence of the RRP in five different methods in a context of conduct sustainability assessment.

Sociedade Brasileira de Pesquisa Operacional Rua Mayrink Veiga, 32 - sala 601 - Centro, 20090-050 Rio de Janeiro RJ - Brasil, Tel.: +55 21 2263-0499, Fax: +55 21 2263-0501 - Rio de Janeiro - RJ - Brazil
E-mail: sobrapo@sobrapo.org.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] *Corresponding author.