ABSTRACT
This study associates graph theory and a multicriteria decision aid technique, presenting a different process for doing the investigation of criminal networks. In the criminal subject, privacy concerns limit identification. For this reason, the database composed of 110 actors, involving criminals and peripheral characters to the network, was identified by numbers, without names and penalties. The discrimination of critical actors in criminal networks can help law enforcement officers to conduct a more detailed investigation for their disruption. Communication between drug traffickers was transformed into different centrality indices for each actor in their social network. Centralities and actors compose a decision matrix, analyzed by the Composition of Probabilistic Preferences to identify the most relevant actors in the criminal network. Results indicated that the five actors highlighted in the real investigation have a clear distinction of importance in the network, which in a way have been ratified.
Keywords:
Social Network Analysis; Composition of Probabilistic Preferences; CPPTRI
1 INTRODUCTION
Police intelligence plays a vital role in preventing and controlling organized crime. The success of law enforcement depends, in large part, on the identification of the criminal network, its nature, structure, methods of operation, among others (Ratcliffe, 2016124. RATCLIFFE JH. 2016. Intelligenceled policing (2nd ed.). Routledge.; Tilley, 2016161. TILLEY N. 2016. Intelligenceled policing and the disruption of organized crime: motifs, methods and morals. In T. Delpeuch & J. E. Ross (Eds.), Comparing the Democratic Governance of Police Intelligence: New Models of Participation and Expertise in the United States and Europe (1st ed., p. 153179). Edward Elgar Publishing.). One of the first studies to explore the socalled “link analysis” in an investigative problem dates back to the precomputational period, almost two decades before the emergence of the Internet (Harper & Harris, 197569. HARPER WR & HARRIS DH. 1975. The application of link analysis to police intelligence. Human Factors, 17(2): 157164.).
Since then, the evolution of data storage, processing and transmission capacity has provided a significant advance in the analysis of social networks, based on graph theory (Wiil et al., 2010165. WIIL UK, GNIADEK J & MEMON N. 2010. Measuring link importance in terrorist networks. 2010 International Conference on Advances in Social Networks Analysis and Mining, 225232.). An increasing number of researchers have applied social network analysis to reveal the dynamics of criminal organizations (Fire & Puzis, 201652. FIRE M & PUZIS R. 2016. Organization mining using online social networks. Networks and Spatial Economics, 16(2): 545578.; Taha & Yoo, 2015155. TAHA K & YOO PD. 2015. A system for analyzing criminal social networks. Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015, 10171023.). The identification of critical network nodes can be directed by law enforcement agencies for a more detailed investigation and subsequent disruption as a result of the association of these critical nodes with the leaders and main links in a criminal network (Burcher & Whelan, 201819. BURCHER M & WHELAN C. 2018. Social network analysis as a tool for criminal intelligence: Understanding its potential from the perspectives of intelligence analysts. Trends in Organized Crime, 21(3): 278294.).
In graph theory, the identification of critical nodes can be obtained through measures of centrality, whose values allow to order from the most to the least important network node, for example (Borgatti & Everett, 200615. BORGATTI SP & EVERETT MG. 2006. A graphtheoretic perspective on centrality. Social Networks, 28(4): 466484.). The different measures of centrality can be especially useful in complex networks, composed of a significant number of actors and links, to clarify vulnerabilities and weaknesses that guide the application of research resources (Burcher & Whelan, 201819. BURCHER M & WHELAN C. 2018. Social network analysis as a tool for criminal intelligence: Understanding its potential from the perspectives of intelligence analysts. Trends in Organized Crime, 21(3): 278294.).
Although graph theory describes measures of centrality using a few basic categories such as degree, closeness and betweenness, the literature records more than two hundred algorithms for calculating these measures (Bloch et al., 201913. BLOCH F, JACKSON MO & TEBALDI P. 2019. Centrality measures in networks. Available at SSRN 2749124.; Das et al., 201830. DAS K, SAMANTA S & PAL M. 2018. Study on centrality measures in social networks: a survey. Social Network Analysis and Mining, 8(1): 13.; Oldham et al., 2019110. OLDHAM S, FULCHER B, PARKES L, ARNATKEVICIŪTE A, SUO C & FORNITO A. 2019. Consistency and differences between centrality measures across distinct classes of networks. PloS One , 14(7).). Depending on the type of centrality used, it is possible that one node in the network calls more attention than another during the network analysis, depending on the variation of the data. Alternatively, the use of a set of centrality measures to determine a list of key actors in a criminal network can be interesting to offer investigators new options, helping the process of choosing targets to monitor. The use of a set of centralities expands the range of options for the optimized application of resources, which can be a differential to investigations.
In this article, the Caviar Project, an antidrug operation, was revisited considering several centrality measures, combined with a methodology to multicriteria decision aid (MCDA) (Morselli, 2009102. MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.). The criminal network was targeted between 1994 and 1996 by a tandem investigation uniting the Montreal Police, the Royal Canadian Mounted Police, the Canadian Police, and lawenforcement agents from England, Spain, Italy, Brazil, Paraguay and Colombia. The purpose of reassessing this real case was to ratify or rectify the existence of important actors in the criminal network, both those identified and those who, eventually, remained hidden from investigators. The motivation to “reopen” this case arose from the fact that investigators exploited only two measures of centrality to analyze the social network, which may have biased or distorted the identification of the main criminals. The proposal to revisit the investigation, through new Operations Research techniques, contributes to the robustness of the process.
Network analysis usually evaluates how actors are connected. In the Caviar Project, analysts used measures of degree and betweenness centralities. Degree centrality emulates a direct connectivity, visibility, and vulnerability within a criminal network. Betweenness centrality shows brokerage positions, revealing a possible sign of strategic involvement in criminal networks (Morselli, 2009102. MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.). It should be noted that the analysis carried out at the time was not mistaken, but here we suggest new centrality measures and a new methodological approach capable of highlighting other actors also important in the criminal network.
The article also brings an innovative approach to Probabilistic Composition of Preferences for problems of ordered classification (CPP TRI) (Sant’Anna, 2015131. SANT’ANNA AP. 2015. Probabilistic Composition of Preferences, Theory and Applications (1st ed.). Springer.). This method allows to allocate the actors of a network in preselected categories. For example, it is possible to distinguish more and less important actors, considering performance measures in multiple criteria. The CPP TRI algorithm verifies if each actor performs above or below parameters, called class profiles, which best define the categories.
Considering the focus in determining the upper class, instead of choosing a set of class profiles, as in the original CPP TRI, in the present study a series of twoclass scenarios with increasing restriction is designed to progressively select the number of upperclass actors. In this way, only actors with the best overall performance in the various measures of centrality remain in the upper class. This procedure gives robustness to the classification procedure, as the alternatives only stay in the main class if they confirm their performance in all scenarios.
This procedure is also interesting because it involves less computational effort, in relation to other variants of CPP. CPP TRI calculates the probabilities of each alternative being above or below one profile at a time, instead of considering all alternatives under each criterion in the calculation of joint probabilities. This reduction in computational effort is useful for dealing with large networks, characterized by thousands (or even millions) of actors and links. Although the Caviar network is made up of 110 actors, the reduction in processing time for calculations with CPP TRI is significant in relation to the original CPP procedure.
2 RELATED RESEARCH ON CRIMINOLOGY
In the field of quantitative criminology, several scientific articles highlight the usefulness of social network analysis to help the elucidation of crimes and identification of key criminals for investigation. Radil et al. (2010122. RADIL SM, FLINT C & TITA GE. 2010. Spatializing social networks: Using social network analysis to investigate geographies of gang rivalry, territoriality, and violence in Los Angeles. Annals of the Association of American Geographers, 100(2): 307326.) compared the geography of rivalry relations that connect territoriallybased criminal street gangs in a section of Los Angeles with a geography of the location of gangrelated violence. Tita & Radil (2011162. TITA GE & RADIL SM. 2011. Spatializing the social networks of gangs to explore patterns of violence. Journal of Quantitative Criminology , 27(4): 521545.) analyzed the spatial distribution of gang violence by considering the relative location of the gangs in space while simultaneously capturing their position within an enmity network of gang rivalries. Davies & Johnson (201531. DAVIES T & JOHNSON SD. 2015. Examining the relationship between road structure and burglary risk via quantitative network analysis. Journal of Quantitative Criminology , 31(3): 481507.) examined the relationship between road structure and residential burglary risk in Birmingham (UK) via network analysis and betweenness centrality measures. Duxbury & Haynie (201839. DUXBURY SW & HAYNIE DL. 2018. The network structure of opioid distribution on a darknet cryptomarket. Journal of Quantitative Criminology , 34(4): 921941.) studied the network structure of opioid distribution on a darknet cryptomarket, using exponential random graph modeling to evaluate which vendor characteristics explain variation in purchasing patterns. Bright et al. (201918. BRIGHT D, KOSKINEN J & MALM A. 2019. Illicit network dynamics: The formation and evolution of a drug trafficking network. Journal of Quantitative Criminology, 35(2): 237258.) examined a methamphetamine manufacture and trafficking network that expanded from small scale social dealing to a largescale profitmotivated business, using dynamic network analysis to evaluate both overall network structure and actor level characteristics. They highlight that “social network analysis should be used by intelligence agencies and law enforcement agencies to facilitate intelligence collection and to guide prevention and intervention strategies”. McCuish et al. (2020)96. MCCUISH E, BOUCHARD M & BEAUREGARD E. 2020. A NetworkBased Examination of the Longitudinal Association Between Psychopathy and Offending Versatility. Journal of Quantitative Criminology. Available at: https://doi.org/https://doi.org/10.1007/s1094002009462w
https://doi.org/https://doi.org/10.1007/...
modeled three network structures that separated the variance in the relationship between features of psychopathy and offending into betweensubjects and withinindividual networks. McMillan et al. (2020)97. MCMILLAN C, FELMLEE D & BRAINES D. 2020. Dynamic patterns of terrorist networks: Efficiency and security in the evolution of eleven islamic extremist attack networks. Journal of Quantitative Criminology , 36(3): 559581. used a variety of descriptive network measures and Separable Temporal Exponential Random Graph Models to find patterns of tie formation across eleven multiwave terrorism networks.
Although the Caviar Project took place in the 1990s, several studies are recurrent on the subject, with the purpose of bringing new possibilities to identify criminal networks. The first studies associating the Caviar network with network analysis methodologies were developed by (Morselli, 2009102. MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.; Morselli et al., 2007103. MORSELLI C, GIGUÈRE C & PETIT K. 2007. The efficiency/security tradeoff in criminal networks. Social Networks , 29(1): 143153.; Morselli & Petit, 2007105. MORSELLI C & PETIT K. 2007. Lawenforcement disruption of a drug importation network. Global Crime, 8(2): 109130.). Later, Morselli et al. (2013)104. MORSELLI C, MASIAS VH, CRESPO F & LAENGLE S. 2013. Predicting sentencing outcomes with centrality measures. Security Informatics, 2(1): 4. reported the ability of centrality measures to predict the verdict (innocent or guilty) and the sentence length in years, using the same network. Skillicorn et al. (2014150. SKILLICORN DB, ZHENG Q & MORSELLI C. 2014. Modeling dynamic social networks using spectral embedding. Social Network Analysis and Mining , 4(1): 182.) used spectral embedding to model the Caviar network. Masías et al. (2016)95. MASÍAS VH, VALLE M, MORSELLI C, CRESPO F, VARGAS A & LAENGLE S. 2016. Modeling verdict outcomes using social network measures: the watergate and caviar network cases. PloS One, 11(1): e0147248. used regression techniques to model criminal trial verdict outcomes using social network measures, exploring the Watergate Case and the Caviar network. Duxbury & Haynie (201940. DUXBURY SW & HAYNIE DL. 2019. Criminal network security: An agentbased approach to evaluating network resilience. Criminology, 57(2): 314342.) used an agentbased model to evaluate criminal network resilience by examining network recovery from disruption in an array of different criminal networks and across different disruption strategies, including the Caviar Project. Taha & Yoo (2019154. TAHA K & YOO PD. 2019. Shortlisting the influential members of criminal organizations and identifying their important communication channels. IEEE Transactions on Information Forensics and Security, 14(8): 19881999.) proposed a new forensic analysis system to infer the highlevel criminals and shortlist the important communication channels in a criminal organization, based on the mobile phone communications of its members.
3 RELATED RESEARCH ON MULTICRITERIA METHODS
Network analysis is a broad area of study, which is based on two variables to conduct the analysis: actors and relationships. The range of methods in network analysis involves quantitative studies, such as social network analysis, and ethnographic studies. In the literature, the methodological procedure followed in this study is called “social network analysis” and “link analysis,” to differentiate quantitative analysis from the current meaning of social networks, often associated with Facebook, Twitter, Instagram applications, among others (Alrashoud et al., 20154. ALRASHOUD M, ALMESHARY M & ABHARI A. 2015. Automatic validation for multi criteria decision making models in simulation environments. Proceedings of the 18th Symposium on Communications & Networking, 4447b.; Liu et al., 201890. LIU L, ZUO W & PENG T. 2018. A reliability and link analysis based method for mining domain experts in dynamic social networks. Journal of Intelligent & Fuzzy Systems, 34(4): 20612073.). The expression “social network analysis” is now used in this study.
The analysis of social networks, and especially their measures of centrality, combined with MCDA models has been explored in the literature, to identify the most influential actors in a network, based on a varied set of indicators. Several authors have explored, for example, the Technique of Ordering Preferences by Similarity with Ideal Solution (TOPSIS) for this purpose (Du et al., 201438. DU Y, GAO C, HU Y, MAHADEVAN S & DENG Y. 2014. A new method of identifying influential nodes in complex networks based on TOPSIS. Physica A: Statistical Mechanics and Its Applications, 399, 5769.; Fox & Everton, 201354. FOX WP & EVERTON SF. 2013. Mathematical modeling in social network analysis: using TOPSIS to find node influences in a social network. Journal of Mathematics and System Science, 3(10): 531.; J. Hu et al., 201670. HU J, DU Y, MO H, WEI D & DENG Y. 2016. A modified weighted TOPSIS to identify influential nodes in complex networks. Physica A: Statistical Mechanics and Its Applications , 444, 7385.; Kermani et al., 201684. KERMANI MAMA, BADIEE A, ALIAHMADI A, GHAZANFARI M & KALANTARI H. 2016. Introducing a procedure for developing a novel centrality measure (Sociability Centrality) for social networks using TOPSIS method and genetic algorithm. Computers in Human Behavior, 56, 295305.).
Associated with TOPSIS for calculating criteria weights or even used in isolation, the Hierarchical Analysis Process (AHP) and its variant with network analysis (ANP) also find applications in research in this area, although a limitation of AHP in networks is the difficulty of dealing with a large number of nodes and links (Choudhary & Singh, 201823. CHOUDHARY P & SINGH U. 2018. Ranking terrorist organizations network in India using combined SnaAhp approach. International Journal of Recent Technology and Engineering, Article, 7(4): 168172.; Fox & Everton, 201555. FOX WP & EVERTON SF. 2015. Using data envelopment analysis and the analytical hierarchy process to find node influences in a social network. The Journal of Defense Modeling and Simulation, 12(2): 157165.; Shoman & Gülgen, 2017145. SHOMAN W & GÜLGEN F. 2017. Centralitybased hierarchy for street network generalization in multiresolution maps. Geocarto International, 32(12): 13521366.). However, note that TOPSIS, AHP and its ANP variant are not MCDA methods designed to face the problem of classifying alternatives in ordered homogeneous groups, defined by an order of preference, such as ELECTRE TRI and CPP TRI. It is also worth mentioning that these MCDA methods are different from traditional clustering methods, not covered in this paper, as they classify the data into ordered groups.
CPPTRI is a variant of the CPP method. The original model was developed by Sant’Anna & Sant’Anna (2001)134. SANT’ANNA AP & SANT’ANNA LAFP. 2001. Randomization as a stage in criteria combining. International Conference on Industrial Engineering and Operations Management  VII ICIEOM, 248256., and was later expanded in Sant’Anna (2015)137. SANT’ANNA AP. 2015. Probabilistic human development indices. Brazilian Journal of Operations & Production Management, 12(1): 136146.. CPP is a probabilistic MCDA method, whose variants apply to different problems of multicriteria decision, which include the ordering of alternatives under different points of view for decision making (Gavião et al., 201758. GAVIÃO LO, MEZA LA, LIMA GBA, SANT’ANNA AP & SOARES DE MELLO JCCB. 2017. Improving discrimination in efficiency analysis of bioethanol processes. Journal of Cleaner Production . Available at: https://doi.org/10.1016/j.jclepro.2017.06.020
https://doi.org/https://doi.org/10.1016/...
), the ordering with Choquet capacities (Souza et al., 2016152. SOUZA RG DE, CLÍMACO JCN, SANT’ANNA AP, ROCHA TB, VALLE R DE AB DO & QUELHAS OL G. 2016. Sustainability assessment and prioritisation of ewaste management options in Brazil. Waste Management .), the dynamic assessment of alternatives based on Malmquist index (Sant’Anna, 2009135. SANT’ANNA AP. 2009. A probabilistic approach to the evaluation and combination of preferences. In Abraham A, Hassanien AE & Snášel V (Eds.), Foundations of Computational Intelligence Volume 5 (p. 107125). Springer.), the assessment of regularity based on the Gini index (Gavião, Sant’Anna, & Lima, 201960. GAVIÃO LO, SANT’ANNA AP & LIMA GBA. 2019. A Composição Probabilística de Preferências com medidas de desigualdade: correlações com os pontos de vista progressista e conservador. Pesquisa Operacional e Sua Atuação Multidisciplinar, 217232. Available at: https://doi.org/10.22533/at.ed.78819110716
https://doi.org/https://doi.org/10.22533...
), among other options that associate these variants (Gavião et al., 201959. GAVIÃO LO, SANT’ANNA AP, ALVES LIMA GB & DE ALMADA GARCIA PA. 2019. Evaluation of soccer players under the Moneyball concept. Journal of Sports Sciences. Available at: https://doi.org/10.1080/02640414.2019.1702280
https://doi.org/https://doi.org/10.1080/...
). In this study, the model of interest is CPP TRI, which performs classification of alternatives in ordered classes predefined by the user, also called sorting. (Sant’Anna et al., 2015133. SANT’ANNA AP, COSTA HG & PEREIRA V. 2015. CPPTRI: a sorting method based on the probabilistic composition of preferences. International Journal of Information and Decision Sciences, 7(3): 193212.; Sant’Anna, 2014136. SANT’ANNA AP. 2014. Aplicação do CPPTri à Classificação dos países pelos Critérios do IDH. Encontro Nacional de Engenharia de Produção  XXXVI ENEGEP.; Sant’Anna et al., 2016138. SANT’ANNA AP, COSTA HG, NEPOMUCENO LDO & PEREIRA V. 2016. A Probabilistic Approach Applied to the Classification of Courses by Multiple Evaluators. Pesquisa Operacional .).
The algorithm introduced here seeks to raise a group of actors of greater relevance in a social network through the ordered classification of its actors. Operational Research brings several methods for sorting alternatives, besides CPP TRI. Table 1 compares CPP TRI with other approaches and their variants, based on the main features of CPP. It is not simple to frame a wide variety of methods under similar characteristics, but the attempt is worthwhile to show how CPP is unique in its classification algorithm. This review tracked methods published in the main Operations Research journals and events, but, given the dynamism and creativity of scientists in creating new methods and variants to the existing ones, it may still be omitting some methods.
The main features of multicriteria methods to sort alternatives are highlighted in Table 1. The first columns show the root method on which the variant used for sorting the alternatives is based. The ‘data type’ column identifies the degree of certainty attributed to data collected from input sources for modeling, e.g., expert evaluations and raw data. Data represent quantitative values, e.g., crisp numbers, probabilistic measures, fuzzy sets, and rough sets. The ‘normalization’ column indicates whether the method requires standardization of the collected raw data, including the use of specific scales. The ‘class profiles’ column indicates whether the model requires class profiles to distinguish the thresholds or features of each class. The ‘intracriterion evaluation’ column indicates how the evaluations are employed in the aggregation algorithm, if previously compared to all the others, or part of the others, or if some criteria comparison is initially performed. The last two columns show the first published version of the model and relevant ulterior applications.
As shown in Table 1, CPP TRI differs from other methods with a similar purpose. In fact, the preference for a specific method depends on the nature of the problem, the format of data, calculation complexity, among other features. In the literature, two methods explore probability densities to deal with uncertain data in sorting problems, defining class acceptability indices with SMAATri, or expressing the percentage that an alternative belong to each class, as CPP TRI (Ishizaka, Tasiou, et al., 202080. ISHIZAKA A, TASIOU M & MARTÍNEZ L. 2020. Analytic hierarchy processfuzzy sorting: An analytic hierarchy processbased method for fuzzy classification in sorting problems. Journal of the Operational Research Society , 71(6): 928947.). Although both methods favor a probabilistic approach to assessments, SMAA is more complex than CPP. SMAA requires Monte Carlo iterations and, for having practical applicability, the complexity of SMAA computations should not be too high with respect to the number of criteria and alternatives (Tervonen & Figueira, 2008157. TERVONEN T & FIGUEIRA JR. 2008. A survey on stochastic multicriteria acceptability analysis methods. Journal of MultiCriteria Decision Analysis, 15(12), 114.). CPP calculation procedures in R and Matlab softwares indicate reduced processing time, without the need to implement Monte Carlo simulations, nor requires any rate between the number of criteria and alternatives (Casado & Silva, 201720. CASADO RSGR & SILVA LC E. 2017. Avaliação de risco no APL de confecção do agreste de Pernambuco: uma abordagem da Composição Probabilística com base na FMEA. Simpósio Brasileiro de Pesquisa Operacional  XLIX SBPO.; Gavião et al., 201861. GAVIÃO LO, SANT’ANNA AP, LIMA GBA & GARCIA PA DE A. 2018. CPP: Composition of Probabilistic Preferences. R package version 0.1.0. (R package version 0.1.0.; p. 124). R Core Team. Available at: https://cran.rproject.org/package=CPP
https://cran.rproject.org/package=CPP...
; Sant’Anna et al., 2012140. SANT’ANNA AP, GOMES LFAM, COSTA FF DA, RANGEL LAD, FARIA MJ DA S, FERREIRA RG, FILHO RMM, RIBEIRO ROA & SENNA V DE. 2012. Análise multicritério baseada em probabilidades de preferência. In V. F. de Oliveira VCavenaghi & F. S. Másculo (Eds.), Tópicos emergentes e desafios metodológicos em Engenharia de Produção: casos, experiências e proposições  Volume V (p. 258). ABEPRO.). Therefore, this paper explores CPP TRI with an application to the Caviar Project, introducing an interesting variant to refine elements that belong to a certain class.
4 MATERIALS AND METHODS
The methodology explored here involved five steps, illustrated by Fig. 1. The 1^{st} step consisted of data collection. In social network analysis, data are organized in a square matrix called “adjacency matrix”. The rows and columns of this matrix are composed of network actors and the internal values correspond to the measures of the links considered in the problem. The 2^{nd} step focused on the centrality algorithms available in the “CINNA” package of the R software (Ashtiani, 20198. ASHTIANI M. 2019. CINNA: Deciphering Central Informative Nodes in Network Analysis. R package version 1.1.53 (p. 1). CRAN. Available at: https://cran.rproject.org/package=CINNA
https://cran.rproject.org/package=CINNA...
). This package provides several functions for calculating, comparing and demonstrating the main measures of centrality in a network. In the 3^{rd} step, Principal Component Analysis (PCA) was used to reduce the problem dimension, by selecting the most representative centrality measures, through a function of the “CINNA” package, called “pca centralities” (Ashtiani, 20198. ASHTIANI M. 2019. CINNA: Deciphering Central Informative Nodes in Network Analysis. R package version 1.1.53 (p. 1). CRAN. Available at: https://cran.rproject.org/package=CINNA
https://cran.rproject.org/package=CINNA...
). The 4^{th} step elaborated a decision matrix with the network nodes as alternatives (matrix rows), the main centralities indicated by PCA as criteria (matrix columns) and the respective measures of centrality as the evaluations of the alternatives by each criterion. The 5^{th} step consisted of the application of CPPTRI, admitting different scenarios of class profiles.
The classification of alternatives by CPPTRI requires prior definition of representative profiles for each class chosen. A profile is a vector with one coordinate for each criterion used as a reference to classify an alternative at different levels of quality. These levels can define, for example, a high and a low performance class. The profiles of classes can be established based on a priori information about the context, by expert opinion, by measures of statistical position, among other possibilities (Sant’Anna, 2015137. SANT’ANNA AP. 2015. Probabilistic human development indices. Brazilian Journal of Operations & Production Management, 12(1): 136146.). Fig. 2 illustrates the marking of class profiles for two categories. In the present study, the profiles are chosen based on percentiles of performance according to each criterion. The dotted line indicates each profile.
The classification algorithm of the 5th Step is based on Equations (1) to (6). The randomization of assessments is the first step of CPP. Exact measures of performance are interpreted as location parameters e.g. mean, mode, or median) of a statistical distribution. The choice of the probabilistic distribution that best fits the data is usually based on a priori information, on successful experiences in similar problems or from a goodnessoffit function. In the absence of such information, it is still possible to use an empirical approach or even arbitrate a probability distribution according to the context.
Fig. 3 illustrates a generic randomization of an exact performance evaluation a _{kj} = 7, by a Beta PERT distribution, with minimum, mode and maximum parameters equal to 0, 7, and 10, respectively, and shape equal to 10. This procedure of randomization is performed for each a _{kj} element of the decision matrix. This transformation of an exact number into a probability distribution takes into account the uncertainty of data collection, recording processes and preference decisions in the intelligence realm.
After the randomization, in a multiplicative composition with one representative profile per class (Sant’Anna et al., 2012140. SANT’ANNA AP, GOMES LFAM, COSTA FF DA, RANGEL LAD, FARIA MJ DA S, FERREIRA RG, FILHO RMM, RIBEIRO ROA & SENNA V DE. 2012. Análise multicritério baseada em probabilidades de preferência. In V. F. de Oliveira VCavenaghi & F. S. Másculo (Eds.), Tópicos emergentes e desafios metodológicos em Engenharia de Produção: casos, experiências e proposições  Volume V (p. 258). ABEPRO.), for i denoting the class, j denoting the criterion, c _{ij} denoting the jth coordinate of the representative profile of class i and X _{kj} denoting the random variable derived from the numerical assessment of the kth alternative by the jth criterion, the classification procedures starts with the computation of the probabilities. Notations A ^{+} and A ^{} indicate the probabilities of alternative A being respectively above and below the values reported for the jth criterion in the ith class profiles.
Then, estimates of joint probabilities of each alternative being below and above each class are computed:
The classification is completed with the computation of the distances (∆).
The kth alternative is classified in a class i _{0} .
The algorithms for classifying alternatives in CPPTRI are available in the “CPP” package of software R (Gavião et al., 201861. GAVIÃO LO, SANT’ANNA AP, LIMA GBA & GARCIA PA DE A. 2018. CPP: Composition of Probabilistic Preferences. R package version 0.1.0. (R package version 0.1.0.; p. 124). R Core Team. Available at: https://cran.rproject.org/package=CPP
https://cran.rproject.org/package=CPP...
). Fig. 4 illustrates the allocation of an alternative to its most appropriate class, considering one alternative (“A”) and two classes.
5 REVISITING THE CAVIAR PROJECT
The Caviar Project was an antidrug investigation based on the analysis of social networks (Morselli, 2009102. MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.). Investigations have raised a hash and cocaine trafficking network operating outside the city of Montreal, Canada. The network was monitored between 1994 and 1996 by officers from the Montreal Police, the Canadian Mounted Police and other national and international police parties.
This investigation was peculiar because the agents seized shipments of drugs without detaining the participants. This atypical context of police intervention was used to observe and analyze the behavior of the actors during the investigation. Thus, although 11 shipments were seized at different times during the investigations, the arrests occurred only at the end of the investigation.
The main source of data was composed of information collected from telephone conversations between network participants, which were intercepted electronically. More than a thousand pages of transcripts were recorded and analyzed to create an adjacency matrix based on the connections between members of the drug trafficking network.
Initially the intercepted data allowed identifying 318 individuals. However, several of them were not involved in trafficking operations and others did not reveal a clear participatory role in the network, among which are family members or legitimate entrepreneurs. With the removal of members with no relevance to the drug trafficking network, the final list of investigated individuals was composed of 110 actors, represented in the following description arbitrarily by integers from “1” to “110”. From this group of 110 actors, 25 were arrested, 22 accused and 14 considered guilty.
The adjacency matrix in Table 2 presents a sample of the collected data from the UCINET software base (Coutinho, 201628. COUTINHO J. 2016. Caviar database. Ucinet. Available at: https://sites.google.com/site/ucinetsoftware/datasets/covertnetworks/caviar
https://sites.google.com/site/ucinetsoft...
), which provides open access to the Caviar network data in a “.csv” file. This matrix is also presented in (Morselli, 2009102. MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.). The full matrix of 110 actors is found in the Appendix. The values of the matrix elements describe the exchanges of communication between drug traffickers, originating from the police wiretapping. These measures represent the level of communication activity. For example, Criminal #3 made 152 calls to Criminal #1, while Criminal #1 made 337 calls to Criminal #3. This characterizes the weighted directionality of the network. It is also possible to verify that the main diagonal of the adjacency matrix is null, since an actor does not make calls for himself.
Based on the entire adjacency matrix, the centralities of the network actors were calculated. The CINNA application returned results for 37 types of centralities. Then, PCA was used to select the most representative types of centrality. Fig. 5 graphically describes the results of PCA, a cutoff line separating the more and less representative types. The cutoff line reflects the intensity that must be exceeded in cumulative percentage of variance of eigenvalues, being adopted the default=80, suggested in CINNA. Therefore, only 10 criteria were kept in the process.
Table 3 presents a sample of the results of the PCA, as shown in Fig. 5. The full matrix of 110 actors is also found in the Appendix. Those results configure the decision matrix to be modeled with CPP TRI. The ten most representative criteria, in order of relevance from highest to lowest contribution to results, are indicated in the columns: (C1) Local Bridging Centrality; (C2) Information Centrality; (C3) EPC  Edge Percolated Component; (C4) Harary Centrality; (C5) DMNC  Density of Maximum Neighborhood Component; (C6) Eccentricity Centrality; (C7) Kcore Decomposition; (C8) Stress Centrality; (C9) Kleinberg’s authority centrality scores; and (C10) Flow Betweenness Centrality. The specifics of these centrality measures and general PCA procedures are detailed in the “CINNA” package of software R (Ashtiani, 20198. ASHTIANI M. 2019. CINNA: Deciphering Central Informative Nodes in Network Analysis. R package version 1.1.53 (p. 1). CRAN. Available at: https://cran.rproject.org/package=CINNA
https://cran.rproject.org/package=CINNA...
).
Finally, the application of CPP TRI to this decision matrix concludes the modeling leading to the results in Table 4. Two classes (1 and 2) were designated to represent the actors with the greatest and least relevance in the network. This identification is important during investigations to raise the most influential actors in the drug trafficking network. Four scenarios were established in which each class is represented by a single profile. The class profiles are constituted by statistic percentiles of the data for each criterion, indicated in parentheses in the heading of Table 4. The increase of 5% in the profiles gives greater selectivity for the classification of the actors. For example, it is possible to verify that Criminal #1 remained in the most important class in the four scenarios, while Criminal #3 remained slightly below Criminal #1 because he was reclassified in the most restrictive scenario (profiles 90% and 65%). The probability distributions were modeled with the “CPP.Tri.Beta” function, available in the “CPP” package of software R (Gavião et al., 201861. GAVIÃO LO, SANT’ANNA AP, LIMA GBA & GARCIA PA DE A. 2018. CPP: Composition of Probabilistic Preferences. R package version 0.1.0. (R package version 0.1.0.; p. 124). R Core Team. Available at: https://cran.rproject.org/package=CPP
https://cran.rproject.org/package=CPP...
).
Initially, Criminals #1, #3, #12, #76 and #87 were highlighted for analysis. The Caviar Project priorized those criminals due to results of the network analysis with degree and betweeness centrality measures (Morselli, 2009102. MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.). Although these two centralities are among the most explored in social networks analyses, the use of ten most representative measures among 37 types of centralities allowed identifying in greater depth the real importance of these actors in the criminal network. Of these five actors, only Criminal #1 and, in a way, Criminal #3 were confirmed the most important in the modeling here explored. Criminals #12 and #76 were classified in the least relevant class in all scenarios, while Criminal #87 obtained an intermediate result between classes 1 and 2.
Further analysis is needed about the Criminal #12. Morselli & Petit (2007105. MORSELLI C & PETIT K. 2007. Lawenforcement disruption of a drug importation network. Global Crime, 8(2): 109130.) identified #1 and #12 as among the most important players in the network. They computed centrality scores for the overall Caviar network to indicate the extent to which Criminals #1, #3 and #12 were positioned as key participants. Criminal #1 was the most central participant (degree centrality = 55; betweenness centrality = 64), followed by Criminal #3 who was equally connected (degree centrality = 25), yet less indirectly connected (betweenness centrality = 11) in his communications than Criminal #12 (degree centrality = 25; betweenness centrality = 29). They also describe Criminal #12 as “the principal coordinator for the cocaine consignments” and therefore logically among the top law enforcement targets.
However, the results regarding criminal #12 in the proposed modeling did not reach the class with the greatest attention to law enforcement. This does not mean to say that it is irrelevant to the investigations, but in the set of several centrality measures, Criminal #12 did not performed in the same way. The mathematical model is just a complement that contributes to fragment the network, indicating other actors that deserve investigation to confirm or not its relevance in the network. It is reasonable to assume, for example, that the connections with a given head of the network are protected and generate less data in a adjacency matrix. In summary, quantitative analysis goes handinhand with qualitative analysis by experts in criminal investigations, so that the puzzle of a criminal organization can be effectively put together.
In Table 4, the criminals under grayshaded lines are also selected for analysis. The results indicated that Criminals #8, #22 and #59 are at the same level as Criminal #1, while Criminals #9, #14, #78, #89 and #106 are at the same level as Criminal #3. In practice, these results would indicate the use of additional resources and special attention to a wider range of actors during the two years of investigation. For some reason, Criminals #8, #22 and #59 have connections as relevant as those of Criminal #1. The same observation applies to criminals at Criminal #3 level.
Fig. 6 complements and illustrates the information presented in Table 4. We see the bands indicating the four scenarios and the actors highlighted in Table 4. In hollow circles are the criminals analyzed in the original study. In full circles the new actors identified in this research. It is also possible to identify the changing of profiles in each scenario, like obstacle barriers, which rise and become more restrictive to classify the actors. As the bar rises, some criminals retain their main class and others are relocated to class 2.
The study continued with additional calculations on the groups indicated in Table 5. Even if they are classified as highly relevant actors in the network, it is possible that their criminal activities are organized and divided by similar tasks, by geographic location or by any other procedure that allows grouping them in clusters. The numbers indicated in the matrix discriminate clusters that join or separate sets of actors, for each technique used. The clustering analysis of the adjacency matrix was performed using the application “igraph” package, from software R (Csardi & Nepusz, 200629. CSARDI G & NEPUSZ T. 2006. The igraph software package for complex network research. InterJournal, Complex Systems, 1695(5): 19.).
Table 5 shows the results of six clustering techniques for social networks, applied to the criminals highlighted in Table 3. Details of the calculation procedures for these techniques are available in the “igraph” package (Csardi & Nepusz, 200629. CSARDI G & NEPUSZ T. 2006. The igraph software package for complex network research. InterJournal, Complex Systems, 1695(5): 19.). Note that the numerical results in Table 5 do not order the groups, they only discriminate against them. The results indicate that Criminals #1, #8, #3, #106 and, probably #9 and #89, belong to the same functional group in the criminal network. By different techniques, these actors remained in the same group, indicated by the result “1”. It can also be seen that Criminals #22 and #14 coincide in five of the six techniques, and they are also likely to act in the same tasks, they are geographically close or even have some kind of special connection. Criminals #59 and #78 coincided in four of the six techniques and, likewise, may have some type of activity in common in the criminal network. Finally, note that mathematical results are useful indicators for law enforcement officers to intensify their investigative work with priority for certain actors. This prioritization can be decisive in criminal networks composed of a high number of actors and links, as it reduces uncertainty, optimizes the use of investigation resources, and brings the focus to the most relevant actors in the network.
The main source consulted about the Caviar Project indicates that 14 criminals were found guilty in the network. However, the 110 actors “numbered” in Morselli (2009102. MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.) are neither associated with their names nor with their penalties. This lack of information does not allow a direct comparison between results from different approaches. Despite that, two advantages of the methodological proposal are visualized. First, the use of a large amount of centrality measures, submitted to PCA to select the most relevant ones, improves the accuracy and quality of criminal identification, because it goes beyond the use of only two centrality measures used in the original research. Second, after computing the centrality measures, the use of a multicriteria decision aid method to select the most important criminals in the network is a natural step. Thus, this new approach highlighted new actors with important participation in the network.
6 CONCLUSION
The analysis of social networks based on measures of centrality is a useful tool to support researchers in problems in criminology. The analysis of social networks is based on graph theory, which presents several types of centrality measures to evaluate the influence of the nodes of a network, based on the quantification of its links. In the specific case of the Caviar network, the nodes were associated with the actors and links were measured based on wiretapping, whose unit of measure was represented by the number of directional calls from actor “x” to actor “y”. These links were then consolidated into a square matrix, called the adjacency matrix, which sets up the initial database for the analysis of social networks.
In this study, a multicriteria decision support methodology was used to classify the main actors in the criminal network, which operated in the drug trade in Canada. CPP TRI is a variant of CPP, being used for the ordered classification of alternatives. The results of the Caviar network investigations were compared and ratified with the results of application of the methodology proposed.
The article also contributed to a new approach to the application of CPP to support decision making in criminology. Graph theory and social network analysis can be enhanced with the use of MCDA methods, what is confirmed by the publication of studies in this regard. CPP, in particular, has interesting variants to bring new perspectives to investigators, identifying actors who may have remained hidden with the use of few measures of centrality. The methodological procedure that associates PCA to CPP TRI and the use of scenarios with different class profiles also proved to be relevant, as it allowed selecting the actors through progressive restrictions, with reduced computational effort. This methodological procedure becomes even more important if criminal networks have large numbers of actors. The Caviar network was composed of 110 actors, but criminal networks can contain thousands of actors, which must be classified, for the optimization and prioritization of limited investigative resources.
New research can deepen the study presented here. Initially, the possibility of including new CPP variants is imagined. It is possible, for example, to analyze the dynamic evolution of the centrality measures with Malmquist indexes. CPP has applications in this context and the assessment of investigations in stages may receive the application of the method. The possibility of performing CPP TRI tests with network dimensions of the order of thousands of components is also visualized to verify the computational performance of the R software. For the time being, the biggest limitation for this type of test is in the accessibility to real data with matrices of adjacent networks of such dimensions, due to the need of preserving data confidentiality.
References

^{1}ALMEIDADIAS J, FIGUEIRA JR & ROY B. 2012. A multiple criteria sorting method where each category is characterized by several reference actions: The Electre TrinC method. European Journal of Operational Research, 217(3): 567579.

^{2}ALMEIDADIAS J, FIGUEIRA JR & ROY B. 2010. Electre TriC: A multiple criteria sorting method based on characteristic reference actions. European Journal of Operational Research , 204(3): 565580.

^{3}ALMEIDADIAS J, FIGUEIRA JR & ROY B. 2012. A multiple criteria sorting method where each category is characterized by several reference actions: The Electre TrinC method. European Journal of Operational Research , 217(3): 567579.

^{4}ALRASHOUD M, ALMESHARY M & ABHARI A. 2015. Automatic validation for multi criteria decision making models in simulation environments. Proceedings of the 18th Symposium on Communications & Networking, 4447b.

^{5}ANGILELLA S & MAZZÙ S. 2015. The financing of innovative SMEs: A multicriteria credit rating model. European Journal of Operational Research , 244(2): 540554.

^{6}ARAÚJO RM. 2015. Multicriteria Classification with TODIMFSE. Procedia Computer Science, 55, 559565. Available at: https://doi.org/10.1016/j.procs.2015.07.043
» https://doi.org/10.1016/j.procs.2015.07.043 
^{7}ARAZ C & OZKARAHAN I. 2005. A multicriteria sorting procedure for financial classification problems: The case of business failure risk assessment. International Conference on Intelligent Data Engineering and Automated Learning, 563570.

^{8}ASHTIANI M. 2019. CINNA: Deciphering Central Informative Nodes in Network Analysis. R package version 1.1.53 (p. 1). CRAN. Available at: https://cran.rproject.org/package=CINNA
» https://cran.rproject.org/package=CINNA 
^{9}ASL FR, SINAEI H & NEISI A. 2021. Forming a Portfolio of Exchange Traded Funds with the Clustering and UTADIS Hybrid Models. Quarterly Journal of Quantitative Economics (Former Economic Studies), 18(1): 7996.

^{10}ASSUMMA V, BOTTERO M, ISHIZAKA A & TASIOU M. 2021. Group Analytic Hierarchy Process Sorting II Method: An Application to Evaluate the Economic Value of a Wine Region Landscape. Environmental Modeling & Assessment, 26(3): 355369.

^{11}BILUCA J, DE AGUIAR CR & TROJAN F. 2020. Sorting of suitable areas for disposal of construction and demolition waste using GIS and ELECTRE TRI. Waste Management, 114, 307320.

^{12}BLASZCZYNSKI J, ALMEIDA FILHO AT DE, MATUSZYK A, SZELAG M & SLOWINSKI R. 2021. Auto loan fraud detection using dominancebased rough set approach versus machine learning methods. Expert Systems with Applications, 163: 113740.

^{13}BLOCH F, JACKSON MO & TEBALDI P. 2019. Centrality measures in networks. Available at SSRN 2749124.

^{14}BOGGIA A, ROCCHI L, PAOLOTTI L, MUSOTTI F & GRECO S. 2014. Assessing rural sustainable development potentialities using a dominancebased rough set approach. Journal of Environmental Management, 144, 160167.

^{15}BORGATTI SP & EVERETT MG. 2006. A graphtheoretic perspective on centrality. Social Networks, 28(4): 466484.

^{16}BOUYSSOU D & MARCHANT T. 2015. On the relations between ELECTRE TRIB and ELECTRE TRIC and on a new variant of ELECTRE TRIB. European Journal of Operational Research , 242(1): 201211.

^{17}BOUYSSOU D, MARCHANT T & PIRLOT M. 2020. A theoretical look at ELECTRE TRInB. ArXiv Preprint ArXiv:2008.09484.

^{18}BRIGHT D, KOSKINEN J & MALM A. 2019. Illicit network dynamics: The formation and evolution of a drug trafficking network. Journal of Quantitative Criminology, 35(2): 237258.

^{19}BURCHER M & WHELAN C. 2018. Social network analysis as a tool for criminal intelligence: Understanding its potential from the perspectives of intelligence analysts. Trends in Organized Crime, 21(3): 278294.

^{20}CASADO RSGR & SILVA LC E. 2017. Avaliação de risco no APL de confecção do agreste de Pernambuco: uma abordagem da Composição Probabilística com base na FMEA. Simpósio Brasileiro de Pesquisa Operacional  XLIX SBPO.

^{21}CERVI AFC & CARPINETTI LCR. 2017. Competências essenciais da organização: avaliação baseada no método ELECTRE TRI. Pesquisa Operacional Para o Desenvolvimento, 9(2): 95108.

^{22}CHAKHAR S, ISHIZAKA A, LABIB A & SAAD I. 2016. Dominancebased rough set approach for group decisions. European Journal of Operational Research , 251(1): 206 224.

^{23}CHOUDHARY P & SINGH U. 2018. Ranking terrorist organizations network in India using combined SnaAhp approach. International Journal of Recent Technology and Engineering, Article, 7(4): 168172.

^{24}COSTA AS, GOVINDAN K & FIGUEIRA JR. 2018. Supplier classification in emerging economies using the ELECTRE TRInC method: A case study considering sustainability aspects. Journal of Cleaner Production, 201, 925947.

^{25}COSTA AS, RUI FIGUEIRA J, VIEIRA CR & VIEIRA IV. 2019. An application of the ELECTRE TRIC method to characterize government performance in OECD countries. International Transactions in Operational Research, 26(5): 19351955.

^{26}COSTA HG, DE OLIVEIRA NEPOMUCENO LD & PEREIRA V. 2018. A proposal for avoiding compensatory effects while using ELECTRE TRI with multiple evaluators. International Conference on Hybrid Intelligent Systems, 470480.

^{27}COSTA HG & DUARTE MB T. 2019. Applying ELECTRE TRI ME for Evaluating the Quality of Services Provided by a Library. Proceedings of the 2019 11th International Conference on Education Technology and Computers, 278281.

^{28}COUTINHO J. 2016. Caviar database. Ucinet. Available at: https://sites.google.com/site/ucinetsoftware/datasets/covertnetworks/caviar
» https://sites.google.com/site/ucinetsoftware/datasets/covertnetworks/caviar 
^{29}CSARDI G & NEPUSZ T. 2006. The igraph software package for complex network research. InterJournal, Complex Systems, 1695(5): 19.

^{30}DAS K, SAMANTA S & PAL M. 2018. Study on centrality measures in social networks: a survey. Social Network Analysis and Mining, 8(1): 13.

^{31}DAVIES T & JOHNSON SD. 2015. Examining the relationship between road structure and burglary risk via quantitative network analysis. Journal of Quantitative Criminology , 31(3): 481507.

^{32}DE SMET Y. 2019. Beyond Multicriteria Ranking Problems: The Case of PROMETHEE. In New Perspectives in Multiple Criteria Decision Making. Springer, p. 95114.

^{33}DE SMET Y. 2013. P2CLUST: An extension of PROMETHEE II for multicriteria ordered clustering. 2013 IEEE International Conference on Industrial Engineering and Engineering Management, 848851.

^{34}DEHRAJ P & SHARMA A. 2020. Autonomic computing based trustworthiness attributes weight estimation using ElectreTri method. Journal of Interdisciplinary Mathematics, 23(1): 2129.

^{35}DEMIR L, AKPINAR ME, ARAZ C & ILGIN MA. 2018. A green supplier evaluation system based on a new multicriteria sorting method: VIKORSORT. Expert Systems with Applications , 114, 479487.

^{36}DEVAUD JM, GROUSSAUD G & JACQUETLAGREZE E. 1980. UTADIS: Une me´thode de construction de fonctions d’utilite´ additives rendant compte de jugements globaux. European Working Group on Multicriteria Decision Aid, Bochum, 94.

^{37}DOUMPOS M & FIGUEIRA JR. 2019. A multicriteria outranking approach for modeling corporate credit ratings: An application of the Electre TrinC method. Omega, 82, 166 180.

^{38}DU Y, GAO C, HU Y, MAHADEVAN S & DENG Y. 2014. A new method of identifying influential nodes in complex networks based on TOPSIS. Physica A: Statistical Mechanics and Its Applications, 399, 5769.

^{39}DUXBURY SW & HAYNIE DL. 2018. The network structure of opioid distribution on a darknet cryptomarket. Journal of Quantitative Criminology , 34(4): 921941.

^{40}DUXBURY SW & HAYNIE DL. 2019. Criminal network security: An agentbased approach to evaluating network resilience. Criminology, 57(2): 314342.

^{41}ESMAELIAN M, SHAHMORADI H & NEMATI F. 2017. PUTADIS: a multi criteria classification method. Intelligent Computational Systems: A MultiDisciplinary Perspective, 1, 214.

^{42}ESMAELIAN M, SHAHMORADI H & VALI M. 2016. A novel classification method: A hybrid approach based on extension of the UTADIS with polynomial and PSOGA algorithm. Applied Soft Computing, 49, 5670.

^{43}FARAJI SABOKBAR H, HOSSEINI A, BANAITIS A & BANAITIENE N. 2016. A novel sorting method TOPSISSORT: an applicaiton for Tehran environmental quality evaluation. Business Administration and Management, XIX(2): 87104. Available at: https://doi.org/10.15240/tul/001/20162006
» https://doi.org/https://doi.org/10.15240/tul/001/20162006 
^{44}FERNÁNDEZ E, FIGUEIRA JR & NAVARRO J. 2019. An indirect elicitation method for the parameters of the ELECTRE TRInB model using genetic algorithms. Applied Soft Computing , 77, 723733.

^{45}FERNÁNDEZ E, FIGUEIRA JR & NAVARRO J. 2020. Intervalbased extensions of two outranking methods for multicriteria ordinal classification. Omega , 95, 102065.

^{46}FERNÁNDEZ E, FIGUEIRA JR, NAVARRO J & ROY B. 2017. ELECTRE TRInB: A new multiple criteria ordinal classification method. European Journal of Operational Research , 263(1): 214224.

^{47}FERNANDEZ E & NAVARRO J. 2011. A new approach to multicriteria sorting based on fuzzy outranking relations: The THESEUS method. European Journal of Operational Research , 213(2): 405413.

^{48}FERNANDEZ E, NAVARRO J, COVANTES E & RODRIGUEZ J. 2017. Analysis of the effectiveness of the THESEUS multicriteria sorting method: theoretical remarks and experimental evidence. Top, 25(2): 314339.

^{49}FERNANDEZ E, NAVARRO J & SALOMON E. 2014. Automatic enhancement of the reference set for multicriteria sorting in the frame of THESEUS method. Foundations of Computing and Decision Sciences, 39(2): 5777.

^{50}FIGUEIRA JJR, DE SMET Y & BRANS JP. 2005. MCDA methods for sorting and clustering problems: Promethee TRI and Promethee CLUSTER. Technical report TR/SMG/2004002.

^{51}FIGUEIRA JR, ALMEIDADIAS J, MATIAS S, ROY B, CARVALHO MJ & PLANCHA CE. 2011. Electre TriC, a multiple criteria decision aiding sorting model applied to assisted reproduction. International Journal of Medical Informatics, 80(4): 262273.

^{52}FIRE M & PUZIS R. 2016. Organization mining using online social networks. Networks and Spatial Economics, 16(2): 545578.

^{53}FONTANA ME & CAVALCANTE CA V. 2013. Electre tri method used to storage location assignment into categories. Pesquisa Operacional, 33(2): 283303.

^{54}FOX WP & EVERTON SF. 2013. Mathematical modeling in social network analysis: using TOPSIS to find node influences in a social network. Journal of Mathematics and System Science, 3(10): 531.

^{55}FOX WP & EVERTON SF. 2015. Using data envelopment analysis and the analytical hierarchy process to find node influences in a social network. The Journal of Defense Modeling and Simulation, 12(2): 157165.

^{56}GAGANIS C, PAPADIMITRI P & TASIOU M. 2020. A multicriteria decision support tool for modelling bank credit ratings. Annals of Operations Research, 130.

^{57}GARCEZ TV, MARESCHAL B & ALMEIDA AT DE. 2012. Inferência dos pesos dos critérios no método de classificação Promethee TRI. Congresso LatinoIberoamricano de Investigacion Operativa. Rio de Janeiro, Brasil, Setembro, 2428.

^{58}GAVIÃO LO, MEZA LA, LIMA GBA, SANT’ANNA AP & SOARES DE MELLO JCCB. 2017. Improving discrimination in efficiency analysis of bioethanol processes. Journal of Cleaner Production . Available at: https://doi.org/10.1016/j.jclepro.2017.06.020
» https://doi.org/https://doi.org/10.1016/j.jclepro.2017.06.020 
^{59}GAVIÃO LO, SANT’ANNA AP, ALVES LIMA GB & DE ALMADA GARCIA PA. 2019. Evaluation of soccer players under the Moneyball concept. Journal of Sports Sciences. Available at: https://doi.org/10.1080/02640414.2019.1702280
» https://doi.org/https://doi.org/10.1080/02640414.2019.1702280 
^{60}GAVIÃO LO, SANT’ANNA AP & LIMA GBA. 2019. A Composição Probabilística de Preferências com medidas de desigualdade: correlações com os pontos de vista progressista e conservador. Pesquisa Operacional e Sua Atuação Multidisciplinar, 217232. Available at: https://doi.org/10.22533/at.ed.78819110716
» https://doi.org/https://doi.org/10.22533/at.ed.78819110716 
^{61}GAVIÃO LO, SANT’ANNA AP, LIMA GBA & GARCIA PA DE A. 2018. CPP: Composition of Probabilistic Preferences. R package version 0.1.0. (R package version 0.1.0.; p. 124). R Core Team. Available at: https://cran.rproject.org/package=CPP
» https://cran.rproject.org/package=CPP 
^{62}GAVIÃO LO, SANT’ANNA AP, LIMA GBA & GARCIA PA DE A. 2020. Evaluation of soccer players under the Moneyball concept. Journal of Sports Sciences , 38(1112 ), 12211247.

^{63}GONÇALVES ATP, ARAÚJO MVP DE, MÓL ALR & ROCHA FAF DA. 2021. Application of the Electre Tri method for supplier classification in supply chains. Pesquisa Operacional , 41.

^{64}GOVINDAN K & JEPSEN MB. 2016. Supplier risk assessment based on trapezoidal intuitionistic fuzzy numbers and ELECTRE TRIC: a case illustration involving service suppliers. Journal of the Operational Research Society, 67(2): 339376.

^{65}GRECO S, MATARAZZO B & SLOWINSKI R. 2000. Multicriteria classification by dominancebased rough set approach. In Laboratory of Intelligent Decision Support Systems. Institute of Computing Science. Available at: http://idss.cs.put.poznan.pl/site/ fileadmin/projectsimages/4emka\ methodology.pdf
» http://idss.cs.put.poznan.pl/site/ fileadmin/projectsimages/4emka\ methodology.pdf 
^{66}GRECO S, MATARAZZO B & SŁOWIŃSKI R. 2007. Dominancebased rough set approach as a proper way of handling graduality in rough set theory. In Transactions on rough sets VII (p. 3652). Springer.

^{67}GÜL M. 2021. A quantitative occupational risk assessment methodology based on TOPSISSort with its application in aluminum extrusion industry. International Journal of Pure and Applied Sciences, 7(1): 163172.

^{68}GUO Q, QIN J & QU, C. 2020. AHPSortllGAIA: A Sorting Method in Model Engineering to Sort Cities According to the Risk of Sustainable. 2020 2nd International Conference on Economic Management and Model Engineering (ICEMME), 797804.

^{69}HARPER WR & HARRIS DH. 1975. The application of link analysis to police intelligence. Human Factors, 17(2): 157164.

^{70}HU J, DU Y, MO H, WEI D & DENG Y. 2016. A modified weighted TOPSIS to identify influential nodes in complex networks. Physica A: Statistical Mechanics and Its Applications , 444, 7385.

^{71}HU Q, CHAKHAR S, SIRAJ S & LABIB A. 2017. Spare parts classification in industrial manufacturing using the dominancebased rough set approach. European Journal of Operational Research , 262(3): 11361163.

^{72}ISHIZAKA A & GORDON M. 2017. MACBETHSort: a multiple criteria decision aid procedure for sorting strategic products. Journal of the Operational Research Society , 68(1): 5361.

^{73}ISHIZAKA A, KHAN SA, KUSISARPONG S & NAIM I. 2020. Sustainable warehouse evaluation with AHPSort traffic light visualisation and postoptimal analysis method. Journal of the Operational Research Society , 118.

^{74}ISHIZAKA A, LOLLI F, GAMBERINI R, RIMINI B & BALUGANI E. 2017. AHPKGDSS: a new sorting method based on AHP for group decisions. 16th International Conference on Modeling and Applied Simulation, MAS 2017, 15.

^{75}ISHIZAKA A & LÓPEZ C. 2019. Costbenefit AHPSort for performance analysis of offshore providers. International Journal of Production Research, 57(13): 42614277.

^{76}ISHIZAKA A & NEMERY P. 2014. Assigning machines to incomparable maintenance strategies with ELECTRESORT. Omega , 47, 4559.

^{77}ISHIZAKA A, PEARMAN C & NEMERY P. 2012. AHPSort: an AHPbased method for sorting problems. International Journal of Production Research , 50(17): 47674784.

^{78}ISHIZAKA A & PEREIRA V. 2019. Utilisation of ANPSort for sorting alternative with interdependent criteria illustrated through a researcher’s classification problem in an academic context. Soft Computing, 112.

^{79}ISHIZAKA A, PEREIRA V & SIRAJ S. 2021. AHPSortGAIA: a visualisation tool for the sorting of alternative in AHP portrayed through a case in the food and drink industry. Annals of Operations Research , 116.

^{80}ISHIZAKA A, TASIOU M & MARTÍNEZ L. 2020. Analytic hierarchy processfuzzy sorting: An analytic hierarchy processbased method for fuzzy classification in sorting problems. Journal of the Operational Research Society , 71(6): 928947.

^{81}JASIŃSKI D, CINELLI M, DIAS LC, MEREDITH J & KIRWAN K. 2018. Assessing supply risks for nonfossil mineral resources via multicriteria decision analysis. Resources Policy, 58, 150158.

^{82}KADZIŃSKI M,TERVONEN T & FIGUEIRA JR. 2015. Robust multicriteria sorting with the outranking preference model and characteristic profiles. Omega , 55, 126140.

^{83}KARABAY S, KÖSE E, KABAK M & OZCEYLAN E. 2016. Mathematical model and stochastic multicriteria acceptability analysis for facility location problem. PrometTraffic&Transportation, 28(3): 245256.

^{84}KERMANI MAMA, BADIEE A, ALIAHMADI A, GHAZANFARI M & KALANTARI H. 2016. Introducing a procedure for developing a novel centrality measure (Sociability Centrality) for social networks using TOPSIS method and genetic algorithm. Computers in Human Behavior, 56, 295305.

^{85}KREJČÍ J & ISHIZAKA A. 2018. FAHPSort: A fuzzy extension of the AHPSort method. International Journal of Information Technology & Decision Making, 17(04): 11191145.

^{86}KÜÇÜKBAY F & METIN İ. 2019. A new methodology for credit evaluation process of Turk Eximbank. Uşak Üniversitesi Sosyal Bilimler Dergisi, 12(2): 122130.

^{87}LABELLA Á, ISHIZAKA A & MARTÍNEZ L. 2021. Consensual GroupAHPSort: applying consensus to GAHPSort in sustainable development and industrial engineering. Computers & Industrial Engineering, 152, 107013.

^{88}LABELLA Á, RODRÍGUEZCOHARD JC, SÁNCHEZMARTÍNEZ JD & MARTÍNEZ L. 2020. An AHPSort II Based Analysis of the Inequality Reduction within European Union. Mathematics, 8(4): 646.

^{89}LIOU JJH & TZENG, G.H. 2010. A dominancebased rough set approach to customer behavior in the airline market. Information Sciences, 180(11): 22302238.

^{90}LIU L, ZUO W & PENG T. 2018. A reliability and link analysis based method for mining domain experts in dynamic social networks. Journal of Intelligent & Fuzzy Systems, 34(4): 20612073.

^{91}LOLLI F, ISHIZAKA A, GAMBERINI R, RIMINI B, BALUGANI E & PRANDINI L. 2017. Requalifying public buildings and utilities using a group decision support system. Journal of Cleaner Production , 164, 10811092.

^{92}LOLLI F, ISHIZAKA A, GAMBERINI R, RIMINI B & MESSORI M. 2015. FlowSortGDSSA novel group multicriteria decision support system for sorting problems with application to FMEA. Expert Systems with Applications , 42(1718), 63426349.

^{93}LÓPEZ C & ISHIZAKA A. 2017. GAHPSort: A new group multicriteria decision method for sorting a large number of the cloudbased ERP solutions. Computers in Industry, 92, 1225.

^{94}MANSHADI ED, MEHREGAN MR & SAFARI H. 2015. Supplier classification using UTADIS method based on performance criteria. International Journal of Academic Research in Business and Social Sciences, 5(2): 31.

^{95}MASÍAS VH, VALLE M, MORSELLI C, CRESPO F, VARGAS A & LAENGLE S. 2016. Modeling verdict outcomes using social network measures: the watergate and caviar network cases. PloS One, 11(1): e0147248.

^{96}MCCUISH E, BOUCHARD M & BEAUREGARD E. 2020. A NetworkBased Examination of the Longitudinal Association Between Psychopathy and Offending Versatility. Journal of Quantitative Criminology. Available at: https://doi.org/https://doi.org/10.1007/s1094002009462w
» https://doi.org/https://doi.org/10.1007/s1094002009462w 
^{97}MCMILLAN C, FELMLEE D & BRAINES D. 2020. Dynamic patterns of terrorist networks: Efficiency and security in the evolution of eleven islamic extremist attack networks. Journal of Quantitative Criminology , 36(3): 559581.

^{98}MEHREGAN MR, SADEGHI MOGHADAM MR & EMAMAT M. 2018. Applying the Clustering and UTADIS Models to form an Investment Portfolio. Financial Research Journal, 20(1): 5374.

^{99}MEI Y, TU Y, XIE K, YE Y & SHEN W. 2019. Internet Public Opinion Risk Grading under Emergency Event Based on AHPSort IIDEMATEL. Sustainability, 11(16): 4440.

^{100}MICCOLI F & ISHIZAKA A. 2017. Sorting municipalities in Umbria according to the risk of wolf attacks with AHPSort II Ecological Indicators, 73, 741755.

^{101}MONTE F DE C. 2019. Classificação do nível de maturidade em gestão de processos de negócios através da aplicação da composição probabilística de preferências tricotômica (CPPTRI) [Universidade Federal de Pernambuco]. Available at: https://attena.ufpe.br/bitstream/123456789/34083/1/DISSERTAOFrancieldeCarvalhoMonte.pdf
» https://attena.ufpe.br/bitstream/123456789/34083/1/DISSERTAOFrancieldeCarvalhoMonte.pdf 
^{102}MORSELLI C. 2009. Inside criminal networks (Vol. 8). Springer.

^{103}MORSELLI C, GIGUÈRE C & PETIT K. 2007. The efficiency/security tradeoff in criminal networks. Social Networks , 29(1): 143153.

^{104}MORSELLI C, MASIAS VH, CRESPO F & LAENGLE S. 2013. Predicting sentencing outcomes with centrality measures. Security Informatics, 2(1): 4.

^{105}MORSELLI C & PETIT K. 2007. Lawenforcement disruption of a drug importation network. Global Crime, 8(2): 109130.

^{106}NAVARRO J, FERNANDEZ E & OLMEDO R. 2013. Handling incomplete information in the framework of THESEUS multicriteria sorting method. Fourth International Workshop on Knowledge Discovery, Knowledge Management and Decision Support, 164168.

^{107}NEMERY P, ISHIZAKA A, CAMARGO M & MOREL L. 2012. Enriching descriptive information in ranking and sorting problems with visualizations techniques. Journal of Modelling in Management.

^{108}NEMERY P & LAMBORAY C. 2008. Flow Sort: a flowbased sorting method with limiting or central profiles. Top , 16(1): 90113.

^{109}NEMERY P & LAMBORAY C. 2007. FlowSort: a sorting method based on flows. Proceedings of the ORP3 Conference, 4560.

^{110}OLDHAM S, FULCHER B, PARKES L, ARNATKEVICIŪTE A, SUO C & FORNITO A. 2019. Consistency and differences between centrality measures across distinct classes of networks. PloS One , 14(7).

^{111}OUHIBI A & FRIKHA H. 2019. CODASSORT: A new CODAS based method for sorting problems. 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT), 855860.

^{112}OUHIBI A & FRIKHA H. 2020a. Intervalvalued intuitionistic fuzzy CODASSORT method: Evaluation of natural resources in Tunisia. 2020 International MultiConference on:“Organization of Knowledge and Advanced Technologies”(OCTA), 15.

^{113}OUHIBI A & FRIKHA HM. 2020b. Evaluating environmental quality in Tunisia using Fuzzy CODAS SORT method. 2020 International Conference on Decision Aid Sciences and Application (DASA), 11151119.

^{114}PASSOS AC & GOMES LF A. M. 2014. TODIMFSE: A multicriteria classification method based on prospect theory. Multiple Criteria Decision Making, 9, 123139.

^{115}PASSOS AC, TEIXEIRA MG, GARCIA KC, CARDOSO AM & GOMES LFAM. 2014. Using the TODIMFSE method as a decisionmaking support methodology for oil spill response. Computers & Operations Research, 42, 4048.

^{116}PELISSARI R, ABACKERLI AJ, AMOR SBEN, OLIVEIRA MC & INFANTE KM. 2021. Multiple criteria hierarchy process for sorting problems under uncertainty applied to the evaluation of the operational maturity of research institutions. Omega , 103, 102381.

^{117}PELISSARI R, BENAMOR S & DE OLIVEIRA MC. 2019. A MultipleCriteria Decision Sorting Model for Pharmaceutical Suppliers Classification Under Multiple Uncertainties. In Pharmaceutical Supply ChainsMedicines Shortages (p. 229247). Springer.

^{118}PELISSARI R, OLIVEIRA MC, AMOR S, BEN & ABACKERLI AJ. 2019. A new FlowSortbased method to deal with information imperfections in sorting decisionmaking problems. European Journal of Operational Research , 276(1): 235246.

^{119}PEREIRA J, DE OLIVEIRA ECB, GOMES LFAM & ARAÚJO RM. 2019. Sorting retail locations in a large urban city by using ELECTRE TRIC and trapezoidal fuzzy numbers. Soft Computing, 23(12): 41934206.

^{120}PEREIRA V. 2019. A proposal for avoiding compensatory effects while using ELECTRE TRI with multiple evaluators . Hybrid Intelligent Systems: 18th International Conference on Hybrid Intelligent Systems (HIS 2018) Held in Porto, Portugal, December 1315, 2018, 923, 470.

^{121}QIN J, ZENG Y & ZHOU Y. 2021. Contextdependent DEASort: A multiple criteria sorting method for ecological risk assessment problems. Information Sciences , 572, 88 108.

^{122}RADIL SM, FLINT C & TITA GE. 2010. Spatializing social networks: Using social network analysis to investigate geographies of gang rivalry, territoriality, and violence in Los Angeles. Annals of the Association of American Geographers, 100(2): 307326.

^{123}RAHMANI ASL F, SINAEI HA & NEIS A. 2021. Forming an portfolio of Exchange Traded Funds with the Clustering and UTADIS Models. Quarterly Journal of Quantitative Economics, 18(1): 7996.

^{124}RATCLIFFE JH. 2016. Intelligenceled policing (2nd ed.). Routledge.

^{125}REMADI FD & FRIKHA HM. 2020. The Intuitionistic Fuzzy Set FlowSort methodology for green supplier Evaluation. 2020 International Conference on Decision Aid Sciences and Application (DASA), 719725.

^{126}REZAEI J, KADZIŃSKI M, VANA C & TAVASSZY L. 2017. Embedding carbon impact assessment in multicriteria supplier segmentation using ELECTRE TRIrC. Annals of Operations Research , 123.

^{127}ROCCHI L, KADZIŃSKI M, MENCONI ME, GROHMANN D, MIEBS G, PAOLOTTI L & BOGGIA A. 2018. Sustainability evaluation of retrofitting solutions for rural buildings through life cycle approach and multicriteria analysis. Energy and Buildings, 173, 281 290.

^{128}ROSENFELD J & SMET YDE. 2019. An extension of PROMETHEE to hierarchical multicriteria clustering. International Journal of Multicriteria Decision Making, 8(2): 133150.

^{129}ROY B & BOUYSSOU D. 1993. Aide multicritère à la décision: méthodes et cas. Economica.

^{130}ŞAHIN M, ULUCAN A & YURDUGÜL H. 2021. Learner classification based on interaction data in Elearning environments: the ELECTRE TRI method. Education and Information Technologies, 26(2): 23092326.

^{131}SANT’ANNA AP. 2015. Probabilistic Composition of Preferences, Theory and Applications (1st ed.). Springer.

^{132}SANT’ANNA AP, COSTA HG & PEREIRA V. 2012. CPPTri: um método de classificação ordenada baseado em Composição Probabilística. Relatórios de Pesquisa Em Engenharia de Produção (UFF), 12(8): 104117.

^{133}SANT’ANNA AP, COSTA HG & PEREIRA V. 2015. CPPTRI: a sorting method based on the probabilistic composition of preferences. International Journal of Information and Decision Sciences, 7(3): 193212.

^{134}SANT’ANNA AP & SANT’ANNA LAFP. 2001. Randomization as a stage in criteria combining. International Conference on Industrial Engineering and Operations Management  VII ICIEOM, 248256.

^{135}SANT’ANNA AP. 2009. A probabilistic approach to the evaluation and combination of preferences. In Abraham A, Hassanien AE & Snášel V (Eds.), Foundations of Computational Intelligence Volume 5 (p. 107125). Springer.

^{136}SANT’ANNA AP. 2014. Aplicação do CPPTri à Classificação dos países pelos Critérios do IDH. Encontro Nacional de Engenharia de Produção  XXXVI ENEGEP.

^{137}SANT’ANNA AP. 2015. Probabilistic human development indices. Brazilian Journal of Operations & Production Management, 12(1): 136146.

^{138}SANT’ANNA AP, COSTA HG, NEPOMUCENO LDO & PEREIRA V. 2016. A Probabilistic Approach Applied to the Classification of Courses by Multiple Evaluators. Pesquisa Operacional .

^{139}SANT’ANNA AP, FARIA F & COSTA HG. 2013. Aplicação da Composição Probabilística e do método das KMédias à classificação de municípios quanto à oferta de creches. Cadernos Do IMESérie Estatística, 34(1): 17.

^{140}SANT’ANNA AP, GOMES LFAM, COSTA FF DA, RANGEL LAD, FARIA MJ DA S, FERREIRA RG, FILHO RMM, RIBEIRO ROA & SENNA V DE. 2012. Análise multicritério baseada em probabilidades de preferência. In V. F. de Oliveira VCavenaghi & F. S. Másculo (Eds.), Tópicos emergentes e desafios metodológicos em Engenharia de Produção: casos, experiências e proposições  Volume V (p. 258). ABEPRO.

^{141}SANT’ANNA AP, MARTINS EF, LIMA GBA & DA FONSECA RA. 2015. Beta distributed preferences in the comparison of failure modes. Procedia Computer Science , 55, 862869.

^{142}SANT’ANNA AP, MARTINS EF, LIMA GBA & SILVA RF DA. 2015. Beta Distributed Preferences in the Comparison of Failure Modes. Procedia Computer Science , 55, 862 869.

^{143}SARRAZIN R, DE SMET Y & ROSENFELD J. 2018. An extension of PROMETHEE to interval clustering. Omega , 80, 1221.

^{144}SEPULVEDA J, GONZALEZ J, ALFARO M & CAMARGO M. 2010. A metricsbased diagnosis tool for enhancing innovation capabilities in SMEs. International Journal of Computers Communications & Control, 5(5): 919928.

^{145}SHOMAN W & GÜLGEN F. 2017. Centralitybased hierarchy for street network generalization in multiresolution maps. Geocarto International, 32(12): 13521366.

^{146}SILVA ACCLM & ALENCAR LH. 2019. Model for sorting partners in construction: a case study in a highway project. 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 15181523.

^{147}SILVA DF DE L & ALMEIDA FILHO AT DE. 2020. Sorting with TOPSIS through boundary and characteristic profiles. Computers & Industrial Engineering, 141, 106328.

^{148}SILVA DF DE L, FERREIRA L & ALMEIDA FILHO AT DE. 2020. A new preference disaggregation TOPSIS approach applied to sort corporate bonds based on financial statements and expert’s assessment. Expert Systems with Applications , 152, 113369.

^{149}SILVA GB DA, BARROS MD DE & COSTA HG. 2016. Abordagem híbrida multicritério mineração de dados aplicada a classificação de unidades da federação com base na população economicamente ocupada . In SPOLM (Ed.), Simpósio de Pesquisa Operacional e Logística da Marinha (Vol. 2, Issue 1, p. 377387). CASNAV.

^{150}SKILLICORN DB, ZHENG Q & MORSELLI C. 2014. Modeling dynamic social networks using spectral embedding. Social Network Analysis and Mining , 4(1): 182.

^{151}SOBRAL MFF & COSTA AP C. S. 2012. Negotiation model for group decision with ELECTRE TRIthe ELECTRE TRING. Journal of Decision Systems, 21(2): 121136.

^{152}SOUZA RG DE, CLÍMACO JCN, SANT’ANNA AP, ROCHA TB, VALLE R DE AB DO & QUELHAS OL G. 2016. Sustainability assessment and prioritisation of ewaste management options in Brazil. Waste Management .

^{153}SZAJUBOK NK, MOTA CM DE M & ALMEIDA AT DE. 2006. Uso do método multicritério ELECTRE TRI para classificação de estoques na construção civil. Pesquisa Operacional , 26(3): 625648.

^{154}TAHA K & YOO PD. 2019. Shortlisting the influential members of criminal organizations and identifying their important communication channels. IEEE Transactions on Information Forensics and Security, 14(8): 19881999.

^{155}TAHA K & YOO PD. 2015. A system for analyzing criminal social networks. Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015, 10171023.

^{156}TERVONEN T. 2014. JSMAA: open source software for SMAA computations. International Journal of Systems Science, 45(1): 6981.

^{157}TERVONEN T & FIGUEIRA JR. 2008. A survey on stochastic multicriteria acceptability analysis methods. Journal of MultiCriteria Decision Analysis, 15(12), 114.

^{158}TERVONEN T, FIGUEIRA JR, LAHDELMA R, DIAS JA & SALMINEN P. 2009. A stochastic method for robustness analysis in sorting problems. European Journal of Operational Research , 192(1): 236242.

^{159}TERVONEN T, LAHDELMA R, DIAS JA, FIGUEIRA J & SALMINEN P. 2007. SMAATRI. In Environmental security in harbors and coastal areas (p. 217231). Springer.

^{160}TERVONEN T, LINKOV I, FIGUEIRA JR, STEEVENS J, CHAPPELL M & MERAD M. 2009. Riskbased classification system of nanomaterials. Journal of Nanoparticle Research, 11(4): 757766.

^{161}TILLEY N. 2016. Intelligenceled policing and the disruption of organized crime: motifs, methods and morals. In T. Delpeuch & J. E. Ross (Eds.), Comparing the Democratic Governance of Police Intelligence: New Models of Participation and Expertise in the United States and Europe (1st ed., p. 153179). Edward Elgar Publishing.

^{162}TITA GE & RADIL SM. 2011. Spatializing the social networks of gangs to explore patterns of violence. Journal of Quantitative Criminology , 27(4): 521545.

^{163}TROJAN F & MORAIS DC. 2012. Using Electre TRI to support maintenance of water distribution networks. Pesquisa Operacional , 32(2): 423442.

^{164}VIANA VFC & ALENCAR LH. 2015. Modelo de classificação de projetos de acordo com a complexidade gerencial para uma empresa de médio porte em Pernambuco. XLVII SBPO, Porto de Galinhas, Pernambuco.

^{165}WIIL UK, GNIADEK J & MEMON N. 2010. Measuring link importance in terrorist networks. 2010 International Conference on Advances in Social Networks Analysis and Mining, 225232.

^{166}XIE K, MEI Y, GUI P & LIU Y. 2019. Earlywarning analysis of crowd stampede in metro station commercial area based on internet of things. Multimedia Tools and Applications, 78(21): 3014130157.

^{167}XU Z, QIN J, LIU J & MARTINEZ L. 2019. Sustainable supplier selection based on AHPSort II in interval type2 fuzzy environment. Information Sciences , 483, 273293.

^{168}YAMAGISHI K & OCAMPO L. 2021. Utilizing TOPSISSort for sorting tourist sites for perceived COVID19 exposure. Current Issues in Tourism, 111.

^{169}YANG R & JIANG Q. 2020. Detecting Falsified Financial Statements Using a Hybrid SMUTADIS Approach: Empirical Analysis of Listed Traditional Chinese Medicine Companies in China. Discrete Dynamics in Nature and Society, 2020.

^{170}YU W. 1992. Aide multicritère à la décision dans le cadre de la problématique du tri: concepts, méthodes et applications. Université Paris IXDauphine.

^{171}ZENG Y. 2019. Urban Public Safety Risk Assessment of 35 Cities in China Based on ContextDependent DEA Approach And DEASort. E3S Web of Conferences, 136, 4094.

^{172}ZHOU Y. 2020. A Study of Country Economic Risks in Countries Along the Belt and Road: A Classification Method Based on DEA Sort. Journal of Physics: Conference Series, 1453(1): 12147.

^{173}ZOPOUNIDIS C & DOUMPOS M. 1999. Business failure prediction using the UTADIS multicriteria analysis method. Journal of the Operational Research Society , 50(11): 1138 1148.

ERRATUM
In the article Multicriteria decision support to criminology by graph theory and composition of probabilistic preferences, with DOI number: 10.1590/01017438.2021.041.00249751, published in the journal Pesquisa Operacional, 41: e249751, page 1, on author Lucio Camara e Silva affiliation,Reads:^{4}Universidade Federal Fluminense (UFF), Rua Passo da Pátria, 156, Bloco D, São Domingos, 24220240 NiteróiRJ, Brazil  Email: luciocsilva@gmail.com  http://orcid.org/0000000160430019Should read:^{4}Universidade Federal de Pernambuco (UFPE), Centro Acadêmico do Agreste, Avenida Campina Grande, Bairro Nova Caruaru, CaruaruPE, Brazil  Email: luciocsilva@gmail.com  http://orcid.org/0000000160430019
Publication Dates

Publication in this collection
26 Nov 2021 
Date of issue
2021
History

Received
12 Mar 2021 
Accepted
02 Sept 2021