A MULTICRITERIA DECISION MODEL FOR RISK MANAGEMENT MATURITY EVALUATION

ABSTRACT This article aims to present a Multicriteria Decision Aiding (MCDA) model for assessing risk management maturity. Therefore, it is proposed to use a Maturity Model (MM) for risk management aligned with the ELECTRE TRI method. The ELECTRE TRI was chosen as the sorting method because it has a strong axiomatic structure based on the relationship of concordance and discordance between the alternative and the profile that delimits each of its classes. To test the proposal, a case study was carried out on a real company in the construction industry. For the development of the risk management maturity assessment model, a questionnaire was applied to collect data related to risk management practices in the organization. After collection, the data were used for modeling in a Decision Support System to apply the ELECTRE TRI, which managed to classify and identify the organization’s risk management maturity at level 3 (managed).


INTRODUCTION
Risk management (RM) is a relevant topic for any organization as it offers integrated strategies for evaluating, controlling, and monitoring decisions that involve risks (Hopkin, 2010).To say that an organization reaches maturity in risk management means there is an evolution towards the full development of risk management processes (Hoseini, Hertogh & Bosch-Rekyeldt, 2019).In this regard, Maturity Models (MMs) are instruments that support the measurement of risk maturity in organizations.
MMs offer organizations a simple and effective way to assess their process development.MMs were developed to evaluate the capability and effectiveness of systems in different situations through a coherent, capabilities-based framework (Macgillivray et al., 2007;Sheehan et al., 2021).Despite being widely diffused initially in software engineering, application areas have been expanding rapidly and research has gained increasing importance (Wendler, 2012;Santos-Neto & Costa, 2019).A Risk Management Maturity Model (RMMM) aims to measure the maturity of risk management in projects and/or organizations.
These MMs can assess the current state of the RM and identify where it should prioritize intervention to reach higher levels of maturity (Zou, Chen & Chan, 2010).The foundation of the RM and maturity assessment supports a company in gaining an understanding of its current ERM implementation, as well as the strong and weak aspects of ERM implementation (Zhao, Hwang &Low, 2016).
The complexity of the evaluation process proposed by some MMs and the lack of operationalization are considered barriers that hinder the use of MMs as a means of management and organizational diagnosis (Röglinger et al., 2012).Santos-Neto & Costa (2019) identified in their literature review that 24% of the articles found on MM do not clearly present the sorting method used to identify the maturity level.
Thus, this article aims to present a Multicriteria Decision Aiding (MCDA) model for assessing risk management maturity.To this end, the application of an RMMM is aligned with the MCDA ELECTRE TRI method as an alternative for sorting the maturity level.It is believed that the use of MCDA can be a way to standardize the set of procedures necessary for the application of MMHence, MCDA methods can support the Decision Maker (DM) in a problem concerning a sorting of maturity levels by comparing information and characteristics of the maturity model through a set of attributes.
The MCDA approach seeks to support the solution of problems that demand complex decisions, which involve multiple criteria, some conflicting with each other when evaluating the actions (Trojan & Morais, 2012;Gonc ¸alves et al., 2021).Furthermore, MCDA methods admit a systematic view of the problem assessment and are efficient in comparing alternatives via multiple attributes allowing the combination of both subjective and objective attributes (Rodrigues et al., 2022;Lacerda, Santos-Neto & Martins, 2021).Besides that, allow to making a decision by choosing the best one from a set of options in the attendance of multiple and conflict attributes (Santos-Neto & Costa, 2023).Thus, it is expected that the use of MCDA provides sci-entificity and robustness of the results found, which facilitates the assessment of maturity for a benchmarking approach.

BACKGROUND
Organizations look for objective ways to monitor and control their own risk management and identify weaknesses and opportunities for improvement (Wibowo & Taufik, 2017;Hoseini, Hertogh & Bosch-Rekyeldt, 2019).This justifies the establishment of standardized procedures to improve the application of maturity models focused on risk management as an alternative to meet these demands.Furthermore, it covers the gap raised by Santos-Neto & Costa (2019) about the scarcity of evaluation models for the application of MM.
For the RM domain, some different MMs have been developed over the years (Santos-Neto & Costa, 2019).Some MMs in RM are shown in Table 1.MCDA is a term that describes a collection of approaches that aims to support individuals or groups in the process of making decisions, taking explicit account of multiple criteria (Sapienza et al., 2016).In this process, the decision maker is a key actor, as he/she is one of the main sources of information and is responsible for establishing the constraints, preferences and assessing each alternative.According to Belton & Stewart (2002), the MCDA approach offers the following advantages: it seeks to clarify all the multiple factors involved in a decision, provides a structured analysis for the problem, helps the decision maker by synthesizing and presenting all the information; and even though the process does not provide an "ideal solution", it allows the decision maker to reach an agreement between his preferences and the possible outcomes.
To test the MCDA method combined with a MM to assess RM, a company in the construction industry was selected to develop a decision model in this study.Therefore, due to the alignment of the MM with the target company's segment of the test, the MM chosen for testing was the RM3 (Risk Management Maturity Model).According to Zou, Chen & Chan (2010), the model was developed through comparison with other similar MMs, compiling the aspects considered most important for identifying the main characteristics of risk management specifically in construction companies.
Although each author gives a name or establishes different aggregations, in RM3 the dimensions can be translated into five main approaches: management (people and leadership) in relation to risk; organizational risk culture; identifying risks; analyzing risks; and standardized management process of risks.
According to Zou, Chen & Chan (2010), civil construction activity strongly depends on the interaction between employees and leadership.While management must ensure productivity and quality, it is necessary to cultivate an organizational climate that encourages employees.Within RM3, these factors (management and culture) are encompassed by the first two attributes, which define the operational part of the organization.The identification, analysis, and standardization of risks, on the other hand, focus specifically on the resolution of events and at the same time consider the possible risks involved.The combination of these factors leads to a general understanding of risk management for an organization focused on civil construction.

RESEARCH METHODOLOGY
For the development of the Multicriteria Decision Model for RM evaluation, we developed a framework for the research model summarized in three phases, as shown in Figure 1.
In the preliminary phase, we characterized the decision maker and defined the evaluation criteria for the decision problem, and performed data collection.The decision maker is the person responsible for the decision and for establishing relationships and judgment of values that influence the decision process.At this stage, the Project Coordinator of the organization that was the focus of the study was identified as the decision maker who was responsible for evaluating and acting prescriptively on the result of the maturity assessment.The decision maker had experience in the position and extensive contact with the preparation and execution of projects developed by the organization.
For the definition of the set of criteria, the dimensions listed in the RM3 model were selected.A total of five criteria were defined, namely Management perspective (people and leadership) concerning risk, Organizational risk culture, Identifying risks, Analyzing risks, and Standardized risk management process.The criteria are measured to assess maturity at four levels: initial, repeated, managed, and optimized, which are the sorts of the proposed model.They define the organization's maturity level in RM.
The attributes proposed in RM3 reflected the fundamentals of risk management and were designed to benefit construction companies in measuring and improving their risk management capabilities.The meaning of each criterion is defined as: e) Standardized risk management process: measure if a standardized risk management process is applied to all projects within the organization.
A survey questionnaire to assess the organization's performance regarding the problem criteria was defined for data collection.The questionnaire was a means of allowing the organization's decision maker and specialists to assess the RM dimensions of each of the processes in a simplified way.The RM3 defined five main dimensions that are translated into five questions/statements each (total of 25), which represented the practices related to RM.
For data collection, five specialists directly in contact with the organization's processes were selected to answer the questionnaire: coordinator (5 years with the company); resident engineer (8 years with the company); engineering assistant (2 years with the company); production supervisor (7 years with the company); engineering assistant (3 years with the company).All specialists were employees of the company and were included daily in the production routine, at different hierarchical levels.
Each of the experts rated each question/statement of the questionnaire according to a fivepoint Likert scale, where 1 meant "The practice is not widely used in the organization" and 5 meant "The practice is widely used in the organization," as shown in Table 2.The application questionnaire to verify the level of applicability of risk management practices is found in Appendix A.

Scale Description 1
The practice is not widely used in the organization.2 There is a strong discussion about using the practice, but no decision.3 There is the decision and action plan to start using the practice.4 The practice has been tested and experience can be gained.5 The practice is widely used in the organization.
In the modeling phase, after data collection, the answers were evaluated using the mode found in each criterion, and the value identified in the criterion was used as the organization's performance in composing the decision matrix.
To assess the maturity level, the ELECTRE TRI method was selected.The ELECTRE TRI method was chosen as the sorting method by observing the non-compensatory rationality of the decision maker.In other words, it is expected that the substandard performance of one criterion is not compensated by the good performance of another, something that is exploited by ELECTRE TRIFurthermore, ELECTRE TRI has a strong axiomatic structure and uses a Decision Support System (DSS) for application.For this study, the DSS MCDA-ULAVAL was used.MCDA-ULAVAL, a Canadian software developed at Laval University, has free access and an open source (ULAVAL, 2018).
Particularly in the ELECTRE TRI method, the sorting of an alternative, a, into a given class, c h (h = 1, 2, ..., p), is made based on the comparison between the alternative and the profile that defines the limit of each category (Brito, Almeida & Mota, 2010).This factor allows the sorting of only one alternative, if necessary, as the comparison is made against the profile and not with other alternatives.
The sorting criteria g j (j = 1, 2, ..., m) are used to judge the alternative and compare it with the profile, b h , which represents the upper limit of class c h and the lower limit of class c h+1 , as depicted in Figure 2 (Mousseau, Figueira & Naux, 2001).(2002) proposed a method and a Decision Support System for defining weights of criteria used in some methods of the ELECTRE family.The described method is based on the use of cards and its application is described in Section 4 of this study.
Credibility Index: Partial concordance: Concordance: Discordance: When measuring credibility indexes, the result can be evaluated in two ways: Pessimistic procedure and Optimistic procedure.The pessimistic approach is made from successive comparisons between a and b h , h = 1, 2, ..., p, with a being allocated to the first class, c h+1 , in which aSb h is verified.The other procedure, optimistic, compares a and b h , h = p, p-1, ..., 1, with a being allocated to the first class c h where b h is preferable to a.More details about the ELECTRE TRI algorithm can be seen in Mousseau, Figueira & Naux (2001).
In the finalization phase, the sensitivity analysis was performed to verify the method's robustness.
For this, the model parameters must be varied to observe possible changes in the model results.
For this study, we performed the sensitivity analysis varying the cutting level λ between 0.7 and 1.Finally, to evaluate the results, the scenario identified by the MM result was clarified with the decision maker to determine its consistency with reality.

EMPIRICAL APPLICATION
The research model was applied in the form of a case study in a Brazilian company in the construction industry.The company is active in all stages of real estate development, from land acquisition, project design, sales, planning, and construction, including transfers and after-sales.
During the process of data collection, all five specialists from the company were invited to answer the survey questionnaire (Appendix A), rating every statement on a scale from 1 to 5 (Table 2).
After that, the decision matrix was constructed using the mode found in the questions related to each of the five assessment criteria for the organization that was the focus of the study.For example, in order to determine the performance of the first dimension, Management Perspective (C1), the mode was applied for the answers to questions 1 to 5. The decision matrix developed for this empirical application can be seen in Table 3.For the application of ELECTRE TRI we defined some parameters along with the decision maker.
To define the weight parameter, SRF 2.2 software (Figueira & Roy, 2002) was used to determine the weight of each attribute that would be used to apply the ELECTRE TRI method.The SRF facilitated the process of defining the degree of importance among the criteria for the decision maker since this decision is not always clear.SFR is based on the use of cards and can be applied in three phases.
1.In the first phase, the decision maker is provided with two sets of cards.The first set contains a card for each criterion in the assessed set, and the second has blank cards of the same size.The number of blank cards will depend on the decision maker's need.
2. In sequence, the decision maker is required to order the set of cards with criteria from least important to most important.If any criterion is of equal importance to another, the card must be placed over the criterion (or criteria) of equal importance.
3. In the third phase, the decision maker is asked to think about the degree of importance between two successive criteria.The determination of weights considers the change in importance between two successive criteria.Then, the decision maker is asked to insert white cards between two successive cards (or subset of successive cards, in case of criteria with equal importance).The greater the difference between the criteria (or the subsets of criteria), the greater the number of white cards.
After these three phases, the DSS provides the normalized weights for each of the criteria.For this application, the decision maker ordered the criteria thus: C3 > C4 > C5 > C1 > C2.The weights given were: 5.8 for C2; 15.4 for C4; 23.1 for C5; 26.9 for C4; and 28.8 for C3.More details about the SFR method and the method's algorithm can be found in Figueira & Roy (2002).Figure 3 illustrates the application of SFR 2.2.
Also, for the application of ELECTRE TRI in the DSS MCDA-ULAVAL according to the ELEC-TRE TRI algorithm, the following parameters were verified with the decision maker: cutting level (λ ) equal to 0.7; veto threshold and both preference (p) and indifference (i) thresholds equals to zero.This means an abrupt transition between the preference range, not considering the weak preference zone or uncertainty (Miranda;De Almeida, 2003).Moreover, the definition of thresholds p, i, and v equal to zero is equivalent to using a true criterion, that is, there is a sudden change between the preference zones for the maturity levels (Rogers & Bruen, 1998).Other parameters defined for the application of the model were the classes and evaluation profiles.
In RM3, the maturity level is defined by the lowest value among the attributes.The attribute with the lowest value is considered the weak point; therefore, improvement actions must prioritize that attribute.For the definition of the level, an evaluation interval [0, 1] is made available for the organization's evaluation.This interval was used in our modeling to compose the assessment profiles for each class (maturity level), as shown in Table 4. Class Profile 1 Initial 0,0 -0,25 2 Repeatable 0,25 -0,50 3 Managed 0,50 -0,75 4 Optimized 0,75 -1,00 Having defined all the necessary parameters to apply the ELECTRE TRI, we ran the DSS MCDA-ULAVAL from the perspective of evaluating the maturity of RM in the organization that was the focus of the case study, which allowed us to compare the performance in the five criteria with the maturity levels represented by the four classes scaled from 0 to 1 divided between profiles.Figure 4 illustrates the application of the model in MCDA-ULAVAL.
The execution of the DSS allowed the realization of two different sortings: one pessimistic and the other optimistic.Table 5 shows the two categorizations for the cutting level λ ∈ [0.7, 1], as defined in the research methodology for the sensitivity analysis.
According to the procedure described in Section 3 of this article, when compared to a cutting level of 0.7 to 0.75, the assessed organization is rated at risk management maturity level 4 for both pessimistic and optimistic assessments.However, above 0.75, the pessimistic assessment gives a rating result for level 3 (Managed).The pessimistic procedure is considered in the analysis because it is more rigorous and, therefore, allows a prescriptive analysis of the evaluation toward improvements for the process.Cutting Level (λ ) Pessimistic Level Optimistic Level 0,7 Optimized (4) Optimized (4) 0,75 Optimized (4) Optimized (4) 0,8 Managed (3) Optimized (4) 0,85 Managed (3) Optimized (4) 0,9 Managed (3) Optimized (4) 0,95 Managed (3) Optimized (4) 1 Managed (3) Optimized (4) According to Zou, Chen & Chan (2010), the RM3 maturity level 3 is named Managed and represents a scenario in which risk management systems and processes are formalized, implemented, and documented.At this level, the benefits of risk management are understood by all hierarchical levels of the organization.Senior management provides strong support, while employees are empowered to implement risk management processes to take risks.Level 3 maturity is considered sufficient for most organizations where risk management has become an integral part of their daily practices.
By making the result available to the decision maker, it was confirmed that the company's RM possesses mostly characteristics compatible with the Managed maturity level.It was reported that the organization undertook practices such as the formalization and documentation of risk management, good RM practices are encouraged, and the understanding of its benefits permeate the entire organization.
However, although managers understand risk management as a competitive differential that must be continuously improved, which is a practice associated with level 4 organizations, RM is still not perceived as an integral part of the company's culture.This disparity can be identified in criterion 2 (Organizational risk culture), had a mode equal to five (maximum value) while the question that assesses the responsibility for RM by all team members (question 8) had a mode equal to 1 (minimum value).Thus, classifying the company's risk management as managed (level 3) is a result compatible with the real state of maturity, which demonstrates the quality in the use of the ELECTRE TRI multicriteria method as an ally to assess maturity in RM.

CONCLUSION
The present paper proposed a Multicriteria Decision Aiding model for assessing risk management maturity.The ELECTRE TRI Method was used as a tool to apply a Maturity Model (MM) for risk management.
As the main contribution of this study, the development of a Multicriteria Decision Model to assess maturity in RM showed us that the MCDA can be used as an important alternative for the application of MMs.Through a case study carried out with the application in a real organization, the proposed model was able to process data to assess the risk management practices proposed by the RM3 model and determine a maturity level in RM aligned with the decision maker's perspective.
Furthermore, this study acts on the gap evidenced by Röglinger et. al. (2012), Becker et. al. (2009), & Santos-Neto and Costa (2019) when using the ELECTRE TRI algorithm to standardize procedures for the operationalization of the MM application.
5. Resources are dedicated to projects in accordance with the severity of risk events identified.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.
The practice has been tested and experience can be gained.
The practice is widely used in the organization.
risk culture 6.There is a build-up of trust within the organization and project teams in relation to risk management.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.
There is the decision and action plan to start using the practice.
The practice has been tested and experience can be gained.
The practice is widely used in the organization.
7. Frequently, team members take risk ownership during project implementation.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
8. Responsibilities for managing risks are distributed and carried out by all team members.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.9. Risk events are openly communicated within the organization.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
10. Risk management is widely accepted and practiced in all levels within the organization.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
Dimension: Identifying risks 11.Potential risks are identified each time for new projects.
The practice is not in the organization.
There is a strong discussion about using the practice, but no decision.
There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
12. A systematic identification method (i.e.FMEA, Preliminary Risk Analysis-PRA, Brainstorming, SWOT) is used to ensure major risks are identified.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
13. Information on risks identified is processed, grouped, and communicated to all project participants.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
14. Risks identified are consistently revised and reevaluated throughout the project process.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
15. Actual risks found are compared against initially identified risks.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
Dimension: Analyzing risk 16.All project participants are capable of basic risk analysis skills such as qualitative or quantitative analysis.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
17.The likelihood of occurrence and magnitude of impacts of a risk is thoroughly assessed upon identification.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
18. Qualitative and/or quantitative risk analysis tools and applications are used to assess identified risks.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.
19.After analyzing the analytical results of risks identified, it is used to aid in decision-making for risk responses.
The practice is not widely used in the organization.
There is a strong discussion about using the practice, but no decision.There is the decision and action plan to start using the practice.The practice has been tested and experience can be gained.
The practice is widely used in the organization.

Figure 1 -
Figure 1 -Framework of the multicriteria decision model for RM evaluation.
a) Management perspective (people and leadership) concerning risk: seeks to assess how much the upper management actively takes part in risk activities, supports and encourages risk management; b) Organizational risk culture: seeks to measure to what extent team members are taking risk ownerships during project implementation; c) Identifying risks: assesses whether the organization has a risk identification procedure; d) Analyzing risks: assesses whether the organization has qualitative and/or quantitative risk analysis tools or procedures;

Figure 3 -
Figure 3 -Weights defined with the help of SRF 2.2.

Table 1 -
Risk Management Maturity Models.

Table 3 -
Mode and normalization of the answers.

Table 5 -
Results and sensitivity analysis.