SELECTING WORKING TEAMS FOR INFORMATION TECHNOLOGY OUTSOURCING PROJECTS THROUGH A COMBINATION OF METHODOLOGIES

ABSTRACT

This paper deals with a problem that Information Technology outsourcing suppliers generally face when selecting a working team technically capable for specific roles in software development projects. A combination of methodologies, interactively integrated, is proposed. They are Soft System Methodology to structure the problem, Repertory Grid for individual interviews and elicitation of the selection criteria, DRV Processes to assess the candidates and to generate knowledge and consensus on the selection process and Linear Programming to assign people to each position. This multimethodology allowed finding a more comprehensive solution than that initially requested by the company, since it helped to establish the necessary transformations for the selection model to operate in the right way, set the competencies to be considered as selection criteria, develop a consensus estimate of the weighted criteria, and award global values to candidates, optimizing the assignment of roles in the group for the project.

Keywords:
outsourcing; information technology; SSM; Repertory grid; DRV Processes; Linear Programming; TODIM

1 INTRODUCTION

Information Technology (IT) outsourcing services is a widespread practice in the field of information technology management (^{Ravindran, Susarla, Mani & Gurbaxani, 2015}); (^{Han & Mithas, 2013}); (^{Willcocks, 2010}). However, this strategy involves different risks for both the client and the supplier of such services, among which is the possibility of an incorrect selection of the professionals or assignment of roles (dos ^{Santos & da Silva, 2015}). Therefore, this paper proposes and applies to a real case a combination of methodologies for the selection of working teams, aimed at increasing the chances of success, from the point of view of the provider of the projects that apply this type of practices.

When working in an outsourced basis, the client organization interested in developing an informatics system, hires for a limited time, the services of a group of professionals who depend on a supplier company. In general, the technical staff then works in direct contact with the client.

Such modality offers great advantages for both parties, such as greater flexibility, capacity to have technical skills, innovation and reduction of overall costs (^{Lacity et al., 2010}); (^{Svejvig & Pries-Heje, 2011}). Nevertheless, it also poses risks to the organizations involved (^{de Sa´ Soares, Soares & Arnaud, 2014}) including the possibility of resignation by some members of the project team.

From the point of view of the supplier, for each requirement it is necessary to select a group of people among its employees, who could meet the conditions to carry out the project.

However, this continuity is always threatened due to the high job rotation of contracted professionals. It should be noted that once the project started, the selected group remains in the client company for several months. Under these conditions, the initial relationship is often undermined and it is common for professionals to be tempted to migrate to the contracting organization, or to other companies. Therefore, suppliers must worry about developing a certain sense of belonging and identification among their professionals.

In other words, this implies that the supplier must pay particular attention to the critical aspects of outsourcing, specially being the interface of the relationships with the client (^{Duhamel et al., 2012}). Following this logic, it is recommended that the management of working teams is done with a systemic approach, both at the time of selecting the technical equipment, and during the post-project monitoring as well. Such an approach must consider the needs and priorities of the professionals involved.

In those terms, the problem faced by IT outsourcing providers is the need to select a team technically capable for specific roles determined by the client and also for the individual and social needs of the team.

In this context, it can be considered a complex problem since the organizations and individuals involved operate in network environments densely interconnected and they are affected by the uncertainty generated by the concurrence of changing perceptions (^{Georgiou, 2008}). Moreover, it is also necessary to observe the positions of different people or entities and to ensure that their decisions are sustained over time (^{Vidal, 2006}). Furthermore, the organizational environment also offers complexity.

Given this evidence, this paper proposes a combination of methodologies - Soft, Hard Operational Research and Statistics - to address the problem, since this type of approach has been recommended for situations of high complexity (^{Franco & Lord, 2011}). Thus, this article does at least two contributions to knowledge: an original methodology to solve a known problem in the management of IT outsourcing and experience with a multimethodology that was not used before.

Regarding this document structure, after the introduction, bibliographic references are presented for the different approaches used to face the problem of selection of working teams. Afterwards, the multimethodological approach adopted and the results of the corresponding implementation are explained. The article continues with the necessary discussion of these results. Finally, conclusions and suggestions for future research are included.

2 BIBLIOGRAPHIC REFERENCES

2.1 Approaches to the problem of selecting working teams

The problem of selecting working teams requires the consideration of the potential members’ competencies. In accordance with ^{Loufrani-Fedida & Missonier (2015}), this document considers competence as “the ability of an individual, a team, or a company, to mobilize and combine resources (i.e., knowledge, skills, and attitudes), in order to implement an activity in situation. Moreover, we understand competence management as the set of managerial actions taken by one or more organizations to identify, construct, and develop competencies”.

Usually, there is no agreement about which skills have to be assessed in this type of analysis because the authors who define the competences required for the performance of certain roles do not agree on their appraisal. For example, there are significant discrepancies characterizing leadership roles (^{Mu¨ ller & Turner, 2010}; ^{Yang, Huang & Wu, 2011}; ^{Brie`re et al., 2015}).

However, since the professional groups that must be selected in this work should be directed to the development of IT Projects, it is necessary to incorporate a Project Management (PM) perspective to the study of the problem. In that area of knowledge, the interest in project management competencies (PMCs) is increasing (^{Bredillet et al., 2015}). Its evolution, both in theory and in practice, has placed the project manager and his/her competencies at the center of a project’s and an organization’s success.

As an example of this interest, Mayumi ^{Takey & de Carvalho (2015}) propose a seven-step method for the PMC map based on the “description of competences and performance criteria, an assessment process, a diagnosis of the current proficiency level, the identification of competence levels that differentiate professional categories, the establishment of expected profiles, a gap analysis and the association between experience and competency development”.

Regarding the way of eliciting skills, ^{Medina & Francis (2015}) present a similar approach to the strategy of this paper. In effect, they use the repertory grid technique to identify the characteristics of the project team members’ associate with a good project manager. They adopt this technique because of its strength in eliciting personal constructs.

However, the competences to be considered are not only technical, but also they have to consider the skills to work in situations of high complexity. ^{Ramazani & Jergeas (2015}) in a qualitative study of project managers, emphasize the importance of “1) developing critical thinking for dealing with complexity, 2) developing softer parameters of managing projects, especially interpersonal skills and leadership as opposed to just technical skills, and 3) preparing project managers to be engaged within the context of real life projects”.

Regarding the evaluation of competences, there are various studies about the applicability of Multicriteria Decision Making (MCDM) methodologies in problems about selection and evaluation of personnel. Among them, one of the methodologies used is the multi-criteria method Analytic Hierarchy Process (AHP). Moreover, there are various approaches using AHP and fuzzy sets to represent uncertainty and inaccuracy of evaluations (Lin et al., 2009).

Different methodological possibilities have been considered. ^{Shahhosseini & Sebt (2011}) propose an interesting combination of neural networks with AHP, where the criteria are measured using linguistic variables. Meanwhile, in ^{Kelemenis et al. (2011}), skills are represented by the adoption of fuzzy sets and subsequent aggregation is done with TOPSIS method.

In general, previous contributions assume that the decision maker is a single person and is not a group that must agree on key issues for the selection to be effective. An exception is the work of ^{Alencar & Almeida (2010}) who use Promethee to analyze individual preferences that are then conveniently aggregated. De ^{Morais, Costa & Almeida (2014}), propose a similar approach to facilitate different activities of decision-making in small groups of organizations with IT outsourcing modality.

However, in these approaches it is not necessary the consensus of the whole group of decision makers. In this sense, it should be clarified that the term consensus refers to an agreement reached by all members of the group. The problem is that the lack of basic arrangement can reduce the level of commitment to the project and, consequently, their chances of success.

In short, it is desirable that the selection of team members working for IT Project consider that they will operate in complex environments. In addition, since it isn’t possible to define the set of skills to be considered, they must be identified for each particular application. Finally, since there are several actors involved, it is appropriate to consider the different views and ensure a certain level of consensus.

2.2 Multimethodologies to support group decision-making

Multicriteria decision methods (MCDM) have been strongly developed in recent decades (^{Velasquez & Hester, 2013}). A significant fraction of the contributions considers that the decider is a unique and individual entity (^{de Almeida et al., 2014}; ^{Gonc¸alo & Alencar, 2014}; ^{Nepomuceno & Costa, 2015}; ^{Santos et al., 2015}; ^{Gomes et al., 2015}).

Within the scope of group decision making, an important area is Soft OR (^{Yolles, 2012}), where methodologies like Soft System Methodology and Strategic Choice Approach can be grouped (^{Georgiou, 2012}; ^{Gomes et al., 2010}). A recognized advantage of this type of approach is the ability to structure problems solidly, considering the views of different stakeholders. At the same time, the exchange of knowledge and subsequent commitment with the adopted decisions is favored.

The evolution of this approach led to the development of strategies oriented to the analysis of complexity through combinations of methodologies (^{Franco & Lord, 2011}; ^{Small & Wainwright, 2014}; ^{Henao & Franco, 2016}). In relation to the application of the multiple methodologies (MM) of Operational Research to real situations, there were not found any records of multimethodological approaches applied to personnel selection processes.

However, reality shows that behavioral aspects are determining factors in decision problems. Its incorporation is analyzed in the stream Behavioral Operational Research (^{Ha¨ma¨la¨inen, Luoma & Saarinen, 2013}) and (^{Franco & Ha¨ma¨la¨inen, 2016}).

On the other hand, the application of group Multicriteria Decision Making (MCDM) contributes to the construction of shared knowledge and favors participation and commitment of the participants in taking decisions (^{Ferreti, 2016}). This includes approaches using Multiattribute Utility Theory Additive (MAUT) (^{Keeney & Raiffa, 1993}).

In that line, it is possible to recognize different strategies; some do not attempt to reduce disturbances that affect the decision process. Indeed, a first set of authors worries about finding reasonable solutions by proposing different types of aggregation, for example through geometric or arithmetic means (^{Forman & Peniwati, 1998}), (^{Dong & Saaty, 2014}) and (^{Wu & Xu, 2012}). Another set of authors is aimed at modeling the noise, but without an attempt of its reduction. An approach inscribed in this line is the family of methods called Stochastic Multicriteria Acceptability Analysis (SMAA) (^{Tervonen, 2014}) and (^{Kontu et al., 2015}).

In this regard, ^{Montibeller & Winterfeld (2015}) highlight that, although attention has been paid to obtain judgments (probabilities, values, weights, etc.) for decision-making and risk analysis, is scarce the attention given to possible distortions in the analysis. However, they emphasize that one should try to reduce these disturbances to increase the chances of success of the agreed proposed action.

This fact is recognized by a third group of researchers whose contributions are interested in reducing noise with different methodological approaches (^{Dias & Climaco, 2005}), (^{Altuzarra et al., 2007}), (^{Altuzarra et al., 2010}) and (^{Fu & Yang, 2012}). In particular, the methodology used in this document: Processes DRV (^{Zanazzi et al., 2014}) and (^{Zanazzi, 2016}); is part of the latter aspect. This method aims to facilitate the identification of solutions, while reducing the effects of group pressure and minimizes disturbances.

This method has advantages over the purposes of this article that make it preferable to other approaches. On one hand, it significantly reduces disturbances, even when assessing the alternatives; on the other hand, it offers multiple opportunities for the exchange of experiences and knowledge among group members, becoming an effective way of training, while it encourages the commitment for further action.

2.2.1 Presentation of the DRV Processes Method

The DRV method is presented in ^{Zanazzi & Gomes (2009}), ^{Zanazzi et al. (2014}) and ^{Zanazzi (2016}). It is the most recently and less widespread method, among the tools used in this multimethodology approach, and for that reason it is considered appropriate to describe its general characteristics.

This methodology has been developed to facilitate the work of groups that share its objectives; therefore, scenarios of non-negotiation or conflict are not contemplated. Moreover, the method does not allow you to assign greater or less weight to certain members of the group.

The dynamic included in the methodology seek to reduce the disturbances that affect the group’s decision processes. Among these problems are both, group pressure and the influence of certain leaders. For this reason, the methodology includes in its stabilization phase, moments of group and individual work. In particular, the determination of weights and preferences is performed individually. This is so because, it is considered important to capture all the individual contributions.

In a working team of N members, the decision problem is structured and it is represented in a tree diagram that can be divided into sub-problems. One of them is the comparison of criteria with each other and the assignment of weights to them; another is the comparison of alternatives in light of each criterion, and the assignment of utilities to each alternative, as shown in Figure 1.

Figure 1
Decision-making tree diagram and associated sub-problems.

The application of DRV processes is done in three phases: stabilization of the sub-problems, aggregation and ordering.

The first phase is where the reduction of both, imprecision and uncertainty of the process, takes place. At the beginning of the sub-problem analysis, it is possible that the knowledge, preferences and hence the group’s priorities are completely different. In order to reach an agreement, an iterative cycle of analysis that allows the exchange of knowledge and experience, takes place. Thus, it contributes to reduce the different views among group members. The stable condition is achieved when the positions of the group may not vary too much, even when the analysis is extended.

In order to verify if this requirement has been met, the group members are asked to assign subjective utilities to the compared elements (^{Keeney & Raiffa, 1993}). Those assigned utilities can be regarded as observations of a multidimensional random variable, with a marginal distribution for each element analyzed.

Following this logic, a situation without consensus can be reflected by Uniform Marginal Distributions. Otherwise, if the members align their positions, assigned utilities are similar and the probability distribution is Normal.

For this reason, the stability condition is verified by normality analysis of each sub-problem valuation and through the IVR (in Spanish: “´Indice de Variabilidad Remanente”) indicator. This compares the sum of squares of the observations, with which would be obtained if the true distribution were uniform.

In practice, IVR values below 25% indicate a condition of stable consensus. If stability is not achieved, the group should repeat the sub-problem analysis in a plenary in order to identify, and if possible, eliminate points of dissent.

When all sub-problems have been stabilized, the second phase called aggregation starts. Then, it is possible to determine global values for each alternative. DRV processes method offers the possibility of aggregation by two modalities: linear weighting and TODIM method formulation.

In the first variant, if W _j represents the weights of the criteria, and U _ij the utilities assigned to candidates under the criterion j , the partial contribution to the priority assigned to a generic alternative i, when considering the criterion j is obtained as the product of the two random variables mentioned, under the expression (1)

(1)

Where the distributions of _^Zij variables can be formulated with the expression (2).

(2)

The global values of a generic alternative (_^Vi ) can be obtained as indicated in the expression (3).

(3)

^{Zanazzi & Gomes (2009}) state that when the distributions of _^Wj and _^Uij variables are Gaussian, then the marginal distributions of the global values of generic alternatives (_^Vi ) can also be considered Normal.

In the second variant, it is possible to aggregate expressions with TODIM method (^{Gomes et al., 2015}; ^{Machado et al., 2015}; ^{Paredes-Frigolett & Gomes, 2016}). Therefore, it is adopted one of the criteria as referential (r criterion) and partial dominance matrices and final dominance matrix are calculated. The global dominance of alternative i, is obtained as the sum of the partial dominances.

This latter option allows improving the representation of the group of decision makers’ preferences, because it provides a method of practical application of the concepts of Prospect Theory. Indeed, this is a way to reflect the risk attitude that participants show, since the impact perceived by decision makers, when losses occur, is usually greater than that generated by profits ones (^{Gomes & Zanazzi, 2012}).

In the third and final phase: ordering, the samples of global values obtained are analyzed in order to define order relations among the alternatives, so hypotheses tests comparing means of dependent variables are applied. Furthermore, this method uses an algorithm (^{Benjamini & Yekutieli, 2001}) to control the probability of type I error.

3 METHODOLOGY

This paper proposes the following combination of methodologies for the problem of selecting working teams, aimed at increasing the chances of success, both from the point of view of the supplier and of the projects that apply this modality.

• Soft System Methodology (SSM): to structure the selection problem.

• Repertory Grid: as support of individual interviews and to elicit the selection criteria.

• DRV Processes: to assess the candidates and generate knowledge and consensus in the selection process.

• Linear programming: to define the person to be assigned to each position.

In Table 1, the methodologies selected are shown as well as the way they complement and provide feedback to each other. The conceptual framework adopted is ^{Mingers & Brocklesby (1997}) which discriminates the dimensions to be considered (social, personal and material) as well as the type of phases required. The interaction of these methods in the dynamics of implementation is highlighted.

Table 1
Configuration Intervention.

	Appreciation of	Analysis of	Assessment of	Action to
Social	Social practices, power relations SSM	Distortions, conflicts, interests	Ways of altering existing structures	Generate empowerment and enlightenment DRV Process
Personal	Individual beliefs, meanings, emotions SSM, Repertory grid	Differing perceptions and personal rationality SSM	Alternative conceptualizations and constructions SSM	Generate accommodation and consensus DRV Process, SSM
Material	Physical circumstances SSM	Underlying causal structure	Alternative physical and structural arrangements	Select and implement best alternatives Linear programming DRV Process

Along all the phases and dimensions, the supplier company and a team of facilitators worked in a collaboratively way. The team was made up with external professionals, who were in charge of the design, interviews management, workshops and the creation of the proposal. The modality adopted for each of the methodologies is described in the following sections.

3.1 Modality adopted for SSM

In order to structure the problem of selection, ^{Checkland’s principles (2000}) were adopted. Some modifications were introduced to achieve management effectiveness, according to ^{Georgiou (2008}, ²⁰¹²), whose version helped to define the context of the decision-making process, namely: problematic situations to be considered, stakeholders, interactions, characterization of the organizational environment and definition of the type of transformations that it was necessary to implement.

The focus was placed on the exploration and expression of the problematic situation. Hence, oral semi structured interviews were conducted with the two Partner-Managers of the company, the Development Manager and twelve people with different roles in the working teams (project leaders and employees with experience in the organization), in order to extract clear information on the context in which the problem was immersed.

Questions and additional activities were combined, such as the construction of rich pictures to graphically represent their perceptions in decision-making process. Three types of analyzes were performed: identification and characterization of roles, socio-cultural dynamics and power relations in the context of the problem, which allowed the identification of those involved, the relations among them, the prevailing values and the systems restrictions.

The rigorous definition of the problematic situation was essential for its resolution, and it was achieved identifying the transformations, where the observed situations (conflicts) and the ideal situation were made explicit; being the former the input of the transformation that returns as output, the expected change.

In order to express these transformations and the context in which they occurred, the CATWOE elements were design in a plenary, jointly with stakeholders and the team of facilitators. The seven elements that constitute its mnemonic name were developed (C: client, those benefited or damaged by the transformation; A: actors, who implement the transformation; T: transformation process, changes a defined input or current situation into a defined output or ideal situation; W: the reasons that justify the transformation; O: owners, who could disrupt or limit the transformation; E: environment restrictions that may interrupt, limit or complicate the transformation).

Each T is transcribed in a phrase that became a declaration, leading the systematic planning of this transformation and the global planning of all as a whole. These transformations should be both: systematically convenient (on the basis of logic models) and culturally feasible for those who are immersed in the problematic situation (^{Mingers & Rosenhead, 2004}).

A root definition was developed for each CATWOE, and a realistic planning of actions that would facilitate its implementation was programmed for each of them.

As a summary, the contributions of SSM helped identify the necessary transformations for the selection model to operate correctly. In this setting, the next step was to establish the competencies to be adopted as selection criteria.

3.2 Modality adopted for the Repertory Grid

In order to elicit the competencies required for the members of the working team, individual interviews were conducted to all the participants involved in the SSM modality (managers, project leaders and employees with experience in the supplier company).

These interviews were structured through the Kelly Repertory Grid; tool that is based on the Personal Construct Theory (^{Kelly, 1955}; ^{Alexander et al., 2010}). According to this theory, people build their individual world from their interpretation of the outside world. Thus, it facilitates the interpretation of how a person understands their experiences, so that it constitutes an appropriate interface for conducting interviews.

A version of the Repertory Grid, called Grid of Rankings was applied, in which the roles to be assessed were disposed; each interviewee indicated bipolar constructs, which according to their perception, operated as evaluation criteria for these roles. Thus, each interviewee proposed a set of criteria to evaluate an employee for a given role.

The Repertory Grid has an important quality, called the principle of commonality. According to this property, if the members of a group that share objectives on the same subject, individually identify their personal constructs, these constructions are similar. That is, group members tend to repeat the constructs proposed by their peers.

In a workshop, the fifteen interviewees and the team of facilitators analyzed the recurrences of Repertory Grids arising from the individual interviews. This analysis generated a recurrence rate of the constructs. On this basis, the participants analyzed the possibility of adopting the dominant constructs as selection criteria. Then, the DRV processes methodology was applied in order to obtain the estimate consensus of weighted competences (criteria) and to determine the global values assigned to the candidates.

3.3 Modality adopted for DRV Processes

In order to value the candidates and produce knowledge and consensus in the selection process, a workshop was organized for N participants with leading roles and experience in the company. Hereinafter, the sub index n (with 1 ≤ n ≤ N) identifies the participants.

The activity was divided into two stages. In the first, criteria weights used in the selection of the four roles to be considered were elicited: functional analyst; testing analyst; developer and referent.

A sub index k, (where 1 ≤ k ≤ K) was assigned to identify each role.

The activity was designed as an instance of construction of group knowledge. For that, the recommendations of teaching specialists were considered in the design. In this respect, ^{Wenger-Trayner & Wenger-Trayner (2015}) emphasize that collaborative learning requires that the group adopt a common language code and develop a shared discourse. That is, linguistic activity is fundamental (^{Wenger et al., 2014}). On the other hand, there are activities that favor the joint construction of knowledge; those include the development of joint definitions, the sharing of knowledge and experience, as well as problem solving as a whole (^{Hmelo-Silver & Barrows, 2008}).

With this idea and for each role, the group began with the analysis of the results obtained with the Repertory Grid, which were elaborated in the individual interviews. First, it was agreed that it was possible that the constructs identified in the interviews were adopted as criteria for the selection to be implemented. Emerging constructs were analyzed and definitions were developed for possible criteria. The revision and improvement of these definitions allowed specifying the criteria to be considered for each role and to agree on the way of valuing them. Group applications were also made, with alternatives of fictitious decision, to favor the construction of sense around each criterion.

When a state of apparent agreement was reached, it continued the individual task. For every role, each group member had to perform the following tasks, with complete independence among partcipants:

• Sort the criteria to be considered, from highest to lowest priority.

• Compare in pairs the adjacent elements in the preorder mentioned, answering the question: “how many times is preferable a criterion, compared to the adjacent one?”. The priority intensity of criterion m, on criterion (m − 1), was expressed by an amount _^hm , defined in the field of real numbers; such that one represents indifference and for example, three means that the first element of the relationship has three times more priority than the other.

• Obtain global utilities for each criterion j , by doing:

(4)

• Standardize the weights performed by each participant, on (0.1) scale, dividing by the sum of the allocations.

(5)

The obtained values W ^* _k,j,n are preliminary values. To be considered definitive, they must comply stability and consensus requirements. According to ^{Zanazzi (2016}), these conditions require that:

• The values W ^* _k,j,n could be assumed to be extracted from a population with Normal Distribution.

• The IVR indicator takes a value close to zero.

When these conditions were verified, the preliminary values were assumed as definitive weights and the process continued with the analysis of other criteria; otherwise, an analysis to improve the criteria definition, had to be taken up in plenary. In this way, the set of weights can be organized in the following matrix:

(6)

Once criteria weights for all roles were determined, it begins the second stage, intended to compare candidates. This part of the activity is summarized in the following iterative process.

• a. The analysis is positioned at the j th criterion.

• b. The I candidates are analyzed, in light of the j criteria considered, until the process stabilizes. This includes comparing, in plenary, the characteristics of the candidates, then assign utilities individually and finally, check if the stability and consensus condition are met. Assigned utilities are represented as _{^{ui, j,n}} , con 1 ≤ i ≤ I.

• c. Return to stage a., in order to analyze another criterion. When all criteria were analyzed, the obtained utilities for each criterion j , were ordered in a matrix:

(7)

Then, it is time to aggregate. Lineal weighting is used to obtain the global values _^vk,i,n , which are determined to each candidate, for each role k, through the expression (4).

(8)

Where sub index k indicates the role considered, sub index j identifies the criterion, sub index i corresponds to the candidate or alternative being analyzed and n indicates the participant who makes the assignations.

• d. Representative global values of each candidate for each role are determined by the expression (5).

(9)

Finally, the optimal assignation of available people to the different roles in the project group is executed.

3.4 Modality adopted to assign people to required roles

When assigning a person to a certain role in the working team, it was considered appropriate to do the selection taking into account that the global evaluation of the group was as high as possible. Since, the employees of the supplier company had a common training; it was assumed that they had the flexibility to adapt to different possible roles.

A classic process of assigning people to jobs, which was resolved through Linear Programming (^{Alberto & Carignano, 2013}) was proposed. The structure of this program is indicated in the system of expressions (6).

(10)

Where the variables _^yk,i have a value of 1, when the candidate i is assigned the role k. When the number of people available in the supplier company is greater than the amount of roles to cover, additional variables (roles) are defined to balance the problem. Obviously, these variables _^yk,i are incorporated with zero utility, and (K + 1) ≤ k ≤ I is fulfilled.

3.5 Summary of the multimethodology used

In order to facilitate understanding of the proposal, Table 2 indicates the methodologies used, the input required for each one, its main products and sub-products to be obtained.

Table 2
Input and products of the methodologies used.

Methodology	Input	Main Product	By-products
SSM	Systems Survey. Identification of problems that may affect the selection process.	Selection model requirements. Necessary transformations for the model to operate accurately.	Organization members’ perceptions and priorities. Build consensus on the action plain.
Kelly’s Repertory Grid	Selection model requirements.	Competences adopted as selection criteria to recruit the working team.	Participants commitment and attitude.
DRV Processes	Competences adopted as selection criteria.	Consensus estimate of the weights of the criteria. Global values assigned to candidates.	Spaces for exchanging knowledge and experiences. Uncertainty reduction in the decision-making process.
Integer Linear Programming	Global values assigned to candidates.	Optimal allocation of people available to the different roles for a project working team.	Objectivity.

4 RESULTS

The case study analyzed in this paper was a company oriented to both, the production of software and outsourcing of services. Initially, the company requested the development and implementation of a methodology, which under a client request of outsourcing service, allowed facilitating the selection of the suitable working team to be assigned to the project, that is to say, the people involved and the roles each should play. At the time of the study, the company had one hundred and twenty employees and fifteen years of experience in the market, so it was a consolidated structure.

As indicated in Table 3, the application of the amended version of SSM according to ^{Georgiou (2008}, ²⁰¹²) allowed detecting five CATWOE elements with their corresponding transformations, related to the recruitment process.

Table 3
CATWOE for the selection process.

	Client	Actors	Transformation		Reasons that justify transformation	Owner	Restrictions
			Initial	Ideal
1	Company-Client	Development Manager	Staff does not share time and physical space	Dynamics of work including moments and space to share	To develop shared spaces and times to increase group cohesion and loyalty	Client Company	Urgency of Projects
	Employees						Geographic distribution
	Outsourcing organization	Partner-Managers				Partner-Managers	Employees schedule
2	Company-Client	Development Manager	Human Capital (HC)	Development of an adequate system of Human Capital (HC)	To facilitate the selection and assignment of HC To explicit nd observe the required competencies	Partner-Managers	Quality System Requirements
	Employees
	Outsourcing organization	Facilitators	Management is not systematic	Management
3	Company-Client	Development Manager	There is no methodology for the process of selection and assignment of HC for a required role in a project	Availability of a methodology that makes it possible to select and then assign appropriate team members for a required role in a project; so that this process is sustained by empirical evidence and improve the reliability of HC	To facilitate and optimize the selection and assignment of HC.	Partner-Managers	Urgency of Projects.
							Employees schedule.
							Quality system.
	Employees	Referents			To explicit, observe and measure the competences required.	Development	Specific characteristics of the generation Y.
							Employees’ permanence.
	Outsourcing organization	Facilitators			To compare employees based on empirical evidence.	Manager	High demand from employees.
							Availability of information.
4	Company-Client	Development	The Referent of the company is not defined.	Development and empowerment of the Referent role.	To collect and process information related to the needs of customers and employees.	Partner-Managers	Specific characteristics of the generation Y.
	Employees	Manager Referents					Employees’ permanence.
	Outsourcing organization	Facilitators			To promete cohesion and loyalty.		Urgency of Projects.
5	Development Manager	Facilitators	No studies have been conducted to determine the impact of economic and non-economic variables in the permanence of an employee.	Availability of a model to assess the impact of economic and non-economic variables in the permanence of an employee.	To predict the behavior of employees in their permanence in the company.	Partner-Managers	Quality system.
							Availability of information
							Urgency Projects
							High demand for employees
							Geographic distribution
	Outsourcing organization				To enhance training activities.		Employee schedule.

The initial problematic situations to be improved were: there was no time sharing and there were not common physical spaces, Human Capital (HC) Management was not systematized, there was no methodology for the recruitment and allocation of suitable employees for a required role in a project, there was not a Referent figure defined and there were not studies to determine the impact of economic and non-economic variables in the permanence of an employee.

Root definitions of each CATWOE element were extracted. For example, as a result of the third row of Table 3, the following root definition emerged: The design of a recruitment and allocation methodology to assign suitable HC for a specific role in a project, based on empirical evidence, will improve the effectiveness and efficiency of the selection process.

For the implementation of the actions resulting from this transformation, it was necessary to define the roles required for the working teams. These were: Functional Analyst, Developer, Testing Analyst, and Referent of the organization. In a plenary session with the group of the fifteen people who were interviewed during the SSM modality, the roles identified were characterized as shown in Table 4.

Table 4
Definition of Roles.

Role	Description
Functional Analyst	Functional Analyst is responsible for data collection, analysis and design of solutions to meet user requirements. May or may not participate in the development itself. He follows-up the project up to the implementation, may design testing instances with users to validate the products. He must know the client’s business since users can not always explain their needs. He is generally responsible for the initial basic training of the user.
Developer	Developer must be able of interpreting the requirements indicated by the Functional Analyst, resolves solely technical issues. He may perform unitary tests when requested by the Functional Analysts, and has nocontact with the user.
Testing Analyst	Testing Analyst is responsible for testing, at the functional level, that user requirements have been correctly interpreted, validates the product, followsup that it has the behavior expected by the user and verifies that it meets standards of development. He may control or facilitate the correct implementation process.
Referent of the company	Referent is the nexus between the business needs and the employee and vice versa, also links the customer’s needs with those of the company. He coordinates daily tasks and activities for the people in charge.

In the fifteen individual interviews, structured through the Grid of Ratings, interviewees suggested between 6 and 12 emerging constructs that in their opinion would allow assessing an employee for a given role.

In a plenary session, the observable similarities in the grids of ratings resulting from the interviews were analyzed, consequently the list of emerging constructs was designed and the recurrence rate was obtained for each, as shown in Table 5.

Table 5
Emerging Constructs and Recurrence Rate.

Emerging Constructs	Recurrence Rate (%)
Communication	100
Experience	100
Leadership	87
Knowledge of the Business	87
Initiative	80
Creativity	67
Commitment	53
Self-confidence	53
Decision-making	40
Working in teams	40
Others	Less than 40

The five constructs with the highest rate were considered, as a set of criteria to assess a person for a given role. Participants agreed that the remaining constructs were included in the five selected. Thus, in agreement they proposed: communication, experience, leadership, business knowledge, and initiative as the competences to guide the assessment. Moreover, the group agreed unanimously that the same criteria could be used for all the roles.

The workshop on DRV Processes was held with the same participants of the previous activities. Initially, they agreed on the need of finding the definitions of the competences adopted as criteria for assessing the suitability of a candidate for a given role. With that purpose in mind, each participant was asked to define individually and in written the criteria to be considered. Then, in plenary each competence was analyzed and re-signified, which allowed the group to adopt the definitions presented in Table 6.

Table 6
Agreements on the definition of criteria (competences to be considered).

Criteria	Definition
Initiative (I)	Proactivity, vocation for innovative and feasible proposals. Willingness to take responsability.
Experience (E)	Concrete experience in exercising the role or related roles.
Knowledge of the Business (N)	Knowledge of the processes of the organization, where he/she would work.
Leadership (L)	Ability to guide, monitors, motivates and transmits the vision of the business to its peers.
Communication (C)	Skills to transmit ideas liston to group members and develop interpersonal relationships.

Once criteria were defined, estimates of weights for different roles were discussed. The estimation process was carried out with the method described in paragraph 3.3, i.e. group members assigned weights to each criterion individually and then the resulting sample was analyzed to infer if stability was achieved. Figure 2 illustrates the results of these activities, and shows the box diagrams obtained in the first cycle of analysis for the position of Developer. By analyzing the figure, it can be detected a strong variability in opinions. The greatest variability is related to the experience (E) and knowledge (N) of the business, while there are positions placed very far from the rest related to initiative (I) and leadership (L).

Figure 2
Box Diagram of Weighted criteria for the role Developer.

The IVR indicator took the value 0.58 or 58%, which is unacceptable. Moreover, normality tests suggest that weights cannot be assumed extracted from Gaussian populations.

It is worthy to explain that an additional step was necessary to achieve stability, as shown in Figure 3. It is interesting to note that in two cycles of analysis, it was possible to reduce the sum of squares of the weights assigned individually, to 24% of the sum of squares taken as reference, that is, the one corresponding to a uniform distribution.

Figure 3
Evolution of the IVR indicator criteria for the role Developer.

Similar results were obtained when assessing the criteria for the remaining roles; in some cases with IVR values lower than 20%. On the other hand, at most three steps were needed to achieve stability. In Table 7 average weights for the five criteria agreed, are summarized for each of the roles.

Table 7
Average weights for the criteria agreed for each role.

Role	Experience	Knowledge of Business	Commmunication	Leadership	Initiative
Functional Analyst	0,29	0,3	0,11	0,2	0,1
Testing Analyst	0,2	0,32	0,27	0,09	0,12
Developer	0,45	0,09	0,15	0,06	0,25
Referent of the organization	0,15	0,23	0,28	0,16	0,18

Based on this information, the recruitment of a working team for a project that required four members (one for each of the roles considered) was performed. They were selected from six employees who were not yet assigned to any project. Utilities were assigned to the six candidates in the light of the criteria adopted. Therefore, the activities formulated in the algorithm in section 3.3 listed from a to e, were performed.

This process was done in similar conditions to those observed during the estimation of the weights of the criteria. Therefore, it was possible to arrive at an acceptable consensus in the five analyses: on two of them, consensus was achieved in a single cycle of study; in other two, two cycles were necessary and three cycles in the remaining. Moreover, all final IVR were located between 15 and 25%.

Once utilities were assigned to the candidates, their global values were calculated (global average utilities for each dependent and for each role: Table 8 these results are presented, where the names have been replaced by the designations X ₁ , X ₂ , ..., X ₆.). In

Table 8
Global Values for candidates to each role.

Candidate	Developer	Functional Analyst	Testing Analyst	Referent of the organization
X 1	0,144	0,17	0,173	0,188
X 2	0,229	0,221	0,171	0,171
X 3	0,157	0,161	0,171	0,154
X 4	0,123	0,144	0,129	0,148
X 5	0,2	0,166	0,202	0,198
X 6	0,148	0,138	0,153	0,14

These values are the input for the linear programming of the expression (6). With these values, the solution to the problem was to assign individual X ₂ to the role Functional Analyst, X ₁ to the role Referent, X ₄ to the role Testing Analyst and finally X ₆ to the role Developer.

The first application of this methodological proposal was managed by the authors of this paper. At the same time, the development manager and one of the professionals were trained, so that they could act as facilitators in the following applications. The authors conducted a face tracking of the two subsequent applications without detecting significant difficulties.

DISCUSSION

This experience started with the requirement, from a supplier company of IT Outsourcing, of a model that would make the selection process of suitable working teams (technically competent and with chances to remain and finish the project). This type of problem is highly complex, because technical, individual and social issues are involved.

Thus, the application of SSM methodology allowed determining that finding an effective solution to the problem, implied going beyond the selection method. In fact, it was detected the need to improve the human capital management, by recognizing the employees needs and perceptions, by setting places and moments to develop and strengthen their relationship with the company and by developing the role of the Referent in the organization.

Regarding the selection process itself, a problem that affected the MCDM was the difficulty to structure the problem and particularly to define the criteria to be used. Therefore, the use of Kelly Grid during personal interviews had an interesting impact, since it was accepted and used without difficulty by the interviewees. Besides, it was successfully verified the principle of commonality, because although some people proposed more than ten competencies in the interviews, only five of them were repeated steadily. It should be noted that these five were finally adopted during the group activity.

The group also agreed to use the same competences to assess the employees in the four roles. Although it is not essential to the proposed methodology, it should be recognized that it is desirable, because then the selection methodology can be structured similarly, with differences only in the weights assigned to the criteria.

Despite the existence of the coincidences mentioned above, the first applications of the DRV Processes proved that there were important differences in priorities and perceptions of participants. In fact, the results showed the lack of consensus and asked for an expansion and improvement of the analysis. Such situations should not be surprising; in plenary meetings often groups express agreements that are not really such. The problem is that these differences introduce imprecision in the valuations and create uncertainty on the outcome of the selection process. However, along these activities, an important reduction of disturbances was achieved. Indeed, the remaining dispersion was estimated in twenty percent of the original. Moreover, it should be noted that the participants recognized that the activities offered an interesting opportunity to exchange knowledge and experiences what it was not frequent in the company.

Regarding the results of the linear programming, they were accepted as natural and reasonable by the members of the company. It was highlighted that those values should be updated with high frequency for the system to work properly.

In conclusion, the combination of methodologies allowed finding a more complete and more appropriate solution than that initially requested by the company. Moreover, the methodologies used seem to complement very well, since each contributed essential information to the remaining ones.

CONCLUSIONS

In this paper, a methodology is proposed to select a group of people who could focus on the development of IT projects. The problem is complex since the decision is affected by high levels of uncertainty and it is necessary to identify and consider both the organization values, and the participant’s personal interests.

In this context, a multimethodological approach that combined different methodological tools from operations research and statistics was adopted. It is interesting the way this combination of methods made it possible to structure a learning process in the organization, where the contributions of the participants interconnect and complement to improve the level of shared knowledge.

The dynamic setting of the SSM methodology helped determine that the problem of selecting the group of participants in a project cannot be considered in isolation, but it is essential to create a favorable organizational context for which it is necessary to implement a set of transformative actions. The Repertory Grid facilitated structuring the decision-making problem, simplified the processing of interviews and made possible a natural derivation of the set of criteria to be used.

In addition, the implementation of DRV processes methodology helped to obtain a consensus weighting of the criteria used and the prioritization of the alternatives being considered. Moreover, the expected key contribution was the possibility of receiving contributions from all

stakeholders, generating a space for joint learning and improvement in the level of commitment to the decision adopted.

Regarding the limitations of the proposal, it is an approach that cannot be applied in emergencies, since it requires a time of considerable analysis and participation of stakeholders.

Conversely, it is possible to transfer this multi-methodological approach without difficulty to different contexts. In fact, it can be adapted to the development of other management systems such as quality or the environment, for example.

ACKNOWLEDGMENTS

The research, on which this paper is based, was financed by Secretaría de Ciencia y Técnica de la Universidad Nacional de Córdoba, Argentina and UBACYT Project 20620130199924BA, Universidad de Buenos Aires, Argentina.

REFERENCES

• ¹
  ALBERTO C & CARIGNANO C. 2013. Apoyo Cuantitativo a las Decisiones. Cuarta Edición. Cooperadora Facultad de Ciencias Económicas, UNC, Argentina.
• ²
  ALENCAR L & ALMEIDA AT. 2010. A Model for Selecting Project Team Members Using Multicriteria Group Decision Making. Pesquisa Operacional, 30: 221-236.
• ³
  ALEXANDER P, VAN LOGGERENBERG J, LOTRIET H & PHAHLAMOHLAKA J. 2010. The Use of the Repertory Grid for Collaboration and Reflection in a Research Context. Group Decision and Negotiation, 19: 479-504.
• ⁴
  ALTUZARRA A, MORENO-JIMÉNEZ JM & SALVADOR M. 2007. A Bayesian priorization procedure for AHP-group decision making. European Journal of Operational Research, 182(1): 367-382.
• ⁵
  ALTUZARRA A, MORENO- JIMÉNEZ JM & SALVADOR M. 2010. Consensus building in AHP-group decision making: a Bayesian approach. Operations Research, 58(6): 1755-1773.
• ⁶
  BREDILLET C, TYWONIAK S & DWIVEDULA R. 2015. What is a good project manager? An Aristotelian perspective. International Journal of Project Management, 33(2): 254-266.
• ⁷
  BRIÈRE S, PROULX D, NAVARO FLORES O & LAPORTE M. 2015. Competencies of project managers in international NGOs: Perceptions of practitioners. International Journal of Project Management, 33: 116-125.
• ⁸
  BENJAMINI Y & YEKUTIELI D. 2001. The Control of the False Discovery Rate in Multiple Testing under Dependency. Annals of Statistics, pp. 1165-1188.
• ⁹
  CHECKLAND P. 2000. Soft Systems Methodology: A Thirty Year Retrospective. Systems Research and Behavioral Science, 17: 11-58.
• ¹⁰
  DE ALMEIDA JAD, ALMEIDA ATD & COSTA AP. 2014. Portfolio selection of information systems projects using PROMETHEE V with C-optimal concept. Pesquisa Operacional, 34(2): 275-299.
• ¹¹
  DE SÁ-SOARES F, SOARES D & ARNAUD J. 2014. Towards a Theory of Information Systems Outsourcing Risk. Procedia Technology, 16: 623-637.
• ¹²
  DIAS L & CLÍMACO J. 2005. Dealing with Imprecise Information in Group Multicriteria Decisions: A Methodology and GDSS Architecture. European Journal of Operational Research, 160: 291-307.
• ¹³
  DONG Q & SAATY TL. 2014. An analytic hierarchy process model of group consensus. Journal of Systems Science and Systems Engineering, 23(3): 362-374.
• ¹⁴
  DOS SANTOS JC & DA SILVA MM. 2015. Mapping Critical Success Factors for IT Outsourcing: The Providers’ Perspective. International Journal of Enterprise Information Systems (IJEIS), 11(1): 62-84.
• ¹⁵
  DUHAMEL F, GUTIÉRREZ-MARTÍNEZ I, PICAZO-VELA S & LUNA-REYES LF. 2012. The Key Role of Interfaces in IT Outsourcing Relationships. International Journal of Information Technologies and Systems Approach (IJITSA), 5(1): 37-56.
• ¹⁶
  FERRETTI V. 2016. From stakeholders analysis to cognitive mapping and Multi-Attribute Value Theory: An integrated approach for policy support. European Journal of Operational Research, 253(2): 524-541.
• ¹⁷
  FORMAN E & PENIWATI K. 1998. Aggregating Individual Judgments and Priorities with the Analytic Hierarchy Process. European Journal of Operational Research, 108: 165-169.
• ¹⁸
  FRANCO L , LORD E. 2011. Understanding Multi-Methodology: Evaluating The Perceived Impact of Mixing Methods for Group Budgetary Decisions.Omega, 39: 362-372.
• ¹⁹
  FRANCO LA & HÄMÄLÄINEN RP. 2016. Behavioural operational research: returning to the roots of the OR profession. European Journal of Operational Research, 249(3): 791-795.
• ²⁰
  FU C & YANG S. 2012. An Evidential Reasoning Based Consensus Model for Multiple Attribute Group Decision Analysis Problems With Interval-Valued Group Consensus Requirements. European Journal of Operational Research, 223: 167-176.
• ²¹
  GEORGIOU I. 2008. Making Decisions In The Absence of Clear Facts. European Journal of Operational Research, 185: 299-321.
• ²²
  GEORGIOU I. 2012. Messing About In Transformations: Structured Systemic Planning For Systemic Solutions To Systemic Problems. European Journal of Operational Research, 223: 392-406.
• ²³
  GOMES LFAM, RANGEL LAD & JERÔNIMO RL. 2010. A study of professional mobility in a large corporation through cognitive mapping. Pesquisa Operacional, 30(2): 331-344.
• ²⁴
  GOMES LFAM, MACHADO MAS, SANTOS DJ & CALDEIRA AM. 2015. Ranking of Suppliers for a Steel Industry: A Comparison of the Original TODIM and the Choquet-extended TODIM Methods. Procedia Computer Science, 55: 706-714.
• ²⁵
  GOMES L & ZANAZZI J. 2012. Análisis Multicriterio Com Multiples Decisores: Aplicación Combinada de los Métodos Todim y Procesos DRV. Revista De Administração do Gestor, 2: 105-136.
• ²⁶
  GONÇALO TEE & ALENCAR LH. 2014. A supplier selection model based on classifying its strategic impact for a company’s business results. Pesquisa Operacional, 34(2): 347-369.
• ²⁷
  HÄMÄLÄINEN, LUOMA & SAARINEN. 2013. On the importance of behavioral operational research: The case of understanding and communicating about dynamic systems. European Journal of Operational Research, 228: 623-634.
• ²⁸
  HAN K & MITHAS S. 2013. Information Technology Outsourcing and Non-IT Operating Costs: An Empirical Investigation. MIS Quarterly, 37(1): 315-331.
• ²⁹
  HENAO F & FRANCO A. 2016. Unpacking multimethodology: Impacts of a community development intervention. European Journal of Operational Research, 253: 681-696.
• ³⁰
  HMELO-SILVER CE & BARROWS HS. 2008. Facilitating collaborative knowledge building. Cognition and instruction, 26(1): 48-94.
• ³¹
  KEENEY RL & RAIFFA H. 1993. Decisions with Multiple Objectives: Preferences and Value TradeOffs. Cambridge University Press.
• ³²
  KELLY G. 1955. The Psychology of Personal Constructs. Vol. 1: A Theory of Personality. Vol. 2: Clinical Diagnosis and Psychotherapy. New York. Norton.
• ³³
  KELEMENIS A, ERGAZAKIS K & ASKOUNIS D. 2011. Support Managers Selection Using an Extension of Fuzzy Topsis. Expert Systems with Applications, 38: 2774-2782.
• ³⁴
  KONTU K, RINNE S, OLKKONEN V, LAHDELMA R & SALMINEN P. 2015. Multicriteria evaluation of heating choices for a new sustainable residential area. Energy and Buildings, 93: 169-179.
• ³⁵
  LACITY MC, KHAN S, YAN A & WILLCOCKS LP. 2010. A Review of the IT Outsourcing Empirical Literature and Future Research Directions. Journal of Information Technology, 25(4): 395-433.
• ³⁶
  LOUFRANI-FEDIDA S & MISSONIER S. 2015. The project manager cannot be a hero anymore! Understanding critical competencies in project-based organizations from a multilevel approach. International Journal of Project Management, 33: 1220-1235.
• ³⁷
  MACHADO MAS, GOMES LFAM, SANTOS DJ & CALDEIRA AM. 2015. Using a Bipolar Choquet Neural Network to Locate a Retail Store. Procedia Computer Science, 55: 741-747.
• ³⁸
  MEDINA A & FRANCIS AJ. 2015. What Are the Characteristics That Software Development Project Team Members Associate With a Good Project Manager? Project Management Journal, 46(5): 81-93.
• ³⁹
  MINGERS J & ROSENHEAD J. 2004. Análisis Racional Reestudiado para un Mundo Problemático: Métodos para Estructurar Problemas en Condiciones de Complejidad, Incertidumbre y Conflicto. Instituto Venezolano de Planificación, España.
• ⁴⁰
  MINGERS J & BROCKLESBY J. 1997 Multimethodology: Towards A Framework For Mixing Methodologies. Omega, 25(5): 489-509.
• ⁴¹
  MINGERS J. 2000. Variety Is the Spice of Life: Combining Soft and Hard OR/MS Methods. International Transactions in Operational Research, 7(6): 673-691.
• ⁴²
  MINGERS J & GILL A. (Eds.) 1997. Multimethodology: Towards Theory And Practice And Mixing And Matching Methodologies. John Wiley & Sons Incorporated.
• ⁴³
  MONTIBELLER G & WINTERFELDT D. 2015. Cognitive and motivational biases in decision and risk analysis. Risk Analysis, 35(7): 1230-1251.
• ⁴⁴
  MORAIS DC, COSTA APC & DE ALMEIDA AT. 2014. Group Decision Model for Outsourcing IT Services. Procedia Technology, 16: 562-568.
• ⁴⁵
  MÜLLER R & TURNER R. 2010. Leadership competency profiles of successful project managers. International Journal of Project Management, 28(5): 437-448.
• ⁴⁶
  NEPOMUCENO LDDO & COSTA HG. 2015. Analyzing perceptions about the influence of a master course over the professional skills of its alumni: a Multicriteria Approach. Pesquisa Operacional, 35(1): 187-211.
• ⁴⁷
  PAREDES-FRIGOLETT H & GOMES LFAM. 2016. A novel method for rule extraction in a knowledge-based innovation tutoring system. Knowledge-Based Systems, 92: 183-199.
• ⁴⁸
  RAMAZANI & JERGEAS 2015 Project managers and the journey from good to great: The benefits of investment in project management training and education. International Journal of Project Management, 33: 41-52.
• ⁴⁹
  RAVINDRAN K, SUSARLA A, MANI D & GURBAXANI V. 2015. Social Capital, Reputation and Contract Duration in Buyer-Supplier Networks for Information Technology Outsourcing. Information Systems Research, Forthcoming.
• ⁵⁰
  SANTOS MARD, SALOMON VAP & MARINS FAS. 2015. Analytic Network Process and Balanced Scorecard applied to the performance evaluation of Public Health Systems. Pesquisa Operacional, 35(2): 353-361.
• ⁵¹
  SHAHHOSSEINI V & SEBT MH. 2011. Competency-Based Selection and Assignment of Human Resources to Construction Projects. Scientia Iranica, 18: 163-180.
• ⁵²
  SMALL A & WAINWRIGHT D. 2014. SSM and Technology Management: Developing Multimethodology through practice. European Journal of Operational Research, 233(3): 660-673.
• ⁵³
  SVEJVIG P & PRIES-HEJE J. 2011. Enterprise Systems Outsourcing Behind The Curtain: A Case Study Showing how Rational and Institutional Explanations Coexist and Complement Each Other. International Journal of Enterprise Information Systems (IJEIS), 7(1): 1-17.
• ⁵⁴
  TAKEY SM & MONTEIRO DE CARVALHO M. 2015. Competency mapping in project management: An action research study in an engineering company. International Journal of Project Management, 33: 784-796.
• ⁵⁵
  TERVONEN, T. 2014. JSMAA: Open source software for SMAA computations. International Journal of Systems Science, 45(1): 69-81.
• ⁵⁶
  VELASQUEZ M & HESTER PT. 2013. An analysis of multi-criteria decision making methods. International Journal of Operations Research, 10(2): 56-66.
• ⁵⁷
  VIDAL RVV. 2006. “Operational Research: A Multidisciplinary Field”. Pesquisa Operacional, 26: 69-90.
• ⁵⁸
  WILLCOCKS L. 2010. The Next Step for the CEO: Moving IT-Enabled Services Outsourcing To the Strategic Agenda. Strategic Outsourcing: An International Journal, 3(1): 62-66.
• ⁵⁹
  WU Z & XU J. 2012. A consistency and consensus based decision support model for group decision making with multiplicative preference relations. Decision Support Systems, 52(3): 757-767.
• ⁶⁰
  WENGER-TRAYNER E & WENGER-TRAYNER B. 2015. Communities of practice. A brief introduction. URI: URI: http://wenger-trayner.com/wp-content/uploads/2015/04/07-Brief introduction-tocommunities-of-practice.pdf Cited, 6, 16.
  » http://wenger-trayner.com/wp-content/uploads/2015/04/07-Brief introduction-tocommunities-of-practice.pdf
• ⁶¹
  WENGER-TRAYNER E, FENTON-O’CREEVY M, HUTCHINSON S, KUBIAK C & WENGER-TRAYNER B. 2014. Learning in Landscapes of Practice: Boundaries, identity, and knowledgeability in practice-based learning. Routledge. a. 62.
• ⁶²
  YANG L, HUANG C & WU K. 2011. The association among project manager’s leadership style, teamwork and project success. International Journal of Project Management, 29(3): 258-267.
• ⁶³
  YOLLES M. 2010. Exploring complex sociocultural situations through Soft Operational Research. Pesquisa Operacional, 30(2): 345-370.
• ⁶⁴
  ZANAZZI J & GOMES L. 2009. La Búsqueda de Acuerdos en Equipos de Trabajo: El Método Decisión con Reducción de la Variabilidad (DRV). Pesquisa Operacional, 29: 195-221.
• ⁶⁵
  ZANAZZI JL, GOMES LFAM & DIMITROFF M. 2014. Group Decision Making applied to Preventive Maintenance Systems. Pesquisa Operacional, 34(1): 91-105.
• ⁶⁶
  ZANAZZI J. 2016. Tesis doctoral: “Toma de decisiones en grupos de trabajo. El método Procesos DRV (Decisión con Reducción de Variabilidad)”. FCEFyN, Universidad Nacional de Córdoba, Argentina. Available in https://rdu.unc.edu.ar/handle/11086/4427
  » https://rdu.unc.edu.ar/handle/11086/4427

Publication Dates

• Publication in this collection
  Jan-Apr 2017

History

• Received
  22 Dec 2015
• Accepted
  16 Mar 2017