A multi-criteria decision model for selecting project portfolio with consideration being given to a new concept for synergies

A multi-criteria decision model for selecting project portfolio with consideration being given to a new concept for synergies

Adiel Teixeira de Almeida^*; Marina D.O. Duarte

Federal University of Pernambuco, Cx. Postal 7462, 50630-970 Recife, PE, Brazil. E-mail: [e-mail]

ABSTRACT

Project Portfolio Selection (PPS) is a kind of problem found in a variety of practical situations, such as research and development planning. Several different approaches have been proposed to deal with Project Portfolio Selection (PPS). However, the consideration on benefits synergies between projects is little addressed in the literature. The main focus of this paper is on synergy between projects, which is related to the interactions between the benefits of the projects. In this paper, a method has been developed to meet some of the major limitations of existing models: the inadequacy of the treatment of multiple criteria and inter-relationships between projects and the absence of the recognition and incorporation of managers' experience and knowledge, concerning to synergy between projects. The decision model is formulated as a non-linear 0-1 optimization problem, which considers the evaluations of projects and the benefit synergies of theses projects.

Keywords: multi-criteria decision aid, Project Portfolio Selection, synergy of projects.

1 INTRODUCTION

Applications of Project Portfolio Selection (PPS) have been published in several areas, particularly for Research and Development (Coffin & Taylor III, 1996; Coldrick et al., 2005; Wang & Hwang, 2005; Carlsson et al., 2006), pharmaceuticals (Prabhu, 1999), financial analysis (Ferreira et al., 2009) and Information Systems (Lee & Kim, 2001; Klapka & Piños, 2002). Despite the large range of papers, only a few of them really address the issue of synergies (Golabi et al., 1981; Hall et al., 1992; Santhanam & Kyparisis, 1995; Chien, 2002).

In general, many methods can be applied to solve the PPS problem, including Economic Analysis, Decision Theory, Optimization and Multi-criteria methodologies. Despite the wide applicability of optimization methods, there is a tendency to apply selection structures to the PPS (see, e.g., Archer & Ghasemzadeh, 1999). This approach can make use of the best characteristics of a combination of existing methods and consists of a way to consider the structure of preferences and experience of the decision makers. Multicriteria methods may also be integrated within PPS, incorporation the analysis of several different objectives. Multicriteria decision models are applied in many different problems and areas, such as: civil construction engineering, (Szajubok et al., 2006), suppliers selection (Alencar & Almeida, 2008; Alencar & Almeida, 2010), water management (Morais et al., 2010), energy planning (Albuquerque et al., 2009), management in oil companies (Meirelles & Gomes, 2009), and evaluation of rental values(Rangel & Gomes, 2007).

With regard to the synergies issue, the main focus of this work, there are several ways of defining it. Although the knapsack approach is seen as a way itself of dealing with synergy, there are other definitions related to the interaction between the projects in the value function, rather than in the constraint. In this case, the synergy is related to the benefits of these projects interactions, although in some cases this is implemented in the constraints, as a modelling procedure.

A small set of papers dealing with this kind of view is below. In this paper a different way of dealing with synergy is proposed based on experiences achieved in the context of R&D (Research and Development) portfolio for power electricity companies.

In the context of synergy, Golabi et al. (1981) express a concern about the existence of this type of project interrelationship, although it is considered only after the application of the selection method, during meetings of the selection committee. Hall et al. (1992) incorporate this concept into a linear programming model by means of constraints that meet the requirements for diversity needed for the R&D portfolio of the National Cancer Institute, in the United States. This can be as an indirect procedure to deal with a multi-objective process; that is, including some objective issues within the constraints.

Another way to consider synergies was used by Santhanam & Kyparisis (1995). They developed a nonlinear 0-1 goal programming model for Information Systems (IS) project selection that assumes the existence of interdependencies among some projects. The benefit-related objective accounts for interdependencies among different projects by considering synergistic benefits derived from implementing related projects. These interdependencies are also considered in the cost-related objective and in the constraint on resources, through the assumption that costs and resources can be shared among one or more projects. The model considers that such synergistic effects occur among three, at most, related projects. These synergies are represented by additional benefits derived from implementing two or three projects together.

Chien (2002) proposes an approach for selecting R&D portfolios that consists of evaluatingindividual projects from a portfolio perspective and linking project and portfolio measurements with the consideration of project interrelation. The method involves three phases: (1) identifying the portfolio objectives by considering the preference for the alternative portfolios and thenselecting the associated portfolio attributes; (2) selecting or constructing scales for measuring portfolio attributes; (3) measuring the alternative portfolios by aggregating the portfolio attributes. A taxonomy of portfolio attributes is provided as a guideline to help formulate portfolio attributes by considering the different types of interrelations between them that can be independent, interrelated or synergistic.

This paper sets out to address the synergies considered in the PPS as added benefits due to the joint presence of projects in the portfolio. An alternative approach for measuring such benefits is proposed. In addition, a multi-criteria methodology will be used to compile the objective function of the developed optimization model, which allows the decision maker's preferences to be incorporated.

The rest of the paper is organized as follows. In Section 2, the PPS problem is defined andthe synergy aspect is explained. In Section 3, the nonlinear 0-1 optimization model is presented. In Section 4, an application is developed. Finally, Section 5 sets out the main conclusions ofthis paper.

2 DEFINITION OF THE PROBLEM

The selection of the project portfolio to be undertaken has a high impact on organizations, especially in strategic terms, as a consistent decision has the power to drive the strategy and generate competitive advantages for the organization. Overall, the selection of the appropriate portfolio must meet the goals in a desirable way without violating the possible constraints.

In this context, some difficulties are observed: usually multiple goals must be considered; it may be necessary to measure strategic benefits; the projects may consider uncertain aspects (the deadline for completion, for example); and the number of feasible portfolios is often high. Moreover, the identification and measurement of synergy contributions to the portfolio deserve great attention, due to the holistic nature of the synergies.

A project portfolio is a set of projects considered for implementation by a particular organization. Thus, project portfolio selection is a regular activity that involves selecting a portfolio, from available project proposals and projects in progress, which achieves organizational goals without exceeding the available resources or violating other constraints (Archer & Ghasemzadeh, 1999). Belton & Stewart (2002) consider portfolio selection a problematic ( i.e. a category of problems) to which the methods of MCDA can be applied. They argue that the portfolio problematic consists of choosing a sub-set of alternatives from a wide range of possibilities, taking into consideration not only the characteristics of the alternatives individually, but also the way they interact and their positive and negative synergies.

The term synergy is full of subjectivity. However, in its essence, it carries the notion of cooperation between parts. The Systems Engineering area considers synergy the multiplier effect that allows the system to achieve levels of performance higher than those obtained by the sum of the performance of each part alone. In the context of PPS, it can be said that there is synergy between projects if the total value of the portfolio containing these projects is greater than the sum of the projects' individual values.

Due to the large range of studies in the literature, it is necessary to distinguish project selection from project portfolio selection. Project selection aims at selecting a sub-set of the set of project proposals available, by considering the characteristics of individual projects and possibly the constraints imposed. Moreover, PPS aims to compile a portfolio, which means that it is intended for choosing a sub-set of projects, taking into account the limitations and characteristics of individual projects as well as the synergy relationships between them.

3 THE MODEL

The model proposed employs the Combinatorial Optimization approach to the PPS problem, where the objective function is an additive aggregation function. Additive models is quite common as an aggregation procedure in multicriteria modeling (Almeida, 2011; Campos & Almeida, 2006; Alencar & Almeida, 2008). As it is a 0-1 model, the problem can be considered a knapsack problem, which is a well-known combinatorial optimization problem (Han & Kim, 2002).

The goal to be maximized includes two aspects: (1) scores, assigned by the decision-maker to projects, obtained through an additive value function, and (2) an indicator of the value of synergy between projects. The synergy consists of the main focus of this paper. It is evaluated by means of the decision maker's judgement on the relative gain proportionate to a project due to its presence jointly with another project in the portfolio.

3.1 Project Evaluation

Consider m projects proposed for implementation in a given period, the evaluation of which is performed on n criteria. Let X be the set of considered projects, such that X = {X₁, X₂, ..., X_m}. It is assumed that projects are indivisible. Therefore, fractions or possible versions of the same proposal are introduced in the model as separate projects. Each project presents an array of scores (z_i), which describes its performance in relation to the criteria.

The set of available projects can be represented by the score matrix Z, the rows of which are the score arrays of each project.

The overall evaluation of each project is expressed as an additive value function, which aggregates the project scores using scaling constants relative to the criteria. Because of the additive form, there must be special care with regard to the assessment scale and the criteria scaling constants, responsible for making the transformation of the assessment scale to a common scale. The scaling constants may be obtained through the elicitation process, using the trade-off concept to build a system of multi-attribute values from the single attribute scores (Keeney & Raiffa, 1976; Almeida, 2005; Almeida, 2011).

Given these considerations, the scaling constants array is represented by w:

For each project X_i the value of its overall evaluation in terms of all the n criteria is expressed by equation 4.

So, ν_i = [ν(X₁), ν(X₂), ..., ν(X_m)]^T is the overall evaluation array of the projects considered.

3.2 Synergy Evaluation

Denote the set of all possible portfolios by P. A project portfolio is a sub-set of all available projects. Thus, P is the power set of X, so the number of elements of P is given by |P| = 2^m. A portfolio can be represented as a binary vector p_r:

for r = 1, ..., 2^m.

To establish a measure of the synergy provided by the projects to the portfolio it is necessary that the decision maker determines the individual contribution added to each project by the joint presence with other projects in the portfolio. We define the elements of the synergy matrix S_ij = ⌊s_ij⌋ as the degree of contribution of the project X_j to the project X_i, in percentage values of X_i.

Therefore, the rows of the matrix S_ij represent the contributions of all other projects to the project X_i and the columns are related to the contributions of the project X_j to all the other projects.

The matrix in (6) is determined by the decision maker (DM), by means of an elicitation procedure. This elicitation procedure consists of an evaluation of each pair of project combination in order to determine how much would be the gain from this combination, assuming that both projects are included in the portfolio. Furthermore, the DM may be supported by experts in the company related to the portfolio management staff, a so common specialization nowadays in the project management career. This support by experts may consider their knowledge in the additional benefits over the multiples objectives that can be achieved when two projects are simultaneously implemented. The application of elicitation procedures is quite usual in applications involving multicriteria decision and may involve the presence of an analyst, who has the role of supporting the DM in the process of given information in a structured way (Almeida, 2011).

The meaning of this matrix is quite clear to the DM. Actually this proposition arisen in the context of a real application of research and development (R&D) portfolio in power electricity companies. The DM felt that for some projects there was a clear improvement in the value of the objectives, by the implementation of some projects simultaneously.

The very concept of synergy implies the order of magnitude of the gain that it represents. It seems quite unlikely that the synergy due to the presence of certain projects in the portfolio generates a gain equal or superior to the value of the project considered. Intuitively, this gain should be small enough to describe its value compared to the benefit provided by the project itself, but high enough to justify the consideration of synergies in the model. Thus, it is necessary to define the β parameter, so that it is not possible that a project's potential contribution to synergies in the portfolio exceeds the value of β.

The use of β is required as a condition in order to examine the consistency of the subjective assessment on the value of the synergies s_ij. In order to choose each β_j the decision maker must consider two main aspects:

(1) the benefit of executing project X_j alone;

(2) the additional benefit that project X_j brings to the portfolio when considered in conjunction with other projects.

The second aspect can be estimated by the decision maker's perception of the interaction between project X_j and the other candidate projects. This assessment must consider the number of interactions, that is, with how many projects X_j can interact, and the degree of influence that it can exert over them. For example, consider a set of projects where X₁ and X₂ are included. Supposing that the synergies of X₁ with the other projects are of the same magnitude of the synergies among X₂ and the others, if X₁ interacts synergistically with more projects than X₂ it has greater potential to contribute synergistically to the portfolio and this reflects on the choice of the β parameters, so that β₁ should possibly be greater than β₂.

The fact of the contributions being measured in relation to the value of the project benefited suggests that in order to obtain the contribution added to each project, the absolute values of the individual contributions should be used. For this purpose, the contribution in synergy for a project X_i is defined as:

However, it is necessary to notice that the existence of the contribution in synergy to a project also makes it a condition that this project is present in the portfolio.

This way of modelling the synergy was found very suitable to practical situations in an actual study conducted in the context of power electricity companies.

3.3 Objective Function

After having established the measures for evaluating projects and synergies, it is possible obtain an overall value for the portfolio, conditioned to the binary decision variables x_i:

Or in terms of ν(X_i) and CS(X_i),

The first parcel of equations 9 and 10 provides a comprehensive evaluation of the projects contained in the portfolio against the criteria, while the second gives a measure of the contribution of the synergies active in the portfolio.

Therefore, what is required is to maximize the value of the portfolio, so that the objective function is given by V(p_r). The problem of optimization can then be formulated as follows:

But for the model better to represent the class of problems of portfolio selection, consideration should still be given to a set of constraints that are capable of capturing some peculiarities of the cases examined.

3.4 Constraints

The literature reveals a considerable body of studies that addresses the most different types of constraints applied to portfolio selection (Bradi et al., 2001; Ghazemzadeh et al., 1999; Klapka & Piños, 2002; Wang & Hwang, 2005). Constraints allow the interdependencies between projects to be incorporated into the model. For each problem, various types of constraints can be considered. Therefore, the list of constraints described below illustrates the main types of constraints on the class of problem, without, however, making the claim that the list is exhaustive.

3.4.1 Resource constraints

In this model, the consumption of resources will be treated as a set of constraints on the sharing of each resource. Consider that q categories of resources are available and each project presents a resource consumption array (c_i):

Consequently, the consumption of all the available projects can be represented by the consumption matrix, the rows of which are the consumption arrays of each project.

In order to set the constraints, consider B the array of available resources:

Therefore, the set of resource constraints will be such that:

3.4.2 Cost constraints

Commonly, limitations on funding projects are considered in the selection process. The cost constraint must ensure that the set of projects being implemented is compatible with the capital provided by the organization for this purpose. Consider the cost array of projects (R):

Denoting by F the available capital, the cost constraint becomes:

3.4.3 Mandatory Project Constraints

In a set of projects considered for selection, there should be projects that must be included in the portfolio, based on certain considerations. Moreover, during periodic portfolio reviews it is common that projects in progress are continued, and should therefore be included. It is important to model the existence of mandatory projects in the portfolio, because these projects compete with others for scarce resources and it may be necessary to develop sensitivity analysis to determine the opportunity cost of including them.

Let S_M be the set of mandatory projects considered in the portfolio selection. The set of constraints related to the mandatory projects are such that

3.4.4 Logical Constraints

Other interdependencies between projects may be considered by a selection model and can be dealt with as constraints. The main relations existing between projects in this context are dependent projects and mutually exclusive projects.

– Dependent Projects: it is said that a project is dependent on another if for the first to be selected, the second necessarily must be included in the portfolio. Moreover, the second project can be included in the portfolio even though the first is not. This type of interdependence between projects will be modelled by the constraint represented in equation 19:

– Mutually Exclusive Projects: a set of projects is considered mutually exclusive if only one of them can be included in the portfolio, which means that, once a project is selected from the set, all the others should be excluded from the portfolio. The constraint that reflects this interdependence is given by equation 20.

3.5 General considerations on the Proposed Model

The portfolio selection model presented may have various formulations, but in general, it is summarised by:

Attention should be given to the role of the decision maker for the model application. The decision maker should participate during the entire process, expressing his values about projects, scaling constants and criteria. In order to the model application be useful to the portfolio selection problem decision making, the analyst must ensure that the evaluations provided depict the preferences of the decision maker. The elicitation process to obtain this information from the decision maker must be comprehensive and thorough. When a group decision making process is considered, the model can be applied to the input level aggregation. This paper considers the situation of individual decision maker. Group decision process involves others issues that are not approached in this paper.

4 NUMERICAL APPLICATION

A numerical application is presented in order to illustrate the use of the model. This application is based on a real context, although real data have not been applied for reasons of confidentiality. The criteria and general structure are also based on a real situation. The objective of using realistic data in this case is to provide a general view of the model to the decision maker, clarifying aspects over the assessment of scores, scaling constants and synergies. It is extremely important the understanding of these concepts so as the decision maker can provide judgments that really portray his preferences.

The problem being addressed is that of choosing a portfolio from ten available R&D projects evaluated against four criteria. The criteria were chosen based on a real case study, as already mentioned, and on several empirical studies found in the literature (Coffin & Taylor III, 1996; Liberatore & Titus, 1983). The criteria considered for this application are expected return, the probability of success associated with the project, the degree of strategic impact on the organization and the degree of impact on the operational processes. The first criterion, the expected return, will be treated as the percentage represented by the estimated net profit on investments. The other three criteria will also be evaluated on a scale of zero to one, because it is considered that the percentage values are easily understood by the decision maker in the context of the criteria presented.

In this application the decision maker provided a preliminary project evaluation and scalingconstants for the four criteria. The projects considered are a subset of the real situation available projects. This limitation was imposed in order to facilitate the decision maker's learning process. These projects were chosen to portray the main types of available projects, creating a situation fairly common in organizations: (1) the existence of a group of projects with a lowprobability of success and a high expected return, known as radical projects, because they represent the possibility of high competitive advantage, if they are successful; (2) a group of projects with a high probability of success, but a low expected return, known as incremental projects, as the results are modest benefits, usually technical incremental steps; (3) a group of projects for which the probability of success and expected return are median, their being at the frontier between the two sets mentioned above. According to this terminology, the projects presented in Table 1 can be classified as follows: P1, P2, P3 and P4 are radical projects; P7, P8, P9 and P10 are incremental projects; P5 and P6 are the intermediary projects. The data presented in Table 1 are the input for the score matrix Z (equation 2). It is valid to note that the extreme values of the assessment scale were not reached by the available projects. Nevertheless, they are considered viable.

The scores elicitation process led to the normalized scaling constants array w (equation 3) presented in Table 2.

Only three resource constraints and the cost constraint are considered. The three resources are:

(1) labour - expressed as the number of individuals needed;

(2) equipment - amount of hours of use of certain equipment;

(3) energy - expressed in megawatts (MW) consumed by the project.

Table 3 presents the consumption of resources and the cost (in $10³) for each project and their availability. These are the input data for the constraints, where from the second to fourth columns are the data related to the resource constraints (equation 15) and the fifth column are the data for the cost constraint (equation 17).

The scenarios are distinguished by their synergy matrix S_ij (equation 6). These scenarios were designed with the purpose of portraying the model operation, thus allowing the decision maker to understand its logic before the start of the application. Therefore, the five scenarios discussed below depict extreme situations for illustrative purposes. This does not mean that they represent common conditions.

Scenario 1 is characterized by the absence of synergies between projects. In scenario 2 just one project (P7) contributes synergistically to others and in scenario 3 only one project (P5) receives synergistic contributions from other ones. Such scenarios allow evaluating the trade-off between project values and synergy gains. Scenario 4 presents mutually synergistic projects, which makes the synergy matrix symmetrical in relation to the main diagonal. Despite the symmetry, it cannot be said that the synergy value added to a project is equal to the value it adds, because the percentages are related to the value of the project that receives the contribution. Scenario 5 portrays the existence of synergy groups by considering two groups: {P1, P8, P10} and {P2, P7}. Given the set of available projects X, a synergy group is a sub-set of X in which the constituent projects interact synergistically with each other.

The value of β used throughout the application is 0.2. Given the explanatory nature of the application, this value was obtained as an appropriate upper bound for the synergy contribution of each project, considering the aspects for the definition of β. These values could be better refined, but this task was left over the decision maker after the first contact with the data. The concern here is to provide parameters for the understanding of the model and the concepts linked to it.

The synergy matrices for each scenario are presented in the ^appendix, except for scenario 1, where they do not exist. It can be observed that due to the fact that the main diagonal of the matrices represents, in theory, the gain from the synergistic interaction of a project with itself, non-zero values make no sense.

The model formulation applied to this problem is given by equation 22, where the input data are specified in the previous tables and the decision variables array p_r consists of ten binary variables related to the presence of the considered projects in the portfolio.

Table 4 presents the main results for the scenarios described. For each scenario were obtained, by applying the model, the portfolio to be selected and the final evaluation for each of these represented in the third column by the Portfolio Value (V_P), which was calculated based on equation 9. The fourth column presents an indicator, the synergy gain, which allows evaluating how much the synergies represent in terms of the sum of the absolute values of the projects to the portfolio. The synergy gain is obtained by dividing the second parcel of equation 9 by its first parcel, according to equation 23. In the other four columns are shown the values obtained for the constraints in each scenario.

In the first scenario, it is observed that the projects selected are those with the greatest overall evaluations and lowest consumption of resources. Despite this, project P4 was not selected due to its high consumption of resources, in addition to the high cost when compared to other projects with a lower overall value, since this does not allow other projects with a good evaluation to be considered in conjunction with P4. Although project P9 has the same overall value as project P7, it consumes fewer resources, and therefore has been selected. The projects with the lowest individual value were not considered in the selected portfolio (P5 and P10).

Scenario 2 makes P7 contribute to more attractive projects for the portfolio, which makes it more attractive than P6, because the synergy gain provided by P7 is greater than the deficit existing in its value. In this case, P7 becomes more attractive than P6 when it provides a gain of 0.17% over the value of the selected set. In the third scenario, what makes P5 attractive is the contribution that it receives from the high-valued projects, notoriously more susceptible to being part of the portfolio (e.g. P2). Therefore, in this situation it seems possible for P5 to appear in the final portfolio when it receives contributions of 0.1% over the value of the selected portfolio. It may be noted that the absence of other gains resulting from synergies in these scenarios is merely illustrative, because in real situations this type of interaction rarely occurs alone.

In scenario 4, the increase of the synergistic contribution occurs due to the comparatively larger amounts of synergistic interactions between the available projects. Consequently, the value of some projects plus the amounts due to the synergies exceeds the gain provided by projects of greatest value, which receive few or no contributions. Moreover, the resulting portfolio consumes fewer resources, compared to those obtained in the first scenario, as Table 4 shows. This fact illustrates that, in this case, the values attributed to the synergy relations are determinants in portfolio selection.

Scenario 5 allows the observation that there are thresholds for synergy values from which the projects of the two groups are considered in the portfolio. This fact occurs because these two synergy groups have low value projects together with other high-value ones. Therefore, the synergy value should be sufficient, when added to the portfolio, to overcome the presence of a high-value project, thus justifying the selection of the entire group.

These scenarios enabled the decision maker to identify the impact of synergies in the value of projects and the trade-off between project values and the synergy contributions in the model. If this analysis were developed, the decision maker could easily identify and measure the existing synergies in his business context, so that the model was easily applied and allowed a satisfactory portfolio to be generated.

5 CONCLUSIONS

The model developed contributes as an alternative approach to the incipient study of synergies between projects and provides a multi-criteria approach for project portfolio selection. The advantage that the optimization approach provides for the model is its versatility for application to different types of problems, as it allows for the particularity of the relationships between projects.

Therefore the main contribution in this model is the proposition for dealing with synergies, which is based on a real situation faced in a study conducted for R&D projects portfolio in the context of power electricity companies. Furthermore, it has been discussed the facility for the decision maker to understand and deal with this view for synergies. The elicitation procedure to obtains the synergies matrix may be supported by experts in the management project staff, an usual organizational issue found in project management nowadays. Moreover, in order to facilitate the elicitation of the synergies, a consistency parameter β has been introduced. The use of the parameter β allows some coherence to be introduced to the model, thus setting limits on the contributions from synergies.

The main constraints for problems of project and portfolio selection have been presented, but the model is not limited to them, so that other constraints can be considered.

There is a certain difficulty in the models that consider synergy in order to measuring its impact on the selection process. Value aggregation to the portfolio has been shown to be a satisfactory solution for incorporating synergies in the portfolio decision, but this does not solve the problem related to the value that should be added. The model presented in this paper offers an alternative measure based on the value of the project considered that seems to be easy for the specialist to deal with.

Further work to be addressed on this kind of problem might focus on investigating the applicability of other types of objective functions that also aggregate the synergy aspect. Another important issue is the situations where the synergies occur due to the interaction between more than two projects. In this case it may be difficult to separate the contributions into individual project measures. So, other measures should be developed in order to deal with this kind of interrelationship.

ACKNOWLEDGMENTS

This study is part of a research program funded by the Brazilian Research Council (CNPq). The authors acknowledge the valuable suggestions made by anonymous reviewers to a previous version of this paper, which have contributed to make this a better final version.

Received December 30, 2008 / Accepted February 15, 2011

• [1] ALBUQUERQUE LL, ALMEIDA AT & CAVALCANTE CAV. 2009. Aplicabilidade da Programação Matemática Multi-Objetivo no Planejamento da Expansão de Longo Prazo da Geracão no Brasil. Pesquisa Operacional, 29: 153-177.
• [2] ALENCAR LH & ALMEIDA AT. 2008. Multicriteria Decision Group Model for Suppliers Selection. Pesquisa Operacional, 28: 321-337.
• [3] ALENCAR LH & ALMEIDA AT DE. 2010. A Model for Selecting Project Team Members Using Multicriteria Group Decision Making. Pesquisa Operacional, 30(1): 221-236.
• [4] ALMEIDA AT. 2011. O Conhecimento e o Uso de Métodos Multicritério de Apoio a Decisão. 2ª Ed. Recife: Editora Universitária UFPE.
• [5] ALMEIDA AT. 2005. Modelagem Multicritério para Seleção de Intervalos de Manutenção Preventiva Baseada na Teoria da Utilidade Multiatributo. Pesquisa Operacional, 25(1): 69-81.
• [6] ARCHER NP & GHASEMZADEH F. 1999. An integrated framework for project portfolio selection. International Journal of Project Management, 17(4): 207-216.
• [7] BELTON V & STEWART TJ. 2002. Multiple Criteria Decision Analysis. Kluwer Academic Publishers.
• [8] BRADI MA, DAVIS D & DAVIS D. 2001. A comprehensive 0-1 goal programming model for project selection. International Journal of Project Management, 19: 243-252.
• [9] CAMPOS VR & ALMEIDA AT DE. 2006. Modelo multicritério de decisão para localização de nova Laguaribara com VIP Analysis. Pesquisa Operacional, 26(1): 91-107.
• [10] CARLSSON C, FULLÉR R, HEIKKILÄ M & MAJLENDER P. 2006. A fuzzy approach to R&D project portfolio selection. International Journal of Approximate Reasoning, s.n.t.
• [11] CHIEN C.-F. 2002. A portfolio evaluation framework for selecting R&D Projects. R&D Management, 32(4): 359-368.
• [12] COFFIN MA & TAYLOR III BW. 1996. Multiple Criteria R&D Project Selection and Scheduling using Fuzzy Logic. Computers & Operations Research, 23(3): 207-220.
• [13] COLDRICK S, LONGHURST P, IVEY P & HANNIS J. 2005. An R&D options selection model for investment decisions. Technovation, 25: 185-193.
• [14] EILAT H, GOLANY B & SHTUB A. 2006. R&D project evaluation: An integrated DEA and balanced scorecard approach. European Journal of Operational Research, doi:10.1016/j.omega.2006.05.002.
• [15] FERREIRA RJP, ALMEIDA-FILHO AT DE & SOUZA FMC DE. 2009. A decision model for portfolio selection. Pesquisa Operacional, 29(2): 403-417.
• [16] GHASEMZADEH F, ARCHER NP & IYOGUN P. 1999. A zero-one model for project selection and scheduling. Journal of the Operational Research Society, 50: 745-755.
• [17] GOLABI K, KIRKWOOD CW & SICHERMAN A. 1981. Selecting a portfolio of solar energy projects using multi-attribute preference theory. Management Science, 27(2): 174-189.
• [18] HALL NG, HERSHEY JC, KESSLER LG & STOTTS RC. 1992. A model for making project funding decisions at the National Cancer Institute. Operations Research, 40: 1040-1052.
• [19] HAN K-H & KIM J-H. 2002. Quantum-inspired evolutionary algorithm for a class of combinatorial optimization. IEEE Transactions on Evolutionary Computation, 6(6): 580-593.
• [20] KEENEY RL & RAIFFA H. 1976. Decision with Multiple Objectives: Preferences and Value Trade-offs, John Wiley & Sons.
• [21] KLAPKA J & PIÑOS P. 2002. Decision support system for multicriterial R&D and information systems projects selection. European Journal of Operational Research, 140: 434-446.
• [22] LEE JW & KIM SH. 2001. An integrated approach for interdependent information system project selection. International Journal of Project Management, 19: 111-118.
• [23] LIBERATORE MJ & TITUS GJ. 1983. The practice of management science on R&D project management. Management Science, 29: 962-974.
• [24] MEIRELLES CLA & GOMES LFAM. 2009. O apoio multicritério à decisão como instrumento de gestão do conhecimento: uma aplicação à indústria de refino de petróleo. Pesquisa Operacional, 29(2): 451-470.
• [25] MORAIS DC, CAVALCANTE CAV & ALMEIDA AT DE. 2010. Priorização de Áreas de Controle de Perdas em Redes de Distribuição de Água. Pesquisa Operacional, 30(1): 15-32.
• [26] PRABHU GN. 1999. Managing research collaborations as a portfolio of contracts: A risk reduction strategy by pharmaceutical firms. International Journal of Technology Management, 18(3): 207-231.
• [27] RANGEL LAD & GOMES LFAM. 2007. Determinação do valor de referência do aluguel de imóveis residenciais empregando o Método TODIM. Pesquisa Operacional, 27(2): 357-372.
• [28] SANTHANAM R & KYPARISIS J. 1995. A multiple criteria decision model for information system project selection. Computers and Operations Research, 22(8): 807-818.
• [29] SZAJUBOK NK, MIRANDA CMG DE & ALMEIDA AT DE. 2006. Uso Do Método Multicritério ELECTRE TRI Para Classificação De Estoques Na Construção Civil. Pesquisa Operacional, 26: 625-648.
• [30] WANG J & HWANG W.-L. 2005. A fuzzy set approach for R&D portfolio selection using a real options valuation model. The International Journal of Management Science, 35: 247-257.

appendix

*

Corresponding author

Publication Dates

• Publication in this collection
  05 Aug 2011
• Date of issue
  Aug 2011

History

• Accepted
  15 Feb 2011
• Received
  30 Dec 2008