The influence of risk management on the project portfolio success – proposal of a risk intensity matrix

A influência da gestão de riscos no sucesso do portfólio de projetos - proposta de uma matriz de intensidade de riscos

Abstract:

This article aims to understand how the risk management influences the project portfolio success. Two methodological approaches were selected in this research: a bibliometric survey followed by a case study. The object of study was the new products project portfolio of an organization from the industrial sector, manufacturer of durable goods. The findings revealed a low intensity of project portfolio risk management. This is aligned with the bibliometric survey results and with the evidence from the investigation performed on the case study unit. In order to evaluate the risk management influence over the portfolio success, this article proposes a matrix which suggests the risk management intensity associated with the project portfolio management processes. The proposed matrix application can be considered a contribution element to deepen the knowledge of the risk management influence on the project portfolio success.

Keywords:
Risk management matrix; Portfolio risk management; Project portfolio success

Resumo:

O objetivo deste artigo é compreender como a gestão de riscos influencia o sucesso do portfólio de projetos. Duas abordagens metodológicas foram adotadas neste trabalho: a bibliometria seguida de um estudo de caso. O caso investigado foi uma organização do setor industrial de bens duráveis e o objeto de estudo foi o portfólio de projetos de novos produtos. A análise dos achados da pesquisa mostrou uma fraca intensidade da gestão de riscos de portfólio de projetos. Esta constatação está aderente ao que diz a literatura vigente no assunto, explorada através do levantamento bibliométrico, e às evidências advindas da investigação feita na unidade objeto do estudo de caso. Para a avaliação da influência da gestão de riscos no sucesso do portfólio, foi proposta uma matriz que sugere a intensidade da gestão de riscos associada aos processos de gestão do portfólio de projetos. A aplicação da matriz proposta pode ser vista como um elemento que contribui para o aprofundamento dos estudos da influência da gestão de riscos no sucesso do portfólio de projetos.

Palavras-chave:
Matriz de gestão de riscos; Gestão de riscos de portfólio; Sucesso do portfólio de projetos

1 Introduction

One of the most evident findings from the literature about project portfolio management is that organizations use projects to achieve their strategic objectives and increase their competitive advantage (Shenhar & Dvir, 2007). It is also known that market dynamism and increasing competitiveness force organizations to run several projects simultaneously, to maintain flexibility and efficiency (Teller, 2013). This scenario requires efficiency in resources management and projects selection from the organizations which, according to Archer & Ghasemzadeh (1999), generates a dispute over the available resources. Consequently, it is necessary to organize the projects around similar objectives, creating the project portfolios to optimize the resources on organizational strategies implementation.

Organizing the projects around similar objectives is clearly insufficient to ensure that the strategic plan is properly implemented, since the projects are permanently exposed to uncertainties (Kerzner, 2009). Market dynamism and internal organizational changes, associated with project uncertainties, can lead to strategic plan reviews, resulting in project portfolio adjustments. Such modifications, of course, can imply in risks if they are not properly performed (Ghasemzadeh et al., 1999).

Considering this situation, keeping the projects permanently aligned with strategic objectives at the lowest risk level is the biggest challenge of project portfolio administrators. This concern was discussed by Archer & Ghasemzadeh (1999), who emphasized the need of managing the project portfolio efficiently, effectively, and optimized, choosing the highest profit and lowest risk projects. The authors' recommendation originates from the Modern Portfolio Theory, introduced by Markowitz (1952) and later adapted for project portfolio management by Meskendahl (2010). Following the same line of reasoning, Mikkola (2001) emphasized that a balanced portfolio allows organizations to achieve their strategic objectives without being exposed to unreasonable risks.

However, having a balanced portfolio is not enough to achieve the strategic objectives. Jonas et al. (2010) associate the success of the project portfolio with the quality of its management. However, Hofman et al. (2017) empirically demonstrated that the success of the project portfolio is also associated with the quality of its risk’s management. Likewise, Teller & Kock (2013) affirmed that risk management in the project portfolio has positive influences on its success through two dimensions: the ability to cope with risks and the transparency in its management.

Therefore, not properly manage the risks seems to imply in having the project portfolio unbalanced. According to Teller et al. (2014) this situation prevents the organization to achieve its strategic objectives and be prepared for the future in terms of reserves.

Despite these findings, risk management is still not well incorporated into project portfolio management, according to Artto et al. (2000) point of view. This is corroborated by Teller et al. (2014) since, in addition to the high costs, the practice has shown that organizations still have difficulties in dealing with risk management at the project portfolio level.

An interesting aspect observed in the literature by Shenhar et al. (2001) is that there is no risk management method available which fits all portfolio types. Therefore, they recommend the customization of risk management to the characteristics of portfolio, the uncertainties and complexities of the projects that comprise it and the market turbulence in which the organization is inserted.

The findings explained so far show the risk management importance in the context of project portfolios and, for this reason, it is imperative to advance and deepen within this phenomenon. However, it is known that a better understanding of this topic is, on the one hand, challenging, but instigating on the other. Motivated by these two aspects, the authors of this paper outlined the following objective: to understand how risk management influences the project portfolio success.

To find convincing answers to this objective, initially a longitudinal bibliometric survey was developed on the subject matter. The result of the survey brought theoretical and conceptual subsidies to develop a case study in an organization of the industrial sector of durable goods.

The results found in the case study showed that portfolio risk management, specifically in the case of this study, presents a low application intensity. This finding was possible because the authors proposed a conceptual risk management intensity matrix, from the reading of the articles identified in the bibliometric survey, and the results could be scored there.

The findings of this work refer to some reflections that can glimpse implications from the academic point of view, such as the possibility of exploring the intensity matrix in different project portfolio environments, as well as examining aspects related to the costs and benefits of applying the project portfolio risk management. Practically speaking, it is expected that the executives and professionals be able to expand risk management, integrating it with project portfolio management processes.

After these introductory aspects, the literature review is presented in section 2. The literature consulted was useful to outline the bibliometric results, as well as to aid to the establishment of variables and constructs of risk management and its implications for the portfolio management success. Section 3 is dedicated to the methodological approaches’ presentation. The bibliometric procedures, as well as the case study protocol and its derivations are presented in this section. After, in the section 4, the results of the research are presented in two different dimensions: one referring to the bibliometric survey and the other referring to the case study. The data analysis is shown in section 5 by the portfolio risk intensity matrix. Finally, the conclusions and final considerations of the research are presented.

2 Literature review

2.1 Project portfolio management

Project portfolio management has received more attention from executives and researchers since the end of the last century, based on the work of McFarlan (1984). Its conception is based on the project portfolio´s value maximization to an acceptable level of risk, considering the alignment of its projects with the organization's strategic objectives. According to PMI (2017b), it is not essential to have dependencies between the projects that compose the portfolio, but they need to be scaled in order to allow a classification and prioritization.

In this perspective, organizing projects around similar goals is not enough to ensure that organization's strategic plan is going to be properly implemented. Archer & Ghasemzadeh (1999) emphasizes that project portfolio must be well-managed. They propose a model to optimize the project selection process and affirm that portfolio management must allow the organization to select and prioritize a group of projects aligned to its strategic objectives, provided that the available resources to conduct them are appropriate to their needs.

Killen et al. (2012) define project portfolio management as a dynamic and complex series of decision-making processes, in which projects are constantly evaluated according to the corporate strategy and can be closed, replaced or have their priority modified. Organizations use this set of processes to quickly adapt to the market changes, pursuing the projects’ success rate maximization and the strategic plan execution, increasing, this way, their competitive advantages.

However, the decision to keep, remove or include a project in the portfolio involves risks and organization conflicts if it is not properly performed (Ghasemzadeh et al., 1999). The explanation to this phenomenon is that, once the projects are grouped around specific objectives, there is a tendency to appear an interdependence relationship between them, creating new risks at the portfolio level (Teller et al., 2014).

According to Teller et al. (2012), it is fundamental to manage the project portfolio in an efficient, effective and optimized way, in order to keep projects permanently aligned with the organization's strategic objectives. This means that the projects which are part of a portfolio must be efficiently managed to reach its goals as planned, whereas the project portfolio management must focus on effectiveness, ensuring the execution of the projects which give the highest return to the organization, considering the risks involved. Finally, the portfolio optimization is related to the synergy of available and usually scarce, resources to run several projects concurrently.

Cooper et al. (1999) argue that effective management is represented by strategic choices in terms of markets, products, technologies and project selection to achieve a balance between the number of active projects and the available resources. According to Cooper & Edgett (2010) project portfolio management is considered effective if the active projects are aligned with the organization's strategic plan, balanced in terms of risks and have the resources suited to their needs.

Considering the definition of Cooper et al. (2002), the portfolio success can be measured using four dimensions: average projects success, resources synergy application, portfolio balancing, and the alignment with strategic objectives.

Differently from the projects, which have a well-defined cycle life with start and finish, the project portfolio management is continuously run through the execution of certain processes. According to PMI (2017b), the project portfolio ends only when the organization disappears or when the portfolio is abandoned. The portfolio management is a continuous business process, usually influenced by external and internal requirements, in which certain activities are invoked periodically, according to the needs of the organization.

In their research, Castro & Carvalho (2010) defined a conceptual model based on a comparative analysis of five different models of project portfolio management from the literature. The literature presents several models with different combinations of these processes. However, the authors recommend that each organization must customize its own management model, adopting the most appropriate processes and combining them according to the characteristics and level of complexity of its portfolio, because there is no model that can be considered effective for all organizations. The customization must also consider the tools and techniques that will support the different portfolio management processes execution.

Due to the specific characteristics and different complexity levels of each portfolio, Jeffery & Leliveld (2004) proposed a classification model of three different levels: Inventory, when the projects are well defined, formally documented and have good costs and benefits estimation; Administration, if there are periodic reviews of the portfolio for measurements and prioritization adjustments, when new projects can be included or active projects can be eliminated if there is a lack of alignment with the organization´s requirements; Optimization, when the project portfolio is optimized and balanced.

Nevertheless, De Reyck et al. (2005) explain that the organizations recognize the benefits from the portfolio management and the negative consequences of its non-realization, regardless of the level of execution and formalization. According to the authors, risk analysis is one of the main elements of portfolio management and must cover not only the project risks, but also the risks from the interactions between projects. The strategic risk management of each one of the portfolio projects and the possible impacts on the strategic objectives assume greater importance when there are interdependence relations between projects with higher priority level, when the risks of one project increase the risks of the others (PMI, 2017b).

Castro & Carvalho (2010) argue that project portfolio management supports decision-making processes, relevant investments validation and strategic objectives implementation, becoming a key activity for the organization’s success in their long-term strategies. Teller et al. (2014) emphasize the importance of this activity stating that the organizations must rely on project portfolio management to run their projects and programs concurrently and to be able to quickly adapt themselves to the external environment changes, ensuring their competitive advantages.

Teller & Kock (2013) state that there is a consensus in the literature concerning the project portfolio management objectives defined by Cooper et al. (2001) and Elonen & Artto (2003): portfolio value maximization, projects alignment with organization's strategic objectives and portfolio balance considering the projects’ characteristics and available resources. These objectives are consistent with Hill & Jones's (1992) definition, according to which an ideal project portfolio is balanced in terms of different types of projects in such a way that it allows the organization to achieve its strategic growth and profit objectives without exposing itself to unwanted risks. Finally, Sanchez et al. (2008) emphasize that the project portfolio value must be maximized in terms of strategic objectives at a level of risk considered acceptable by the organization.

2.2 Portfolio risk management

Based on this literature review, gains relevance the assertion of Courtney et al. (1997) that underestimating the large number of uncertainties involved in managing the project portfolio may guide the organization to define strategies that do not protect it against the threats, preventing the achievement of the desired competitive advantages from existing opportunities.

Wideman (1992) defines uncertainty as a lack of knowledge about future events, whether favorable or unfavorable, and argues that risk management should act on a region of uncertainties delimited by total certainty in one extreme, and by total uncertainty, at the opposite end. Still, according to the author, most decisions are made in a scenario of incomplete or non-existent information, thus increasing the level of uncertainty in their results.

According to Ward & Chapman (2003) uncertainties are part of projects and must be considered as indecisive situation, hesitation, or lack of complete information. Uncertainty refers to situations of ambiguity and variability. Ambiguities are related to lack of information (lack of project data, stakeholder’s behavior, premises, estimates, trends). The variability is linked to the possible variations of project's performance (cost, life cycle, functional requirements fulfillment). Since risks arise from uncertainties, the authors suggest that managing uncertainties rather than risks provides a much broader view, allowing the identification of risks and opportunities at the origin.

Risk management contributes to a more conscious decision-making process, through the risk identification, analysis and treatment (Flanagan & Norman, 1993). Consequently, risk management must be initiated together with the project and must become an ongoing process throughout its life cycle (Ward & Chapman, 1991).

However, managing project risks alone is not enough to ensure that strategic objectives will be achieved, since organizations usually must manage multiple projects simultaneously to maintain their flexibility and efficiency (Olsson, 2008). Hence, risk management must be strategically conducted and continuously follow the portfolio management to reduce uncertainties and contribute to the success of strategic plan implementation of the organizations (Sanchez et al, 2008).

According to the literature reviewed in this article, risk management provides different benefits to the organizations when integrated in the project portfolio. Among the benefits provided, Sanchez et al. (2008) said that there is an increase of information transparency, Lee et al. (2009) argue that it improves the company's ability to deal with risks and McFarlan (1981) emphasizes that risk management provides deeper information for decision making, improving its outcome.

Deepening the benefits analysis, Teller & Kock (2013) affirm that risk management becomes essential for project portfolio management because it allows the organizations to improve the management of new opportunities and threats. Moreover, it helps the organization to consolidate the activities, eliminating duplicated work. Hence, risk management supports the resources alignment and distribution between projects, contributing with the organization to take more conscious decisions.

Kock et al. (2016) argue that risks and opportunities are always very close and that risk-taking can encourage organizations to quickly implement new ideas, making better use of available resources. This agility can become a strategic differential over its competitors, through the launch of new products and services. However, new ideas can become a competitive advantage for organizations only when they are successfully implemented.

Therefore, it is necessary to keep a coherent view of risks and opportunities (Teller, 2013; Ward & Chapman, 2003), which leads organizations to manage the uncertainties associated with projects, so that their negative effects (risks) are mitigated or eliminated, and their positive effects (opportunities) are stimulated and expanded (Petit, 2012).

Consequently, managing project portfolio risks avoid failures and increase success possibilities (Reyck et al., 2005; McFarlan, 1981). Additionally, Teller et al. (2014) conclude that neglecting risks identified during the project portfolio management may lead organizations to have an unbalanced portfolio, not being properly prepared in terms of reserves, which can compromise their future.

Portfolio risk management must provide assessments such as: verify whether portfolio assumptions remain valid; identify significant changes of risks previously assessed; provide the continuity of risk management procedures already defined; and verify a perfect alignment between cost and timeline with contingency reserves concerning the identified risks (PMI, 2017a).

Reyck et al. (2005) was one of the first studies about portfolio risk management and pointed to risk analysis as one of the key elements of project portfolio management. Subsequently other studies also addressed portfolio risk management, like the works from Sanchez et al. (2008) and Hasna & Raza (2010). However, only from 2013 onwards the subject became widely and intensely addressed by other authors, starting with the articles from Teller & Kock (2013) and Teller et al. (2014).

In the field of project risk management there is already a significant number of studies. Some of them analyzed the relation between risk management intensity and project success, based on risk factors presence. Others gave more attention to risk management processes.

Specifically, in the field of infrastructure projects, Flyvbjerg (2007) identified that the project’s success possibilities increase as the intensity and extent of performance evaluation are higher. Redmill (2002) and Jordão et al. (2015) argue that identifying and categorizing risks into risk factors can prevent negative impact on the project results. Similarly, studies by Chapman & Ward (2004) and Mu et al. (2014) showed that increasing the intensity of risk management implies in a reduction of risk factors presence. Likewise, the articles from Raz et al. (2002) and Zwikael & Ahn (2011) indicated that projects with a greater intensity in risk management have a lower number of risk factors and, consequently, perform better.

Concerning risk management processes, Carvalho & Rabechini (2015) carried out an extensive analysis on the available literature and defined six main processes: planning, identification, qualitative analysis, quantitative analysis, monitoring and responses to risks.

In the project portfolio risk management field, however, there are not many studies that deal specifically with processes. However, the few articles identified point to the same processes used in project risk management. PMI (2017b) adopts 4 processes: planning, identification, evaluation and responses. Teller et al. (2014) studied portfolio risk management in a larger scope and proposed a conceptual model, based on the same processes defined by PMI (2017b) and argues that there is a positive relation between the competence of the organization to perform risk management through its processes and the quality of portfolio risk management.

3 Methodological procedures

Two methodological procedures were used to reach the proposed objective of this research: the bibliometric survey and the case study.

3.1 Bibliometric survey

The bibliometric survey method was defined, at the end of last century, as the application of some techniques in order to quantify the written communication process (Ikpaahindi, 1985). According to Small (2003), the bibliometric survey enables the identification of the literature tendencies about specific topics based on existing publications characteristics. Through the application of this research method, it is possible to identify patterns, the most profitable journals on a specific topic, the evolution of the literature over a period and the authors' networks by the number of citations (Prasad & Tata, 2005).

Meanwhile, it is not possible to identify the influence of recent articles and authors through bibliometric surveys because this relevance becomes evident after some time, when the articles are cited by other authors (Zupic & Čater, 2015). This characteristic is, therefore, a restriction of the method.

Carlos et al. (2017) explained the three laws of bibliometric. According to Zipf’s law, it is possible to demonstrate the importance of certain words by their repetition along the text. Lotka's law identifies the productivity of the authors and universities concerning the subject studied. Finally, the Bradford's law enables to estimate the journals’ degree of relevance in a certain area of ​​knowledge. According to this law, the publication of the first articles about a certain subject in a journal attracts new articles about the same subject, creating a cycle and the identification of the journal with the area of ​​knowledge in question.

The bibliometric survey was performed by combining the keywords “RISK” and “UNCERTAINTY” with the terms “PROJECT PORTFOLIO MANAGEMENT” and “PROJECT PORTFOLIO SUCCESS” and its variations, resulting in the following searching keyword: (“Management of risk*” OR “managing risk*” OR “management of uncertaint*” OR “managing uncertaint*” OR “uncertaint* management”) AND (“project portfolio management” or “project portfolio success”). This keyword definition was possible after analyzing some articles about the subject and performing several tests with each of the terms separately.

The keywords were written in English, once it is the most usual language used in international journals. Furthermore, using the terms in English increased the scope of the research, as there are many more articles about this issue published in international journals.

The survey was carried out in two different databases, also with the aim of increasing the scope of the result. The ISI Web of Knowledge (Web of Science) and Scopus databases were consulted due to the resources offered for metadata gathering (abstract, citations, authors, institutions, and countries) and their relevance and tradition in the academic field, once they consolidate relevant journals (Carlos et al., 2017). According to Archambault et al. (2009), ISI Web of Knowledge and Scopus are the most popular databases in academic research.

Concerning the publication type, only articles from academic journals were selected. Articles usually precede books and are considered a trusted source for studies, due to the methodological rigor required for this kind of paper (Moran et al., 2010). It was not established any limit in terms of date which the articles were issued.

3.2 Case study

McNamara (2006) recommends making use of interviews when it is necessary to gather in-depth information about people's opinions, thoughts, experiences and feelings. Interviews can be categorized into three types (structured, semi-structured and unstructured or informal) and the most appropriate one depends on the researcher's goals. Semi-structured interviews, for example, are recommended when the researcher wants to go deeper in a specific topic. Consequently, the authors adopted a semi-structured interview in this case study.

The questions were previously formulated and clustered according to the topics studied. Additional questions were asked to the respondents, as needed, in order to gather enough information for the full understanding of the topics, concerning the research´s objectives. An interview protocol was established aiming to ensure uniformity in data collection. Both the questionnaire and the protocol were tested, adjusted and validated before the interviews. The pre-test aims to improve the instrument and increase its reliability and validity, thereby ensuring a perfect adherence with the research purpose (Martins & Theóphilo, 2007).

Regarding the definition of the people to be interviewed, Gil (2011) recommends interviewing only qualified, experienced people (specialists) and seen as typical to answer a questionnaire, concerning the context studied. Following this recommendation, some criteria were previously established to aim selecting the people to be interviewed. The interviewees should: perform the role of project leader, be in the company for more than 6 months and have more than 5 years of professional experience concerning the kind of projects studied. The definition of basic conditions ensured the collection of relevant and reliable data from people with a solid knowledge about internal procedures of the company, characteristics of the business and of the portfolio´s projects.

Following the adopted criteria, the managers of all the active projects of the portfolio were interviewed as well as the manager of the business unit, totalizing 12 interviews, performed between July and August of 2018. The average time of stay in the company of the interviewees was 15,2 years. On the other hand, the average time of experience in project management was 4,8 years. The interviews were conducted inside the company and, in the end, there were 22 hours of recordings. The business unit manager, who accumulates the responsibility for the project portfolio management as well, was the first interview. Consequently, it was possible to know which projects formed the portfolio and to identify all portfolio management processes. The remaining interviews were performed randomly, according to the project managers' availability.

According to Yin (2015), triangulation of data collected from different sources allows the researcher to converge to more robust conclusions through the analysis of different historical and behavioral aspects. Following this recommendation, the main documents of the active projects, available during the interviews, were analyzed. Consequently, it was possible to deepen the analysis and stimulate the discussions with the interviewees.

The spoken language, stored from a recording, can be transformed into text through a transcription process, however, the original meaning of the speech must be preserved (Marshall & Rossman, 2006). Following this recommendation, all transcriptions were transcribed, checked and compared with the original audio to ensure the meaning preservation. Then they were sent to the interviewees for validation.

The data analysis was done with the aim of Atlas.ti. Yin (2015) recommends the use of computer aided quantitative data analysis software (CAQDAS) to code and categorize large quantities of data. Following the strategy adopted, the code’s creation in Atlas.ti was based on the researched themes.

The Content Analysis technique supported the data analysis after codification process. According to Bardin (2011), this technique is performed through the execution of 5 subsequent steps: a) analysis organization; b) data coding; c) data categorization; d) data treatment; e) results interpretation. Friese (2012) states that this text analysis method implies a recursive and iterative process of reading, coding, checking, recording, reviewing and carefully analyzing meaningful patterns, known as themes.

Specifically speaking about the results interpretation, according to Bardin (2011, p. 41), is a “[...] logical operation, by which a proposition is admitted due to its connection with other propositions already accepted as true [...]”, which means that this step is performed by inferences from the researcher.

The presence or absence of elements can transmit a meaning to the data analysis, according to Bardin (2011). The frequency which an element is repeated in a discourse depicts its importance in that context. Frequency is the most commonly used measure in Content Analysis. Another important measure is the co-occurrence index, explicating the simultaneous presence of two or more registration units in a part of the discourse. Atlas.ti automatically calculates the co-occurrence index, which can range from 0 (without co-occurrence) to 1 (both codes appear together everywhere in the hermeneutic unit).

4 Results

4.1 Bibliometric survey

The research was carried out in the ISI Web of Knowledge and Scopus databases, in October 2018. All the non-scientific articles and even the ones written in a language different from Portuguese and English were excluded from the results.

Articles related to specific categories were also eliminated from the list, as they were not aligned with the scope of this research. Concerning Scopus database, the articles in the following categories were eliminated: Earth and Planetary Sciences, Energy, Environmental Science and Mathematics. Regarding Web of Knowledge database, articles from the following categories were disqualified: Computer Science Artificial Intelligence, Computer Science Interdisciplinary Applications, Energy Fuels, Engineering Petroleum and Geosciences Multidisciplinary.

The final list resulted in 21 articles, considering the search keywords and elimination of the documents not qualified to this study. After reading all the abstracts, some articles not related to project portfolio management were also excluded from the list, culminating in 18 validated articles, which are presented in Table 1.

Table 1
Validated articles list.

Item	Title	Authors	Number of Citations		Annual Average
			WoS	Scopus	WoS	Scopus
1	The impact of project portfolio management on information technology projects	Reyck et al. (2005).		141		10,1
2	An empirical investigation on how portfolio risk management influences project portfolio success	Teller & Kock (2013)	42	45	7,0	7,5
3	Identifying, framing and managing uncertainties in project portfolios	Martinsuo et al. (2014)	31	35	6,2	7,0
4	Risk Management in Project Portfolios Is More Than Managing Project Risks: A Contingency Perspective on Risk Management	Teller et al. (2014)	27	35	5,4	7,0
5	Portfolio Risk Management and Its Contribution to Project Portfolio Success: An Investigation of Organization, Process, and Culture	Teller (2013)	27	27	4,5	4,5
6	A Project Portfolio Risk-Opportunity Identification Framework	Sanchez et al. (2008)	27		2,5
7	Managing project portfolios: balancing flexibility and structure by improvising	Jerbrant &, Karrbom Gustavsson (2013)		20		3,3
8	Management Control of Project Portfolio Uncertainty: A Managerial Role Perspective	Korhonen et al. (2014)	15	17	3,0	3,4
9	A Contingency Approach on the Impact of Front-End Success on Project Portfolio Success	Kock et al. (2016)	13		4,3
10	Business Case Control in Project Portfolios-An Empirical Investigation of Performance Consequences and Moderating Effects	Kopmann et al. (2015)	9		2,3
11	Preparedness for the Future in Project Portfolio Management: The Roles of Proactiveness, Riskiness and Willingness to Cannibalize	Rank et al. (2015)		5		1,3
12	A Maturation Model for Project-Based Organizations With Uncertainty Management as an Ever-present Multi-project Management Focus	Jerbrant (2014)	0	2	0,0	0,4
13	Integrated PPM Process: Scale Development and Validation	Padovani & Carvalho (2016)	1		0,3
14	The Status of Project Portfolio Management Practices Adoption and Awareness in Gulf Cooperation Counties: an Empirical Study	Hasna & Raza (2010)		1		0,1
15	Modeling Project Criticality in IT Project Portfolios	Neumeier et al. (2018)	0	0	0,0	0,0
16	R&D Portfolio Management Practices in Brazilian Electric Power Utilities	Yamakawa et al. (2018)	0		0,0
17	Information Technology Project Portfolio Implementation Process Optimization Based on Complex Network Theory and Entropy	Wang et al. (2017)	0		0,0
18	Project Portfolio Management Capability: The Case of Iran's Power Industry Innovation Projects	Khameneh et al. (2016)		0		0,0

• Source: Authors, from Web of Science and Scopus databases.

This bibliometric survey enabled a longitudinal analysis of the publications about this topic since 2005. According to this survey, the publication by Reyck et al. (2005) is the most cited one until this research was concluded, considered, therefore, one of the references on the topic studied, as stated by Zupic & Čater (2015), that the number of citations indicates the relevance of an article.

The publications evolution analysis over the time demonstrated an intensification after 2013, as shown in Figure 1. This corroborates with the statement by Reyck et al. (2005), Teller & Kock (2013) and Teller et al. (2014) that this topic is recent to the academy and still little explored.

Figure 1
Publication’s evolution over the time. Source: Authors, from Web of Science and Scopus databases.

The number of citations evolution, presented in Figure 2, demonstrates a significant expansion since 2013, as observed with the publications. This ramp up confirms the recent interest increase, even considering the bibliometric survey restriction of demanding some time to disclose relevant articles.

Figure 2
Citation’s evolution over the time. Source: Authors, from Web of Science and Scopus databases.

The repetition of specific words in the text indicates their relevance to the subject studied, according to Zipf's law. Figure 3 shows the predominance of some words, which are linked to different characteristics of the topics studied: 1) The project portfolio management is associated with the strategic plan execution of an organization and is performed by the portfolio manager, supported by the project managers; 2) Risks must be considered at the project and portfolio levels; 3) The risk management processes must be associated with the portfolio management ones.

Figure 3
Cloud of relevant words. Source: Authors, from Web of Science and Scopus databases.

The word repetition analysis was performed on the Introduction section of the articles, using Iramuteq software. The terms used in the search keywords (project, portfolio, management, risk, uncertainty and success) were eliminated, once they were evidenced in the preliminary analysis. The aim was to show in which contexts these topics were already studied by the academy.

Based on the Lotka's law, three authors can be considered more relevant on the portfolio risk management studies, as shown in Table 2.

Table 2
Relevant authors.

Author	Number of publications
	WoS	Scopus
Kock, Alexander	4	2
Teller, Juliane	3	3
Gemuenden, Hans Georg	3	2
Killen, Catherine P.	2
Korhonen, Tuomas	2	2
Laine, Teemu	2	2
Martinsuo, Miia	2	2
Carvalho, Marly M.	1
Jerbrant, A.		2
Calderine, S. R.		1
De Reyck, B.		1
Garizy, T.Z.		1

• Source: Authors, from Web of Science and Scopus databases.

In terms of journal’s relevance, the analysis indicated a concentration in two important periodicals, which published most of the articles about ​​portfolio risk management, as shown in Table 3. Following the Bradford´s law, this result demonstrates the importance of the theme for the project management area, since these journals are considered important in this category of subject.

Table 3
Relevant journals.

Journal	Number of publications
	WoS	Scopus
Project Management Journal	5	3
International Journal of Project Management	4	5
IEEE Transactions on Engineering Management	1
Entropy	1
Benchmarking an International Journal	1
South African Journal of Economic and Management Sciences	1	1
International Business Management		1
International Journal of Managing Projects in Business		1
International Journal of Project Organisation and Management		1
	13	12

• Source: Authors, from Web of Science and Scopus databases.

4.2 Description of the studied portfolio

The analysis unit adopted in this research is one of the project portfolios of a multinational company, established in Brazil about 100 years ago. The organization operates in the corporate and residential segments and is currently one of the top 3 largest worldwide players in its segment.

The analyzed portfolio has 13 active projects, which are managed by 11 different project managers. All project managers accumulate this function with their obligations and responsibilities for the routine activities defined by the functional structure, what means that they do not work exclusively on the projects. The management of this project portfolio began several years ago.

The strategic objectives determined by the organization to this portfolio is to achieve a growth rate of 10% per year by 2020, considering two dimensions: gross sales and contribution margin of the products.

The portfolio management cycle is performed through seven consecutive processes: 1) project candidates are defined by Sales and their alignment with the organization's strategy are verified; 2) when aligned with the corporate strategy, they are classified according to the amount of investment required; 3) once, classified, they are individually evaluated through a business case study; 4) if the study results in a reasonable financial feedback, the projects are submitted to the board to be selected and prioritized, according to the organization's strategic planning; 5) the necessary resources are defined to the selected projects; 6) but they are allocated only when the projects is started, becoming active projects; 7) the seventh process is the portfolio monitoring, which is continuously performed. However, there is a deeper review every 6 months, considering the defined objectives, when the projects are reassessed according to their performance and alignment with corporate strategy. New projects can be started, and active projects can be stopped or canceled at any time during the portfolio management cycle, if resources are available. Resource’s reallocations are also possible and frequent. Projects are continuously monitored through specific performance indicators.

The management processes of the analyzed portfolio are represented in Figure 4, adapted from the conceptual model proposed by Castro & Carvalho (2010).

Figure 4
Project portfolio management processes. Source: Adapted by the authors, from Castro & Carvalho (2010, p. 307).

4.3 Portfolio success analysis

Based on the definition of portfolio success by Cooper et al. (2002), four dimensions were considered in this study: average projects success, synergy in the use of available resources, portfolio balancing and alignment with the strategic objectives.

The success evaluation of the active projects demonstrated that only 40% of them can be considered successful. The other 60% do not reach the goals established in, at least, one of the success dimensions previously established: cost, schedule, project scope and customer satisfaction, based on the Shenhar et al. (2001) definition of project success.

The synergy in resources utilization among the projects was assessed by the Atlas.ti co-occurrence analysis of the code SP_Sinergy with all the portfolio active projects, according to Table 4. The criteria considered some synergy between projects where the co-occurrence index was greater than zero. Therefore, 75% of the projects presented some resources employment synergy, even at a low intensity.

Table 4
Synergy in resources utilization among the projects.

Projects	SP_Sinergy
Pr01_Instalações_MOD kits	0.05
Pr02_Instalações_PPG	0.02
Pr03_Instalações_FHB	0.06
Pr04_Instalações_FPY	0.05
Pr05_Instalações_SS	0.02
Pr06_Produtos_BX7	0.01
Pr07_Produtos_MB	0.00
Pr08_Produtos_SLD	0.00
Pr09_Produtos_SM	0.01
Pr10_Produtos_Redesign	0.00
Pr11_Ferramentas_SHAPE	0.06
Pr12_Vendas_SPA	0.03
Pr13_Vendas_PROMOD	0.02

• Source: Adapted by authors, from Atlas.ti co-occurrence analysis.

Regarding the strategic alignment dimension of portfolio success, a detailed analysis of the active projects scopes indicated that one of them, in fact, was a continuous improvement process. This element was excluded from the portfolio for subsequent analysis. On the other hand, another project had to be included on the portfolio analysis because it was not prioritized by the board and this fact was causing a negative impact on the strategic objectives.

According to the portfolio balance analysis, demonstrated in Table 5, it can be observed a good balance considering some project characteristics: type, source and strategic objectives. However, 50% of the projects will be finished between May and July/2019. Considering that the usual lead time between sales and effective installation is around 8 months, these projects can contribute only to the strategic objectives of 2020, provided that no further delays occur until their completion. As some of them had already delayed, it is possible to consider that there is a negative impact on the strategic plan fulfillment.

Table 5
Portfolio balance analysis.

Project ID	Type / Classification	Source	PCP Gate	Cycle time	Strategic Objectives		Due date
					C1	OR
Pr01		Local		1 year	X		june/19
Pr02		Local		1 year	X		june/19
Pr03	Mandatory	Global		1 year	X		apr/19
Pr04		Local		1 year	X		dec/18
Pr05	Mandatory	Global		2,5 years	X		mar/20
Pr06	Regulation / Strategic	Global	C6	2 years		X	june/19
Pr07	Strategic	Local	C6 / C4	2 years		X	july/19
Pr08	Strategic	Local	C6	2 years		X	mar/19
Pr09	Strategic	Local	C4	2 years		X	june/19
Pr10		Local	C4	2 years		X	may/19
Pr11	Mandatory	Global	C6	2,5 years		X	dec/19
Pr12		Local		1 year		X	dec/18
Pr13		Local		1 year		X	mar/19

• Source: Authors.

The portfolio success analysis, based on the assessed dimensions, indicated that the projects deadline, associated with the delays of some of them, can jeopardize the long-term goals. In terms of resource employment, 25% of the active projects do not demonstrate any synergy with other projects and 42% presented little synergy (co-occurrence index between 0.01 and 0.02).

Regarding the project’s performance, individually assessed, 60% of them cannot be considered successful because they do not achieve all the performance goals, considering analyzed success dimensions.

5 Result analysis: proposition of a matrix of portfolio risk management intensity

A matrix of portfolio risk management intensity was proposed to allow a more detailed data analysis. The matrix construction was based on the literature exposed in this research and aims to show how deep the risk management is performed in each one of the portfolio management processes, considering two dimensions: risk management amplitude and depth.

The quantity of risk management processes performed in each of the portfolio management processes was assumed to give the risk management amplitude perception. The risk processes considered for this analysis was those adopted by PMI (2017b) and corroborated by Teller et al. (2014): risk identification, risk evaluation, responses planning and implementation control. The scale to measure the risk management amplitude is based on the premise that the risk management processes must be executed in a logical sequence. Therefore, the following amplitude grades was established: (0) no evidence of risk management detected; (1) if the identification process has been used to obtain a list of risks; (2) whether the identified risks were analyzed; (3) when there is a formal response plan for the analyzed risks; and (4) whether the response plan was monitored, i.e., whether there is a follow-up of the actions defined and recorded in the response plan.

The challenge of amplitude measurement is to detect how many processes of risk management have been performed in each portfolio management process. For example, if only the Risk Identification process (corresponding to grade 1) was performed in all the 7 processes of a portfolio management cycle, the amplitude value is 7 (7x1). Following this logical reasoning, the maximum amplitude grade to be achieved is 28, i.e., the presence of all risk management processes (grade 4, by our premise) multiplied by the 7 portfolio management processes (7x4).

The depth was defined considering the effort allocated (dedication, hours of work) in risk management in each one of the portfolio management processes.

Figure 5 shows the proposed matrix, which aims to understand the intensity of the risk management applied in project portfolio management processes. Intensity is conceptually understood as the product of amplitude versus depth.

Figure 5
Portfolio Risk Management Intensity Matrix. Source: Authors.

According to its positioning in the matrix, a portfolio can be considered Incipient, Superficial, Deep or Mature in terms of risk management intensity.

An Incipient portfolio, represented in Figure 5 as the lower left quadrant, happens when risk management is not widely used (amplitude) and performed in a superficial way (depth), without much dedication or effort from the team.

A Superficial portfolio is represented by a shallow risk management execution, but with a wide application over the portfolio management processes. In this case there is integration of risk management in the various portfolio management processes, which tends to increase its efficiency.

The quadrant which represents a portfolio considered Deep shows the risk management processes performed with commitment and dedication of the team. However, the risk management processes are partially performed over the portfolio management cycle, in other words, the portfolio risk management is not integrated in all the processes of the portfolio management cycle.

Finally, in the Mature portfolio, risk management is deeper and more comprehensive, therefore, risks are identified, analyzed, treated and monitored in the various processes of the portfolio management cycle, increasing their chances of success.

The scales standardization in each of the matrix dimensions is necessary in order to enable a comparison analysis of different portfolios with coherence. As a starting point for the definition of the scales, a graduation from 0 to 4 was considered, applied based on the criteria adopted in each of the dimensions.

Given the definition of the risk intensity matrix described above, the analysis of the portfolio studied could be performed. Codes were created in Atlas.ti for each of the portfolio management and risk management processes. The co-occurrence analysis between these codes, based on interview transcriptions, indicated small signs of risk management in the processes of Strategic Alignment and Resource Allocation only, as shown in Table 6.

Table 6
Integration of risk management and portfolio management processes.

Processes	RM_Analysis	RM_Identificatio n	RM_Monitoring	RM_Response Plan
PPM_Strategic Alignement	0.01	0.01	0	0.01
PPM_Resource Allocation	0.01	0.01	0	0
PPM_Project individual analysis	0	0	0	0
PPM_Classification	0	0	0	0
PPM_Portfolio control	0	0	0	0
PPM_New projects identification	0	0	0	0
PPM_Projects selection and prioritization	0	0	0	0

• Source: Adapted by authors, from Atlas.ti co-occurrence analysis.

For the amplitude calculation it was considered that the Strategic Alignment process corresponded to the index (3), since the existence of a response plan for the identified risks was detected. The process Resource Allocation earned an index (2) because risk management was done until the analysis stage only.

Considering that the analyzed portfolio is managed through 7 processes (28 for maximum amplitude), the value associated to the risk management intensity was 17,8% (result of the ratio 5/28).

To estimate the risk management depth in the analyzed portfolio, the average of the co-occurrence indexes found in each of the processes was considered, as shown in Table 6, since there was no way to quantify the commitment (dedication, work) of the team in the analyzed portfolio. Considering that the co-occurrence index can vary between 0 and 1, the value of 0,01 corresponds to 1%. Therefore, the following result is obtained: ((0.01 x 5) / 5) x 100 = 1%.

Consequently, this portfolio was considered “Incipient”, as shown in Figure 6. According to the proposed definition, this portfolio has little integration of risk management in its management processes, which leaves it exposed to situations such as the existence of projects which are not aligned with the corporation´s strategic objectives and the possibility of presenting balancing or synergy problems, reducing, this way, their chances of success in executing the strategic plan. As the depth rate is also low, there is a risk that some of the projects will not succeed.

Figure 6
Portfolio positioning in the portfolio risk management intensity matrix. Source: Authors.

The statements made by the interviewees, which referred to risks of the portfolio management processes, were coded as “PPM_Risk of the processes”. The co-occurrence analysis between this code and the ones of the portfolio management processes identified the existence of risks in some of these processes, as indicated in Table 7. As there is no integration of risk management in the portfolio management processes, part of the identified risks is not managed.

Table 7
Risks on portfolio management processes.

Processes	PPM_Risk processes
PPM_Strategic Alignement	0.15
PPM_Resource Alocation	0.15
PPM_Project individual analysis	0
PPM_Classification	0
PPM_Portfolio control	0.01
PPM_New projects identification	0
PPM_Projects selection and prioritization	0.31

• Source: Adapted by authors, from Atlas.ti co-occurrence analysis.

The result of the portfolio's success analysis is therefore consistent with that expected for an Incipient portfolio. This argument supports the possibility of using the proposed matrix of portfolio risk management intensity.

To close this argument regarding the data analysis, it is worth making the considerations about the findings of this investigation.

Therefore, it was possible to notice that most of the projects of the analyzed portfolio are not successful, which is consistent with the shallow depth of risk management in portfolio control. This may not be the only cause of project failures, but risk management could help to avoid some of the problems.

The project selection and prioritization process presented the highest risk level, but the lack of management of these risks leads the organization to select wrong projects, while priority projects are not included in the list of active projects. The same applies to the resource allocation process. Projects with low synergy level are exactly the largest projects that end in the middle of 2019, giving a low contribution to the organization's strategic objectives.

Therefore, the portfolio analyzed has many problems and a low-risk management intensity. It is not possible to assert that increasing the risk management intensity would ensure the full success of the strategic plan. However, the results enable to state that not managing the portfolio risks has negative consequences for the portfolio´s result, which corroborates with Teller et al. (2014) statement that neglecting the risks identified in the project portfolio management may lead organizations to have an unbalanced portfolio, not been adequately prepared for the risks in terms of reserves, thereby compromising their future.

The proposed matrix has an analytical aspect, allowing diagnosis of how the risk management is integrated to the processes of portfolio management. Through cross-analysis with the portfolio result, it is possible to confirm the points in which there are problems. But this analysis was possible in this case only because a content analysis was made in the speeches of project managers. This practice is unfeasible in organizations' daily lives. Therefore, it is necessary to develop new artifacts to identify portfolio management vulnerabilities.

It is an easy-to-apply model that can help organizations to assess the level of risk management in their project portfolio. Knowing the portfolio management vulnerabilities helps the organizations to make the necessary customization and achieve positive results. However, it is necessary to consider that portfolio management is typically performed at a more strategic level in the organizations. Therefore, the tools and techniques to be developed to assist portfolio management need to be simple and practical, despite the complexity of the subject.

6 Conclusion

This research demonstrated that it is possible to understand how risk management influences the project portfolio success. Considering the results presented, it is possible to conclude that the risk management causes little influence on the portfolio success of the company studied. It is a portfolio with little intensity (still incipient) in terms of risk management application.

This conclusion was substantiated through the application of the intensity matrix proposed by the authors. Despite being useful to conclude this research, the matrix, as a proposed analysis instrument, needs to be validated both in academy and in practice.

From the findings of this research, it was possible to reflect on the academic and practical implications.

Regarding the academic point of view, the bibliometric survey results confirmed that portfolio risk management is a topic which has drawn attention more recently, but has been little explored yet, despite its importance. Furthermore, the combination of two possible constructs - risk management and portfolio management – can be potentially better explored in the literature. In addition, with the intensity matrix proposition new research opportunities arise. Among them, the possibility of exploring the intensity matrix in different project portfolio environments and deepening the cost and benefit relations of the risk management application in project portfolio. Topic already suggested in other studies.

In practical terms, the study brought up the possibility of using portfolio risk management in favor of better results. In this aspect it is possible to implement risk management gradually, process by process, in the portfolios of similar organizations to the one studied in this case.

Obviously, the results achieved in this research are neither generalizable nor serve as the basis for any institutional recommendation. In fact, there are important limitations that must be observed in this regard, such as the fact of being a single case study. Even though it was a deep study, it is not possible to make comparisons between cases given this restriction.

• Financial support: None.
• How to cite: Zanfelicce, R. L., & Rabechini Jr., R. (2021). The influence of risk management on the project portfolio success – proposal of a risk intensity matrix. Gestão & Produção, 28(2), e5264. https://doi.org/10.1590/1806-9649-2020v28e5264

References

• Archambault, E., Campbell, D., Gingras, Y., & Lariviere, V. (2009). Comparing bibliometric statistics obtained from the Web of Science and Scopus. Journal of the American Society for Information Science and Technology, 60(7), 1320-1326. http://dx.doi.org/10.1002/asi.21062
  » http://dx.doi.org/10.1002/asi.21062
• Archer, N., & Ghasemzadeh, F. (1999). An integrated framework for project portfolio selection. International Journal of Project Management, 17(4), 207-216. http://dx.doi.org/10.1016/S0263-7863(98)00032-5
  » http://dx.doi.org/10.1016/S0263-7863(98)00032-5
• Artto, K. A., Kahkonen, K., & Pitkanen, P. (2000). Unknown soldier revisited: a story of risk management. Project Management Association Finland Helsinki, Finland: PMA Finland.
• Bardin, L. (2011). Análise de conteúdo São Paulo: Edições 70.
• Carlos, L., Serio, D., Serio, M. D. I., & Serio, M. D. I. (2017). Uma análise bibliométrica multifontes sobre a geração de valor ao acionista e a Gestão de Projetos, Programas e Portfólios. Revista de Gestão e Projetos - GeP, 8(2), 87-108. https://doi.org/10.5585/gep.v8i2
  » https://doi.org/10.5585/gep.v8i2
• Carvalho, M. M., & Rabechini, R., Jr. (2015). Impact of risk management on project performance: the importance of soft skills. International Journal of Production Research, 53(2), 321-340. http://dx.doi.org/10.1080/00207543.2014.919423
  » http://dx.doi.org/10.1080/00207543.2014.919423
• Castro, H. G., & Carvalho, M. M. (2010). Gerenciamento do portfólio de projetos (PPM): estudos de caso. Produção, 20(3), 303-321. http://dx.doi.org/10.1590/S0103-65132010005000044
  » http://dx.doi.org/10.1590/S0103-65132010005000044
• Chapman, C., & Ward, S. (2004). Why risk efficiency is a key aspect of best practice projects. International Journal of Project Management, 22(8), 619-632. http://dx.doi.org/10.1016/j.ijproman.2004.05.001
  » http://dx.doi.org/10.1016/j.ijproman.2004.05.001
• Cooper, R., & Edgett, S. (2010). Developing a product innovation and technology strategy for your business. Research Technology Management, 53, 33-40. http://dx.doi.org/10.1080/08956308.2010.11657629
  » http://dx.doi.org/10.1080/08956308.2010.11657629
• Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (1999). New product portfolio management: practices and performance. Journal of Product Innovation Management, 16(4), 333-351. http://dx.doi.org/10.1111/1540-5885.1640333
  » http://dx.doi.org/10.1111/1540-5885.1640333
• Cooper, R., Edgett, S., & Kleinschmidt, E. (2001). Portfolio management for new products New York: Basic Books.
• Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2002). Portfolio management: fundamental to new product success. In P. Belliveau, A. Griffin & S. Somermeyer (Eds.), The PDMA toolbook for new product development New York: John Wiley & Sons.
• Courtney, H., Kirkland, J., & Viguerie, P. (1997). Strategy under uncertainty. Harvard Business Review, 75(6), 67-79.
• Reyck, B. D., Grushka-Cockayne, Y., Lockett, M., Calderini, S. R., Moura, M., & Sloper, A. (2005). The impact of project portfolio management on information technology projects. International Journal of Project Management, 23(7), 524-537. http://dx.doi.org/10.1016/j.ijproman.2005.02.003
  » http://dx.doi.org/10.1016/j.ijproman.2005.02.003
• Elonen, S., & Artto, K. A. (2003). Problems in managing internal development projects in multi-project environments. International Journal of Project Management, 21(6), 395-402. http://dx.doi.org/10.1016/S0263-7863(02)00097-2
  » http://dx.doi.org/10.1016/S0263-7863(02)00097-2
• Flanagan, R., & Norman, G. (1993). Risk management and construction Victoria, Australia: Blackwell Science Pty Ltd.
• Flyvbjerg, B. (2007). Policy and planning for large-infrastructure projects: problems, causes, cures. Environment and Planning. B, Planning & Design, 34(4), 578-597. http://dx.doi.org/10.1068/b32111
  » http://dx.doi.org/10.1068/b32111
• Friese, S. (2012). Qualitative data analysis with Atlas.ti London: SAGE Publications Ltd.
• Ghasemzadeh, F., Archer, N., & Iyogun, P. (1999). A zero-one model for project portfolio selection and scheduling. The Journal of the Operational Research Society, 50(7), 745-755. http://dx.doi.org/10.1057/palgrave.jors.2600767
  » http://dx.doi.org/10.1057/palgrave.jors.2600767
• Gil, A. C. (2011). Como elaborar projetos de pesquisa (5a. ed.). São Paulo: Atlas.
• Hasna, A., & Raza, S. (2010). The status of project portfolio management practices adoption and awareness in Gulf Cooperation Counties: an empirical study. International Journal of Project Organisation and Management, 2(2), 174-192. http://dx.doi.org/10.1504/IJPOM.2010.033661
  » http://dx.doi.org/10.1504/IJPOM.2010.033661
• Hill, W., & Jones, R. (1992). Strategic management theory: an integrated approach (2nd ed.). USA: Houghton Mifflin.
• Hofman, M., Spalek, S., & Grela, G. (2017). Shedding new light on project portfolio risk management. Sustainability, 9(10), 1798. http://dx.doi.org/10.3390/su9101798
  » http://dx.doi.org/10.3390/su9101798
• Ikpaahindi, L. (1985). An overview of bibliometrics: its measurements, laws and their applications. Libri, 35(2), 163-176.
• Jeffery, M., & Leliveld, I. (2004). Best practicies in IT Portfólio Management. MIT Sloan Management Review, 45(3), 41-49.
• Jerbrant, A. (2014). A maturation model for project-based organizations - with uncertainty management as an ever-present multi-project management focus. South African Journal of Economic and Management Sciences, 17(1), 33-51.
• Jonas, D., Kock, A., & Gemünden, H. G. (2010). The impact of portfolio management quality on project portfolio success. In Paper presented at EURAM 2010 Rom, Italy.
• Jordão, R. V. D., Pelegrini, F. G., Jordão, A. C. T., & Jeunon, E. E. (2015). Fatores críticos na gestão de projetos: um estudo de caso numa grande empresa latino-americana de classe mundial. Gestão & Produção, 22(2), 280-294. http://dx.doi.org/10.1590/0104-530X1091-13
  » http://dx.doi.org/10.1590/0104-530X1091-13
• Jerbrant, A., & Karrbom Gustavsson, T. (2013). Managing project portfolios: balancing flexibility and structure by improvising. International Journal of Managing Projects in Business, 6(1), 152-172. http://dx.doi.org/10.1108/17538371311291071.
  » https://doi.org/10.1108/17538371311291071
• Kerzner, H. (2009). Project management: a system approach to planning, scheduling, and controlling (10a ed). New Jersey: John Wiley and Sons.
• Killen, C. P., Jugdev, K., Drouin, N., & Petit, Y. (2012). Advancing project and portfolio management research: applying strategic management theories. International Journal of Project Management, 30(5), 525-538. http://dx.doi.org/10.1016/j.ijproman.2011.12.004
  » http://dx.doi.org/10.1016/j.ijproman.2011.12.004
• Khameneh, A., Sobhiyah, M. H., & Hosseini, S. H. K. (2016). Project portfolio management capability: the case of Iran’s power industry innovation projects. International Business Management, 10(12), 2329-2336.
• Kock, A., Heising, W., & Gemünden, H. G. (2016). A contingency approach on the impact of front-end success on project portfolio success. Project Management Journal, 47(2), 115-129. http://dx.doi.org/10.1002/pmj.21575
  » http://dx.doi.org/10.1002/pmj.21575
• Kopmann, J., Kock, A., Killen, C. P., & Gemünden, H. G. (2015). Business case control in project portfolios: an empirical investigation of performance consequences and moderating effects. IEEE Transactions on Engineering Management, 62(4), 529-543. http://dx.doi.org/10.1109/TEM.2015.2454437.
  » https://doi.org/10.1109/TEM.2015.2454437
• Korhonen, T., Laine, T., & Martinsuo, M. (2014). Management Control of Project Portfolio Uncertainty: A Managerial Role Perspective. Project Management Journal, 45(1), 21-37. http://dx.doi.org/10.1002/pmj.21390.
  » https://doi.org/10.1002/pmj.21390
• Lee, K. C., Lee, N., & Li, H. (2009). A particle swarm optimization-driven cognitive map approach to analyzing information systems project risk. Journal of the American Society for Information Science and Technology, 60(6), 1208-1221. http://dx.doi.org/10.1002/asi.21019
  » http://dx.doi.org/10.1002/asi.21019
• Markowitz, H. M. (1952). Portfolio selection. The Journal of Finance, 7(1), 77-91.
• Marshall, C., & Rossman, G. B. (2006). Data collection methods. In C. Marshall & G. B. Rossman. Designing qualitative research (pp. 97-141). Thousand Oaks, Califórnia, EUA: Sage publications.
• Martins, G. A., & Theóphilo, C. R. (2007). Metodologia da investigação científica para ciências sociais aplicadas São Paulo: Editora Atlas SA.
• Martinsuo, M., Korhonen, T., & Laine, T. (2014). Identifying, framing and managing uncertainties in project portfolios. International Journal of Project Management, 32(5), 732-746.
• McFarlan, F. W. (1981). Portfolio approach to information systems. Harvard Business Review, 59(5), 142-150.
• McFarlan, F. W. (1984). Information technology changes the way you compete. Harvard Business Review, 62(3), 98-103.
• McNamara, C. (2006). General guidelines for conducting research interviews Retrieved June 13, 2017, from http://managementhelp.org/businessresearch/interviews.htm
  » http://managementhelp.org/businessresearch/interviews.htm
• Meskendahl, S. (2010). The influence of business strategy on project portfolio management and its success: A conceptual framework. International Journal of Project Management, 28(8), 807-817. http://dx.doi.org/10.1016/j.ijproman.2010.06.007
  » http://dx.doi.org/10.1016/j.ijproman.2010.06.007
• Mikkola, J. H. (2001). Portfolio management of ReD projects: implications for innovation management. Technovation, 21(7), 423-435. http://dx.doi.org/10.1016/S0166-4972(00)00062-6
  » http://dx.doi.org/10.1016/S0166-4972(00)00062-6
• Moran, M. R., Souza, F. F. A., Boaventura, J. M. G., Marinho, B. L., & Fischmann, A. A. (2010). Alianças Estratégicas: uma análise bibliométrica da produção científica entre 1989 e 2008. Revista de Ciências da Administração, 12(27), 63-85. http://dx.doi.org/10.5007/2175-8077.2010v12n27p63
  » http://dx.doi.org/10.5007/2175-8077.2010v12n27p63
• Mu, S., Cheng, H., Chohr, M., & Peng, W. (2014). Assessing risk management capability of contractors in subway projects in mainland China. International Journal of Project Management, 32(3), 452-460. http://dx.doi.org/10.1016/j.ijproman.2013.08.007
  » http://dx.doi.org/10.1016/j.ijproman.2013.08.007
• Neumeier, A., Radszuwill, S., & Garizy, T. Z. (2018). Modeling project criticality in IT project portfolios. International Journal of Project Management, 36(6), 833-844.
• Olsson, R. (2008). Risk management in a multi‐project environment. International Journal of Quality & Reliability Management, 25(1), 60-71. http://dx.doi.org/10.1108/02656710810843586
  » http://dx.doi.org/10.1108/02656710810843586
• Padovani, M., & Carvalho, M. M. (2016). Integrated PPM Process: scale development and validation. International Journal of Project Management, 34(4), 627-642.
• Petit, Y. (2012). Project portfolios in dynamic environments: organizing for uncertainty. International Journal of Project Management, 30(5), 539-553. http://dx.doi.org/10.1016/j.ijproman.2011.11.007
  » http://dx.doi.org/10.1016/j.ijproman.2011.11.007
• Prasad, S., & Tata, J. (2005). Publication patterns concerning the role of teams/groups in the information systems literature from 1990 to 1999. Information & Management, 42(8), 1137-1148. http://dx.doi.org/10.1016/j.im.2005.01.003.
  » http://dx.doi.org/10.1016/j.im.2005.01.003.
• Project Management Institute. (2017a). A guide to the project management body of knowledge (PMBOK ® Guide) (6th ed.). Newtown Square, PA: PMI.
• Project Management Institute. (2017b). The standard for portfolio management (4th ed.) Newtown Square, PA: PMI.
• Rank, J., Unger, B. N., & Gemünden, H. G. (2015). Preparedness for the future in project portfolio management: the roles of proactiveness, riskiness and willingness to cannibalize. International Journal of Project Management, 33(8), 1730-1743.
• Raz, T., Shenhar, A. J., & Dvir, D. (2002). Risk management, project success, and technological uncertainty. R & D Management, 32(2), 101-109. http://dx.doi.org/10.1111/1467-9310.00243
  » http://dx.doi.org/10.1111/1467-9310.00243
• Redmill, F. (2002). Risk analysis: a subjective process. Engineering Management Journal, 12(2), 91-96.
• Sanchez, H., Robert, B., & Pellerin, R. (2008). A project portfolio risk-opportunity identification framework. Project Management Journal, 39(3), 97-109. http://dx.doi.org/10.1002/pmj.20072
  » http://dx.doi.org/10.1002/pmj.20072
• Shenhar, A. J., Dvir, D., Levy, O., & Maltz, A. C. (2001). Project success: A multidimensional strategic concept. Long Range Planning, 34(6), 699-725. http://dx.doi.org/10.1016/S0024-6301(01)00097-8
  » http://dx.doi.org/10.1016/S0024-6301(01)00097-8
• Shenhar, A. J., & Dvir, D. (2007). Why your business success depends on projects—excerpted from reinventing project management: the diamond approach to successful growth and innovation Boston, Massachusetts: Harvard Business Press.
• Small, H. (2003). Paradigms, citations, and maps of science: a personal history. Journal of the American Society for Information Science and Technology, 54(5), 394-399. http://dx.doi.org/10.1002/asi.10225
  » http://dx.doi.org/10.1002/asi.10225
• Teller, J., Unger, B. N., Kock, A., & Gemünden, H. G. (2012). Formalization of project portfolio management: the moderating role of project portfolio complexity. International Journal of Project Management, 30(5), 596-607. http://dx.doi.org/10.1016/j.ijproman.2012.01.020
  » http://dx.doi.org/10.1016/j.ijproman.2012.01.020
• Teller, J. (2013). Portfolio risk management and its contribution to project portfolio success: an investigation of organization, process and culture. Project Management Journal, 44(2), 36-51. http://dx.doi.org/10.1002/pmj.21327
  » http://dx.doi.org/10.1002/pmj.21327
• Teller, J., & Kock, A. (2013). An empirical investigation on how portfolio risk management influences project portfolio success. International Journal of Project Management, 31(6), 817-829. http://dx.doi.org/10.1016/j.ijproman.2012.11.012
  » http://dx.doi.org/10.1016/j.ijproman.2012.11.012
• Teller, J., Kock, A., & Gemunden, H. G. (2014). Risk management in project portfolios is more than managing project risks: a contingency perspective on risk management. Project Management Journal, 45(4), 67-80. http://dx.doi.org/10.1002/pmj.21431
  » http://dx.doi.org/10.1002/pmj.21431
• Wang, Q., Zeng, G., & Tu, X. (2017). Information technology project portfolio implementation process optimization based on complex network theory and entropy. Entropy (Basel, Switzerland), 19(6), 287. http://dx.doi.org/10.3390/e19060287.
  » https://doi.org/10.3390/e19060287
• Ward, S. C., & Chapman, C. B. (1991). Extending the use of risk analysis in project management. International Journal of Project Management, 9(2), 117-123. http://dx.doi.org/10.1016/0263-7863(91)90071-3
  » http://dx.doi.org/10.1016/0263-7863(91)90071-3
• Ward, S., & Chapman, C. (2003). Transforming project risk management into project uncertainty management. International Journal of Project Management, 21(2), 97-105. http://dx.doi.org/10.1016/S0263-7863(01)00080-1
  » http://dx.doi.org/10.1016/S0263-7863(01)00080-1
• Wideman, M. (1992). Project and program risk management: a guide to managing project risks and opportunities. Newton Square Pennsilvania, USA: Project Management Institute.
• Yamakawa, E. K., Sousa-Zomer, T. T., Cauchick-Miguel, P. A., & Killen, C. P. (2018). R&D portfolio management practices in Brazilian electric power utilities. Benchmarking, 25(6), 1641-1655. http://dx.doi.org/10.1108/BIJ-10-2016-0159.
  » https://doi.org/10.1108/BIJ-10-2016-0159
• Yin, R. K. (2015). Estudo de caso: planejamento e métodos (5a. ed.). Porto Alegre: Bookman.
• Zupic, I., & Čater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429-472. http://dx.doi.org/10.1177/1094428114562629
  » http://dx.doi.org/10.1177/1094428114562629
• Zwikael, O., & Ahn, M. (2011). The effectiveness of risk management: an analysis of project risk planning across industries and countries. Risk Analysis, 31(1), 25-37. http://dx.doi.org/10.1111/j.1539-6924.2010.01470.x PMid:20723146.
  » http://dx.doi.org/10.1111/j.1539-6924.2010.01470.x

Publication Dates

• Publication in this collection
  16 June 2021
• Date of issue
  2021

History

• Received
  14 Jan 2019
• Accepted
  23 Apr 2019