The problem of research project portfolio selection in educational organizations: a case study

Ribeiro, Maria Celeste de Carvalho Ressiguier; Alves, Alex da Silva

doi:10.1590/0104-530X2089-16

Acessibilidade / Reportar erro

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Articles • Gest. Prod. 24 (1) • Jan-Apr 2017 • https://doi.org/10.1590/0104-530X2089-16 copy

The problem of research project portfolio selection in educational organizations: a case study

Authorship SCIMAGO INSTITUTIONS RANKINGS

Abstract:

This paper proposes a model to support decisions in scientific-research project portfolio selection in an educational institution. We used the Analytic Hierarchy Process (AHP) method for development of this model, combining relative and absolute measurement techniques. That combination allowed us to handle a large number of projects and to measure, in an easier way, all variables in the selection. Our research procedure was a case study based on exploratory research and followed by a quantitative modeling approach. Additionally, we performed documental research and conducted non-structured interviews to better understand the research setting and institutional goals. This contributes to propose a more consistent tailoring grant-request and research-funding project selection criteria. This paper contributes to sort out research project proposals that best adhered to the organization's goals, allowing educational-organization managers to handle the problem of limited-resource allocation in the context of large number of project requests.

Keywords:
Multicriteria support to decision-making; AHP; Project portfolio management; Scientific research evaluation; Project selection

	C₁	C₂	...	C_n
C₁	a₁₁	a₁₂	...	a_1n
...	...	...	...	...
C_n	a_n1	a_n2	...	a_nn

Degree of importance	Reasons
Numerical scale	Conceptual Scale
1	The same	Both compared elements equally contribute to achieving the goal.
3	Moderate	The compared element is slightly more important than the other.
5	Strong	Experience and the judgment strongly favors this element against the other.
7	Very Strong	The element being compared is much stronger against the element in the comparison and this strength can be observed in practice.
9	Absolute	The compared element has the highest level of evidence possible on its side.
2,4,6,8	Intermediate values between both judgments, used when the decision maker is having a hard time to decide between two neighboring degrees of importance.

	C₁	C₂	...	C_n	PLs
C₁	w₁₁= a₁₁ / t₁	w₁₂=a₁₂ / t₂	...	w_1n=a_1n / t_n	w₁=(w₁₁+...+w_1n)/ n
C₂	w₂₁=a₂₁ / t₁	w₂₂ =a₂₂ / t₂	...	w_2n=a_2n / t_n	w₂=(w₂₁+...+w_2n)/ n
...	...	...	...	...	...
C_n	w_n1=a_n1 / t₁	w_n2 = a_n2 / t₂	...	w_nn=a_nn / t_n	w_n=(w_n1+...+w_nn)/ n

n	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
RI	0.00	0.00	0.58	0.90	1.12	1.24	1.32	1.41	1.45	1.49	1.51	1.48	1.56	1.57	1.59

Major goal (Obj#1)	Criterion	Aspect	Code
Research is committed to local and state development	Scientific and Technological Advance	Provides new scientific/technological knowledge (new laws, theories, concepts, models) or a new approach relying on previous knowledge.	Obj#1_1
		Provides the creation and/or improvement of methods, processes, materials and/or services potentially applicable to the economic, political and/or social sectors.	Obj#1_2
		Encourages the acquiring of research skills by the young scientist grantees, technical staff, college-degree staff, master degree and Ph.D. students, Masters' degree holders, Ph.D. degree holders and Post-Ph. D. degree holders.	Obj#1_3
	Social commitment	Encourages new job opportunities, opportunities for traineeship, training courses, products and services, develops professional skills of employees and students and contributes to the critical awareness of citizens.	Obj#1_4
	Social commitment	Cooperates direct and indirectly to sustainable development, adopting and/or encouraging sustainable practices from an environmental, economic and social perspective.	Obj#1_5
	Links to education, research and outbound courses	Plans to tie up the teaching, research and outbound courses activities at IFFluminense.	Obj#1_6
	Links to education, research and outbound courses	Plans to review the application of knowledge resulting from the research to solve a social problem.	Obj#1_7
	Links to the research sectors at IFF	The project's research field corresponds to one of the priority research themes at IFFluminense.	Obj#1_8
	Links to the research sectors at IFF	The theme being researched matches the lines of research in the research hub to which it is associated.	Obj#1_9

Major goal (Obj#2)	Criterion	Aspect	Code
Evaluation of the scientific research	Originality	This is a new theme that debates a new approach to an existing topic.	Obj#2_1
	Technical quality of project	Problem is clearly stated	Obj#2_2
		Methodological matching	Obj#2_3
		Theoretical foundation	Obj#2_4
		Procedures to publicize and use the findings by society	Obj#2_5
	Quality wording and text organization	Spelling	Obj#2_6
		Grammar use	Obj#2_7
		Clarity	Obj#2_8
		Objectivity	Obj#2_9
		Formal Structure	Obj#2_10
	Grantees' plan of activities matches the goals and schedule of the project	Project's activities are aligned with its goals	Obj#2_11
		Students' activities match the schedule of the project	Obj#2_12
	Feasibility of carrying out the project	Availability of material resources	Obj#2_13
		Financial support from another funding entity	Obj#2_14
		Sizing-up of schedule	Obj#2_15
	Scientific and technological productivity in the thematic area of the project, as stated in the researcher's Lattes CV	Titles of the researcher	Obj#2_16
		Publication (article in indexed journals, book or special edition, book chapter) or filed patent.	Obj#2_17
		Publication in congress/events proceedings	Obj#2_18
		Advisor in Young Scientist Programs or in dissertations (undergraduation or minor master's)	Obj#2_19
		Advisor in Master's or Ph.D. degrees	Obj#2_20
		Participation in benches (graduation dissertations, Master's Degree or Ph. D. dissertations)	Obj#2_21

Major goal: Research is committed to state and local development (RC = 0.00)
Criterion / Aspects		Weight	Item under evaluation		Weight of the answer options
Technological and Scientific Advances Weight = 0.3507 (RC=0.03)
Obj#1_1 (RC=0.00)		0.1992	1.1	very much		fairly much	little	not at all
				0.5375		0.3027	0.1055	0.0543
Obj#1_2 (RC=0.00)		0.0344	1.2	very much		fairly much	little	not at all
				0.5375		0.3027	0.1055	0.0543
Obj#1_3 (RC=0.00)		0.1171	1.3	very much		fairly much	little	not at all
				0.5375		0.3027	0.1055	0.0543
Social commitment weight = 0.3507 (RC=0.00)
Obj#1_4 (RC=0.00)		0.2630	2.1	very much		fairly much	little	not at all
				0.5375		0.3027	0.1055	0.0543
Obj#1_5 (RC=0.00)		0.0877	2.2	very much		fairly much	little	not at all
				0.5375		0.3027	0.1055	0.0543
Linked to the Education, Research and Outbound Courses weight = 0.1892 (RC=0.00)
Obj#1_6 (RC=0.00)		0.1419	3.1	very much		fairly much	little	not at all
				0.5375		0.3027	0.1055	0.0543
Obj#1_7 (RC=0.00)		0.0473	3.2	very much		fairly much	little	not at all
				0.5375		0.3027	0.1055	0.0543
Linked to the lines of research at IFF weight=0.1093 (RC=0.00)
Obj#1_8 (RC=0.00)		0.0364	4.1	Yes		No
				0.9000		0.1000
Obj#1_9 (RC=0.00)		0.0729	4.2	High		Average	Low	None
				0.5375		0.3027	0.1055	0.0543
Total	1.0000

Major goal: evaluation of the scientific research (RC=0.01)
Criterion/aspects	Weight	Item under evaluation	Weights of the answer options
Originality of the research Weight = 0.0580
Obj#2_1 (RC=0.00)	0.0580	1.1	yes	partially	No
			0.7606	0.1577	0.0817
Technical quality of the project weight = 0.2204 (RC=0.00)
Obj#2_2 (RC=0.00)	0.0773	2.1	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_3 (RC=0.00)	0.0773	2.2	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_4 (RC=0.00)	0.0417	2.3	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_5 (RC=0.00)	0.0241	2.4	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Quality of wording and text organization weight = 0.0800 (RC=0.01)
Obj#2_6 (RC=0.00)	0.0114	3.1	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_7 (RC=0.00)	0.0114	3.2	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_8 (RC=0.00)	0.0314	3.3	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_9(RC=0.00)	0.0186	3.4	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_10 (RC=0.00)	0.0071	3.5	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Plan of activities for grantees matches the goals/schedule of the project. Weight=0.2106 (RC=0.00)
Obj#2_11(RC=0.00)	0.1580	4.1	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Obj#2_12 (RC=0.00)	0.0527	4.2	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Feasibility to carry out the project Weight= 0.2106 (RC=0.01)
Obj#2_13 (RC=0.00)	0.1135	5.1	High	Medium	Low
			0.7606	0.1577	0.0817
Obj#2_14 (RC=0.00)	0.0345	5.2	High	Enough	Little	None
			0.5375	0.3027	0.1055	0.0543
Obj#2_15(RC=0.00)	0.0626	5.3	Great	Good	Regular	Awful
			0.5375	0.3027	0.1055	0.0543
Scientific and technological productivity of the researcher in the thematic area the project belongs to, based on the researcher's Lattes CV Weight=0.2204 (RC=0.05)
Obj#2_16(RC=0.03)	0.0264	6.1	Post Doctor Degree	Ph.D.	Master's Degree	Minor Master's	Undergraduation
			0.4867	0.2720	0.1370	0.0661	0.0382
Obj#2_17 (RC=0.03)	0.0982	6.2	great	good	Regular	low	None
			0.4867	0.2720	0.1370	0.0661	0.0382
Obj#2_18 (RC=0.03)	0.0084	6.3	great	good	Regular	low	None
			0.4867	0.2720	0.1370	0.0661	0.0382
Obj#2_19 (RC=0.03)	0.0311	6.4	great	good	Regular	low	None
			0.4867	0.2720	0.1370	0.0661	0.0382
Obj#2_20 (RC=0.03)	0.0480	6.5	great	good	Regular	low	None
			0.4867	0.2720	0.1370	0.0661	0.0382
Obj#2_21 (RC=0.03)	0.0084	6.6	great	good	Regular	low	None
			0.4867	0.2720	0.1370	0.0661	0.0382
Total 1.0000

Obj#1: alignment with institutional goals (first ranking)			Obj#2: fulfillment of the scientific research criteria (final ranking)
Project	Score	Ranking	Project	Score	Ranking
3	0.550684	1st	49	0.511931	1st
49	0.550684	2nd	5	0.509791	2nd
2	0.523192	3rd	7	0.481797	3rd
30	0.480854	4th	6	0.447083	4th
...	...	...	...	...	...
12	0.164049	58th	38	0.163276	51th
53	0.131822	59th	25	0.150343	52th
10	0.130795	60th	23	0.129835	53th
59	0.112414	61th	61	0.114163	54th

Universidade Federal de São Carlos Departamento de Engenharia de Produção , Caixa Postal 676 , 13.565-905 São Carlos SP Brazil, Tel.: +55 16 3351 8471 - São Carlos - SP - Brazil
E-mail: gp@dep.ufscar.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Source: Adapted from Vargas (2010)Vargas, R. V. (2010). Utilizando a programação multicritério (Analytic Hierarchy Process – AHP) para selecionar e priorizar projetos na gestão de portfólio. In PMI Global Congress 2010. Washington. Recuperado em 12 de janeiro de 2016, de http://www.ricardo-vargas.com/articles/analytic-hierarchy-process/#portuguese
http://www.ricardo-vargas.com/articles/a... .