A process management benchmarking model for higher education institutions

Um modelo de benchmarking de gestão de processos para instituições de ensino superior

ABSTRACT

Purpose:

This research presents a multicriteria benchmarking participatory model able to classify Federal Higher Education Institutions (FHEIs) into three levels of process management maturity.

Design/methodology/approach:

The research was conducted into three stages: (i) assessment model development using the Delphi technique, (ii) data collection through a self-assessment process, and (iii) classification of the FHEIs using PROMSORT.

Findings:

The results showed that, regardless of the adoption of an optimistic or pessimistic perspective, most FHEIs (51.6% in the optimistic perspective and 54.8% in the pessimistic one) were classified as regular. It is also noteworthy that approximately 80% of the research participating FHEIs maintained their classifications in the sensitivity analysis. Among the six alternatives that presented classification variations, only three varied significantly, confirming the results obtained stability.

Research limitations/implications:

The use of a participatory approach promotes a consistent benchmark in terms of indicators and metrics to measure performances.

Practical implications:

PROMSORT provided flexibility to the model, since it is possible to modify the parameters and thresholds in order to adjust the model strictness.

Originality/value:

The development of a model through which the Federal Higher Education Institutions (FHEIs) can be continually evaluate their process management maturity level.

Keywords:
PROMSORT; Delphi technique; Multi-criteria Decision Aid (MCDA); Participatory approach; Public administration

RESUMO

Objetivo:

Esta pesquisa apresenta um modelo participativo de benchmarking multicritério capaz de classificar as Instituições Federais de Ensino Superior (IFES) em três níveis de maturidade de gestão de processos.

Desenho/metodologia/abordagem:

A pesquisa foi conduzida em três estágios: (i) construção do modelo de avaliação utilizando a técnica Delphi, (ii) coleta de dados através de um processo de autoavaliação, e (iii) classificação das IFES utilizando o PROMSORT.

Resultados:

Os resultados mostraram que, independentemente da adoção de uma perspectiva otimista ou pessimista, a maioria das IFES (51,6% na perspectiva otimista e 54,8% na pessimista) foram classificadas como regulares. Também é notável que aproximadamente 80% das IFES participantes da pesquisa mantiveram suas classificações na análise de sensibilidade. Entre as seis alternativas que apresentaram variações de classificação, apenas três variaram significativamente, confirmando os resultados obtidos em termos de estabilidade.

Limitações/implicações da pesquisa:

O uso de uma abordagem participativa promove uma referência consistente em termos de indicadores e métricas para medir o desempenho.

Implicações práticas:

O PROMSORT proporcionou flexibilidade ao modelo, uma vez que é possível modificar os parâmetros e limiares a fim de ajustar o rigor do modelo.

Originalidade/valor:

Desenvolvimento de um modelo através do qual as Instituições Federais de Ensino Superior (IFES) podem avaliar continuamente seu nível de maturidade em gestão de processos.

Palavras-chave:
PROMSORT; Técnica Delphi; Auxílio à decisão multicritério (MCDA); Abordagem participativa; Administração pública

1 INTRODUCTION

The Federal Higher Education Institutions (FHEIs), as well as other organizations, work with scarce resources, requiring an efficient management of those. From this perspective, the Business Process Management (BPM) can contribute to a better performance of this type of organization, so that it provides a service that meets the citizen's expectations (Carvalho & Sousa, 2017).

The same authors understand that the inefficiency of these organizations, represented by administrative slowness and services that do not meet the user's needs, is due to the adoption of a management model with a rigid and hierarchical structure, focused on rules, without analyzing the process. Thus, process management can be a tool to overcome these challenges, given its intrinsic characteristic of organizing processes focused on meeting the user's demands, making these institutions more agile and flexible in fulfilling their social function (Carvalho & Sousa, 2017).

According to Fettke, Zwicker & Loos (2015), the adoption of BPM as a management model is a key element for the transformation of public administration in service-oriented organizations, providing a more efficient and effective service to society. Paiva et al. (2017) conclude that the adoption of process management in public institutions has generated positive effects, able to reduce bureaucracy in work methods and improve the quality of service to internal and external customers of the institution.

The Brazilian Ministry of Planning has as one of its competencies the coordination of the actions of the National Program for Public Management and Debureaucratization - GesPública, established by Decree 5.378, of February 23, 2005. Among the methods and solutions offered by the GesPública Program is the Process Management (Brazil, 2011).

The need for development of GesPública originated from the realization that the various initiatives conducted in the government related to BPM lack integration, making it difficult or impossible to share results between institutions. As consequences for the successful implementation of priority initiatives of the Brazilian Public Ministry, there is the Decree 6932/2009, created to simplify the citizen related services and align processes and data (Brazil, 2011).

Ordinance No. 321 of November 30, 2015 of the Brazilian Federal Court of Accounts (TCU) guides the FHEIs in their management reports to adopt process management mechanisms such as the identification of the finalistic macro processes, main activities, products/services, responsible units, inputs/suppliers, and main customers and partners.

Considering all the implementation challenges and the importance of process management in HEIs that have been highlighted by Brazilian control agencies, this study presents a model to assess the maturity of process management in this kind of institutions, reducing the subjectivity and filling a research gap in a universe of descriptive works by introducing the following question: is it possible to assess and classify the maturity of process management in HEIs through a participatory approach?

Hence, this research proposes a multi-criteria participatory benchmarking model to classify Higher Education Institutions as to their level of maturity in process management application. It has practical implications for HEIs, since it has the potential to subsidize the creation of benchmarking platforms shared among institutions that can contribute to identify performance gaps for improvement in different areas and foster a culture of continuous improvement.

The following sections describe the literature review, with issues related to BPM in educational institutions and benchmarking, followed by methodology and results, articulating our findings with theory. Finally, the last section presents the conclusions of this research.

2 THEORETICAL REFERENCE

The requirements established by the control agencies regarding the management of processes in the FHEIs is a recent phenomenon, which resulted in several researches focusing on this theme. Dantas et al. (2009) worked on the perspective of process mapping in a unit of a public agency. Mückenberger et al. (2013) analyzed the applicability of BPM in the management of a process in the internationalization sector of a Higher Education Institution (HEI). Lorena (2015) analyzed how ombudsmen could assist process management in an HEI. Branco (2016) developed a framework that serves as a guide for the construction of process architecture in an FHEI, a fundamental element for the implementation of BPM. Costa & Moreira (2018) mapped and analyzed the processes of the personnel administration division of an FHEI in the Brazilian Northeast. Figueiredo et al. (2017) present the experience of the implementation of a process office in an FHEI, concluding that process management is the instrument to accomplish process management in an organization.

In addition, Barbalho et al. (2017) conducted an action research through process mapping for process improvement in a lean perspective. Costa & Moreira (2018) identified that the implementation of management methodologies and process mapping in the personnel administration directorate of a university in the Brazilian Northeast brought agility and standardization to procedures, reduction of errors, improvements in interpersonal communication and direct impact on service delivery to the user. Aganette, Maculan & Lima (2018) implemented process management at the School of Science and Information at the Federal University of Minas Gerais. Yet, Barreto, Vasconcelos & Marques (2019) implemented process management in the budget division of an FHEI, proposing a model and improvements in the processes, allowing this sector and subsidizing the planning and budget execution of the institution.

Bührig, Schoorman & Kanckstedt (2018) research revealed that implementing BPM across three campuses of a German university led to processes standardization improvement across campus locations, adoption of best practices, strengthened university’s overall team spirit, reduced resistance to using process models and documentation and the development of a BPM-supportive culture. In addition to that, Sujanawati, ER & Wibowo (2021) found that a BPM implementation is a key element for good data governance and consequently a good quality of the data that is obtained through processes in a HEI environment.

In order to provide a national panorama on process management in the FHEIs, Andrade, Rasoto & Carvalho (2018) characterized institutions regarding initiatives in process management. Since the mentioned research is an analysis of the FHEIs in a given time frame, the authors found the need to evaluate the evolution of process management maturity in this type of institution. Similarly, Matos, Forte, and Forte (2020) used a multicriteria approach to evaluate organizational management in corporate universities based on the knowledge of Brazilian experts and contributed to reducing the gap between theory and practice, however both researches are fundamentally descriptive.

Furthermore, Szelągowski & Berniak-Woźny (2022) proposed a two-stage organizational process maturity assessment model that uses a system of 5 levels of maturity measured in 6 main areas of BPM and set as a practical future research need the development of a detailed methodology and creation of a constantly updated tool for BPM maturity assessment for conducting BPM development in an organization. In their research, Opletalova & Tučekc (2017) defined four levels of maturity to determine BPM maturity of Czech universities. The results of the survey provided insight regarding the process management of Czech HEIs, especially to the process maturity level and character of process ownership and revealed that institutions which have implemented and developed their BPM governance system have better results in BPM maturity. However, the survey is limited by the size of the sample which does not allow generalizing the findings.

Although there are some researches evaluating the maturity level of these institutions in terms of process management, considering the limitations of these studies in terms of measurability and sample size, it is still necessary a reference model through which the HEIs can be continuously evaluated, identifying the reference institutions in this management model. In this direction, Santos, Santana & Alves (2012) emphasize the need of good practices guides in process management, a gap reinforced by Paiva et al. (2017), who found the need for the identification of similarities and differences in the use of process management in the various institutions.

Also, latest research remarked that one of the most important aspects of university development is to teach staff and university employees to share knowledge and experience through various projects and programs that promote the improvement and discovery of new approaches and methods of both teaching and conducting scientific activities (Taskymbayeva, Shaikh & Salimbayeva, 2022). For that purpose, benchmarking is a useful approach to attain this need, since it is a tool that seeks the best practices to achieve superior performance (Camp, 1994). The use of benchmarking can be understood as a way to improve the performance and competitiveness of public and private organizations of various sectors and sizes, being characterized as a tool increasingly used in decision making (Albertin, Kohl & Elias, 2015).

3 METHODS

Figure 1 presents the method stages for developing the classification model of higher education institutions according to their maturity level in process management. The methodology is organized in three stages: (i) construction of the assessment model, (ii) data collection, and (iii) classification of the HEIs as to their level of maturity in process management.

Figure 1
Method structure for classifying higher education institutions according to their maturity level in process management

In step 1, the criteria used in the evaluation model of the maturity level in process management of HEIs were established. To this end, the Delphi technique (Oliveira et al., 2008) was used. Systematized and anonymous interviews were conducted with a set of seven experts in the area of process management of the HEIs, and the criteria for evaluating the performance of the institutions were defined starting from a preliminary list of criteria and suggesting the inclusion of others. Among the consulted experts, two have Doctoral degrees, three have Master's degrees, and two have Specialist degrees, with an average of approximately five years of experience in process management.

Based on suggestions obtained in the first round of interaction, a second Delphi round was conducted to refine and identify, amongst the criteria suggested in the previous stage, those that should be included in the model for assessing the level of maturity in process management at an HEI. The need for a new round occurs when new criteria emerge in the immediately preceding round.

Then, the preference elicitation process was carried out to define the weights of each criterion (relative importance) through a meeting via Google Meet video conference platform mediated by the researcher. In this phase, the experts were invited to discuss and express their opinions regarding the relative importance of the criteria obtained in the previous phase. The Swing Weights (Olson, 1996) method was used for this purpose.

Initially, a hypothetical alternative with the worst performance was designated, and then the experts chose a criterion to be prioritized so that this alternative would be improved. The criterion chosen was considered the most important. Next, the experts define the second most important criterion. This procedure is used for all the other criteria, and a ranking of importance is obtained. In the sequence, the experts rank the criteria in a vector with a scale of 0 to 100 points (Olson, 1996). Finally, a normalization procedure of the weights is performed using Equation 1.

$\frac{W_j}{\sum _{i=1}^n{W}_j}$ (1)

Where:

jj: set of criteria

w_j,w_j: weight used to represent the importance of each criterion jj

nn: number of criteria

In the second step, based on the criteria established in the previous step, a self-assessment model was developed so that the institutions could self-assess their performance in each criterion. In order to reduce the subjectivity involved in the data collection process, descriptors were established. They were built with the help of a process management expert and correspond to ordinal classification scales that will be chosen in the self-assessment by the HEIs, in order to represent the situation that best fits their reality in each criterion. Data collection was conducted through e-mail from the HEIs throughout Brazil.

In the third step, the classification of the HEIs was performed using the PROMSORT (PROMETHEE Sorting) method, an extension of the PROMETHEE (Preference Ranking Organization Method for Enrichment Evaluation) family. PROMETHEE is a method that is based on performing binary comparisons, by pairs, generating outranking relations. The ranking of the alternatives is established from the balance of the net flow in the criteria, the difference between the positive flow (overcoming strength of an alternative in relation to another) and the negative flow (weakness of an alternative to be overcome in relation to another) (Araz & Ozkarahan, 2007; Brans & Mareschal, 2005).

Carmo et al. (2011) understand there are many multicriteria techniques that can be used for ranking/classifying/selecting alternatives and anyone can be considered the best. Carmo et al. (2020) identify that this approach aims to identify compromise solutions according decision-makers (Carmo et al., 2020; Do Carmo et al., 2021). As such, it was chosen a method from the PROMETHEE family, given the following intrinsic characteristics of this family of methods: i) non-compensatory characteristics; ii) freedom of choice of approach for defining the importance of criteria; iii) reduced cognitive effort made by decision makers (Behzadian, Kazemzadeh, Albadvi & Aghdasi, 2010).

This way, to determine the flows, the weights of the criteria from the first step were used, and an automated spreadsheet was developed with programmed functionalities using the JavaScript programming language to obtain the flows. Table 1 presents the equations for calculating the degree of preference and flows.

Table 1
Preference degree, positive flow, negative flow and net flow equations

Preference degree	$\pi \left(a,b\right)=\sum _{j=1}^k{P}_j\left(a,b\right)W_j$
	$\pi \left(b,a\right)=\sum _{j=1}^k{P}_j\left(b,a\right)W_j$
Positive flow	${\varphi }^{+}\left(a\right)=\frac{1}{n-1}\sum _{x\in A}\pi \left(a,x\right)$
Negative flow	${\varphi }^{-}\left(a\right)=\frac{1}{n-1}\sum _{x\in A}\pi \left(x,a\right)$
Net flow	$\varphi \left(a\right)={\varphi }^{+}\left(a\right)-{\varphi }^{-}\left(a\right)$

• Source: Adapted from Brans and Mareschal (2005)
• Where:
• Pj (a, b)P j (a, b): Preference of alternative a over alternative b
• kk: number of criteria
• nn: number of alternatives
• AA: set of alternatives

Using the flows generated as input data, the phase of the classification of alternatives was initiated through the bh thresholds of preference, indifference and incomparability of PROMSORT (Gonçalo & Alencar, 2014). The bh is the threshold established for ranking the alternatives through the relations of their flows. Table 2 presents the equations used for the classification of the FHEIs.

Table 2
Equations for determining PROMSORT method’s bh classification limits

a is preferred to bh if:	$$\begin{gathered} {\varphi }^{+}\left(a\right)>{\varphi }^{+}\left(bh\right)and{\varphi }^{-}\left(a\right)<{\varphi }^{-}\left(bh\right),or \\ {\varphi }^{+}\left(a\right)={\varphi }^{+}\left(bh\right)and{\varphi }^{-}\left(a\right)<{\varphi }^{-}\left(bh\right),or \\ {\varphi }^{+}\left(a\right)>{\varphi }^{+}\left(bh\right)and{\varphi }^{-}\left(a\right)={\varphi }^{-}\left(bh\right) \end{gathered}$$
a is indifferent to bh if:	${\varphi }^{+}\left(a\right)={\varphi }^{+}\left(bh\right)and{\varphi }^{-}\left(a\right)={\varphi }^{-}\left(bh\right)$
a is incomparable to bh if:	$$\begin{gathered} {\varphi }^{+}\left(a\right)>{\varphi }^{+}\left(bh\right)and{\varphi }^{-}\left(a\right)>{\varphi }^{-}\left(bh\right),or \\ {\varphi }^{+}\left(a\right)<{\varphi }^{+}\left(bh\right)and{\varphi }^{-}\left(a\right)<{\varphi }^{-}\left(bh\right) \end{gathered}$$

• Source: Adapted from Araz and Ozkarahan (2007)

Given that bh bounds assign an alternative to a category only if it exhibits a strictly preferable or non-preferable preference relationship (Araz & Ozkarahan, 2007), the second stage of the classification deals with the incomparability or indifference relationships generated from the alternatives that could not be classified directly using bh bounds.

Table 3
Equations for treating the alternatives not classified by the bh classification limits

Outranking character	$d_k^{+}=\sum _{x\in X_t}\left(\varphi \left(a\right)-\varphi \left(x\right)\right)$
Outranked character	$d_k^{-}=\sum _{x\in X_{t+1}}\left(\varphi \left(x\right)-\varphi \left(a\right)\right)$
Distance function	$d_k=\frac{1}{n_t}{d}_k^{+}-\frac{1}{n_{t+1}}{d}_k^{-}$
Comparative relations for classification	$$\begin{gathered} \mathrm{If}{d}_k>ba\in C_{t+1} \\ \mathrm{If}{d}_k<ba\in C_t \end{gathered}$$

• Source: Adapted from Araz and Ozkarahan (2007)
• Where:
• dk+dk+: measures the outranking character of alternative a over all alternatives assigned to category CtCt
• dk-dk-: measures the outranked character of alternative a over all alternatives assigned to category Ct+1Ct+1
• XtXt: reference set of alternatives for category tt
• ntnt: number of reference alternatives of category CtCt

To this end, alternatives not yet classified were treated as from the comparison between the function dk, which represents the relative distance of the alternative not assigned to the other alternatives already assigned to a category, and a cut-off point b established by the analyst who conducts the study (Araz & Ozkarahan, 2007). In this research two points of view were employed, an optimistic, in which b assumes the value 0, and a pessimistic, in which b assumes the value 1. This means that, under the optimistic perspective, all unassigned alternatives will be allocated to the best possible class for their respective case. In the pessimistic case, they will be assigned to the worst possible class for their specific case. Table 3 presents the equations used in this classification.

Three sets were established to classify the institutions: a) mature institutions in process management, b) regular institutions in process management and, c) immature institutions in process management. Two classification bh thresholds were adopted, a mild one (which adopts the worst performance in criteria that have two descriptors and intermediate performance in criteria with three descriptors) and a restrictive one (which adopts the best performance in criteria that have two descriptors and intermediate performance in criteria with three descriptors).

Finally, the sensitivity analysis was performed through the simulation of scenarios established from the variation of the criteria weights (+20% and -20%) and the points of view adopted for the classification (Optimistic and Pessimistic), resulting in a total of 46 simulations. Thus, for the optimistic perspective, 23 scenarios were analyzed: 1 scenario without weight variation, 11 with weight variation of +20% and 11 with weight variation of -20% in each criterion. For the pessimistic perspective, another 23 scenarios were analyzed: 1 scenario without weight variation, 11 with weight variation of +20% and 11 with weight variation of -20% in each criterion.

4 RESULTS

In this section the results will be presented and discussed. For better understanding, it was divided into three parts. The first one will present the evaluation model (criteria definition and preferences’ elicitation) development. The second will present the FHEIs performance data collection process through a self-assessment questionnaire. Finally, the third part will show the FHEIs classification in three levels of process management maturity through the PROMSORT method.

4.1 Building the evaluation model: defining criteria and preferences’ elicitation

The definition of the criteria adopted in the classification process was carried out through the Delphi technique with the seven experts who composed the model construction stage. The profile of these experts is presented in Table 4.

Table 4
Profile of the experts who participated in the definition of the evaluation criteria for the FHEIs

Expert	Educational degree	Position	BPM experience (years)
E1	Specialist Degree	Accountant	3
E2	Master's Degree	I.T. Analyst	11
E3	Master's Degree	Production Engineer	10
E4	Specialist Degree	Manager	3
E5	PhD	I.T. Analyst	5
E6	Master's Degree	Process and Risk Coordinator	2
E7	PhD	I.T. Analyst	2

In the first round of Delphi interaction, nine criteria were previously proposed: (i) Business process management office level of formalization (OLF), (ii) Methodology for process mapping level of standardization (MLS), (iii) Process automation level (PAL), (iv) Performance measurement capability (PMC), (v) Process updateability (PUA), (vi) Management tools technology level (MTL), (vii) Level of strategic alignment - IDP and value chain (LSA), (viii) Process transparency level - repository adoption (PTL) and (ix) Process culture level (PCL).

Of the proposed criteria, eight were approved: MLS, PMC, PUA, LSA and PTL with 100% consensus and OLF, PAL and PCL with 71.4% consensus. The MTL criterion was disapproved for obtaining only 28.6%, below the minimum required to be inserted in the list. As established by the Delphi technique, alternatives that obtain less than 60% agreement will be excluded (Santos, 2001).

At the end of this round, four new criteria were suggested by the experts: (x) Process Transformation Effectiveness level (TEL), (xi) Level of alignment with the institution's risk management (LAR), (xii) Institution Coverage Capacity (ICC) and (xiii) Upper Management’s Commitment level (MCL). In the second round of Delphi interaction, the CCI criterion was disapproved, having obtained 57.1% consensus, and the others were approved with 85.7% consensus. The approved criteria are shown in Figure 2.

Figure 2
Evaluation criteria defined using the Delphi technique

Considering a consensus perspective, the seven experts were invited to a virtual meeting mediated by the researcher, where they had to agree on the importance ranking of the criteria identified by Delphi and their allocation on a 0 - 100 points scale, generating the weights through the normalization procedure, as provided in the Swing Weights method, obtaining the criteria weights (Figure 3).

Figure 3
Relative importance of the evaluation criteria of the FHEI's defined using the swing weights method through the experts’ opinion

During the preference elicitation process, it was observed that the experts found it easy to reach a common agreement for the OLF, TEL and MCL criteria. For the other criteria (MLS, PAL, PMC, PUA, LSA, PTL, PCL and LAR), the need for further debate was observed, at which point they began to try to influence each other's judgments, presenting personal experiences and opinions.

Among these criteria, the MCL and TEL were classified respectively as first and second most important. According to the specialists, the MCL was judged as the most important because the support from top management is considered fundamental for the success of any initiative related to process management, that is, this criterion was also perceived as a determinant for good performance in the other criteria.

On the other hand, the OLF criterion obtained the lowest importance, under the argument that process-oriented initiatives that have support and involvement from the leaders are more effective for the success in process management than a formalized initiative that lacks such support. TEL, however, according to the experts, ranked second because of its effectiveness in generating improvements, adding value to processes and contributing to conquer or maintain the support of top management.

For positions 3 to 10, the criteria were classified in pairs due to their interdependence, which generated debate among the experts during the meeting. The third position was occupied by the LSA criterion, given its importance in prioritizing process improvements in the specialists' view. Next, in fourth position, was ranked the LAR due to its great proximity with the previous criterion. In the fifth and sixth positions are respectively the MLS and the PCL, where the MLS presented greater relevance because it is considered that a standardization is necessary to support a good processes culture. The PMC and PUA criteria were assigned to the seventh and eighth position, respectively, for considering that process updating should be based on performance indicators. Thus, the experts established equivalent weights for these two criteria. Finally, PTL was classified in the ninth position and then PAL, where transparency stood out in the order of importance due to its relevance for the integration of processes in the institution, an important characteristic in the context of process automation.

4.2 Self-assessment of the FHEIs’ performances

For each of the established criteria, descriptors were built (statements for the FHEIs to choose those that best represented them), considering the self-assessment perspective. Figure 4 presents the descriptors for each of the criteria defined by the Delphi technique.

Figure 4
Descriptors of the criteria established for the self-assessment of the FHEI’s

Legend: OLF - Business process management office level of formalization; MLS - Methodology for process mapping Level of standardization; PAL - Process automation level; PMC - Performance measurement capability; PUA - Process updateability; LSA - Level of strategic alignment - IDP and value chain; PTL - Process transparency level - repository adoption; PCL - Process Culture level; LAR - Level of alignment with the institution's risk management; MCL - Upper Management’s Commitment level; TEL - Process Transformation Effectiveness level

To collect data, exploratory research was carried out on the official websites of the FHEIs in order to gather as much contact information from the sectors responsible for process management in these institutions as possible. This research was done in an attempt to maximize the alternatives involved in the model. Furthermore, the information resulting from this research was added to a contact list provided by the office of processes of the Universidade Federal Rural do Semi-Árido (UFERSA) and formed a contact base referring to 44 FHEIs. With this, all the institutions in this contact base were contacted via email so that they could answer the self-assessment performance questionnaire. The responding FHEIs were automatically included as alternatives in the model.

Figure 5
Distribution of the responding FHEI’s in the Brazilian territory

Through the self-assessment questionnaire, performance data was collected for each of the 11 criteria identified in the first stage of the survey from 31 FHEIs (29 Federal Universities and two Federal Institutes of Education, Science and Technology) from 18 Brazilian states. From these institutions, four (12.90%) are from the north region of the country, ten (32.26%) are from the northeast region, three (9.68%) from the midwest region, seven (22.58%) from the southeast region and another seven (22.58%) from the south region. Figure 5 shows the distribution of the responding FHEIs (alternatives) in the national territory, their respective states and regions.

With the data collected from the participating HEIs, it was possible to observe the performance per criterion in each one of them, as illustrated in Figure 6:

Figure 6
Participating FHEIs performances in each evaluation criterion of the maturity level in process management

4.3 Classification of the FHEIs as to maturity level in process management

The performances presented in Figure 4 were used as input for the PROMSORT method along with the weights established for each criterion. Thus, the FHEIs were classified as immature, regular or mature as to the level of maturity in process management. Figure 7 shows the classification by optimistic and pessimistic point of view, and Figure 8 presents the classification by regions of Brazil for each point of view.

Figure 7
Classification of the FHEIs as Immature, Regular or Mature from the optimistic and pessimistic point of view

Figure 8
Classification of the FHEIs as Immature, Regular or Mature distributed by regions in Brazil

Observing Figure 8 from an optimistic point of view, it can be seen that the Midwest and South regions do not present FHEIs classified as immature, while the North and Northeast regions present one immature institution each, and the Southeast region presents the largest number of immature FHEIs, with two such classifications.

In relation to institutions classified as regular, it is verified that the North and Midwest regions tied with two classifications, followed by the Southeast region with three, the South region with four and, finally, the Northeast region with five regular FHEIs. In addition, the Northeast region also had the most institutions classified as mature under the optimistic perspective, closely followed by the South region, which had three mature classifications. The Southeast region presented two FHEIs classified as mature and was ahead of the North and Midwest regions, which presented only one mature classification each.

In the pessimistic perspective, only the Northeast and South regions presented variations in their classifications in relation to the optimistic point of view. Thus, two FHEIs from the Northeast, which from an optimistic point of view are classified as mature, become regular from a pessimistic point of view, and the South region starts to present an immature FHEI, which from an optimistic point of view is regular. Thus, from a pessimistic point of view, the Northeast region loses its position as leader in the number of mature FHEIs to the South region. Furthermore, the Midwest region becomes the only one that does not present an institution classified as immature.

To evaluate the robustness of the results in relation to their behavior and stability, a sensitivity analysis was performed, which took into account, in addition to the 11 established criteria, the variation of weights and points of view. Figure 9 presents the results of the sensitivity analysis.

Figure 9
Sensitivity analysis of the FHEIs classification

Through the results illustrated in Figure 9, it can be seen that no FHEI presented variations that comprehend the three possible classifications, meaning that all variations included only two classifications. The alternatives that presented variations in classification were: IFRJ (10.87% as mature and 89.13% as regular), UFC (50% as mature and 50% as regular), UFCA (50% as mature and 50% as regular), UFG (2.17% as mature and 97.83% as regular), UFPE (8.7% as mature and 91.3% as regular) and UFSC (54.35% as regular and 45.65% as immature).

Therefore, the remaining alternatives did not present variations and were classified in all scenarios, regardless of the variation in point of view or criteria weights, as mature (UFERSA, UFJF, UFMT, UFPA, UFPR, UFRPE, UFSM, UFV and Unipampa), regular (IFMT, UFABC, UFAM, UFBA, UFFS, UFRGS, UFRN, UFS, UFSCar, UFT, UNILAB and UTFPR) or immature (UFAC, UFAL UFMG and UFRJ).

In the optimistic perspective, the FHEIs IFRJ and UFPE were classified as mature in 21.74% and 17.39% of the scenarios. UFG was classified as mature in only one scenario. The other alternatives did not present variations and were classified in all optimistic scenarios as mature (UFC, UFCA, UFERSA, UFJF, UFMT, UFPA, UFPR, UFRPE, UFSM, UFV and Unipampa), regular (IFMT, UFABC, UFAM, UFBA, UFFS, UFRGS, UFRN, UFS, UFSC, UFSCar, UFT, UNILAB and UTFPR) or immature (UFAC, UFAL UFMG and UFRJ). On the other hand, the pessimistic viewpoint indicates a slight variation in ranking only for UFSC, which presented itself as regular in only 2 (8.7% of the pessimistic simulations) pessimistic scenarios.

Thus, it is perceived that, regardless of adopting an optimistic or pessimistic perspective, most FHEIs (51.6% in the optimistic perspective and 54.8% in the pessimistic one) were classified as regular, and that 25 of the 31 (80.6% of the FHEIs) maintained their classifications regardless of the scenarios studied. Among the six alternatives that presented rating variations, only three (UFC, UFCA, UFSC) showed significant variability. These alternatives, despite not being sensitive to variations in the criteria weights, presented greater sensitivity as to the point of view adopted, which explains the variation. Hence, the stability of the results obtained through the evaluation model developed is evident.

5 CONCLUSIONS

This research proposed a participatory benchmarking approach to classify the FHEIs as to the level of maturity in process management. The model presented can be used to identify benchmark institutions in process management, whether they are public or private institutions, due to the fact that the main input of the method is the value judgment of the experts and the self-assessment of the institutions.

As a result, the study indicated that, in general, participating FHEIs were classified as regular, whether from an optimistic or pessimistic perspective. This observation possibly indicates that Brazilian FHEIs have BPM initiatives, but, in many cases, they still lack the necessary practices to reach a significant maturity. Furthermore, through the simulation of scenarios performed in the sensitivity analysis, stability was observed in the classification of alternatives, evidencing solidity in the results obtained.

It was perceived as an advantage of the model the conception of a dynamic benchmark for presenting the panorama of maturity of Brazilian institutions, with progressive potential to become even more consistent, since the more institutions and specialists contribute and participate in the benchmarking, the more alternatives will be compared, refining the classifications. In addition, the use of the PROMSORT method made it possible, by changing parameters and thresholds in order to adjust the level of requirement of the criteria, making the model flexible and a constantly updated tool, accomplishing the research gap established by Szelągowski & Berniak‑Woźny (2022).

It is recommended for future studies the increase of the number of alternatives and of the body of experts in process management participants. This way, the inclusion of more variables and different points of view can be favored. Finally, the viability of using this model as a subsidy for the development of a benchmarking web platform to share knowledge and experience is perceived. Such platform would work as virtual environment that is in line with Taskymbayeva, Shaikh & Salimbayeva (2022) thinking, promoting the search for best practices in process management in HEIs through the exchange of information and performance data from institutions, becoming a reference for decision making for HEI managers and leaders who define related public policies.

REFERENCES

• Aganette, E. C., Maculan, B. C. M. dos S., & Lima, G. Â. de. (2018). BPM acadêmico: mapeamento de processos e de fluxos informacionais na ECI/UFMG. Pesquisa Brasileira Em Ciência Da Informação e Biblioteconomia, 13(1). https://doi.org/10.22478/ufpb.1981-0695.2018v13n1.39607
  » https://doi.org/https://doi.org/10.22478/ufpb.1981-0695.2018v13n1.39607
• Albertin, M. R., Kohl, H., & Elias, S. J. B. (2015). Manual do Benchmarking: Um guia para implantação bem-sucedida (I. U. da U. F. do C. (UFC) (ed.)).
• Andrade, E., Rasoto, V. I., & Carvalho, H. A. de. (2018). Gerenciamento de processos nas Instituições Federais de ensino superior Brasileiras. Revista Brasileira de Planejamento e Desenvolvimento, 7(2), 171. https://doi.org/10.3895/rbpd.v7n2.5706
  » https://doi.org/https://doi.org/10.3895/rbpd.v7n2.5706
• Araz, C., & Ozkarahan, I. (2007). Supplier evaluation and management system for strategic sourcing based on a new multicriteria sorting procedure. International Journal of Production Economics, 106(2), 585-606. https://doi.org/10.1016/j.ijpe.2006.08.008
  » https://doi.org/https://doi.org/10.1016/j.ijpe.2006.08.008
• Barbalho, S. C. M., Nitzsche, M. C. M., & Dantas, A. S. (2017). Process Improvement on Public Administration: an Action. Revista Produção Online, 17, 406-439.
• Barreto, L. R., Vasconcelos, G. M. R. De, & Marques, A. B. (2019). Aplicação da Gestão de Processos em uma Instituição Federal de Ensino Superior: o caso da Divisão de Orçamento da UFERSA. Anais IV EEP - Encontro dos Escritórios de Processos.
• Behzadian, M., Kazemzadeh, R. B., Albadvi, A., & Aghdasi, M. (2010). PROMETHEE: A comprehensive literature review on methodologies and applications. European Journal of Operational Research, 200(1), 198-215. https://doi.org/10.1016/j.ejor.2009.01.021
  » https://doi.org/https://doi.org/10.1016/j.ejor.2009.01.021
• Branco, G. M. (2016). Proposta de framework para construção da arquitetura de processos: o caso de uma Instituição Federal de Ensino Superior.
• Brans, J.-P., & Mareschal, B. (2005). PROMETHEE methods. In: Multiple criteria decision analysis: state of the art surveys. Springer , New York, NY, 163-186. https://doi.org/10.1007/978-1-4939-3094-4_6
  » https://doi.org/https://doi.org/10.1007/978-1-4939-3094-4_6
• Brasil. Ministério da Fazenda. Ministério do Planejamento Orçamento e Gestão. (2011). Guia de Gestão de Processos de Governo - Áreas de integração para governo eletrônico arquitetura e-ping de interoperabilidade. 93.
• Bührig, J., Schoorman, T., & Kanckstedt, R. (2018). Business Process Management in German Institutions of Higher Education: The Case of Jade University of Applied Science. Business Process Management Cases, Management for Professionals. https://doi.org/10.1007/978-3-319-58307-5_31
  » https://doi.org/https://doi.org/10.1007/978-3-319-58307-5_31
• Camp, R. C. (1994). Benchmarking. Carl Hanser Verlag.
• Carmo, B. B. T., Barros Neto, J. F., Dutra, N. G. S. (2011). Análise do impacto nos custos de transporte de um modelo de seleção de fornecedores baseado em variáveis socioambientais e de competitividade. Production, v. 21, n. 3. https://doi.org/10.1590/S0103-65132011005000013.
  » https://doi.org/https://doi.org/10.1590/S0103-65132011005000013
• Carmo, B. B. T. do, Margni, M., & Baptiste, P. (2020). Ranking product systems based on uncertain life cycle sustainability assessment: a stochastic multiple criteria decision analysis approach. Revista De Administração Da UFSM, 13(4), 850-874. https://doi.org/10.5902/1983465955294
  » https://doi.org/https://doi.org/10.5902/1983465955294
• Carvalho, K. A. de, & Sousa, J. C. (2017). Gestão por Processos: Novo Modelo de Gestão para as Instituições Públicas de Ensino Superior. Revista Administração Em Diálogo - RAD, 19(2), 1. https://doi.org/10.20946/rad.v19i2.25298
  » https://doi.org/https://doi.org/10.20946/rad.v19i2.25298
• Costa, M. T. P., & Moreira, E. A. (2018). Gestão e mapeamento de processos nas instituições públicas: um estudo de caso em uma Universidade Federal. Revista Gestão Universitária Na América Latina - GUAL, 162-183. https://doi.org/10.5007/1983-4535.2018v11n1p162
  » https://doi.org/https://doi.org/10.5007/1983-4535.2018v11n1p162
• Dantas, A. S., Queiroz, F. C., Queiroz, B. P., & Viegas, J. (2009). Gestão De Processos E Avaliação De Desempenho No Setor Público Brasileiro: Um Estudo De Caso Em Uma Instituição De Ensino Superior Pública. 1-15.
• do Carmo, B.B.T., de Oliveira Castro, G., Gonçalo, T.E.E., Ugaya, C.M.L. (2021). Participatory approach for pertinent impact subcategory identification: Local community. Int J Life Cycle Assess 26, 950-962. https://doi.org/10.1007/s11367-021-01892-3
  » https://doi.org/https://doi.org/10.1007/s11367-021-01892-3
• Fettke, P., Zwicker, J., & Loos, P. (2015). Business Process Maturity in Public Administrations. In Handbook on Business Process Management 2: Strategic Alignment, Governance, People and Culture, Second Edition. https://doi.org/10.1007/978-3-642-45103-4_6
  » https://doi.org/https://doi.org/10.1007/978-3-642-45103-4_6
• Figueiredo, F. S. de, Honório, A., Lima, S. da S. M. de, & Papa, R. G. valente. (2017). A trajetória do surgimento da área de processos: o estudo da implantação do escritório de processos em uma Instituição Federal de Ensino Superior. XI Workshop de Tecnologia Da Informação e Comunicação Das Instituições Federais de Ensino Superior , 1.
• Gonçalo, T. E. E., & Alencar, L. H. (2014). A supplier selection model based on classifying its strategic impact for a company’s business results. Pesquisa Operacional, 34(2), 347-369. https://doi.org/10.1590/0101-7438.2014.034.02.0347
  » https://doi.org/https://doi.org/10.1590/0101-7438.2014.034.02.0347
• Hrabala, M., Opletalova, M., & Tučekc, D. (2017). Bussiness Process Management in Czech Higer Educatinon. Journal of Applied Enginerring Science, 411, 35-44. https://doi.org/ 10.5937/jaes15-12171
  » https://doi.org/https://doi.org/ 10.5937/jaes15-12171
• Lorena, A. L. F. de. (2015). A Contribuição Estratégica da Ouvidoria Pública para a Gestão de Processos de Negócios nas IFES. XV Colóquio Internacional de Gestão Universitária - CIGU, Desafios Da Gestão Universitária No Século XXI, 1-12.
• Matos, C. L. B., Forte, S. H. A. C., & Forte, S. A. B. (2020). Organizational assessment methodology of corporate universities. Rev. Adm. UFSM, 13, 709-727. https://doi.org/10.5902/1983465932836
  » https://doi.org/https://doi.org/10.5902/1983465932836
• Mückenberger, E., Togashi, G. B., de Pádua, S. I. D., & Miura, I. K. (2013). Process management applied to the establishment of international bilateral agreements in a brazilian public institution of high education. Producao, 23(3), 637-651. https://doi.org/10.1590/S0103-65132012005000076
  » https://doi.org/https://doi.org/10.1590/S0103-65132012005000076
• Oliveira, J. de S. P. de, Costa, M. M., Wille, M. F. de C., & Marchiori, P. Z. (2008). Introdução ao Método Delphi.
• Olson, D. L. (1996). Decision Aids for Selection Problems.
• Paiva, M. B. M., Barbosa Feitosa, P. P., De Aquino Cabral, A. C., & Dos Santos, S. M. (2017). Barreiras e facilitadores na gestão de processos de trabalho em instituição federal de ensino superior. Revista Gestão Universitária Na América Latina - GUAL, 47-71. https://doi.org/10.5007/1983-4535.2017v10n4p47
  » https://doi.org/https://doi.org/10.5007/1983-4535.2017v10n4p47
• Santos, A. C. (2001). O uso do método Delphi na criação de um modelo de competências. Revista de Administração, 36(2), 25-32.
• Santos, H. M., Santana, A. F., & Alves, C. F. (2012). Análise De Fatores Críticos De Sucesso Da Gestão De Processos De Negócio Em Organizações Públicas. Revista Eletrônica de Sistemas de Informação, 11(01). https://doi.org/10.5329/resi.2012.1101003
  » https://doi.org/https://doi.org/10.5329/resi.2012.1101003
• Sujanawati, R. P., ER, M., & Wibowo, R. P. (2021). Analysis of business process management (BPM) effects on data and information quality improvement at higher education institutions: a literature study. IPTEK The Journal of Technology and Science, 31(3). https://doi.org/10.12962/j20882033.v31i3.6260
  » https://doi.org/https://doi.org/10.12962/j20882033.v31i3.6260
• Szelągowski, M., & Berniak-Woźny, J. (2022). How to improve the assessment of BPM maturity in the era of digital transformation. Information Systems and e-Business Management, 171-198. https://doi.org/10.1007/s10257-021-00549-w
  » https://doi.org/https://doi.org/10.1007/s10257-021-00549-w
• Taskymbayeva, L. A., Shaikh, A. A., & Salimbayeva, R. A. (2022). Application of business process management methods in higher education institutes. Central Asian Economic Review, (3), 45-55. https://doi.org/10.52821/2789-4401-2022-3-45-55
  » https://doi.org/https://doi.org/10.52821/2789-4401-2022-3-45-55

• Plagiarism Check

  The ReA/UFSM maintains the practice of submitting all documents approved for publication to the plagiarism check, using specific tools, e.g.: Turnitin.

Publication Dates

• Publication in this collection
  22 May 2023
• Date of issue
  2023

History

• Received
  16 Apr 2022
• Accepted
  11 Dec 2022
• Published
  29 Mar 2023