Unveiling undergraduate production engineering students’ comprehension of process flow measures

Paper aims: This study analyzes the comprehension of production engineering students about the influence of some key variables on the process performance measures in a service process. Originality: This paper points out the need for educators to re-evaluate their approaches to teaching the Operations Management (OM) principles related to process flow measures. Research method: This study used scenario-based role-playing experiments with 2×2×2 between-subject factorial design with three independent variables (variability of activities, capacity utilization, and resource pooling) and four dependent variables related to key internal process performance measures (Flow Time, Overall Quality of service, Quality of service employees, and Queue Size). The sample was composed of 178 undergraduate production engineering students from a large university in Brazil from various institution units. Main findings: These results show that students perceived the use of resource pooling as an impactful practice. However, the students did not correctly identify the effects of increasing resource utilization and the variability on flow time and queue size when activities are pooled. Implications for theory and practice: The teaching of basic concepts of OM requires the support of computational tools. Undergraduate courses that contemplate subjects in the field of OM should work more intensely on simulation-based learning.


Conceptual background and hypotheses
The classical work of the authors Anupindi et al. (2011) was chosen for defining the basic concepts of OM to be addressed in this study. This book discusses three internal measures (flow time, flow rate, and inventory) to control and improve any process. These measures captured the essence of process flow, serving as leading indicators of customer satisfaction and financial performance. Moreover, many OM manuals discuss these measures as fundamental elements to be managed by service and manufacturing organizations (Jacobs & Chase, 2012;Krajewski et al., 2009;Reid & Sanders, 2005;Slack et al., 2013b;Stevenson, 2014). Therefore, these three process flow measures have been defined as a reference point for establishing the basic concepts of the OM field that students must appropriately understand. The literature in this field points out that these three process flow measures are governed by some principles and formulas (production delay curve, queuing formula, Little's law, and Pooling). In addition, these principles and formulas are affected by some key variables (Process Variability, Capacity Utilization, and Resource Pooling) that were worked out in this study. Figure 1 shows these relationships.
Based on the above points, some principles and formulas of OM are exposed next, and hypotheses related to key internal process performance measures are presented.

Process variability
A critical element in any production process that affects its performance is variability. It can be defined as the degree to which activities vary in their timing or nature in a process (Slack et al., 2013a).
There are different sources of variability in processes. If one takes a simple process as a reference, it can identify four sources of variability: i) Variability in processing times; ii) Variability coming from the inflow of flow units; iii) Random routing exists when there are multiple flow units in the process. If the path a flow unit takes through the process is itself random, the arrival process at each resource is subject to variability; and iv) Random availability of resources because resources are subject to random machine downtime (Cachon & Terwiesch, 2013).
Authors Nigel Slack, Alistair Brandon-Jones, and Robert Johnston (Slack et al., 2013a) point out that greater process variability results in longer waits and greater resource underutilization, reducing resource efficiency. In addition, more significant variability can disrupt the flow and make the process synchronization harder. Controlling process variability is of great importance for the effectiveness of operations management, and the use of simulation models can improve the quality of managers decisions in many ways. If taken as an example the bullwhip effect, as a multi-echelon phenomenon (affecting different agents simultaneously), the problem of lack of demand visibility seems to be quite critical, with material and information flows accounting for different levels of supply chain uncertainty (Yao et al., 2021). Such importance was recognized in Poornikoo & Qureshi (2019), with the merit of forwarding a simulation model using system dynamics method and fuzzy rule-based inference system to evaluate the bullwhip effect in a single-product, three-echelon, multi-stage supply chain system. In the same path, Yin (2021) has described the simulation method as a suitable tool to predict supply chain behaviors and reinforced that many scholars have obtained some useful results studying the bullwhip effect by using simulation methods.

Capacity utilization
The utilization of process resources is the proportion of available time that the resources within the process are doing work and can be calculated by the flow rate divided by the processing capacity. Therefore, this measure indicates how much the process produces relative to how much it could produce if running at full capacity. External constraints or external bottlenecks such as low flow rate (due to low demand rate) or low input rate (due to low supply rate) affect the capacity efficiency (Anupindi et al., 2011).

Little's Law
Little's law shows a mathematical relationship between flow time, flow rate, and inventory, stating that the average number of flow units in a queuing system is the product of the average time an item waits in the system and the average rate at which flow units leave the system. This law is instrumental, working for any stable process (Anupindi et al., 2011;Little, 2011).
In the context of the service delivery processes, flow time and waiting time have a meaningful impact on how customers experience the service (De Pourcq et al., 2021;Lemke et al., 2011). Regarding these crucial dimensions, Kingman´s equation or VUT equation shows that the time in queue increases rapidly with the growth of capacity utilization and variability (Hopp & Spearman, 2008), as shown as follows: Where: - Kingman´s equation has a broad application because it does not require processing times or inter-arrival times to follow a specific distribution and is only valid for stationary processes. The equation shows that queueing time is the product of three terms: processing time, utilization, and variability. This formula also shows that the queue grows vigorously when utilization is close to 100%. Furthermore, it can observe that the waiting time grows with an increase in variability (Cachon & Terwiesch, 2013). This formula portrays that flow time, flow rate, inventory, and resource utilization are interrelated (Slack et al., 2013b). This formula shows that more significant variability in processes increases the queue time, even if the process is working at a utilization level of less than 100 percent. Using this formula, it can be verified the occurrence of an inherent trade-off between greater resource utilization and the responsiveness of the service (Cachon & Terwiesch, 2013). Therefore, service managers face the daily challenge of reconciling supply and demand without compromising the expected level of quality (Armistead & Clark, 1994;Chase, 1978;Kandampully, 2000;Pullman & Moore, 1999).
Also, the Lean manufacturing philosophy allows us to understand the negative effect of variability and high resource utilization for processes. Beyond the eight forms of waste (Muda) in the Lean vision, Mura and Muri's terms can also be comprehended as waste (Liker, 2021).
The term Mura is synonymous with irregularity resulting from an irregular production schedule or fluctuating production volumes due to internal problems such as missing parts, defects, and downtime. Irregularity increases the variation in quality, cost, or delivery of an operation (Liker, 2021;Sayer & Williams, 2007). Thus, the process generates waste when activities do not occur in a swift and even flow (Schmenner, 2012). On the other hand, the unevenness leads to a situation where the process sometimes gets too little work or is overworked at other times -leading directly to Muri. This term is synonymous with overdoing and can be understood as unnecessary or unwarranted overloading of people, equipment, or systems by demands that exceed capacity. Furthermore, the Lean philosophy states that overburdening people results in safety and quality problems, and overburdening equipment causes breakdowns and defects (Liker, 2021;Sayer & Williams, 2007). The problems pointed out by the Mura, and Muri terms complement the problems arising from the increased variability and degree of resource utilization discussed earlier using the Kingman equation.
Based on the topics presented in this section, some hypotheses were formulated. These hypotheses aim to verify whether undergraduate production engineering students understand the core concepts discussed in this study.
H 1 -The perceived impact on process performance measures (flow time, queue size, overall quality of service, and customer satisfaction with service employees) is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.
H 1a : The perceived impact on flow time is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.
H 1b : The perceived impact on queue size is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.
H 1c : The perceived impact on the overall quality of service is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.
H1 d : The perceived impact on customer satisfaction with service employees is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.
H 2 -The perceived impact on process performance measures (flow time, queue size, overall quality of service, and customer satisfaction with service employees) is higher when resource utilization escalates.
H 2a : The perceived impact on flow time is higher when resource utilization escalates.
H 2b : The perceived impact on queue size is higher when resource utilization escalates.
H 2c : The perceived impact on the overall quality of service is higher when resource utilization escalates.
H 2c : The perceived impact on customer satisfaction with service employees is higher when resource utilization escalates.
In our experiment, we defined as "perceived impact" the amount of difference in one performance dimension (flow time, queue size, overall quality of service, and the responsiveness, assurance, and empathy of the service employees) produced by the change from the current situation to a new situation described in the scenario.

Resource pooling
The pooling of resources or activities offers several advantages for improving process performance by offering a better service than individual processes, effectively using the available capacity, preventing idle productive resources. At the same time, the other faces a backlog of work. Therefore, in some situations, a service organization can benefit from resource pooling by reducing customer waiting time without hiring extra employees (Anupindi et al., 2011;Cachon & Terwiesch, 2013). On the other hand, production resources must handle more variation in demand to implement this practice. Thus, this practice should be used if it is not expensive to employ resources with such flexibility (Cattani & Schmidt, 2005).
Pooling of activities can be better understood if the configuration of a given process is modified by combining several activities into one extensive activity forming a single, more flexible resource pool that performs all these activities. This redesign alternative can eliminate starvation and blocking, reducing lead time and improving capacity (Cachon & Terwiesch, 2013). Authors Mansar & Reijers (2005) refer to this business process redesign practice as order assignment. Regarding the pooling practice, we can formulate the following hypothesis: H 3 -The perceived impact on process performance measures (flow time, queue size, overall quality of service, and customer satisfaction with service employees) is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.
H 3a : The perceived impact on flow time is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.
H 3b : The perceived impact on queue size is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.
H 3c : The perceived impact on the overall quality of service is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.
H 3d : The perceived impact on customer satisfaction with service employees is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity. Figure 2 shows the relationships between the three factors with two levels and some process performance measures (flow time, queue size, overall service quality, and customer satisfaction with service employees). The students evaluated these relationships to verify the hypotheses presented in section 2. Figure 2 shows the relationship between the proposed conceptual model and the hypotheses developed in this study. The authors used a scenario-based behavioral experiment to test the three hypotheses proposed. Scenariobased experiments (SBE) are well suited for identifying causal relationships involving factors that guide choice and decision-making by individuals and organizations. In addition, they involve low execution costs compared to other alternatives. Experiments also feature unique settings derived from meticulously constructed and realistic scenarios to assess dependent variables, including attitudes, behaviors, and intentions. Thus, it increases experimental realism and allows researchers to manipulate and control independent variables (Eckerd, 2016;Rungtusanatham et al., 2011).

Overview of studies
The authors used SBE to assess students' understanding of some basic OM concepts by evaluating three hypotheses. The authors defined the variables to be observed and changed in the proposed scenarios so that the hypotheses could be adequately evaluated. The participants should easily manipulate and clearly understand the proposed variables, enabling the experimental method to be successful. Consequently, the scenarios should address a simple process composed of a few activities to ensure a greater understanding by the participants.
The authors used a scenario development process to achieve these goals that encompassed two phases. In the first phase, the scenarios were designed and then tested computationally, verifying the impact of changes in the manipulated factors on the process performance concerning flow time and queue size. This procedure allowed us to adjust the scenarios so that changes in the manipulated factors had a noticeable effect on the process performance.
In the second phase, the scenarios were initially elaborated based on insights from the OM literature and vignettes were developed consisting of written text and pictures that illustrated the proposed process and the changes in the manipulated variables. Two academic professors from OM reviewed this proposal. After making appropriate adjustments, each scenario was presented to eight production engineering students, and feedback was requested regarding readability, length, and realism. Then, further adjustments were based on their feedback.
Next, it was conducted a factorial experiment (2 × 2 × 2) with 178 undergraduate students from different units of a large educational institution on students who have attended more than half of the Production Engineering undergraduate program.

Simulated scenarios
The scenarios aimed to reproduce a service process in which customers appear with different arrival rates and are served by three employees who perform three activities. The Extend 09 software was used to generate two models: i) model A, in which each employee performs one activity, and ii) model B, in which each employee performs all three activities. After some adjustments, the scenarios were built with three factors using the values and resources presented in Table 1. These adjustments allowed the construction of scenarios that showed significant changes in process performance.
Four production delay curves ( Figure 3) were elaborated with the two computationally simulated models. Figure 3 also indicates the eight scenarios used in the experiment and the initial service scenario before the proposed changes.

Factor
Level -Level + The extent of change in resource utilization (A) 60% to 85% Arrival process with interarrival time following an exponential distribution The simulations also enabled the evaluation of the effect of the introduced changes in the scenarios for lead time and queue length (Table 2). These data were used to assess the degree of correctness of the respondents' answers.

Scenarios used
Scenarios were designed to stimulate the respondent to put themselves in decision situations to test our theoretical hypotheses. The scenarios described a customer service process of one sports club that has experienced increased demand. This fact generates a high or medium expansion in resource utilization (Factor A), impacting process performance. In response to this change, the club makes some modifications that imply a change in how the activities performed are distributed to employees (Factor B) and in their time variation (Factor C).
After reading the scenario, participants responded to a questionnaire that assessed the impact of the operational changes described in the scenario on some process performance variables. The scenario manipulations are provided in Appendix 1, and the individual scale items from the questionnaire are provided in Appendix 2.

Measures
The authors chose four dependent variables related to key internal process performance measures: 1) flow time, 2) overall quality of service, 3) the responsiveness, assurance, and empathy of the service employees, and 4) queue size. Likert-type seven-point response scale with anchors of "Decrease a lot"(1), "It remains the same"(4), and "Increase a Lot"(7) was utilized.
Seven-point Likert scale was used to access manipulation checks, realism checks, and the question that measured the perception regarding the degree of knowledge of the OM topics addressed in this research. This scale used anchors of "Strongly Disagree"(1), "Neither agree nor disagree"(4), and "Strongly Agree"(7). All the scales are provided in Appendix 2.

Demand characteristics
The design of this experiment took some cautiousness to reduce the potential confounding effects of demand characteristics (Christensen et al., 2015). The authors used a between-subjects design that is less prone to demand characteristics. Also, it were adopted the good practices recommended in the Belmont Report of respecting people or treating participants as autonomous to make decisions about their participation in research (Ryan et al., 1979). Therefore, it was provided information about the purpose of the study, the benefits and risks of participation, and their rights to refuse participation or cease participation in the study.

Design used in the experiment
Considering the scenarios created in the computer simulation, the experiment consisted of a 2 × 2 × 2 between-subject factorial design, which allowed us to take into account interaction effects explicitly (Shadish & Cook, 2002). Three independent variables or factors (variability of activities, change in capacity utilization, and use of resource pooling) with two levels were used (Table 3).

Sample and assignment
Participants were randomly assigned to one of the eight treatment conditions in the factorial design. Random assignment was used to minimize the likelihood of systematic between-group differences and maximize the internal validity of the experiment (Aguinis & Bradley, 2014). The randomly assigned procedure also attempts to eliminate the influence of all potential confounding third variables on the dependent variables because the influence of any extraneous variables is equal in the experimental conditions (Cozby & Bates, 2011).
The sample for this study was composed of undergraduate production engineering students from a large university in Brazil. As inclusion criteria, all participants must have completed courses involving basic OM knowledge, such as Operations Research, Production and Operations Management, and Supply Chain Management. Initially, a total of 178 respondents participated in the study. However, eight were discarded due to missing data. Thus, the final sample size was 170 participants, with an average of 21 participants and a slight variation in the number of participants per scenario. These values seem suitable since Hair et al. (2016) recommend a minimum size of 20 observations per cell with the sample sizes per cell approximately equal. Table 4 illustrates the scenarios used and shows the number of participants per scenario.

Results
This section shows the results obtained from the experiment. The calculations and the statistical analysis were performed with R 4.0.5 (R Core Team, 2021).

Manipulation checks
Manipulation checks were performed to determine if the participants responded as planned to the independent variable manipulations in the experimental treatment conditions (Webster Junior & Sell, 2014 = p < 0.001). Based on these results, it was found that participants perceived significant differences between the treatment conditions in the scenarios presented.

Realism check
A quantitative realism check was also performed in this study to check whether participants could recognize and respond to treatment conditions. Therefore, the items developed by Dabholkar (1994) were deployed to evaluate the realism of the scenarios in the experimental design (Appendix 2). Participants were asked if the scenario was realistic and if they could imagine themselves in that situation. The realism check indicated that participants considered the scenarios to be engaging and realistic, with an average score of 5.56 on a seven-point scale. This finding suggests that participants' perceptions of the scenario manipulations were realistic enough to evoke authentic behavioral responses.

MANOVA and ANOVA results
Multivariate Analysis of Variance (MANOVA) was conducted on the dependent variables in this study (Table 5). Given the large sample size and the robustness of MANOVA to depart from multivariate normality, violations of multivariate normality were not expected to be severe. In addition, the shape of histograms, normal-probability plots, skewness, and kurtosis for each dependent measure were reviewed which showed no severe departures from normality (Hair et al., 2016). Another assumption underlying MANOVA is equality of variance (Hair et al., 2016). This assumption was tested using the Levene test. It was found a value of 1.68 on statistical testing and a p-value of 0.1174 for the dependent variable flow time. It was also found a value of 1.22 on statistical testing and a p-value of 0.2958 for the dependent variable overall quality of service; and a value of 1.45 on statistical testing and a p-value of 0.1891 for the dependent variable customer satisfaction with service employees. Finally, it was identified a value of 1.18 on statistical testing and a p-value of 0.3142 for the dependent variable queue size. Because the data were collected during the same period, there was no need to evaluate the independence assumption. Just one main effect of the independent variable resource pooling was observed (Wilks' lambda = 0.657; F = 20.78; p < 0.001). Additional univariate tests showed that the use of resource pooling led to a decrease in flow time (F = 25.09; p < 0.001) and queue size (F = 19.89; p < 0.00l). These tests also showed that the use of resource pooling led to an increase in overall quality of service (F = 48.07; p < 0.001) and in customer satisfaction with service employees (F = 58.25; p < 0.001).
The univariate tests also showed that the increase of the coefficient of variation displayed in the scenarios had a significant effect on the decrease in the overall quality of service (F = 7.44; p < 0.01) and in customer satisfaction with service employees (F = 5.13; p < 0.05).
The univariate tests showed the existence of two factors interaction. The independent variables resource pooling (Factor B) and coefficient of variation (Factor C) interact on the dependent variable's overall quality of service (F = 4.43; p < 0.05). Also, the independent variables resource pooling (Factor B) and the extent of resource utilization change (Factor A) interact on the dependent variable customer satisfaction with service employees (F = 4.094; p < 0.05). These interactions are illustrated in Figure 4.  Table 6 exhibits the mean values obtained for each independent variable at the two levels used in the study. The Spearman correlation coefficient was used to assess the respondents' perception of the typical trade-off between overall quality of service and queue size  Note: All items were measured using scales ranging from "1" = Decrease a lot, to "7" = Increase a lot. Significant values are indicated in bold.
The authors verified the degree of correctness of the students' answers concerning the dependent variables flow time and queue size worked on in this experiment. This verification was done by comparing the student's answers with the computer simulation results of the scenarios presented in Table 2. Despite the high knowledge declared by the students about operations management (average of 5.8 points on a scale from 1 to 7), the results showed a moderate rate for the degree of correctness (34% for flow time and 43% for queue size). Table 7 reports the hypotheses proposed in this paper and the results obtained in terms of statistical significance. Table 7. Summary of the results of tests of hypotheses.

Hypothesis Findings
H 1 -The perceived impact in process performance measure … is higher in a situation including changes in the variability of activities duration rather than in a situation with no changes. These findings corroborate the hypotheses H 1b , H 1c , H 2c , H 3a , H 3b , H 3c , and H 3d . The results show a lack of understanding of the effects of increasing the resource utilization in processes (H 2 ) and increasing the variability on flow time (H 1a ) and queue size (H 1d ). However, these results indicate that students accurately perceived the effects of resource pooling on the performance dimensions proposed by the study (H 3 ).
These research findings will be appropriately addressed in the conclusions and final remarks section.

Conclusions and final remarks
The efforts in designing and modeling business processes necessarily comprise the identification of different elements, such as physical and immaterial resources, decision points, flow rates, activities and tasks, information flow, time flows, location of buffers, unit loads, amongst others (Bammert et al., 2020). Not rarely, processes can be very complex in their architecture, requiring a deeper understanding of their functioning so that optimization efforts may be more effective (Gall & Rinderle-Ma, 2020).
The data we obtained from this experiment suggests that our sample, composed of undergraduate engineering students, perceived resource pooling as an impactful practice (Tables 5 and 6). So, even in scenario #8 that showed a considerable increase in the level of resource utilization and a noteworthy increase in the variability of activities, 62.5% of the students reported that this practice has a positive impact enhancing the perceived quality of service (the mean grades are above 4 points). One possible reason that may explain an overestimation of this practice is the widespread acknowledgment of practices related to it. Thus, several academic texts of OM emphasize the importance of terms like "job enlargement," "job enrichment," "empowerment," and "multiskilling." These terms use the resourcing pool practice that is associated with various management programs such as Quality Improvement, Lean Production, and Business Process Management (Jacobs & Chase, 2012;Krajewski et al., 2009;Slack et al., 2013a;Stevenson, 2014).
A common point in the scenarios is the increased workload reflected in the increased degree of resource utilization (A) that was driven by increased demand for the service. The variables resource pooling (B) and coefficient of variation (C) influenced the performance of the process. They allowed the construction of scenarios that showed trade-offs such as increased productivity by increasing the utilization of productive resources and the decreasing responsiveness of the service, and the trade-off between the increased productivity and the erosion of service (Cachon & Terwiesch, 2013;Oliva, 2001). Notwithstanding the existence of these trade-offs, students did not adequately identify the impact of these changes in the process on the performance indicators. This fact was present in the students independently of the respondents' perceived degree of knowledge about OM, indicating that this problem has a more general constitution. However, the perception of the influence of these variables is something important that students and managers of production engineering cannot neglect. The strong impact of the higher resource utilization rate on the processing time coming from the queue theory is a crucial concept for the production engineer who deals daily with the effects of actions taken in the process in search of higher productivity. This fault indicates an urgency in discussing new teaching approaches in the area. Gibbs et al. (1997) point out the need for teaching in engineering to develop skills in experimenting, designing, analyzing, and solving problems as essential for effective learning. The challenges of teaching production engineering are tremendous and go beyond the simple teaching of an established body of knowledge. There is a need for teaching strategies that focus on applying this knowledge to real-life situations.
In this experiment, the authors simulated the proposed scenarios using the computational simulation of discrete events, allowing the use of the Design of Experiments (DOE) to evaluate the effects between the three independent variables (A, B, and C) and the variables Flow Time and Queue size. It was verified through the ANOVA the significant occurrence of main effects, two-way interaction, and three-way interaction for all independent variables. Thus, given the existence of second and third-order interactions, the DOE presented a complex model to portray the relationship between the three independent variables and the two responses variables worked on in this text. Therefore, models of this nature are difficult to accurately design by people without computational tools. This fact shows that the basic concepts of OM in this text require the support of computational tools to be better understood. This statement is congruent with some authors who argued that simulation-based learning offers a superior method of helping students learn how to apply theoretical principles. Scholars in various disciplines further assert that computer simulations offer unique advantages in creating a problem-focused, engaging, and active learning environment. (e.g. Berends & Romme, 1999;Larreche, 1987;Medina-López et al., 2011;Salas et al., 2009;Showanasai et al., 2013). Some articles point to the need for educators to re-evaluate their approaches to teaching OM. They should be encouraged to adopt new teaching methods (Brandon-Jones et al., 2012). Thus, these facts reinforce the need for undergraduate courses that contemplate subjects in the field of OM to work more intensely on simulation-based learning.
On the other hand, in our favor, there is a good variety of teaching methods that provide significant benefits for the understanding and practice of our vast body of knowledge. Thus, methods such as group exercises, experiential teaching methods, conventional lectures, business simulations, role-plays, live cases, and virtual learning environments can and should be used to enhance our students' learning (Brandon-Jones et al., 2012). The challenges are substantial, but we have several successful examples of these strategies.
The results found in the survey are confined to production engineering students from a single university. This fact certainly limits the external validity of this research. External validity can be increased by applying this method to other production engineering students from other universities in Brazil and abroad. Finally, it seems important to strongly encourage extending this experience to professionals who are directly involved in service operations, as they are expected to deal with different assignments related to process management and ultimately to the proposal and evaluation of process performance metrics and process outcomes.