Unveiling undergraduate production engineering students' comprehension of process flow measures

Abstract

Paper aims

This study analyzes the comprehension of production engineering students about the influence of some key variables on the process performance measures in a service process.

Originality

This paper points out the need for educators to re-evaluate their approaches to teaching the Operations Management (OM) principles related to process flow measures.

Research method

This study used scenario-based role-playing experiments with 2×2×2 between-subject factorial design with three independent variables (variability of activities, capacity utilization, and resource pooling) and four dependent variables related to key internal process performance measures (Flow Time, Overall Quality of service, Quality of service employees, and Queue Size). The sample was composed of 178 undergraduate production engineering students from a large university in Brazil from various institution units.

Main findings

These results show that students perceived the use of resource pooling as an impactful practice. However, the students did not correctly identify the effects of increasing resource utilization and the variability on flow time and queue size when activities are pooled.

Implications for theory and practice

The teaching of basic concepts of OM requires the support of computational tools. Undergraduate courses that contemplate subjects in the field of OM should work more intensely on simulation-based learning.

Keywords
Operations management; Process flow measures; Learning; Scenario-Based Experiment

1. Introduction

Production organizations seek greater competitiveness in an environment characterized by greater competition, constant change, and high instability. Thus, several organizations seek a higher quality of their products and lower production costs through various management programs such as Total Quality Management (TQM), Six Sigma, Fitness Program, and Lean Production (Ferdows & Thurnheer, 2011; Gryna, 2004; Min & Smyth, 2014; Negrão et al., 2017). These and various other initiatives adopted by productive organizations in the manufacturing and service sectors impose new challenges for the teaching and learning of the Operations Management (OM) field. If considered the service sector, academic interest in this area continues to expand, creating new opportunities to advance its body of knowledge (Field et al., 2018; Heineke & Davis, 2007; Machuca et al., 2007) and creating new challenges for better understanding and dissemination of knowledge about OM.

The body of knowledge encompassed by the Operations Management (OM) field is extensive and cannot be mastered by a professional who usually specializes in one or more areas of this field. However, production engineers and other professionals should master some basic principles in this area because they are part of their essential background. This body of knowledge is usually called “Principles of Operations Management.” The importance of managers and students understanding these principles is emphasized by Wallace J. Hopp and Mark L. Spearman (Hopp & Spearman,2008). These authors state that the principles governing the behavior of production systems and their understanding can improve management practice. The authors found widespread confusion about what works and what does not in operations and supply chain in their extensive professional practice as consultants and professors in this field. The academic community and industrial engineers have lost their way since various Lean Production methodologies and Six Sigma methodologies are often misapplied. There is a strong tendency to implement numerous production techniques, neglecting the complexity of production processes (Pound et al., 2014; Spearman & Hopp, 2009). A better understanding of these principles is vital for production engineers and professionals who work as managers in service organizations. In addition to the domain of disciplines such as accounting and finance, they must also be familiar with the principles that govern business processes operations.

It is also important to highlight, in this context, the fact that learning environments have been improving rapidly by the utilization of technology in the field of science and engineering, and teaching concepts through simulation labs is of great value and extremely important in engineering education, particularly with respect to learning levels of knowledge, comprehension, application, analysis synthesis and evaluation of complex problems by engineering students (Jasti et al., 2021).

In this direction, Principles of OM related to the queue theory and process flow measures, such as capacity, throughput time, resource utilization, and stock/work in the process, affect directly critical process performance indicators (e.g., cost, response time, and quality) and were selected to be worked in this study. They seem to be critical to solving operational problems in the services and manufacturing sector (Anupindi et al., 2011), thus revealing themselves as fundamental programmatic content for the development of competencies and abilities for undergraduate students, particularly in the areas of production engineering and administration.

These considerations lead us to reflect on how vital teaching is and understand these principles properly. Therefore, our study aimed to analyze the comprehension of production engineering students about the influence of some key variables (the degree of variability in activities, the rate of resource utilization, and the occurrence of pooling of activities) on the process performance measures (quality, flow time, and inventory) in a service process evaluated using a scenario-based experiment methodology. The proposed scenarios allowed us to work indirectly with several principles derived from queue theory, enabling the extraction and verification of three hypotheses that reveal students' comprehension concerning some basic principles of operations management associated with these process flow measures.

The other sections of the paper are structured as follows. First, the authors present the theoretical framework and the hypotheses that were tested and verified in the study. In the next section of the paper, there is a description of the research methodology. Then, in the following two sections of the document, the authors present the data analysis and conclude with a review of the main results and their implications for Operations Management theory and in particular to the body of knowledge of process management in service operations.

2. Conceptual background and hypotheses

The classical work of the authors Anupindi et al. (2011) was chosen for defining the basic concepts of OM to be addressed in this study. This book discusses three internal measures (flow time, flow rate, and inventory) to control and improve any process. These measures captured the essence of process flow, serving as leading indicators of customer satisfaction and financial performance. Moreover, many OM manuals discuss these measures as fundamental elements to be managed by service and manufacturing organizations (Jacobs & Chase, 2012; Krajewski et al., 2009; Reid & Sanders, 2005; Slack et al., 2013b; Stevenson, 2014). Therefore, these three process flow measures have been defined as a reference point for establishing the basic concepts of the OM field that students must appropriately understand. The literature in this field points out that these three process flow measures are governed by some principles and formulas (production delay curve, queuing formula, Little's law, and Pooling). In addition, these principles and formulas are affected by some key variables (Process Variability, Capacity Utilization, and Resource Pooling) that were worked out in this study. Figure 1 shows these relationships.

Figure 1
Relationships discussed in this section.

Based on the above points, some principles and formulas of OM are exposed next, and hypotheses related to key internal process performance measures are presented.

2.1. Process variability

A critical element in any production process that affects its performance is variability. It can be defined as the degree to which activities vary in their timing or nature in a process (Slack et al., 2013a).

There are different sources of variability in processes. If one takes a simple process as a reference, it can identify four sources of variability: i) Variability in processing times; ii) Variability coming from the inflow of flow units; iii) Random routing exists when there are multiple flow units in the process. If the path a flow unit takes through the process is itself random, the arrival process at each resource is subject to variability; and iv) Random availability of resources because resources are subject to random machine downtime (Cachon & Terwiesch, 2013).

Authors Nigel Slack, Alistair Brandon-Jones, and Robert Johnston (Slack et al., 2013a) point out that greater process variability results in longer waits and greater resource underutilization, reducing resource efficiency. In addition, more significant variability can disrupt the flow and make the process synchronization harder. Controlling process variability is of great importance for the effectiveness of operations management, and the use of simulation models can improve the quality of managers decisions in many ways. If taken as an example the bullwhip effect, as a multi-echelon phenomenon (affecting different agents simultaneously), the problem of lack of demand visibility seems to be quite critical, with material and information flows accounting for different levels of supply chain uncertainty (Yao et al., 2021). Such importance was recognized in Poornikoo & Qureshi (2019), with the merit of forwarding a simulation model using system dynamics method and fuzzy rule-based inference system to evaluate the bullwhip effect in a single-product, three-echelon, multi-stage supply chain system. In the same path, Yin (2021) has described the simulation method as a suitable tool to predict supply chain behaviors and reinforced that many scholars have obtained some useful results studying the bullwhip effect by using simulation methods.

2.2. Capacity utilization

The utilization of process resources is the proportion of available time that the resources within the process are doing work and can be calculated by the flow rate divided by the processing capacity. Therefore, this measure indicates how much the process produces relative to how much it could produce if running at full capacity. External constraints or external bottlenecks such as low flow rate (due to low demand rate) or low input rate (due to low supply rate) affect the capacity efficiency (Anupindi et al., 2011).

2.3. Little’s Law

Little's law shows a mathematical relationship between flow time, flow rate, and inventory, stating that the average number of flow units in a queuing system is the product of the average time an item waits in the system and the average rate at which flow units leave the system. This law is instrumental, working for any stable process (Anupindi et al., 2011; Little, 2011).

In the context of the service delivery processes, flow time and waiting time have a meaningful impact on how customers experience the service (De Pourcq et al., 2021; Lemke et al., 2011). Regarding these crucial dimensions, Kingman´s equation or VUT equation shows that the time in queue increases rapidly with the growth of capacity utilization and variability (Hopp & Spearman, 2008), as shown as follows:

Time in queue = $variability\times utilization\times time=V\times U\times T$ (1)

Time in queue = $\left(\frac{C_a^2+C_p^2}{2}\right)\times \left(\frac{u}{1-u}\right)\times t_e$ (2)

Where:

- Coefficient of variation of interarrival time $(C_a)=\frac{Standarddeviationofinterarrivaltime}{Averageinterarrivaltime}$ (3)

- Coefficient of variation of processing time $(C_p)=\frac{Standarddeviationofprocessingtime}{Averageprocessingtime}$ (4)

- Utilization $\left(u\right)=\frac{FlowRate}{Capacity}$ (5)

- Effective process time $(t_e)$ (6)

Kingman´s equation has a broad application because it does not require processing times or inter-arrival times to follow a specific distribution and is only valid for stationary processes. The equation shows that queueing time is the product of three terms: processing time, utilization, and variability. This formula also shows that the queue grows vigorously when utilization is close to 100%. Furthermore, it can observe that the waiting time grows with an increase in variability (Cachon & Terwiesch, 2013).

This formula portrays that flow time, flow rate, inventory, and resource utilization are interrelated (Slack et al., 2013b). This formula shows that more significant variability in processes increases the queue time, even if the process is working at a utilization level of less than 100 percent. Using this formula, it can be verified the occurrence of an inherent trade-off between greater resource utilization and the responsiveness of the service (Cachon & Terwiesch, 2013). Therefore, service managers face the daily challenge of reconciling supply and demand without compromising the expected level of quality (Armistead & Clark, 1994; Chase, 1978; Kandampully, 2000; Pullman & Moore, 1999).

Also, the Lean manufacturing philosophy allows us to understand the negative effect of variability and high resource utilization for processes. Beyond the eight forms of waste (Muda) in the Lean vision, Mura and Muri's terms can also be comprehended as waste (Liker, 2021).

The term Mura is synonymous with irregularity resulting from an irregular production schedule or fluctuating production volumes due to internal problems such as missing parts, defects, and downtime. Irregularity increases the variation in quality, cost, or delivery of an operation (Liker, 2021; Sayer & Williams, 2007). Thus, the process generates waste when activities do not occur in a swift and even flow (Schmenner, 2012). On the other hand, the unevenness leads to a situation where the process sometimes gets too little work or is overworked at other times - leading directly to Muri. This term is synonymous with overdoing and can be understood as unnecessary or unwarranted overloading of people, equipment, or systems by demands that exceed capacity. Furthermore, the Lean philosophy states that overburdening people results in safety and quality problems, and overburdening equipment causes breakdowns and defects (Liker, 2021; Sayer & Williams, 2007). The problems pointed out by the Mura, and Muri terms complement the problems arising from the increased variability and degree of resource utilization discussed earlier using the Kingman equation.

Based on the topics presented in this section, some hypotheses were formulated. These hypotheses aim to verify whether undergraduate production engineering students understand the core concepts discussed in this study.

• H₁ - The perceived impact on process performance measures (flow time, queue size, overall quality of service, and customer satisfaction with service employees) is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.

  H_1a: The perceived impact on flow time is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.

  H_1b: The perceived impact on queue size is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.

  H_1c: The perceived impact on the overall quality of service is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.

  H1_d: The perceived impact on customer satisfaction with service employees is higher in a situation that includes changes in the variability of duration of activities than in a situation with no changes.

• H₂ - The perceived impact on process performance measures (flow time, queue size, overall quality of service, and customer satisfaction with service employees) is higher when resource utilization escalates.

  H_2a: The perceived impact on flow time is higher when resource utilization escalates.

  H_2b: The perceived impact on queue size is higher when resource utilization escalates.

  H_2c: The perceived impact on the overall quality of service is higher when resource utilization escalates.

  H_2c: The perceived impact on customer satisfaction with service employees is higher when resource utilization escalates.

In our experiment, we defined as “perceived impact” the amount of difference in one performance dimension (flow time, queue size, overall quality of service, and the responsiveness, assurance, and empathy of the service employees) produced by the change from the current situation to a new situation described in the scenario.

2.4. Resource pooling

The pooling of resources or activities offers several advantages for improving process performance by offering a better service than individual processes, effectively using the available capacity, preventing idle productive resources. At the same time, the other faces a backlog of work. Therefore, in some situations, a service organization can benefit from resource pooling by reducing customer waiting time without hiring extra employees (Anupindi et al., 2011; Cachon & Terwiesch, 2013). On the other hand, production resources must handle more variation in demand to implement this practice. Thus, this practice should be used if it is not expensive to employ resources with such flexibility (Cattani & Schmidt, 2005).

Pooling of activities can be better understood if the configuration of a given process is modified by combining several activities into one extensive activity forming a single, more flexible resource pool that performs all these activities. This redesign alternative can eliminate starvation and blocking, reducing lead time and improving capacity (Cachon & Terwiesch, 2013). Authors Mansar & Reijers (2005) refer to this business process redesign practice as order assignment. Regarding the pooling practice, we can formulate the following hypothesis:

• H_{3 -}The perceived impact on process performance measures (flow time, queue size, overall quality of service, and customer satisfaction with service employees) is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.

  H_3a: The perceived impact on flow time is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.

  H_3b: The perceived impact on queue size is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.

  H_3c: The perceived impact on the overall quality of service is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.

  H_3d: The perceived impact on customer satisfaction with service employees is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.

3. Methodology

3.1. Overview of studies

Figure 2 shows the relationships between the three factors with two levels and some process performance measures (flow time, queue size, overall service quality, and customer satisfaction with service employees). The students evaluated these relationships to verify the hypotheses presented in section 2. Figure 2 shows the relationship between the proposed conceptual model and the hypotheses developed in this study.

Figure 2
Proposed conceptual model and the hypotheses concerning the perceived impact on process performance in the context of service processes.

The authors used a scenario-based behavioral experiment to test the three hypotheses proposed. Scenario-based experiments (SBE) are well suited for identifying causal relationships involving factors that guide choice and decision-making by individuals and organizations. In addition, they involve low execution costs compared to other alternatives. Experiments also feature unique settings derived from meticulously constructed and realistic scenarios to assess dependent variables, including attitudes, behaviors, and intentions. Thus, it increases experimental realism and allows researchers to manipulate and control independent variables (Eckerd, 2016; Rungtusanatham et al., 2011).

The authors used SBE to assess students' understanding of some basic OM concepts by evaluating three hypotheses. The authors defined the variables to be observed and changed in the proposed scenarios so that the hypotheses could be adequately evaluated. The participants should easily manipulate and clearly understand the proposed variables, enabling the experimental method to be successful. Consequently, the scenarios should address a simple process composed of a few activities to ensure a greater understanding by the participants.

The authors used a scenario development process to achieve these goals that encompassed two phases. In the first phase, the scenarios were designed and then tested computationally, verifying the impact of changes in the manipulated factors on the process performance concerning flow time and queue size. This procedure allowed us to adjust the scenarios so that changes in the manipulated factors had a noticeable effect on the process performance.

In the second phase, the scenarios were initially elaborated based on insights from the OM literature and vignettes were developed consisting of written text and pictures that illustrated the proposed process and the changes in the manipulated variables. Two academic professors from OM reviewed this proposal. After making appropriate adjustments, each scenario was presented to eight production engineering students, and feedback was requested regarding readability, length, and realism. Then, further adjustments were based on their feedback. Next, it was conducted a factorial experiment (2 × 2 × 2) with 178 undergraduate students from different units of a large educational institution on students who have attended more than half of the Production Engineering undergraduate program.

3.2. Simulated scenarios

The scenarios aimed to reproduce a service process in which customers appear with different arrival rates and are served by three employees who perform three activities. The Extend 09 software was used to generate two models: i) model A, in which each employee performs one activity, and ii) model B, in which each employee performs all three activities. After some adjustments, the scenarios were built with three factors using the values and resources presented in Table 1. These adjustments allowed the construction of scenarios that showed significant changes in process performance.

Table 1
The independent variables and their levels and values used in scenario simulation.

Factor	Level -	Level +
The extent of change in resource utilization (A)	60% to 85% Arrival process with interarrival time following an exponential distribution $\mu =5^{\prime }changeto3.8\text{'}$ (average time) or $\lambda =12\frac{units}{hour}changeto16\frac{units}{hour}$ (arrival rate)	60% to 95% Arrival process with interarrival time following an exponential distribution $\mu =5^{\prime }changeto3.2\text{'}$ $\mu =5^{\prime }$ change to ${\mathrm{3,8}}^{\prime }$ (average time) or $\lambda =12\frac{units}{hour}changeto19\frac{units}{hour}$ (arrival rate) $\mu =5^{\prime }$ change to ${\mathrm{3,2}}^{\prime }$
Resource pooling (B)	No change Activities separated	Change Activities grouped
Change in the variability of activities (C)	No change Low variation $\mu =3^{\prime }$ $\mu =5^{\prime }$ change to ${\mathrm{3,8}}^{\prime }$ (average time) Activities time following a constant distribution Coefficient of variation: 0%	Change Low variation to high variation $\mu =3^{\prime }$ $\mu =5^{\prime }$ change to ${\mathrm{3,8}}^{\prime }$ (average time) or $\lambda =20\frac{units}{hour}\left(\text{processing rate}\right)$ Activities time following an exponential distribution. Coefficient of variation: 100%

Four production delay curves (Figure 3) were elaborated with the two computationally simulated models. Figure 3 also indicates the eight scenarios used in the experiment and the initial service scenario before the proposed changes.

Figure 3
The throughput delay curve using computational simulation data.

The simulations also enabled the evaluation of the effect of the introduced changes in the scenarios for lead time and queue length (Table 2). These data were used to assess the degree of correctness of the respondents’ answers.

Table 2
Mean queue size for each of the eight scenarios.

Scenario	Mean flow time			Mean queue size			(A) Utilization change	(B) Resource pooling	(C) Coefficient of variation
	Simulation's value	Percentage growth compared to the baseline scenario	Degree of change	Simulation's value	Percentage growth compared to the baseline scenario	Degree of change
Baseline	13.0	0%	-	0.93	0%	-	60% (no change)	No resource pooling	Low Variation
1(I)	22.0	70%	Increase moderately	1.94	109%	Increase moderately	60 to 85%	No resource pooling	Low Variation
2(a)	30.2	132%	Increase moderately	4.61	396%	Increase a lot	60 to 95%	No resource pooling	Low Variation
3(b)	15.3	18%	Increase slightly	0.75	-20%	Decrease slightly	60 to 85%	Resource pooling	Low Variation
4(ab)	20.6	59%	Increase moderately	2.24	141%	Increase a lot	60 to 95%	Resource pooling	Low Variation
5(c)	43.1	232%	Increase a lot	2.97	219%	Increase a lot	60 to 85%	No resource pooling	High Variation
6(ac)	57.4	342%	Increase a lot	6.47	596%	Increase a lot	60 to 95%	No resource pooling	High Variation
7(bc)	16.8	29%	Increase slightly	0.86	-7%	Decrease slightly	60 to 85%	Resource pooling	High Variation
8 (abc)	22.0	69%	Increase moderately	2.88	210%	Increase a lot	60 to 95%	Resource pooling	High Variation

3.3. Scenarios used

Scenarios were designed to stimulate the respondent to put themselves in decision situations to test our theoretical hypotheses. The scenarios described a customer service process of one sports club that has experienced increased demand. This fact generates a high or medium expansion in resource utilization (Factor A), impacting process performance. In response to this change, the club makes some modifications that imply a change in how the activities performed are distributed to employees (Factor B) and in their time variation (Factor C).

After reading the scenario, participants responded to a questionnaire that assessed the impact of the operational changes described in the scenario on some process performance variables. The scenario manipulations are provided in ^{Appendix 1}, and the individual scale items from the questionnaire are provided in ^{Appendix 2}.

3.3.1. Measures

The authors chose four dependent variables related to key internal process performance measures: 1) flow time, 2) overall quality of service, 3) the responsiveness, assurance, and empathy of the service employees, and 4) queue size. Likert-type seven-point response scale with anchors of “Decrease a lot”(1), “It remains the same”(4), and “Increase a Lot”(7) was utilized.

Seven-point Likert scale was used to access manipulation checks, realism checks, and the question that measured the perception regarding the degree of knowledge of the OM topics addressed in this research. This scale used anchors of “Strongly Disagree”(1), “Neither agree nor disagree”(4), and “Strongly Agree”(7). All the scales are provided in ^{Appendix 2}.

3.3.2. Demand characteristics

The design of this experiment took some cautiousness to reduce the potential confounding effects of demand characteristics (Christensen et al., 2015). The authors used a between-subjects design that is less prone to demand characteristics. Also, it were adopted the good practices recommended in the Belmont Report of respecting people or treating participants as autonomous to make decisions about their participation in research (Ryan et al., 1979). Therefore, it was provided information about the purpose of the study, the benefits and risks of participation, and their rights to refuse participation or cease participation in the study.

3.4. Design used in the experiment

Considering the scenarios created in the computer simulation, the experiment consisted of a 2 × 2 × 2 between-subject factorial design, which allowed us to take into account interaction effects explicitly (Shadish & Cook, 2002). Three independent variables or factors (variability of activities, change in capacity utilization, and use of resource pooling) with two levels were used (Table 3).

Table 3
The independent variables and their levels used in this scenario.

Factor	Level -	Level +
The extent of change in resource utilization (A)	60% to 85% (41.7% increment)	60% to 95% (58.3% increment)
Resource pooling (B)	No change Activities separated	Change Activities grouped
Change in the variability of activities (C)	No change Low variability	Change Low to high variability

3.5. Sample and assignment

Participants were randomly assigned to one of the eight treatment conditions in the factorial design. Random assignment was used to minimize the likelihood of systematic between-group differences and maximize the internal validity of the experiment (Aguinis & Bradley, 2014). The randomly assigned procedure also attempts to eliminate the influence of all potential confounding third variables on the dependent variables because the influence of any extraneous variables is equal in the experimental conditions (Cozby & Bates, 2011).

The sample for this study was composed of undergraduate production engineering students from a large university in Brazil. As inclusion criteria, all participants must have completed courses involving basic OM knowledge, such as Operations Research, Production and Operations Management, and Supply Chain Management. Initially, a total of 178 respondents participated in the study. However, eight were discarded due to missing data. Thus, the final sample size was 170 participants, with an average of 21 participants and a slight variation in the number of participants per scenario. These values seem suitable since Hair et al. (2016) recommend a minimum size of 20 observations per cell with the sample sizes per cell approximately equal. Table 4 illustrates the scenarios used and shows the number of participants per scenario.

Table 4
Number of participants per scenario.

Scenario #	The extent of resource utilization change (A) (percent)	Resource pooling (B)	Variability of activities (C)	Sample size
1	60 to 85	Activities remain separated	Remains in low variability	20
2	60 to 95	Activities remain separated	Remains in low variability	19
3	60 to 85	Activities grouped	Remains in low variability	22
4	60 to 95	Activities grouped	Remains in low variability	21
5	60 to 85	Activities remain separated	Low to high variability	22
6	60 to 95	Activities remain separated	Low to high variability	21
7	60 to 85	Activities grouped	Low to high variability	21
8	60 to 95	Activities grouped	Low to high variability	24

4. Results

This section shows the results obtained from the experiment. The calculations and the statistical analysis were performed with R 4.0.5 (R Core Team, 2021).

4.1. Manipulation checks

Manipulation checks were performed to determine if the participants responded as planned to the independent variable manipulations in the experimental treatment conditions (Webster Junior & Sell, 2014). Tests show that the manipulations in this experiment worked as intended. There was a significant effect of the activities variability manipulation (F = 19.25; ${\mu }_{HighVariability}=3.4<{\mu }_{LowVariability}=4.85;$p < 0.001). The shift in the extent of change in utilization level manipulation was also significant (F = 7.76; ${\mu }_{Highchange}=6.07>{\mu }_{Moderatechange}=5.38;$p < 0.01). Finally, there was a significant effect of using resource pooling manipulation (F = 302.6;${\mu }_{resourcepooling}=1.92<{\mu }_{withoutresourcepooling}=6.35;$p < 0.001). Based on these results, it was found that participants perceived significant differences between the treatment conditions in the scenarios presented.

4.2. Realism check

A quantitative realism check was also performed in this study to check whether participants could recognize and respond to treatment conditions. Therefore, the items developed by Dabholkar (1994) were deployed to evaluate the realism of the scenarios in the experimental design (^{Appendix 2}). Participants were asked if the scenario was realistic and if they could imagine themselves in that situation. The realism check indicated that participants considered the scenarios to be engaging and realistic, with an average score of 5.56 on a seven-point scale. This finding suggests that participants’ perceptions of the scenario manipulations were realistic enough to evoke authentic behavioral responses.

4.3. MANOVA and ANOVA results

Multivariate Analysis of Variance (MANOVA) was conducted on the dependent variables in this study (Table 5). Given the large sample size and the robustness of MANOVA to depart from multivariate normality, violations of multivariate normality were not expected to be severe. In addition, the shape of histograms, normal-probability plots, skewness, and kurtosis for each dependent measure were reviewed which showed no severe departures from normality (Hair et al., 2016). Another assumption underlying MANOVA is equality of variance (Hair et al., 2016). This assumption was tested using the Levene test. It was found a value of 1.68 on statistical testing and a p-value of 0.1174 for the dependent variable flow time. It was also found a value of 1.22 on statistical testing and a p-value of 0.2958 for the dependent variable overall quality of service; and a value of 1.45 on statistical testing and a p-value of 0.1891 for the dependent variable customer satisfaction with service employees. Finally, it was identified a value of 1.18 on statistical testing and a p-value of 0.3142 for the dependent variable queue size. Because the data were collected during the same period, there was no need to evaluate the independence assumption.

Table 5
Results of MANOVA.

Effect	Wilks’ lambda	F 4, 159	p-value
Main effects
The extent of resource utilization change (A)	0.973	1.102	0.357
Resource pooling (B)	0.657	20.78	<0.001
Coefficient of variation (C)	0.946	2.260	0.065
Two-way interaction
A*B	0.972	1.145	0.337
A*C	0.983	0.693	0.598
B*C	0.967	1.367	0.248
Three-way interaction
A*B*C	0.983	0.701	0.592

Just one main effect of the independent variable resource pooling was observed (Wilks’ lambda = 0.657; F = 20.78; p < 0.001). Additional univariate tests showed that the use of resource pooling led to a decrease in flow time (F = 25.09; p < 0.001) and queue size (F = 19.89; p < 0.00l). These tests also showed that the use of resource pooling led to an increase in overall quality of service (F = 48.07; p < 0.001) and in customer satisfaction with service employees (F = 58.25; p < 0.001).

The univariate tests also showed that the increase of the coefficient of variation displayed in the scenarios had a significant effect on the decrease in the overall quality of service (F = 7.44; p < 0.01) and in customer satisfaction with service employees (F = 5.13; p < 0.05).

The univariate tests showed the existence of two factors interaction. The independent variables resource pooling (Factor B) and coefficient of variation (Factor C) interact on the dependent variable's overall quality of service (F = 4.43; p < 0.05). Also, the independent variables resource pooling (Factor B) and the extent of resource utilization change (Factor A) interact on the dependent variable customer satisfaction with service employees (F = 4.094; p < 0.05). These interactions are illustrated in Figure 4.

Figure 4
Interaction two-factor plots of the dependent variables overall quality of service and customer satisfaction with service employees.

Table 6 exhibits the mean values obtained for each independent variable at the two levels used in the study.

Table 6
Mean values obtained for each independent variable.

Independent variable	Flow time		The overall quality of service		Customer satisfaction with service employees		Queue size
	Mean	SE Mean	Mean	SE Mean	Mean	SE Mean	Mean	SE Mean
The extent of resource utilization change (A)
60% to 85%	4.05	0.134	4.35	0.166	4.52	0.136	4.56	0.159
60% to 95%	3.89	0.157	4.53	0.178	4.28	0.165	4.46	0.19
Resource pooling (B)
No resource pooling	4.48	0.114	3.68	0.152	3.68	0.133	5.06	0.149
Resource pooling	3.50	0.152	5.15	0.153	5.07	0.130	4.00	0.179
Coefficient of variation (C)
Low variation	3.94	0.141	4.74	0.161	4.62	0.133	4.43	0.188
High variation	4.00	0.150	4.16	0.176	4.19	0.163	4.59	0.164
No resource pooling
The low extent of utilization change	4.50	0.157	3.74	0.213	4.00	0.170	4.98	0.197
High extent of utilization change	4.45	0.168	3.62	0.220	3.35	0.195	5.15	0.225
Resource pooling
The low extent of utilization change	3.60	0.194	4.95	0.221	5.02	0.181	4.16	0.235
High extent of utilization change	3.40	0.234	5.33	0.211	5.11	0.189	3.84	0.270
No resource pooling
Low coefficient of variation	4.51	0.137	3.74	0.190	3.90	0.168	4.16	0.235
High coefficient of variation	4.44	0.180	3.63	0.235	3.49	0.201	3.84	0.270
Resource pooling
Low coefficient of variation	3.42	0.211	5.65	0.156	5.28	0.142	4.16	0.235
High coefficient of variation	3.58	0.221	4.67	0.240	4.87	0.212	3.84	0.270

• Note: All items were measured using scales ranging from “1” = Decrease a lot, to “7” = Increase a lot. Significant values are indicated in bold.

The Spearman correlation coefficient was used to assess the respondents’ perception of the typical trade-off between overall quality of service and queue size $(\rho =-\mathrm{0.35,}p<0.001)$ and between the overall quality of service and flow time $(\rho =-\mathrm{0.44,}p<0.001)$ The data confirmed the perception of these trade-offs by the respondents.

The authors verified the degree of correctness of the students' answers concerning the dependent variables flow time and queue size worked on in this experiment. This verification was done by comparing the student's answers with the computer simulation results of the scenarios presented in Table 2. Despite the high knowledge declared by the students about operations management (average of 5.8 points on a scale from 1 to 7), the results showed a moderate rate for the degree of correctness (34% for flow time and 43% for queue size).

Table 7 reports the hypotheses proposed in this paper and the results obtained in terms of statistical significance.

Table 7
Summary of the results of tests of hypotheses.

Hypothesis		Findings
H1 -The perceived impact in process performance measure … is higher in a situation including changes in the variability of activities duration rather than in a situation with no changes.
	a) In flow time	Not supported
	b) In queue size	Supported
	c) In overall quality of service	Supported
	d) In customer satisfaction with service employees	Not supported
H2 – The perceived impact in process performance measure … is higher when resource utilization escalates
	a) In flow time	Not supported
	b) In queue size	Not supported
	c) In overall quality of service	Supported*
	d) In customer satisfaction with service employees	Not supported
H3 – The perceived impact in process performance measure … is higher in a situation in which each resource performs several activities than in a situation in which each resource performs just one activity.
	a) In flow time	Supported
	b) In queue size	Supported
	c) In overall quality of service	Supported
	d) In customer satisfaction with service employees	Supported

• ^*
  Indicates this hypothesis was regarded as significant if we consider the concept of effect heredity.

These findings corroborate the hypotheses H_1b, H_1c, H_2c, H_3a, H_3b, H_3c, and H_3d. The results show a lack of understanding of the effects of increasing the resource utilization in processes (H₂) and increasing the variability on flow time (H_1a) and queue size (H_1d). However, these results indicate that students accurately perceived the effects of resource pooling on the performance dimensions proposed by the study (H₃).

These research findings will be appropriately addressed in the conclusions and final remarks section.

5. Conclusions and final remarks

The efforts in designing and modeling business processes necessarily comprise the identification of different elements, such as physical and immaterial resources, decision points, flow rates, activities and tasks, information flow, time flows, location of buffers, unit loads, amongst others (Bammert et al., 2020). Not rarely, processes can be very complex in their architecture, requiring a deeper understanding of their functioning so that optimization efforts may be more effective (Gall & Rinderle-Ma, 2020).

The data we obtained from this experiment suggests that our sample, composed of undergraduate engineering students, perceived resource pooling as an impactful practice (Tables 5 and 6). So, even in scenario #8 that showed a considerable increase in the level of resource utilization and a noteworthy increase in the variability of activities, 62.5% of the students reported that this practice has a positive impact enhancing the perceived quality of service (the mean grades are above 4 points). One possible reason that may explain an overestimation of this practice is the widespread acknowledgment of practices related to it. Thus, several academic texts of OM emphasize the importance of terms like “job enlargement,” “job enrichment,” “empowerment,” and “multi-skilling.” These terms use the resourcing pool practice that is associated with various management programs such as Quality Improvement, Lean Production, and Business Process Management (Jacobs & Chase, 2012; Krajewski et al., 2009; Slack et al., 2013a; Stevenson, 2014).

A common point in the scenarios is the increased workload reflected in the increased degree of resource utilization (A) that was driven by increased demand for the service. The variables resource pooling (B) and coefficient of variation (C) influenced the performance of the process. They allowed the construction of scenarios that showed trade-offs such as increased productivity by increasing the utilization of productive resources and the decreasing responsiveness of the service, and the trade-off between the increased productivity and the erosion of service (Cachon & Terwiesch, 2013; Oliva, 2001). Notwithstanding the existence of these trade-offs, students did not adequately identify the impact of these changes in the process on the performance indicators. This fact was present in the students independently of the respondents' perceived degree of knowledge about OM, indicating that this problem has a more general constitution. However, the perception of the influence of these variables is something important that students and managers of production engineering cannot neglect. The strong impact of the higher resource utilization rate on the processing time coming from the queue theory is a crucial concept for the production engineer who deals daily with the effects of actions taken in the process in search of higher productivity. This fault indicates an urgency in discussing new teaching approaches in the area. Gibbs et al. (1997) point out the need for teaching in engineering to develop skills in experimenting, designing, analyzing, and solving problems as essential for effective learning. The challenges of teaching production engineering are tremendous and go beyond the simple teaching of an established body of knowledge. There is a need for teaching strategies that focus on applying this knowledge to real-life situations.

In this experiment, the authors simulated the proposed scenarios using the computational simulation of discrete events, allowing the use of the Design of Experiments (DOE) to evaluate the effects between the three independent variables (A, B, and C) and the variables Flow Time and Queue size. It was verified through the ANOVA the significant occurrence of main effects, two-way interaction, and three-way interaction for all independent variables. Thus, given the existence of second and third-order interactions, the DOE presented a complex model to portray the relationship between the three independent variables and the two responses variables worked on in this text. Therefore, models of this nature are difficult to accurately design by people without computational tools. This fact shows that the basic concepts of OM in this text require the support of computational tools to be better understood. This statement is congruent with some authors who argued that simulation-based learning offers a superior method of helping students learn how to apply theoretical principles. Scholars in various disciplines further assert that computer simulations offer unique advantages in creating a problem-focused, engaging, and active learning environment. (e.g. Berends & Romme, 1999; Larreche, 1987; Medina-López et al., 2011; Salas et al., 2009; Showanasai et al., 2013). Some articles point to the need for educators to re-evaluate their approaches to teaching OM. They should be encouraged to adopt new teaching methods (Brandon‐Jones et al., 2012). Thus, these facts reinforce the need for undergraduate courses that contemplate subjects in the field of OM to work more intensely on simulation-based learning.

On the other hand, in our favor, there is a good variety of teaching methods that provide significant benefits for the understanding and practice of our vast body of knowledge. Thus, methods such as group exercises, experiential teaching methods, conventional lectures, business simulations, role-plays, live cases, and virtual learning environments can and should be used to enhance our students' learning (Brandon‐Jones et al., 2012). The challenges are substantial, but we have several successful examples of these strategies.

The results found in the survey are confined to production engineering students from a single university. This fact certainly limits the external validity of this research. External validity can be increased by applying this method to other production engineering students from other universities in Brazil and abroad. Finally, it seems important to strongly encourage extending this experience to professionals who are directly involved in service operations, as they are expected to deal with different assignments related to process management and ultimately to the proposal and evaluation of process performance metrics and process outcomes.

Appendix 1 Scenarios Content.

Read the scenario described carefully and answer the following questions.

Current Scenario

Imagine that you are responsible for the customer service area of one Sports Club. This area also includes support for clients' financial issues. All customer support is managed by dedicated software.

On average, 12 customers/hour (1 customer every 5 minutes) arrive at the customer service area. Most customer requests refer to the payment of late fees. Resolving these situations involves 3 sectors of the customer service area. The customer has to go through each of these sectors sequentially to solve their problem and thus resolve their debt. Each sector spends about 3 minutes on average, to serve each customer. Thus, in this situation, the level of employee utilization is around 60% (3/5).

Given the high degree of maturity of the current version of software used, the time of each of the activities involved in this process presents the average value of 3 minutes and a small variation in its resolution time.

New Scenario

A great expansion of the Club was finalized, which made it possible to increase the number of members. Consequently, this change has led to an increase in the number of customers seeking the customer service area. So, on average, arrive about {[16 customers / hour (1 customer every 3.75 minutes)] OR [19 customers / hour (1 customer every 3.16 minutes)]}.

Faced with this new situation, and in order to cope with the increased demand in the customer service area, the management acquired a new version of the software, which includes new features. In addition, the management hired a consulting firm specializing in Process Management that {[maintained the current customer service process, in which customers run sequentially across various sectors] OR [changed the current process, pooling all activities to an employee, which were previously performed by various sectors/ employees. In this situation, a single workstation performs all activities previously performed by three sectors]}.

Each sector spends on average of about 3 minutes to service each customer. Due to the increase in demand, the degree of utilization increased {[reasonably] OR [a lot]}. Changed from {[60% (3/5) to 80% (3 / 3.75)] OR [60% (3/5) to 95% (3/3,16)]} approximately.

The report by this consulting firm pointed out that this change will {[not affect the variation in the processing time of each of the activities involved in this process,] OR [also imply an expressive increase in the variation in the processing time of each of the activities involved in this process]} which continue with the average value of 3 minutes.

The following figure illustrates the change made*.

*One figure illustrating the current situation and new situation in the process has been added to facilitate the understanding of the scenario. This figure was used to describe the scenario 8.

Appendix 2 Questions submitted to the respondents.

DEPENDENT VARIABLES

Based on the information described in the text, evaluate the implications of these changes for the Club in the following dimensions:

− The total time of order fulfillment (Wait Time + Resolution Time).

− The degree of customer satisfaction with the service of order fulfillment.

− The degree of customer satisfaction with the service provided by the service employees.

− The number of people waiting for service.

Based on the information described in the text, answer the following question:

− I think that I have full knowledge of the basic concepts of operations management discussed in this text.

MANIPULATION CHECK VARIABLES

Based on the information described in the text, answer the following questions:

− In the new scenario, the time of each of the activities involved in this process remains with a small variation in its resolution time, keeping its variability low.

− In the new situation, the degree of utilization has increased dramatically, almost reaching 100%.

− In the new scenario, customers will still have to go through three sectors to finalize the fulfillment of their orders.

REALISM CHECK VARIABLES

− The situations described in these two scenarios are realistic.

− I can imagine myself as a customer in both scenarios described.

• How to cite this article: Torres Júnior, N., Azevedo, A. L., Simões, A. C., Ladeira, M. B., Sousa, P. R., & Freitas, L. S. (2022). Unveiling undergraduate production engineering students' comprehension of process flow measures. Production, 32, e20220020. https://doi.org/10.1590/0103-6513.20220020.

References

• Aguinis, H., & Bradley, K. J. (2014). Best practice recommendations for designing and implementing experimental vignette methodology studies. Organizational Research Methods, 17(4), 351-371. http://dx.doi.org/10.1177/1094428114547952
  » http://dx.doi.org/10.1177/1094428114547952
• Anupindi, R., Chopra, S., Deshmukh, S. D., Van Mieghem, J. A., & Zemel, E. (2011). Managing Business Process Flows: principles of operations management (3rd ed.). New Delhi: Pearson Education.
• Armistead, C., & Clark, G. (1994). The “coping” capacity management strategy in services and the influence on quality performance. International Journal of Service Industry Management, 5(2), 5-22. http://dx.doi.org/10.1108/09564239410057654
  » http://dx.doi.org/10.1108/09564239410057654
• Bammert, S., König, U. M., Roeglinger, M., & Wruck, T. (2020). Exploring potentials of digital nudging for business processes. Business Process Management Journal, 26(6), 1329-1347. http://dx.doi.org/10.1108/BPMJ-07-2019-0281
  » http://dx.doi.org/10.1108/BPMJ-07-2019-0281
• Berends, P., & Romme, G. (1999). Simulation as a research tool in management studies. European Management Journal, 17(6), 576-583. http://dx.doi.org/10.1016/S0263-2373(99)00048-1
  » http://dx.doi.org/10.1016/S0263-2373(99)00048-1
• Brandon‐Jones, A., Piercy, N., & Slack, N. (2012). Bringing teaching to life. International Journal of Operations & Production Management, 32(12), 1369-1374. http://dx.doi.org/10.1108/01443571211284142
  » http://dx.doi.org/10.1108/01443571211284142
• Cachon, G. G., & Terwiesch, C. (2013). Matching supply with demand : an introduction to operations management (3rd ed.). New York: McGraw Hill.
• Cattani, K., & Schmidt, G. M. (2005). The pooling principle. INFORMS Transactions on Education, 5(2), 17-24. http://dx.doi.org/10.1287/ited.5.2.17
  » http://dx.doi.org/10.1287/ited.5.2.17
• Chase, R. B. (1978). Where does the customer fit in a service operation? Harvard Business Review, 56(6), 137-142. PMid:10239167.
• Christensen, L. B., Johnson, R. B., & Turner, L. A. (2015). Research methods, design, and analysis (12th ed.). Harlow: Pearson Education.
• Cozby, P. C., & Bates, S. C. (2011). Methods in behavioral research (11th ed.). New York: McGraw Hill.
• Dabholkar, P. A. (1994). Incorporating choice into an attitudinal framework: analyzing models of mental comparison processes. The Journal of Consumer Research, 21(1), 100. http://dx.doi.org/10.1086/209385
  » http://dx.doi.org/10.1086/209385
• De Pourcq, K., Verleye, K., Larivière, B., Trybou, J., & Gemmel, P. (2021). Implications of customer participation in outsourcing non-core services to third parties. Journal of Service Management, 32(3), 438-458. http://dx.doi.org/10.1108/JOSM-09-2019-0295
  » http://dx.doi.org/10.1108/JOSM-09-2019-0295
• Eckerd, S. (2016). Experiments in purchasing and supply management research. Journal of Purchasing and Supply Management, 22(4), 258-261. http://dx.doi.org/10.1016/j.pursup.2016.08.002
  » http://dx.doi.org/10.1016/j.pursup.2016.08.002
• Ferdows, K., & Thurnheer, F. (2011). Building factory fitness. International Journal of Operations & Production Management, 31(9), 916-934. http://dx.doi.org/10.1108/01443571111165820
  » http://dx.doi.org/10.1108/01443571111165820
• Field, J. M., Victorino, L., Buell, R. W., Dixon, M. J., Meyer Goldstein, S., Menor, L. J., Pullman, M. E., Roth, A. V., Secchi, E., & Zhang, J. J. (2018). Service operations: what’s next? Journal of Service Management, 29(1), 55-97. http://dx.doi.org/10.1108/JOSM-08-2017-0191
  » http://dx.doi.org/10.1108/JOSM-08-2017-0191
• Gall, M., & Rinderle-Ma, S. (2020). Assessing process attribute visualization and interaction approaches based on a controlled experiment. International Journal of Cooperative Information Systems, 29(04), 2050007. http://dx.doi.org/10.1142/S0218843020500070
  » http://dx.doi.org/10.1142/S0218843020500070
• Gibbs, G., Gregory, R., & Moore, I. (1997). Teaching more students: 7-labs and practicals with more students and fewer resources. Oxford: Oxford Centre for Staff Development.
• Gryna, F. M. (2004). Work overload! Redesigning jobs to minimize stress and burnout. Wisconsin: ASQ Quality Press.
• Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2016). Multivariate data analysis (7th ed.). Harlow: Pearson Education.
• Heineke, J., & Davis, M. M. (2007). The emergence of service operations management as an academic discipline. Journal of Operations Management, 25(2), 364-374. http://dx.doi.org/10.1016/j.jom.2006.11.003
  » http://dx.doi.org/10.1016/j.jom.2006.11.003
• Hopp, W. J., & Spearman, M. L. (2008). Factory Physics (3rd ed.). Illinois: Waveland Press.
• Jacobs, F. R., & Chase, R. B. (2012). Administração de operações e da cadeia de suprimentos (13th ed.). Porto Alegre: McGraw Hill Brasil.
• Jasti, N. V. K., Kota, S., & Venkataraman, P. B. (2021). An impact of simulation labs on engineering students’ academic performance: a critical Investigation. Journal of Engineering Design and Technology, 19(1), 103-126. http://dx.doi.org/10.1108/JEDT-03-2020-0108
  » http://dx.doi.org/10.1108/JEDT-03-2020-0108
• Kandampully, J. (2000). The impact of demand fluctuation on the quality of service: A tourism industry example. Managing Service Quality, 10(1), 10-19. http://dx.doi.org/10.1108/09604520010307012
  » http://dx.doi.org/10.1108/09604520010307012
• Krajewski, L. J., Ritzman, L. P., & Malhotra, M. (2009). Administração da produção e operações (8th ed.). São Paulo: Prentice Hall Brasil.
• Larreche, J.-C. (1987). On simulations in business education and research. Journal of Business Research, 15(6), 559-571. http://dx.doi.org/10.1016/0148-2963(87)90039-7
  » http://dx.doi.org/10.1016/0148-2963(87)90039-7
• Lemke, F., Clark, M., & Wilson, H. (2011). Customer experience quality: an exploration in business and consumer contexts using repertory grid technique. Journal of the Academy of Marketing Science, 39(6), 846-869. http://dx.doi.org/10.1007/s11747-010-0219-0
  » http://dx.doi.org/10.1007/s11747-010-0219-0
• Liker, J. K. (2021). The Toyota Way: 14 management principles from the world’s greatest manufacturer (2nd ed.). New York: McGraw-Hill Education.
• Little, J. D. C. (2011). Little’s law as viewed on its 50th anniversary. Operations Research, 59(3), 536-549. http://dx.doi.org/10.1287/opre.1110.0940
  » http://dx.doi.org/10.1287/opre.1110.0940
• Machuca, J. D., González-Zamora, M. D. M., & Aguilar-Escobar, V. G. (2007). Service Operations Management research. Journal of Operations Management, 25(3), 585-603. http://dx.doi.org/10.1016/j.jom.2006.04.005
  » http://dx.doi.org/10.1016/j.jom.2006.04.005
• Mansar, S. L., & Reijers, H. A. (2005). Best practices in business process redesign: validation of a redesign framework. Computers in Industry, 56(5), 457-471. http://dx.doi.org/10.1016/j.compind.2005.01.001
  » http://dx.doi.org/10.1016/j.compind.2005.01.001
• Medina-López, C., Alfalla-Luque, R., & Arenas-Márquez, F. J. (2011). Active learning in Operations Management: interactive multimedia software for teaching JIT/Lean Production. Journal of Industrial Engineering and Management, 4(1), 31-80. http://dx.doi.org/10.3926/jiem.2011.v4n1.p31-80
  » http://dx.doi.org/10.3926/jiem.2011.v4n1.p31-80
• Min, B. S., & Smyth, R. (2014). Corporate governance, globalization and firm productivity. Journal of World Business, 49(3), 372-385. http://dx.doi.org/10.1016/j.jwb.2013.07.004
  » http://dx.doi.org/10.1016/j.jwb.2013.07.004
• Negrão, L. L. L., Godinho Filho, M., & Marodin, G. (2017). Lean practices and their effect on performance: a literature review. Production Planning and Control, 28(1), 33-56.
• Oliva, R. (2001). Tradeoffs in responses to work pressure in the service industry. California Management Review, 43(4), 26-43. http://dx.doi.org/10.2307/41166099
  » http://dx.doi.org/10.2307/41166099
• Poornikoo, M., & Qureshi, M. A. (2019). System dynamics modeling with fuzzy logic application to mitigate the bullwhip effect in supply chains. Journal of Modelling in Management, 14(3), 610-627. http://dx.doi.org/10.1108/JM2-04-2018-0045
  » http://dx.doi.org/10.1108/JM2-04-2018-0045
• Pound, E. S., Bell, J. H., & Spearman, M. L. (2014). Factory physics for managers: how leaders improve performance in a post-lean six sigma world. New York: McGraw Hill.
• Pullman, M. E., & Moore, W. L. (1999). Optimal service design: Integrating marketing and operations perspectives. International Journal of Service Industry Management, 10(2), 239-260. http://dx.doi.org/10.1108/09564239910264361
  » http://dx.doi.org/10.1108/09564239910264361
• R Core Team (2021). R: a language and environment for statistical computing. R Foundation for Statistical Computing. Retrieved in 18 Februart 2022, from https://www.R-project.org/
  » https://www.R-project.org/
• Reid, R. D., & Sanders, N. R. (2005). Gestão de operações. Rio de Janeiro: LTC.
• Rungtusanatham, M., Wallin, C., & Eckerd, S. (2011). The vignette in a scenario-based role-playing experiment. The Journal of Supply Chain Management, 47(3), 9-16. http://dx.doi.org/10.1111/j.1745-493X.2011.03232.x
  » http://dx.doi.org/10.1111/j.1745-493X.2011.03232.x
• Ryan, K. J., Joseph, V. B., Cooke, R. E., Height, D. I., Jonsen, A. R., King, P., & Lebacqz, K. (1979). The Belmont Report: ethical principles and guidelines for the protection of human subjects of research. In The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. Washington, DC: Dhew Publication.
• Salas, E., Wildman, J. L., & Piccolo, R. F. (2009). Using simulation-based training to enhance management education. Academy of Management Learning & Education, 8(4), 559-573.
• Sayer, N. J., & Williams, B. (2007). Lean for dummies. Hoboken: Wiley Publishing, Inc.
• Schmenner, R. W. (2012). Getting and staying productive: applying swift, even flow to practice. Cambridge: Cambridge University Press. http://dx.doi.org/10.1017/CBO9781139108775
  » http://dx.doi.org/10.1017/CBO9781139108775
• Shadish, W. R., & Cook, T. D. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.
• Showanasai, P., Lu, J., & Hallinger, P. (2013). Developing tools for research on school leadership development. Journal of Educational Administration, 51(1), 72-91. http://dx.doi.org/10.1108/09578231311291440
  » http://dx.doi.org/10.1108/09578231311291440
• Slack, N., Brandon-Jones, A., & Johnston, R. (2013a). Operations management (7th ed.). Harlow: Pearson.
• Slack, N., Chambers, S., Johnston, R., & Betts, A. (2013b). Gerenciamento de operações e de processos: princípios e práticas de impacto estratégico (2ª ed.). Porto Alegre: Bookman.
• Spearman, M. L., & Hopp, W. J. (2009). Teaching operations management from a science of manufacturing. Production and Operations Management, 7(2), 132-145. http://dx.doi.org/10.1111/j.1937-5956.1998.tb00445.x
  » http://dx.doi.org/10.1111/j.1937-5956.1998.tb00445.x
• Stevenson, W. J. (2014). Operations management. England: McGraw-Hill Education.
• Webster Junior, M., & Sell, J. (2014). Laboratory experiments in the social sciences (2nd ed.). London: Academic Press.
• Yao, Y., Duan, Y., & Huo, J. (2021). On empirically estimating bullwhip effects: measurement, aggregation, and impact. Journal of Operations Management, 67(1), 5-30. http://dx.doi.org/10.1002/joom.1090
  » http://dx.doi.org/10.1002/joom.1090
• Yin, X. (2021). Measuring the bullwhip effect with market competition among retailers: a simulation study. Computers & Operations Research, 132, 105341. http://dx.doi.org/10.1016/j.cor.2021.105341
  » http://dx.doi.org/10.1016/j.cor.2021.105341

Publication Dates

• Publication in this collection
  09 Sept 2022
• Date of issue
  2022

History

• Received
  18 Feb 2022
• Accepted
  10 Aug 2022