Debriefing's Influence on Learning in Business Game: An Experimental Design

ABSTRACT

This paper studies the influence of debriefing in the learning of the participants of business games. Through quasi-experimental study we examined the self-declarations of 112 undergraduate Business Management students undertaking the 8^th term, divided into two groups: experimental, exposed to debriefing; and control, not exposed to the debriefing. Mann-Whitney´s tests revealed that the quantum of learning perceived by the members of the experimental group was statistically significantly higher than the members of the control group in seven out of nine learning variables assessed (p < 0.05). The average effect size (d = 0.45) shows an average improvement of 18%. These results suggested that the debriefing positively influence on experiential learning cycle promoted by business games. In the perspective of Kolb's experiential learning cycle (Kolb, 1984), our findings suggest that Reflective observation and Abstract conceptualization stages can be reinforced by subsequent debriefing activities to the simulation rounds, in order to enhance continuous processes of action and reflection of the participants, according to the spiral experiential learning cycle.

Keywords:
Business game; Moderation; Experiential learning; Experiment

RESUMO

Analisa-se a influência na aprendizagem dos participantes de jogos de empresas do debriefing (etapa na qual os participantes fazem reflexões pós-ação). Por meio de estudo quase experimental foram examinadas as autodeclarações de 112 alunos do 8º período do curso de graduação em Administração de duas instituições de ensino, divididos em dois grupos: experimental, exposto ao debriefing; e de controle, não exposto ao debriefing. Os resultados (teste de Mann-Whitney) evidenciaram que o quantum da aprendizagem percebida pelos membros do grupo experimental foi superior ao dos membros do grupo de controle de forma estatisticamente significativa em sete das nove variáveis de aprendizagem avaliadas (p < 0,05). A média do tamanho do efeito (d = 0,45) mostra uma melhoria média de 18%, indicando que o debriefing influencia positivamente no ciclo de aprendizagem vivencial promovido por jogos de empresas. Sob a lente do ciclo da aprendizagem vivencial de Kolb (1984), os achados sugerem que as etapas Observação reflexiva e Conceituação abstrata podem ser reforçadas por atividades de debriefing posteriores às rodadas da simulação, potencializando processos contínuos de ação e reflexão dos participantes, consoante com o ciclo em espiral da aprendizagem vivencial.

Palavras-chave:
Jogos de empresas; Moderação; Aprendizagem vivencial; Experimento

1. INTRODUCTION

With the support of computer science and theories of learning, business games have become educational alternatives in the teaching-learning process. Specifically in Brazil, the use of business games is present in almost half of the courses of Business Management, according to Araújo, Brito, Correia, Paiva and Santos (2015).

Several studies have demonstrated that business games can contribute to the learning of its participants, many of which are based on the experiential learning theory as theoretical approach, markedly from the experiential learning cycle (e.g. Ben-Zvi & Carton, 2008, Crookall & Thornagate, 2009, Meij, Leemkuil & Li, 2013, Dias, Sauaia & Yoshizaki, 2013, Araújo et al., 2015). The experiential learning cycle considers learning as a spiral quadric cycle where people learn through experience, which supports the translation of experiences into concepts, allowing their application in new experiences: concrete experience, abstract conceptualization, reflexive observation and active experimentation (Kolb, 1984).

Research that evaluates the contributions of business games in learning for business administration students (focus of this study) used variables operationalized in terms of management training as a proxy for learning. These variables can be grouped into three dimensions: knowledge, skills and attitudes (e.g. Sauaia, 1995, Lacruz & Villela, 2006, Dias, Moreira & Stosick, 2013, Fitó-Bertran, Hernández-Lara, Serradell-López, 2015).

Studies suggest a broad set of variables that can affect the perception of the participants regarding their learning, such as gender, cognitive style, complexity of business game, duration of the simulation, previous contact with another business game, method to evaluate the performance of business game etc. In this perspective, there are authors who advocate debriefing (also termed as after-action reviews) as a critical stage of business games in relation to participants' learning (Peters &Vissers, 2004, Ben-Zvi &Carton, 2008, Kriz, 2010, Meij et al., 2013). The central argument is that during debriefing the participants promote after-action reflections in the quest to understand what happened and the reason why, so that they could learn from the ex post reflection. Although there is also abundant literature on debriefing (e.g. Meij et al., 2013, Tannenbaum & Cerasoli, 2013), no empirical research on the moderating role of debriefing in business games was identified in previous studies, so little is known, objectively, about its influence. From what we can question: When adopting games with and without debriefing, can we expect asymmetry in participants' learning? Would there be asymmetry in relation to the learning dimensions perceived by the participants and/or in relation to the stages of the experiential learning cycle? Can business games participants, without debriefing, not close the experiential learning cycle due to lack of reflective activities? Or, in the opposite direction, would the elements present in business games lead the participants to promote reflections during the decision-making process and in the monitoring of results that would make the effect of after-action reflection during the debriefing into a marginal effect? The observation of this research gap motivated the accomplishment of this study, in the search for the improvement of the learning process with business games.

In this connection, we seek to contribute with the advancement of this knowledge, exploring the influence of debriefing on business games, more specifically when applied to undergraduates in Business Administration. We clarify that the teaching-learning strategies are not considered as mutually exclusive, advocating in favor of business games to the detriment of other approaches, since business games are considered as an alternative of teaching-learning that, along with other teaching-learning techniques and tools (expository classes, case studies, junior enterprise, extension activities etc.), can collaborate so that learning is more effective and that students have greater satisfaction during the process.

The results of this study bring important advances to the field. Although many studies suggest relationships between debriefing and learning (e.g. Gentry, 1990, Peters & Vissers, 2004, Ben-Zvi & Carton, 2008, Kriz, 2010, Meij et al., 2013), we identified no empirical study that objectively evaluated the possible influences. Meij et al. (2013), in comparing two conditions (individual and collaborative self-debriefing) from the application of a set of individual business game with high school students in Taiwan, evidenced that the individual performance in the participants' game in individual self-debriefing increased more than that of the participants in the condition of collaborative self-debriefing.

This study expands the scope when analyzing the influences considering different learning dimensions in conditions with and without debriefing. To our knowledge, these relationships have not yet been examined, so this study advances in the discussions of the effect of debriefing on learning in business games.

2. THEORETICAL BACKGROUND AND RESEARCH PROBLEM

From the 1950s, with the development of micro-computing, simulators for didactic support emerged that allowed the development of teaching-learning experiences termed as business games. Its use as a teaching tool in Higher Education Institutions (HEI) dates back to 1956, with the launch of Top Management Decision Simulation in the United States (Keys& Wolfe, 1990). Since then, the use of business games as a teaching-learning tool in business schools and as a research theme has been increasing. Its usage as a teaching-learning tool with students in the business area has been growing progressively around the world and the main reason for its popularity seems to be associated with the view that business games are tools that allow students to learn from the playful experiences that turn participants into central actors of the teaching-learning process (Sauaia, 1995,Gentry, 1990, Peach & Hornyak, 2000).

Business games, such as educational technique, are designed to provide participants with a learning experience, serving as a bridge between academia, past experience and the business environment, from the representation of reality (specific situations in the business area), by means of simulation techniques (portraying laboratory conditions of a given reality, not just a simulation of the company, but the market) and through experience with game participants (bringing interactivity and teamwork) in decision-making processes (Lacruz, 2004). In other words, in business games business environments are simulated in which several groups manage different virtual companies competing in the same industry, allowing participants to learn from their own experience.

There are many studies on business games and from different perspectives. Faria (2001), based on articles published in the Association for Business Simulation and Experiential Learning between 1975 and 2001, identified that research on business game are focused on the factors that lead to good performance in business games; on the effectiveness and efficiency of business games; on the skills that can be learned through the adoption of business games; and on the study of how this learning occurs.

Several studies have supported the experiential learning cycle proposed by Kolb (1984) as support for the use of business games as a teaching-learning tool (e.g. Ben-Zvi &Carton, 2008, Crookall &Thornagate, 2009, Meij et al., 2013, Dias et al, 2013, Araújo et al., 2015). In Kolb's (1984, p.38) words, "Learning is the process whereby knowledge is created through the transformation of experience". At the core of this conceptualization is the tension between dialectical dimensions (concrete/abstract and active/reflective) which is solved by mental operations of experience prehension and its transformation, by considering learning as a spiral quadric cycle in which people learn through experience, that support the translation of experiences into concepts, allowing their application in new experiences: concrete experience, abstract conceptualization, reflexive observation and active experimentation (Figure 1).

Figure 1
Experiential Learning Cycle.

Kolb (1984) explains that in the process of holding people grasp (apprehension) and seize (understanding) the experience through concrete experience and abstract conceptualization.

• - concrete experience: refers to the experiences that occurred during simulation that lead to the immediate knowledge (apprehension) for the search of solutions to the dilemmas proposed in the experience.

• - abstract conceptualization: characterized by the formation of concepts derived from the analysis of life experiences (comprehension), from the organization of knowledge obtained and/or rescued through experience to form a generalizable conceptual framework.

And that the process of transformation leads to the creation of meaning for living, through reflective observation and active experimentation.

• - reflective observation: in regards to the personal examination, and in the context of ideas, life experiences (faced dilemmas, possibilities of choices, performed tasks, results obtained etc.), in order to transform the knowledge resulting from prehension.

• - active experimentation: it deals with the connections with what is real, in an external movement, from analogies and comparisons with the aspects of the experience projected in other situations, in order to bring meaning to what has already been understood.

In short, in the experiential learning cycle, apprehension and transformation are combined by the comprehension and transformation of experience, in which people exercise the role of actor (action) and observer (reflection).

In this connection, Crookall and Thorngate (2009) advocate that the simulation is a means capable of linking tacit knowledge to explicit and harmonizing action and knowledge, in a cycle in which action (experience) leads to knowledge (conceptualization) and knowledge enables and perfects action, according to experiential learning. Ben-Zvi and Carton (2008) evaluated the approximation between business games and experiential learning. For the authors, participation in business games would promote learning, in line with the theoretical lens of experiential learning, provided that certain important operational parameters are assured, such as prior guidance, debriefing and the adoption of a passive role by the teacher. The process would have the following dynamics: initially the professor's prior guidance (game moderator) teaches the terminology. Then, the students progress through integrating knowledge from several disciplines and the activities of the game promote their internalization. As students need to engage in decision-making without greater support from the teacher, they are forced to rely on self-knowledge. They have to analyze various situations and evaluate the outcome of their decisions based on their knowledge. Finally, at a more abstract level, the dynamics lead to self-questioning by students on how and why they made such decisions, allowing them to identify their weaknesses and strengths and thus to be equipped with metacognitive knowledge.

In business games, the experiential learning cycle begins with concrete experience, which unfolds in effects with which the participants make contact in the simulation that lead to apprehension of the conditions of the immediate experience. Then, from reflection on the example arising from immediate experience, it is possible to examine and select actions that can be taken in similar circumstances by projecting plausible outcomes of these actions, which leads to comprehension and general understanding of the life experience of a generalized nature, in which explanatory hypotheses emerge not only for the particular example of that experience. Finally, when the general principle is understood, learning outcomes can be tested, with active experimentation, within the possibilities offered by generalization, from which the learning cycle is continuously renewed. Dias et al.(2013) argue that in this process there is a risk that participants will not close the learning cycle due to lack of reflective activities; in view of the above, they suggest the adoption of steps that stimulate the analysis of the results of the game rounds so that reflective observations are made that contribute to the closing of the experiential learning cycle.

Many studies have shown that participants in business games point to them as being the method that provides learning with greater involvement and participation (e.g. Peach & Hornyak, 2003) and that contributed greatly to learning in management training (e.g. Sauaia, 1995, Lacruz & Villela, 2006, Dias et al., 2013, Fitó-Bertran et al.2015). From this we assume that participation in business games contributes to learning in terms of management training.

And in much of the research the quantum of learning resulting from participation in business games was measured by the participants' self-declaration, in accordance with the suggestion by Gentry (1990) that, from the perspective of the experiential learning theory, the evaluation of learning is done by the participants themselves, as an integrated part of the learning process; from which it is assumed that the quantum of learning resulting from participation in business games can be measured by the participants' self-declaration.

It should be noted that, despite agreeing with such assumptions, the complexity involved in defining and measuring learning is not ignored - which in business games has been defined in most studies based on Bloom's et al. (1956) taxonomy and the experiential learning cycle by Kolb (1984) - and measured by participants' self-declaration. On the other hand, there are sufficient numbers of studies that allow arguing that business games are a valid method to teach management.

We also add that we acknowledge the existence of a wide set of variables that can affect the participants' perception regarding the simulation, their performance and their learning, some of which have been empirically tested: as gender (e.g. Florea et al., 2003, Apesteguia, Azmat & Iriberri, 2012), cognitive style (e.g. Peters & Vissers, 2004), previous academic performance (e.g. Gosen & Washbush, 1999, Sauaia, 2006), complexity of the simulator (e.g. Teach & Murff, 2008, Hall &Cox, 1994). However, for other variables, we did not identify empirical studies that verified its influence on the perception of participants in business games: related to the subject-participant, as prior contact with another business game; and the dynamics of the exercise, such as the duration of the game, the method to evaluate participants' performance, and the debriefing stage.

There are authors who advocate debriefing as a critical stage in business games in relation to participants' learning (e.g. Gentry, 1990, Peters & Vissers, 2004, Ben-Zvi & Carton, 2008, Kriz, 2010, Meij et al., 2013). Tannenbaum and Cerasoli (2013) conducted a meta-analysis research, considering its application in several areas (medical, educational, psychological, military, organizational etc.), to examine the empirical support for the effectiveness claimed for debriefing. The search in the literature resulted a total of 31 studies (with 29 being published and 2 unpublished), with 46 samples (n = 2,136), the majority of whom were involved in comparisons between the same group and a sample physicians (61%). Their analysis showed that debriefing produces a significant advantage over conditions without debriefing. An average improvement of approximately 25% (d = 0.67) in the learning outcomes was found. However, since most of the data came from quasi-experimental designs, causal inferences must be elaborated carefully. We emphasize that we did not identify in this study evaluation involving business games.

It should be noted that there are several ways to promote debriefing: expert-led or self-debriefing; oral or written; individual or collaborative; after the completion of the game or at the end of each round; either guided or not by the facilitator; with or without the participation of external observers as an additional feedback element (Kriz, 2010, Meij et al., 2013,Tannenbaum & Cerasoli, 2013). The abundant diversity of debriefing studies reveals that there may be subtle but relevant, differences in how debriefing is defined. In this article, in harmony with most of the business game applications, and based on the description by Fanning and Gaba (2007), the debriefing set-up is characterized as a facilitated or guided reflection in the experiential learning cycle, structured around a set of questions that invites participants to reflect on their experience in business games, in order to involve them in an analytical process that revolves around the review and analysis of the events that occurred during the business game; which can be characterized as an oral, collaborative self-debriefing, guided by the moderator, carried out at the end of each round of the business game and without the participation of external observers.

In spite of numerous studies that evidenced the contributions of business games to the learning of their participants, however, the influence of debriefing on this process is unclear. Can we expect asymmetry in relation to the learning dimensions, or in relation to the stages of the experiential learning cycle, when games with and without debriefing are adopted? As a result, there is a need for more research to identify the real benefits of debriefing in business games, in terms of student learning development. The incipient empirical evidence regarding the influence of debriefing on business games motivated this research.

So, considering the motivation for this study and that participation in business games contributes to learning in terms of management training of their participants, and the quantum of learning resulting from participation in business games can be measured by participants' self-declaration, the research problem can be summarized by means of a question: Does the debriefing stage in business games applied to undergraduates in Business Administration influence the learning of their participants?

Due to the research problem, and the findings of other debriefing studies, we propose the following guiding hypothesis: - the quantum of perceived learning as coming from the stake in business game with debriefing is greater than that of business games without debriefing.

3. METHODS

We use in this descriptive study, characterized by the quantitative approach, a cross-sectional study to investigate the influence of debriefing on the learning perception of participants in business games and can be characterized as a quasi-experiment.

For Campbell and Stanley (1979), experiment is a type of research in which a variable (independent) is manipulated by the researcher and its effects on another (dependent) variable are observed, with all other factors constant. This research can be classified as a quasi-experiment, with two groups, one that was exposed to treatment (debriefing stage) and another control group (which was not exposed to treatment), whereby both measures were performed only after treatment. We clarify that the pairing of the test units was conducted in order to guarantee greater similarity between the groups in relation to age and gender, for we acknowledge that these variables can affect the participants' perception of their learning. Figure 2 illustrates the research project from Campbell and Stanley's (1979) classic notation system.

Figure 2
Research Project.

Note: Exposure to an event is represented by "X" and the record of measurement by "O". The continuous line represents the temporal order.

3.1. Selection of business games

In order to select the business game, we excluded non-computerized business games, assuming that the complexity and interactivity of these types of business games would not satisfy those sought in the study; non-interactive, i.e., those which the decisions of each team did not influence or were influenced by the others; applied at a distance, without the direct assistance of the teacher, who would establish limitations to debriefing (as defined in this study); functional, i.e., that they focused only on one area of the company; who did not have a manual; and where the investment necessary for its acquisition or use were an impediment.

We selected the GI-MICRO business game (version 6 demo - limited to four rounds processing) because it is a game of medium complexity, and has been used in several studies (e.g. Mecheln, 2003, Lacruz & Vilella, 2006, Gerber, 2006, Souza &Cardoso, 2012).

3.2. Construction of the instrument and measures

the support on which this study is developed is the teaching-learning process in the Administration area, more specifically under the theoretical perspective of the experiential learning theory. In similar research with business games as a teaching-learning instrument in Business Administration courses, and under the same theoretical approach, we used questionnaires of data collection instruments that sought to identify the main dimensions present in business games and learning levels of participants (e.g. Sauaia, 1995, Peach &Hornyak, 2003). From these references, and the experience with the application of business games, we developed the data collection instrument that comprised aspects as seen in Table 1.

Table 1
Operationalization of Variables

Variable					Theoretical background
Control	Age range		vDem_1	Ordinal	-
	Gender		vDem_2	Nominal	Florea et al. (2003). Apesteguia. Azmat and Iriberri (2012)
Learning	Knowledge	Acquiring new knowledge	vOp_1	Ordinal	Sauaia (1995). Lacruz and Villela (2006). Dias et. al. (2013). Fitó-Bertran et al. (2015)
		Integrating knowledge from various areas of business administration	vOp_2
		Updating already existing knowledge	vOp_3
	Skills	Identifying problems. Evaluating alternatives. Formulating and implementing solutions and evaluating their results	vOp_4
		Developing a systemic view (holistic)	vOp_5
	Attitudes	Adapting to new situations (flexibility)	vOp_6
		Encouraging teamwork	vOp_7
		Developing/Improving leadership	vOp_8
		Resolving conflicts	vOp_9

Source: Authors own elaboration.

The learning perception was measured using a Likert scale, adopting it with intervals, under the premise that respondents will treat the differences between adjacent categories as equals and their adequacy for self-perception measurement (Malhotra, 2006).

3.3. Preparation of the debriefing script

By using the experiential learning cycle (Kolb, 1984), we designed a semi-structured debriefing script based on a model proposed by Sims (2002), which is guided by four stages: the first phase (concrete experience) involves questions that ask participants to describe their perceptions and feelings during business games. In the second phase (reflexive observation), participants are invited to consider these experiences from different points of view (e.g., in relation to values and cultures). Subsequently, in the third phase (abstract conceptualization), the participants are encouraged to reflect on concepts and models. And finally, in the last phase (active experimentation), participants are asked to come up with some guidelines that can be considered in future actions. For each topic, we formulated guiding debriefing questions so that participants could reconstruct their experiences, valuing their own perspectives.

3.4. Sample and data colletion

The GI-MICRO business game was applied to undergraduates of the 8^th term of the Business Administration course of two private HEIs in Minas Gerais (Brazil). We emphasize that the test units were matched so that the groups' profiles were similar in relation to age and gender. Moreover, that after the pairing we randomly selected the test units, using the SPSS software.

We collected the data through a structured self-completion questionnaire using the Survey Monkey online platform. We applied the questionnaire at the end of the business game, with the presence and supervision of the game's moderator. All 112 respondents answered the questionnaire online. After the analysis of missing data, the final sample consisted of 108 cases (96% of the total), with 55 from the game with debriefing and 53 from the game without debriefing (Table 2).

Table 2
Sample Description

Groups	Simulated Companies	Participants	Respondents	Valid Questionnaires	Response rate
Experimental (with debriefing)	14	56	56	55	98%
Control (without debriefing)	14	56	56	53	95%
Total	28	112	112	108	96%

Source: Authors own elaboration.

We clarify that a pre-test of the questionnaire was performed, through a protocol analysis, in which the respondent "thinks out loud" (Malhotra, 2006), in order to identify misunderstandings in the data collection instrument more clearly. Eight students participated in the pre-test (divided into 2 groups of 4 students) of one of the HEI composing the sample after having participated in 4 rounds of the same set of business game. We elucidate that these participants were exclusively involved in the pre-test. The research universe for the pre-test was constituted by a sample that consisted of 7% of the predicted population (112).

We reinforce that the groups' profile in relation to gender and age was similar (which was guaranteed by the pairing process), as seen in Table 3.

Table 3
Distribution of Groups by Age and Gender.

Groups	Size	Age		Gender
		Mean	Standard Deviation	Women	Men
Experimental (exposed to debriefing)	55	24.62	3.19	64%	36%
Control (not exposed to debriefing)	53	22.89	1.93	68%	34%

Source: Authors own elaboration.

3.5. Application of the business game

The GI-MICRO business game was applied in two different ways in each of the HEIs: with and without debriefing. Both applications were conducted by the same moderator, to different groups of students, and circumscribed 4 rounds of the business game in the second half of 2015. 112 students participated in the business game (64 from HEI_1 and 48 from HEI_2), equally distributed in the business game with and without debriefing. In both cases, only students who stated that they did not have previous experience with business games participated and were organized in groups composed of 4 students (as indicated in the GI-MICRO manual, in order for each student to take up position in the board of directors: general, marketing, financial and production).

We chose to use the same set of business game, with the same duration, for students with no previous experience with business games, of the last term of the Business Administration course, in the same HEI (experimental group x control group), with similar profiles in relation to gender and age, and with the same moderator, to reduce influences of extraneous variables that could interfere in the results.

One of the authors of this study was the moderator of all the games (pre-test, business game with debriefing and business game without debriefing), assuring uniformity and consistency both in the preparation and in the implementation of the business game and also in the procedures for collecting data. We also assured the use of the operational parameters in all business games, as indicated by Ben-Zvi and Carton (2008) as important for the approximation between business games and the experiential learning cycle on providing learning of their participants: prior guidance, debriefing (only for the experimental group) and adoption of a passive role by the moderator.

The moderator initially made the presentation of the GI-MICRO business game, emphasizing the rules and scenario of the simulation, as well as the dynamics of the activity. He clarified that participants received the director's manual in advance. Then each group prepared the planning for their simulated company, and the decision rounds were started. In each of the four rounds, after decisions are processed by the business game software, the groups received feedback on the results of the decisions through reports and market information by the newspaper and a partial ranking. With the fourth round concluded, we defined the final ranking, with the accumulated profit as criterion. When applying the business game to the experimental group, debriefing was conducted at the end of each round. At the end, the questionnaire was applied to the participants.

3.6. Analysis

We used the Cronbach's alpha coefficient to estimate the reliability of the scale used in the applied questionnaire. And to verify if the data of each item could come from a variable with normal distribution we applied Shapiro-Wilk test.

We describe the opinion variables in terms of mean, standard deviation, minimum and maximum. We used Spearman's correlation coefficient to evaluate the strength of the relationship between opinion variables and control variables (age and gender), in order to verify if age and gender could influence in a relevant way the perception of learning, even though the groups were composed in a very similar way (Table 3).

Through the Mann-Whitney test we verified whether the quantum of learning perceived by the members of the experimental group was higher than that of the members of the control group; and by Cohen's d, Cohen's and CL (common language effect size statistic), the effect size. We assumed the level of significance of 0.05 and we used the SPSS software (Cronbach's alpha, Shapiro-Wilk test, Spearman correlation coefficient and Mann-Whitney U test) and GPower 3.1 (statistical power of Cohen's d test). We calculated Cohen's and CL indices manually.

In order to clarify the problem of the research, a schematic drawing of the study is shown, which summarizes the relationships analyzed in this study: (i) we assume that business games contribute to participants' learning, (ii) the moderating role of the debriefing step in this relationship is evaluated, and (iii) the influence of age and gender (control variables) of participants in their learning (Figure 3).

Figure 3
Conceptual Structure of the Study.

4. RESULTS AND DISCUSSION

Prior to initiating the measurement extraction procedures, we estimated the reliability of the scale used in the questionnaire using Cronbach's alpha coefficient. The result obtained for both samples (experimental group = 0.81 and control group = 0.65) indicates that the scale used to measure participants' perception was considered adequate, because it is above the threshold (> 0.6) from which the value is considered appropriate for non-causal studies (Hair et al. 2009). Then, the variables of opinion were characterized (Table 4).

Table 4
Descriptive Statistics of Opinion Variables.

Variable	Experimental group				Control group
	Mean	Standard Deviation	Minimum	Maximum	Mean	Standard Deviation	Minimum	Maximum
vOp_1	4.56	5.00	0.69	3.00	5.00	4.34	4.00	0.73
vOp_2	4.53	5.00	0.63	3.00	5.00	4.25	4.00	0.85
vOp_3	4.49	5.00	0.69	3.00	5.00	4.17	4.00	0.85
vOp_4	4.62	5.00	0.56	3.00	5.00	4.21	4.00	0.79
vOp_5	4.49	5.00	0.54	3.00	5.00	4.08	4.00	0.70
vOp_6	4.47	5.00	0.72	3.00	5.00	4.30	4.00	0.70
vOp_7	4.45	5.00	0.69	3.00	5.00	4.42	5.00	0.84
vOp_8	4.27	4.00	0.76	2.00	5.00	3.92	4.00	0.98
vOp_9	4.45	5.00	0.81	3.00	5.00	4.15	4.00	0.69

Source: Authors own elaboration.

Both the group exposed to debriefing and the non-exposed pointed to participation in the business game with a strong contribution to learning (lowest mean = 3.92) and homogeneously (higher standard deviation = 0.98). We observed that the mean of the experimental group in all variables was higher than the mean of the control group, in line with literature that debriefing contributes to learning. Also, with the exception of the variables Adapting to new situations (vOp_6) and Resolving Conflicts (vOp_9), the perception of the members of the experimental group is more uniform (lower standard deviations); insinuating that debriefing contributes so that learning in business games also occurs in a more homogeneous manner among the participants.

The highest mean of perceived learning by the group exposed to the debriefing in relation to the non-exposed one (Table 4) is in line with the findings of other debriefing studies - although not as an element of the formative cycle with business games - taking into account the recent meta-analysis by Tannenbaum and Cerasoli (2013). In the course of business games, participants need to make use of a variety of generic and specific management skills, and debriefing seems to contribute triggering this process by promoting a reflection of the results in view of the actions taken. It is possible that the experience with business games will unfold without the activity of evaluating decisions and results actually taking place and with debriefing the reflection begins to compose the cycle of formation of the participants.

Then, we verified by the Shapiro-Wilk test that the null hypothesis (which assumes that the sample was extracted from a normally distributed population) could be refuted (p-value 0.000) in all nine items, so that it should not be assumed that there is normality in the data source variable.

Then, we verified the association between control variables (age and gender) and opinion. By the Spearman's ρ correlation coefficients, it was evidenced that the few statistically significant correlations are classified as weak (0.2 < ρ < 0.4), and from this we can assume that the statistically significant differences between the conditions of age and gender between the groups (experimental and control) did not have a relevant influence on the participants' perception of learning.

Lastly, we processed the non-parametric Mann-Whitney U test to test the general hypothesis of this study. Table 5 summarizes Mann-Whitney U-tests for independent samples, along with the effect size (Cohen's d, Cohen's and CL) and the statistical power of the test.

Table 5
Summary of Results.

Variable	Description	Mann-Whitney's U test	p-value a	Effect size				Test power	Decision
				Cohen's d		Cohen's U3	CLb
vOp_l	Acquiring new knowledge	1182.0	0.026	0.31	Small	62.4	58.7	0.47	Inconclusive
vOp_2	Integrating knowledge from various areas of business administration	1212.5	0.046	0.37	Small	65.0	60.5	0.60	H0 Rejected
vOp_3	Updating already existing knowledge	1154.0	0.021	0.41	Small	66.5	61.6	0.67	H0 Rejected
vOp_4	Identifying problems. Evaluating alternatives. Formulating and implementing solutions and evaluating their results	1043.5	0.002	0.60	Medium	74.0	66.8	0.92	H0 Rejected
vOp_5	Developing a systemic view (holistic)	998.5	0.001	0.66	Medium	76.2	68.4	0.95	H0 Rejected
vOp_6	Adapting to new situations (flexibility)	1243.5	0.080	0.24	Small	59.6	56.7	0.33	H0 Not rejected
vOp_7	Encouraging teamwork	1438.0	0.457	0.04	Insignificant	51.6	51.0	0.07	H0 Not rejected
vOp_8	Developing/Improving leadership	1171.5	0.030	0.40	Small	66.0	61.2	0.64	H0 Rejected
vOp_9	Resolving conflicts	1038.5	0.006	0.40	Small	65.9	61.2	0.64	H0 Rejected

• Note: Experimental Group (n = 55), control group (n = 53) and α = 0.05.
• ^a
  Corrected for ties (one-tailed)
• ^b
  Common language effect size statistic
Source: Authors own elaboration.

The results refute the hypothesis of equality (p-value< 0.05) perceived learning between groups from 7 to 9 variables (vOp_1-5 and vOp_8-9). Except for the variable vOp_1, the statistical power of the test was higher than the threshold (1 - β > 0.5) from which value is considered appropriate in social science (Hill & Hill, 2000, Murphy & Myors, 1988). In relation to the variable vOp_1, given the level of significance adopted in the study (α = 0.05), the low power of statistics is related to the sample size and the effect size. Effect size was determined by Cohen's d (since the size of the two groups is similar and the standard deviations are similar), and may be considered small for the variables vOp_1-3 and vOp_8-9, according to the classification criterion proposed by Cohen (1998) for testing the difference between means of independent groups. As a result, the results in relation to acquiring new knowledge (vOp_1) show that although there is evidence to reject the null hypothesis that there is no difference in the quantum of perceived learning by participants in business games with and without debriefing, the probability of correctly rejecting the null hypothesis when the alternative hypothesis is true (1 - β) is low, which makes the results inconclusive (Cohen, 1988). On the other hand, this result allows estimating the sample size needed for other studies to obtain statistical power from the appropriate test given a small effect size. Considered d = 0.30, 0.35 and 0.40, the minimum sample size would be, respectively, 290, 214 and 164 observations, and this evidences the sensitivity of sample size in face of variations in effect size.

It should also be noted that for the variables Op_4 and vOp_5 we accepted the alternative hypothesis that the quantum of perceived learning by the participants of business games with debriefing (experimental group) is superior to that of those who participated in games without debriefing (control group) and that the effect size was considered medium (0.49 < d < 0.8 = medium). Regarding the effect size, caution should be exercised in interpreting the classification. We adopted the Cohen's (1998) classification, because particularly new results are explored and could not be compared with other findings in the literature, except to a limited extent, with the studies by Tannenbaum and Cerasoli (2013), whose meta-analysis returned a d = 0.67, and Lipsey et al. (2012), for which intervention studies in education rarely have an effect size above 0.3. In this respect, by interpreting the effects in the light of the research area, the average efficacy of the studies analyzed by Tannenbaum and Cerasoli (2013) was similar to that obtained in this study regarding variables vOp_4 (d = 0.60) and vOp_5 (d = 0.66), which in turn had effects above the threshold proposed by Lipsey et al. (2012) for educational studies. However, the comparison is limited by different contexts. In view of this, we need to ask: what are the practical meanings of these results?

It should be noted that in relation to the contribution of business games to the development of systemic view (vOp_5), which was the variable with the largest effect size (d = 0.66), 76% of participants in business games with debriefing perceived this aspect in a superior way to the participants in business games without debriefing. That is, the average of the participants exposed to the debriefing position is in the 76th percentile of the group not exposed to debriefing (Cohen's ). Or, from another perspective, in relation to this dimension, the probability of participants in business games with debriefing perceiving learning in a superior way than the participants in business games without debriefing is of 69%, as shown by the CL. Equally relevant was the result in relation to the variable Practicing decision making (vOp_4), with d = 0.60, Cohen's = 74 and CL = 66.8. In the business game dynamics, the participants find themselves as problem situations that require capacity for interpretation and critical analysis, which involve concepts related to various disciplines and links with the areas of the company and its social environment, which contributes to the development of a holistic vision, as solutions require the simultaneous application of concepts and tools from various areas in the process of problem identifying, evaluating alternatives, formulating and implementing solutions and evaluating their results. It can be proposed that debriefing will contribute to participants' understanding of the connections between different contents and functional areas, reinforcing the systemic and complex nature of the simulated activity, because promoting ex post reflection contributes to the need to simultaneously apply various concepts and tools, either by their self-reflection, or by the sharing of reflections of other participants. In addition, in debriefing there is the opportunity to promote self-criticism on the extent to which it was able to instrumentalize knowledge that was available or that were acquired during the simulation in the situations experienced.

In analyzing only from the point of view of the size of the effect, considering the lowest result (d = 0.31) between the variables with p < 0.05 (vOp_1), 62% of participants in business games with debriefing found that they acquired new knowledge in the simulated activity in a way superior to the participants in business games without debriefing (Cohen's ); and the probability of participants in business games with debriefing perceiving learning in a superior way than the participants in business games without debriefing was of 59% (CL). Therefore, even in considering the smallest size of the effect, it seems to us that the cost of including debriefing in business games (more time in the preparation and execution of the activity) is of little relevance in the face of potential benefits for participants' learning. In this study the average improvement of debriefing in learning perception was 18% (Cohen medium = 68).

It is important to emphasize, in analyzing only in relation to effect size, that in relation to the variable Adapting to new situations (vOp_6), the average improvement in the perception of learning in relation to this dimension was 10% (Cohen = 59.6). We recognize, therefore, that even if the difference was not statistically significant and the power of statistics was small (at the level of the adopted statistical significance), debriefing contributed in a relevant way in the perception of learning in relation to this factor, with the cost-benefit being evaluated. The same cannot be assumed, however, in relation to the variable Encouraging teamwork (vOp_7), whose effect size was insignificant (d = 0.04). That is, business games contribute to stimulating teamwork (mean = 4.45 [experimental] and 4.42 [control]), but debriefing does not influence significantly statistically speaking, nor in any relevant way, in this process. This, perhaps, due to the way debriefing was operationalized, in which the participants discussed their personal impressions about the simulation.

And, finally, there is evidence that debriefing reinforces elements characteristic of business games, especially those related to the abilities of the participants to apply in concrete-hypothetical cases knowledge that they already had or that were acquired or updated in the simulation. On the other hand, in relation to effective teamwork among group participants, the insights of a personal nature regarding the way the group interacted and the posture of how to be a better listener and member of a group (vOp_7) were not influenced by debriefing, whereas the posture as a mentor and seller of ideas (vOp_8) seems to have been influenced, since the effect size was relevant d = 0.40, Cohen's = 66 and CL = 61.2.

In summary, we suggest that participants in business games may not reflect during the simulation about the challenges experienced, the decisions taken and the effects arising from their action in an in-depth manner, often due to the pressure on the time limits established for the delivery of decisions, to the involvement with the character of competition between the teams, so that the debriefing marks the moment for reflection on the actions taken and feelings experienced, contributing to the achievement of insights and generalizations resulting from the simulation.

5. CONCLUSIONS

The objective of this study was to verify the moderating role of debriefing in the learning of participants in business games, using as lens the experiential learning theory. The proposed analytical framework suggests that debriefing reinforces elements present in business games, since the participants promote ex-reflections in the decision-making process, monitor the implementation of the proposed solutions ex cursum and debriefing reinforces the evaluation of ex post results, contributing to the formation of a virtuous learning cycle. The reflexive observation and abstract conceptualization stages are planned in the perspective of experiential learning, which in relation to business games can be reinforced by debriefing activities after the rounds of the simulation, in order to potentiate continuous processes of action and reflection of the participants, depending on the spiral experiential learning cycle.

In business games we have on one side the desirable creativity of thinking and on the other the need of doing, in complementary relations, according to other studies that highlight business games as a tool that favors the learning of doing. We propose, from the findings of the study, that the debriefing implies reflections that lead to learning to learn, at the abstract level of the experiential learning cycle, in which the dynamics of the debriefing contributes to the participants' questioning on why and how they made the decisions, as well as the reasons for the results, allowing them to identify fragilities and forces in a kind of "autophagy" of the processes that allows them to go through the experiential learning cycle.

Thus, the results bring two important advances to the area. First, although many studies suggest relationships between debriefing and learning (e.g. Gentry, 1990, Peters & Vissers, 2004, Ben-Zvi & Carton, 2008, Kriz, 2010, Meij et al., 2013), the evaluation of possible influences in conditions with and without debriefing constituted a research gap. This article analyzes the influences considering different dimensions of learning in conditions with and without debriefing, advancing the discussions of the effect of debriefing on learning in business games, revealing the influence of debriefing on the Reflective observation and abstract Conceptualization stages of Kolb's (1984) experiential learning cycle. Second, the results being presented with effect size and statistical power allow other studies to compare the average effectiveness of the model developed in this study in the light of their area of research, conferring practical significance to the effect size. This is particularly unusual in studies on business games in Brazil, in which even in the experimental studies the size of the effect and the power of statistics is not presented or the power of statistics is presented from an arbitrary average effect size. This compromises the interpretation of results because one does not know the probability of correctly rejecting the null hypothesis when the alternative hypothesis is true.

However, this study, like most, has limitations. Whereas the reliability of the scale used seems to be acceptable for non-causal studies (Cronbach's alpha of > 0.6), its validity must be determined. In addition, comparisons were made of groups of students assumed to be similar, but which may not present similarities in terms of previous training, experiência profissional, cognitive styles and cultural traits (aspects that were not considered as control variables in this study).

Another limitation relates to the way of measuring participants learning in the business game by self-declaration. We acknowledge its limitation as a tool for measuring learning, because the perception of learning may be associated with the emotional dimension that the game triggers by placing participants as active and central elements of the learning process, causing a "good feeling" of what a halo effect may elicit in terms of measuring perceived learning (Gentry et al., 1998). On the other hand, findings from attribution theory suggest that perceptions of behavior may result in actual behaviors (Kelly, 1971, Martinko, 1995).

Finally, the results apply to the observations of students from two higher education institutions and only one set of business game in particular, as in other studies (e.g. Dias, Sauaia & Yoshizaki, 2013, Meij et al., 2013, Mayer et al., 2011), of which we can not refute that their results are dependent on that particular game, there is no possibility of generalization for business games. On the other hand, the results suggest relationships that can be used in future studies.

Thus, to reach a wider range of conclusions, we suggest replicating this research in order to compare the results, both in relation to the hypotheses and the effect size. Investigations could also be conducted to explore the implications of debriefing on different educational arrangements, such as distance learning, and levels of education, MBA for example. In addition, researchers could examine the differences between participants with different learning styles, for example, from the inventory of learning styles of Kolb (2011), elaborated on the experiential learning theory by Kolb (1984), and in business games that promoted debriefing of different forms, like guided and not guided by the moderator. Finally, we hope that this study will collaborate with future researches and the construction of lesson plans with business games that take into account debriefing in the process of management qualification.

6. REFERENCES

• Apesteguia, J., Azmat, G.,& Iriberri, N. (2012). The impact of gender composition on team performance and decision-making: evidence from the field. Management Science, 58(1), 78-93.
• Araújo, U. P., Brito, M. J. de, Correia, L. F., Paiva, , F. D.,& Santos, A. de O. (2015).Simulação de negócios no ensino da administração em centro de educação brasileiro. Revista Iberoamericana sobre calidad, eficacia y cambio en educación, 13(2), 99-130
• Ben-Zvi, T.,& Carton, T. C. (2008).Applying Bloom's revised taxonomy in business games. Developments in business simulation and experiential exercises, 35, 265-272.
• Bloom, B. S., Engelhart, M. D., Frust, E. J., Hill, W. H.,& Krathwohl, D. R. (1956).Taxonomy of educational objectives. New York: David McKay.
• Campbell, D. T.,& Stanley, J. C. (1979).Delineamentos experimentais e quase experimentais de pesquisa. São Paulo: EPU.
• Cohen, J. A.(1998). Power primer. In: Kazdin, A. E. Methodological issues and strategies in clinical research. Washington: APA.
• Crookall, T.,& Thorngate, W. (2009). Acting, knowing, learning, simulating, gaming. Simulation & Gaming, 40(1), 8-26.
• Dias, G. P. P., Sauaia, A. C. A.,& Yoshizaki, H. T. Y. (2013). Estilos de aprendizagem Felder-Silverman e o aprendizado com jogos de empresas. Revista de administração de empresas,53(5), 469-484.
• Dias, C. M. Jr, Moreira, B.,&Stosick, E. (2013). Um estudo sobre o desenvolvimento de competências em ambiente simulado. Novas Tecnologias na Educação, 11(1), 10-23.
• Fanning, R. M.,& Gaba, D. M. (2007). The role of debriefing in simulation-based learning. Simulation in Healthcare, 2(2), 115-125.
• Faria, A. J. (2001). The changing nature of business simulation/gaming research: a brief history. Simulation & Gaming, 32(1), 97-110.
• Fitó-Bertran, A., Hernández-Lara, A. B.,& Serradell-López, E. (2015). The effect of competences on learning results and educational experience with a business simulator. Computers in human behavior, 51, 910-914.
• Florea, N. B.,Boyer, M. A., Brown, S. W., Butler, M. J.,Hernandez, M., Weir, K., Meng, L., Johnson, P. R., Lima, C.,& Mayall, H. J. (2003). Negotiating from Mars to Venus: gender in simulated international negotiations. Simulation & Gaming, 34(2), 226-248.
• Gentry, J. W. (1990). What is experiential learning? In: Gentry, J. W. (Coord) Guide to business gaming and experiential learning. London: Nichols.
• Gentry, J.W., Commuri, S. R., Burns, A. C.,& Dickinson, J. R. (1998). The second component to experiential learning: a look back at how ABSEL has handled the conceptual and operational definitions of learning. Developments in business simulation and experiential exercises,25, 62-68.
• Gerber, J. Z. (2006). Jogo de empresas e a formação de líderes empresariais(Tese de doutorado). Universidade Federal de Santa Catarina, Florianópolis, SC, Brasil.
• Gosen, J.,& Washbush, J. (1999). Perceptions of learning in TE simulations. Developments in business simulation and experiential exercises, 26, 170-175.
• Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E.,& Tatham, R. L. (2009). Análise multivariada de dados. Porto Alegre: Bookman.
• Hall, J.,& Cox, B. (1994). Complexity: it is really that simple. Developments in business simulations and experiential exercises, 21, 30-34.
• Hill, M. & Hill. (2000). A investigação por questionário. Lisboa: Sílabo.
• Kelly, H. H. (1971). Attribution in social interaction. New Jersey: Learning Press.
• Keys, J. B.,& Wolfe, J. (1990). The role of management games and simulations for education and research. Journal of management, 16(2), 307-336.
• Kolb, D. A. (1984). Experiential learning: experience as source of learning and development. New Jersey: Prentice Hall.
• Kolb, D. A. (2011). Kolb learning style inventory:version 4.0. Boston: Hay Resources Direct.
• Kriz, W. C. (2010). A systemic-constructivist approach to the facilitation and debriefing os fimulations and games. Simulation & Gaming, 41(5), 663-680.
• Lacruz, A. J. (2004). Jogos de empresas: considerações teóricas. Caderno de pesquisas em administração, 11(4), 93-109.
• Lacruz, A. J.,& Vilella, L. E. (2006). Percepção dos participantes de jogos de empresas quanto às condições facilitadoras para o aprendizado em programas de simulação empresarial: um estudo exploratório. Anais do Seminários em Administração, São Paulo, SP, 9.
• Lipsey, M. W., Puzzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., Roberts, M., Anthony, K. S.,& Busick, M. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. Washington: IES.
• Malhotra, N. K. (2006). Pesquisa de marketing. Porto Alegre: Bookman.
• Mayer, B. W., Dale, K. M., Fraccastoro, K. A., & Moss, G. (2011). Improving transfer of learning: relationship to methods of using business simulation. Simulation and Gaming, 42(1), 64-84.
• Martinko, M. (1995). Attribution theory: an organizational perspective. Florida: St. Lucie Press.
• Meij, H. van der, Leemkuil, H., & Li, J. (2013). Does individual or collaborative self-debriefing better enhance learning from game?Computers in human behavior, 29(6), 2471-2479.
• Mecheln, P. J. von (2003). Jogo de empresas, ambiente interativo e agentes computacionais mediadores (Tese de doutorado). Universidade Federal de Santa Catarina, Florianópolis, SC, Brasil.
• Murphy, K., & Myors, B. (1998). Statistical power analysis: a simple and general model for traditional and modern hypothesis tests. New Jersey: Erlbaum.
• Peach, B. E.,& Hornyak, M. (2003). What are simulations for? Learning objectives as a simulation device. Developments in business simulation and experiential learning, 30, 220-224.
• Peters, V.,& Vissers, G. (2004). A simple classification model for debriefing simulation games. Simulation & Gaming, 35(1), 70-84.
• Sauaia, A. C. A. (1995). Satisfação e aprendizagem em jogos de empresas: contribuições para a educação gerencial (Tese de doutorado). Universidade de São Paulo, São Paulo, SP, Brasil.
• Sauaia, A. C. A. (2006). Conhecimento versus desempenho das organizações: um estudo empírico com jogos de empresas. Revista eletrônica de administração, 12(1), 1-17.
• Sims, R. R. (2002). Debriefing experiential learning exercises in ethics education. Teaching business ethics, 6(2), 179-197.
• Souza, T. M. P. de & Cardoso, A. M. P. (2012). Diretrizes de interface para jogos de empresas. Revista de informação, 13(6).
• Tannenbaum, S. I.,& Cerasoli, C. P. (2013). Do team and individual debriefs enhance performance? A meta-analysis. Human factors, 55(1), 231-245.
• Teach, R.,& Murff, E. (2008). Are the business simulations we play too complex? Developments in business simulation and experiential learning, 35, 205-211.

Publication Dates

• Publication in this collection
  Mar-Apr 2018

History

• Received
  28 Apr 2016
• Reviewed
  31 May 2016
• Accepted
  21 Mar 2017