Adoption of business game under the perspective of an excellence model in an MBA course: multicriteria peer evaluation analysis

Abstract

This article describes a business game, one of the last courses in an MBA syllabus. The database analyzed consists of outcomes from 47 classes and 1410 students. The aim of the game is to integrate the skills and abilities acquired during the program. The game considerers an evaluation system with 39 items based on an Excellence Model that covers important aspects of the management process. Only six evaluation items are financial goals; the remaining 33 go through a peer evaluation. These items deal with consumer-supplier aspects and management issues. A reduction in dimensionality was attained through a Factor Analysis that yielded 14 factors. Is it possible to identify the relevant factors for winning the game? Two distinct tests (significance of a logit model parameters and t-test for equality of means) are performed to answer this question. A generalized linear model using a logit link function and a Bernoulli distribution was fitted to the data, confirming that most factors with statistically significant differences in the mean between winners and other participants were also significant in the model. Participants’ feedback confirmed that the game was well received by the students: it boosted students’ morale and interest in the subject, fostered entrepreneurship and less absenteeism, increased the degree of class participation, awareness of teamwork, and planning, and created an environment that favored a discussion on business ethics.

Keywords:
MBA; Business game; Business excellence model; Factor analysis; Logit model

Resumo

Este artigo descreve um jogo de negócios aplicado em cursos do tipo Master of Business Administration (MBA) como uma das últimas disciplinas do currículo e analisa os resultados de sua implementação em 47 turmas, envolvendo 1.410 alunos. O jogo de negócios tem por objetivo integrar as habilidades e competências desenvolvidas ao longo do curso e considera um sistema de avaliação com 39 quesitos baseado em um modelo de excelência que abrange todos os aspectos importantes no processo de gestão. Somente 6 quesitos de avaliação são financeiros e os outros 33 são avaliados pelos próprios alunos, em uma avaliação por pares, e dizem respeito à relação cliente-fornecedor e à gestão de cada empresa. Uma redução de dimensionalidade foi obtida por meio de uma análise fatorial, resultando em 14 fatores. Dentre esses fatores, é possível identificar quais são relevantes para o sucesso no jogo? Esta questão é respondida averiguando por meio de 2 métodos estatísticos distintos (significância dos parâmetros de um modelo logito e teste t para diferença de médias) quais são esses fatores. Ajustou-se um modelo linear generalizado com uma função de ligação logito e uma distribuição Bernouilli, confirmando que a maioria dos fatores com diferenças estatisticamente significativas das médias de vencedores e demais participantes também foram estatisticamente significativas no modelo. A avaliação dos alunos participantes confirmou que o jogo de negócios foi bem recebido: levantou o moral dos alunos; despertou o interesse pelo assunto; fomentou o empreendedorismo; diminuiu o absenteísmo; aumentou a participação em sala, a consciência do trabalho em grupo e o planejamento; e criou um ambiente favorável às discussões sobre ética nos negócios.

Palavras-chave:
Master of Business Administration; Jogos de negócios; Modelo de excelência da gestão; Análise fatorial; Modelo logito

Resumen

Este artículo describe un juego de negocios aplicado en los cursos de MBA, como una de las últimas asignaturas del programa y analiza los resultados de su implantación en 47 clases y 1.410 alumnos. El juego tiene como objetivo integrar las habilidades y competencias desarrolladas a lo largo del curso y considera un sistema de evaluación con 39 cuestiones basado en un modelo de excelencia que abarca todos los aspectos importantes del proceso de gestión. Sólo seis cuestiones de la evaluación son financieras y las otras 33 son evaluadas por los propios alumnos, en una evaluación por pares, y se refieren a la relación cliente-proveedor y a la gestión de cada empresa. Mediante un análisis factorial se obtuvo una reducción de dimensionalidad que resultó en 14 factores. Para responder la pregunta ¿entre esos factores, es posible identificar cuáles son relevantes para el éxito en el juego? se aplicaron dos tests (significancia de los parámetros de un modelo logit y prueba t para igualdad de medias). Se ajustó un modelo lineal generalizado con una función de enlace logit y una distribución Bernouilli, que confirman que la mayoría de los factores con diferencias estadísticamente significativas de los promedios de ganadores y demás participantes, también fueron estadísticamente significativas en el modelo. Los comentarios de los alumnos participantes confirmaron que el Juego fue bien recibido: levantó la moral de los alumnos, despertó el interés por el tema, fomentó el espíritu emprendedor, disminuyó el absentismo, aumentó la participación en el aula, la conciencia del trabajo en grupo y la planificación, y creó un ambiente favorable a las discusiones de grupo sobre ética en los negocios.

Palabras clave:
MBA; Juegos de negocios; Modelo de excelencia de la gestión; Análisis factorial; Modelo logit

INTRODUCTION

A revolution has been witnessed in the value creation process since the end of the 20th century, which can be partly attributed to the confluence of globalization and technological advances in the transition from an industrial economy to an information-based economy. This context involves much complexity, instability, and uncertainty. Thus, organizational changes have come to be viewed as one of the main ways to structure and explore the new business world. This exploration of new organizational methods requires higher education institutions to strategically “play” with their teaching, research, and outreach abilities to bring the academic world closer to the business world.

In this context, Master of Business Administration (MBA) programs have experienced — especially in recent decades — a progressive and marked expansion, reinforcing their social and economic importance as an instrument for updating, qualifying, and training professionals for the labor market. These programs play an important role in the economic growth of a country, which involves financial aspects and stimuli for new investments, whether they be in infrastructure, the construction of appropriate environments, the acoustics of these environments, the acquisition of multimedia equipment, the payment of teachers and employees, travel and accommodation of teachers, and others. MBA programs are powerful generators of jobs and gatherers of service, commerce, and industry activities. This new concept is based on the idea that MBA students, most of whom are professionals operating in the market, are not simply recipients of information (the basic assumption in traditional classes) but, rather, intangible assets who should be treated as such.

The relevant function that MBA programs have been performing generates a need to use new teaching techniques that aim, on the one hand, for a re-evaluation of the work of the teacher who is now viewed as a facilitator of learning and, on the other hand, for the total involvement of the student with the teaching-learning process — known as experiential learning (KOLB, 1984).

Studies related to experiential learning (WOLFE and KEYS, 1997) have established a connection between it and business simulations and business games, known internationally as simulation and gaming (S&G).

The use of S&G has grown in recent years in Brazil (BARÇANTE, PITHON and BROCHADO, 2010) in both academic and business environments. The prediction of Rosas and Sauaia (2006) has come to fruition in graduate programs: the most important business schools in Brazil have adopted disciplines such as business games, company games, or business simulations in their programs, which have generally been positively evaluated by students.

These disciplines provide a classroom environment that is closer to the environment of a competitive market and represents teaching techniques that stimulate creativity, promote the exercise of communication, enable the exchange of experiences, and stimulate the experience of new roles and decision making in an environment of risk (VERSIANI and FACHIN, 2007). These techniques have experienced a significant expansion at the world level — including in Brazil — over the last 40 years (BRAGGE, THAVIKULWAT and TÖYLI, 2011; FARIA, HUTCHINSON and WELLINGTON, 2009). The international journal Simulation & Gaming, between 1970 (v. 1, n. 1) and 2017 (v. 48, n. 5), published 2,360 articles (SIMULATION & GAMING: AN INTERDISCIPLINARY JOURNAL, 1970-2017).

The number of publications in Brazilian congresses is much more modest. Table 1 provides the number of articles on business games presented at six annual events — three in Business Administration and three in Engineering. These events were chosen because they are the most frequent forums for publications by researchers on the theme. The information was collected from the online annals of the following events: Brazilian Congress of Engineering (COBENGE, 2017) from 1999 to 2016; National Production Engineering Meeting (ENEGEP, 2017) from 1994 to 2016; Production Engineering Symposium (SIMPEP, 2017) from 2000 to 2016; Meeting of the National Association of Graduate Education and Research in Business Administration (ENANPAD, 2017) from 1990 to 2017; Meeting of the National Association of Undergraduate Programs in Business Administration (ENANGRAD, 2017) from 2004 to 2016; and Business Administration Seminars (SEMEAD, 2017) from 1998 to 2016.

Table 1
Articles about business games published in Brazilian congresses

	ENEGEP 1994-2016	SIMPEP 2000-2016	COBENGE 1999-2016	ENANPAD 1990-2017	ENANGRAD 2004-2016	SEMEAD 1998-2016	TOTAL
Up to 2000	18	1	5	8	-	1	33
2001-2010	52	22	15	52	8	43	191
2011-2014	48	15	15	10	-	26	114
2015-2017	7	3	4	3	4	7	28
TOTAL	125	41	38	73	12	77	366

• Source: Elaborated by the authors.

The present article uses the results of the applications of a business game in 47 MBA classes, with a total of 1,410 students allocated among 402 companies. Using two different statistical methods (significance of the parameters of a logit model and t-test for difference of means), the factors relevant to the success of the companies in the game are determined. The evaluation of the companies in the game is based on what business excellence models advocate.

Business games

The importance of business simulation is a permanent topic in the scientific world and has been debated exhaustively in a non-univocal manner (GOSEN and WASHBUSH, 2004; GROSSLER, 2004; FARIA, 2001; MALIK and HOWARD, 1996). In discussing the validity of using business games, near unanimity exists over the point that a business game depends heavily on the quality of the model used and its implementation. According to Stainton, Johnson and Borodzicz (2010), three sets of suggestions based on the available literature and the experience of business game designers can be adopted to improve the games. These suggestions concern characteristics of the environment, the content, and the implementation of the game.

The game environment (ELGOOD, 1993; NORRIS, 1986; ALESSI, 1988) must be:

1. .verisimilar, that is, close to the real business world environment;

2. . complex and flexible to changes, without becoming confusing; and,

3. . competitive and strategic.

The content of the game (KNOWLES, 1980) must be:

1. .comprehensive and relevant enough to make the exercise stimulating and the participants motivated to get involved;

2. . versatile and treated with the appropriate depth to be able to achieve various learning objectives; and,

3. . foster the development of the participants’ learning.

The game moves (FARIA, 2001; BIGGS, 1999; GENTRY, 1990; KOLB, 1984) must:

1. .show that participants’ decisions impact the team’s performance;

2. . show that the team’s performance is related to its outcome in the game;

3. . develop the cognitive processes through the participatory management theory;

4. . show that the learning process can be behavioral through learning by doing;

5. . enable cooperation and partnerships and making the exercise more enjoyable and rewarding;

6. . be experiential to enable learning through the application of theory in practice;

7. . be dynamic and allow the problems that arise to be promptly resolved by the participants; and,

8. . integrate knowledge already acquired.

In addition to the mentioned suggestions, others should be taken into account. According to Hall (2004), the duration of each game must be envisaged in such a way that sufficient time exists for an analysis of the results, as well as decision making, debates, and reflection; the game manual should be written clearly and objectively to provide the necessary information to the participants; and the teachers — viewed as facilitators of learning — need to be trained and have enough technical and business knowledge to provide support and, in particular, feedback to the participants (GOSEN, 2004; HALL, 2004; WOLFE, 1997). Finally, the supporting material (physical or computational) must undergo pilot testing to ensure that its use is reliable (OAKSHOTT, 1997).

Business excellence models

At least 94 awards were associated with business excellence models used in 83 countries worldwide (MOHAMMAD, MANN, GRIGG et al., 2011; LU, BETTS and CROOM, 2013). The Business Excellence Model (BEM; known in Portuguese as Modelo de Excelência da Gestão or MEG), created in 1991 by the Brazilian Quality Foundation (Fundação Nacional da Qualidade - FNQ), serves as the basis for the FNQ award. The BEM has been improved to adjust to market changes, and it can be used as a guide to make companies more competitive. At the time of application of the games, the BEM was composed of 13 fundamentals and eight criteria (FNQ, 2013) and was the model used as a reference for the definition of the weights and the elaboration of the evaluation items described in this article. The model has already been used with this objective in undergraduate courses (BELTRÃO and BARÇANTE, 2015). In 2016, the BEM underwent an update (FNQ, 2016), which innovated by being formatted into eight fundamentals that replaced the former eight criteria. The fundamentals break down into themes that, in turn, materialize into processes. Some tools and methodologies are suggested for each theme. It is worth mentioning that the new version of the BEM is still not prescriptive regarding the tools, the structure, or the manner in which the business is managed. It is up to each organization to find the most appropriate way to meet the needs and expectations of its stakeholders and to use the information coming from them to formulate their strategic planning — aligning, integrating, sharing, and directing the entire organization to ensure that it operates with excellence in the value chain and manages results for all stakeholders (FNQ, 2013, 2016).

Description of the game as a discipline

For undergraduate Business Administration programs, Motta, Quintella and Melo (2012, p. 445) identified two different approaches for the Business Games discipline: “the first deals with training for the use of games as a teaching-learning tool, whereas the second deals with the integration of the various specific business disciplines.” In MBA programs, given students’ experience and prior knowledge, this discipline mainly follows the knowledge integration approach.

Since 1999, MBA programs of a higher education institution operating nationwide have been introducing the Business Games discipline into their course offerings. A business game will be analyzed within this realm (BARÇANTE and CORREA, 2001). The GAME (as the implemented business game is referred to in this article) was adapted to MBA programs. We use the results of the GAME applied to a total of 47 classes and 1,410 students in 21 cities distributed throughout nine Brazilian states.

The GAME lasts 16 class hours, divided into four distinct four-hour encounters. It simulates a pull production chain because the production chain is driven by the customer (DAVIS, 2010). This chain is composed of three types of companies (Mining, Goldsmiths, and Jewelers) — role played by teams of students acting as suppliers and customers, and the Exporter — role played by the teacher, who also assumes the role of the government, thereby ensuring the smooth progress of the GAME.

GAME model

Figure 1 shows the GAME model, emphasizing that the supplier-customer chain — consisting of three sectors (Mining, Goldsmiths, and Jewelers) — starts with the demand defined by the Exporter, who engages the Jewelers. In turn, the Jewelers order the jewels from the Goldsmiths who, to make them, acquire raw material from the Mines. Buying and selling relationships are represented by the blue arrows. Companies’ performance is evaluated (represented by the red arrows) by the stakeholders in accordance with an evaluation form (items listed in the first column of Table 2) whose results determine the winner of the GAME.

Figure 1
GAME model

The teams consist of the following.

• Two to three Mines — they receive and pay (consignment or otherwise) for raw material lots that will be sold as interchangeable parts for the production of consumer goods (jewelry). They sell to the Goldsmiths and evaluate them in the customer-supplier relationship in terms of previously defined items included in the evaluation form presented beforehand to the participants. Each Mine is formed by two students.

• Three to four Goldsmiths — they purchase from the Mines interchangeable parts to produce jewelry that is sold to the Jewelers. They evaluate both the Mines and Jewelers in the customer-supplier relationship in terms of the previously defined items. The Jewelers are evaluated as customers and the Mines as suppliers. For each item, the companies typically receive scores from more than one evaluator. The final score of the evaluation items of a company is the mean of the scores given by the different evaluators. Each Goldsmith is formed by at least three students.

• Two to four Jewelers — they purchase jewelry from the Goldsmiths according to the demand defined by the Exporter, and they evaluate the Goldsmiths in the customer-supplier relationship in terms of the previously defined items. Each Jeweler is formed by two students.

• Exporter — played by the teacher. Negotiates for high volumes of jewelry (demand from the GAME) with each Jeweler without determining the color or design of the jewels but only the amount of raw material used to make each jewel. The Exporter first determines the jewel lots for the Jewelers. As the GAME proceeds, if the Jewelers make counterproposals, the Exporter negotiates the number of jewels and the prices on a case-by-case basis.

• Government — also played by the teacher. Takes measures of a general nature to ensure that the GAME favors the production and consequent management of each business.

• Audit Committee — formed by a group of students. Its functions include evaluating the companies of the GAME as stakeholders regarding specific items, representing other stakeholders (e.g., government bodies and segments of society), and assisting the teacher with feedback.

The play

Magic Pins (Elka Plásticos Ltda, São Paulo, Brazil) were used as raw material to be handled in the GAME. These pins are produced in five colors and four designs and were chosen because it is an affordable national product that is available in the market.

The colors are white, yellow, red, green, and blue. In the game, they represent precious metals and gems — platinum, gold, rubellite, tourmaline, and aquamarine, respectively. The pins are also classified in terms of the design: 4-hole pins, 3-hole pins, 1-hole pins, and half-wheels. It is worth mentioning that no green half-wheel pins are produced. Figure 2 shows jewelry made from pins of various colors and designs.

Figure 2
Jewelry made with magic pins of various designs and colors

In addition to selling, buying, and handling the pins and manufacturing jewelry, the companies have to manage their business as a whole. This effort includes paying the payroll, taxes, and fees, managing storage costs and variable costs, investing in other companies’ stocks, and determining the remuneration of the board, among other things. All of this information is contained in the GAME Manual distributed to the students on the first day of class.

The four GAME encounters

Regarding MBA programs (graduate programs of at least 360 hours), the basic assumption is that the students already have the theoretical knowledge and the skills and competencies necessary for a sound execution of the GAME — from the disciplines already studied, the knowledge acquired in undergraduate courses, or the training administered by companies or acquired via professional activities. Thus, during the first encounter, the manual is distributed, the rules of the GAME are presented (including the form used to evaluate the companies), the teams are formed, and the spreadsheets for collecting data are installed on the students’ computers. The second encounter is a general rehearsal at which all activities concerning purchases, production, sales, and evaluations are performed simultaneously by the students and the teacher. During this encounter, the students become familiar with their team, the instruments (pins and spreadsheets), and the decisions and actions they need to take to develop the production chain. The results of this round are not considered. In the third encounter, the actual game occurs in which the teams’ performances are also evaluated — but this time, to define the final rank and the winner. These evaluations are the object of the analysis conducted herein. The feedback directed by the teacher occurs in the fourth and last encounter, in which the students are distributed into groups (not coinciding with the teams) and are urged to answer some questions with individual and collective responses and discuss their and their teams’ performances. The answers are used in this process, which also includes the final outcome of the GAME.

Data structure (form)

As already mentioned, the weights and the items of the GAME evaluation form were based on the excellence criteria of the BEM (FNQ, 2013). The eight excellence criteria at the time were as follows: Leadership, Strategy and Plans, Customers, Society, Information and Knowledge, People, Processes, and Results. In the evaluation for the award, these eight criteria were subdivided into 23 evaluation items (all with some internal breakdown), 17 of which were related to managerial processes and six to organizational results. In the GAME evaluation form (the items are listed in the first column of Table 2 for which the first digit refers to one of the eight criteria and the second digit refers to their breakdown). Some BEM criteria are evaluated with more than one item, totaling 39 — 33 related to managerial processes and six to economic-financial results. Despite the new nomenclature, the BEM continues to be based on the same “internalized values and principles in world-class organizations,” and all of the questionnaire items are consistent and included in the new version.

Each team receives a score of 1 to 5 for each of the 33 items related to management processes. In the six economic-financial items, the value of the score is given by ordering the results within each sector, except for item 8.2 Profitability, which receives a score of zero if it is negative (Table 2). The items have different weights, consistent with the BEM score (see Table 2 for the weights). The team with the highest weighted mean is the winner of the GAME.

The scores assigned by the AC to each item were standardized by the median of each item in the class to control for the possible subjectivity of the evaluators, particularly from the Audit Committee (AC).

Analysis

This section presents the results of a factor analysis and, as an example, the distribution of the values obtained for the first two factors. A generalized linear model is subsequently fitted to the data using the factors identified to explain the likelihood of success in the GAME.

Factor analysis

A factor analysis was conducted with 39 items. Fourteen factors were extracted that explained 72% of the variance of the data. The information about the factor loading associated with this decomposition is shown in Table 2. Graph 1 indicates the factor loadings of the items as a function of the first two factors. The items that served as references for naming the factors are indicated by red ellipses. For Factor 1 (Focus on the customer), the items were as follows: 2.1 — The company conducts customer satisfaction surveys; 2.2 — Measures are taken regarding customer satisfaction and complaint surveys; 3.1 — The company knows the expectations and the current and future needs of the consumer markets; 3.2 — The company evaluates and improves its product in accordance with the information from consumer markets; 3.3 — The company has a product line differentiated by quality; 5.3 — Customer information is used by the company; and 8.1 — The company has met all the expectations and needs of the consumer market. For Factor 2 (Focus on the supplier), the items were as follows: 2.7 — The company seeks partnerships with supplier markets; 2.8 — Measures are taken regarding the search for partnerships with the supplier markets; and 8.6 — Ranking of companies that negotiated best with the supplier market.

Graph 1
Factor loadings of items as a function of the first two factors

Graph 2 shows Factor 1 (Focus on the customer) versus Factor 2 (Focus on the supplier) for the 402 participating companies, with an indication of the outcomes (winner/others). The points corresponding to the winners are mainly concentrated in the first and fourth quadrants, indicating that Factor 1 seems to be relevant for success in the GAME, whereas Factor 2 is not. These findings are corroborated in the comparative histograms of the factors (see Graphs 3 and 4) and in the tests presented in Table 3.

Graph 2
Factor 1 (Focus on the customer) versus Factor 2 (Focus on the supplier), with indication of the outcome

2
Factor loadings of evaluation items, with evaluators in brackets and weights in parentheses

EVALUATION ITEMS [Evaluators] (weight)	Components
	1	2	3	4	5	6	7	8	9	10	11	12	13	14
1.1 The Vision defined by senior management is clear. [AC] (8.0)	.046	-.050	.052	.007	-.056	.083	.829	-.008	.132	.090	.018	-.073	.036	.063
1.2 The Mission defined by senior management considers the values of the organization, the focus on the customer, mutual respect and trust, ethical behavior, participation of people, and expectations of high performance. [AC] (8.0)	.034	-.010	.017	.161	.019	-.018	.840	-.064	.159	-.076	.093	.018	.014	.063
1.3 The leadership style is focused on results. [AC] (6.0)	-.046	.071	.278	.520	.009	.141	.322	-.020	-.060	.106	-.071	.121	-.044	.180
2.1 The company conducts customer satisfaction surveys (post sale). [CM] (1.5)	.698	.065	-.069	.103	-.090	.254	-.060	-.128	.034	.108	-.292	-.149	.314	.062
2.2 Measures are taken regarding customer satisfaction and complaint surveys. [CM] (1.5)	.732	.120	-.062	.095	-.108	.231	-.088	-.139	.061	.100	-.228	-.141	.296	.121
2.3 The company seeks partnerships with its competitors. [AC] (1.5)	009	.006	.150	.036	-.110	.189	-.139	.146	.226	-.324	-.045	-.448	-.123	.316
2.4 The company takes action in accordance with the competitor analysis. [AC] (1.5)	.014	.010	.754	.013	.029	-.098	.022	.038	.255	.093	.074	.085	.127	.105
2.5 The company analyzes the competitive environment in search of new opportunities. [AC] (1.5)	-.007	-.033	.814	.239	-.019	.056	.038	-.002	-.023	.130	-.012	-.050	.027	.015
2.6 Actions are taken in relation to the analysis of the competitive environment. [AC] (1.5)	.007	-.019	.820	.167	.092	-.079	.036	.055	-.053	-.023	-.037	.046	-.077	-.041
2.7 The company seeks partnerships with Supplier Markets. [SM] (1.5)	.112	.917	-.005	-.038	-.079	.035	.000	-.014	-.001	.055	-.007	-.040	.056	.007
2.8 Measures are taken regarding the search for partnerships with the Supplier Markets. [SM] (1.5)	.157	.891	-.012	.013	.003	.103	-.001	.027	.064	.049	-.016	-.080	.041	.015
3.1 The company knows the expectations and the current and future needs of the Consumer Markets (presale). [CM] (1.5)	.801	.070	.028	-.013	.105	.042	.058	.160	-.014	.005	.070	-.057	-.019	-.095
3.2 The company evaluates and improves its product in accordance with the information from Consumer Markets. [CM] (4.0)	.878	.137	.035	.023	.005	.086	.015	.013	-.019	.006	.028	.073	.036	.036
3.3 The company has a product line differentiated by quality. [CM] (4.0)	.810	.068	.033	.001	-.057	.015	-.014	-.082	.099	.053	-.002	.195	-.028	.057
4.1 Are people from the workforce, suppliers, and other stakeholders aware of and involved in issues related to social and environmental responsibility? [AC] (3.0)	.000	.074	.016	.169	-.064	.013	.142	-.045	.745	.086	-.102	.041	.092	-.043
4.2 The company has an established code of conduct. [AC] (3.0)	.036	-.031	.058	-.007	-.006	.079	.172	.001	.743	.009	.075	-.077	-.137	.135
4.3 Does the company communicate to society the impacts and corresponding information associated with products, processes, and facilities? [A] (3.0)	.117	.147	-.003	.008	-.030	.911	.082	-.034	.079	.002	.002	.064	-.043	.062
4.4 Has the community's degree of satisfaction with the company been identified and evaluated? [A] (3.0)	.219	.150	-.119	-.009	-.019	.894	.013	.018	.032	.040	.038	-.022	.011	-.010
5.1 Information about competitors is used by the company. [AC] (2.0)	.097	-.003	.365	-.048	.106	.059	-.021	.014	.420	.194	.392	-.066	-.017	-.176
5.2 Information about product quality is used by the company. [AC] (2.0)	.108	.038	.036	.051	.053	.014	-.024	-.007	.085	.785	-.001	.058	.076	.258
5.3 Information about customers is used by the company. [CM] (3.0)	.861	.058	-.037	-.043	.057	-.002	.083	.037	-.013	.004	.090	-.016	-.140	.005
5.4 Information about operational performance is used by the company. [AC] (2.0)	-.005	-.035	.188	.190	-.010	.064	.074	.178	.055	.731	.090	-.022	-.074	-.065
5.5 Information about financial performance is used by the company. [AC] (3.0)	.063	.009	.114	.402	.132	-.036	-.093	.062	.341	.386	-.043	.126	-.226	-.177
6.1 People are satisfied with what they do. [AC] (4.5)	.079	-.013	.069	.729	.126	.020	.053	.097	.044	.122	.080	-.008	-.068	-.026
6.2 The Business area has autonomy to act. [AC] (4.5)	-.022	-.044	.140	.601	-.030	-.004	.126	.082	-.049	.131	.292	-.119	.085	-.265
6.3 The Production area has autonomy to act. [AC] (4.5)	.012	-.022	-.025	.179	-.103	.040	.113	-.045	-.046	.051	.797	-.062	.056	.056
6.4 Communication between the Business and Production areas is clear, objective, and without interference. [AC] (4.5)	.-.036	.014	.018	.364	.042	-.009	-.133	-.097	.251	-.070	.445	.235	.215	.460
7.1 Best average price paid when purchasing raw material (Supplier Market). [G] (4.5)	.006	-.037	.065	-.017	.243	-.050	.051	.153	-.065	-.024	.101	.058	.797	-.106
7.2 Best average price received when selling raw material in the product (Consumer Market). [G] (4.5)	-.008	-.041	.028	.047	.894	-.014	-.033	-.031	.058	.066	-.027	-.040	-.043	-.024
7.3 Value added to the product (Final sale price of product / cost of the raw material acquired). [G] (4.5)	.046	-.050	.031	.055	.852	.008	.013	.078	-.019	.058	-.005	.049	.301	-.060
7.4 Lead time for delivery (time elapsed from the moment the buyer confirms the purchase until receiving it). [CM] (4.5)	.235	-.110	.114	.070	.008	.101	-.100	.128	.010	.018	-.073	.776	.028	.023
7.5 The criteria used to select and train suppliers are clear and well defined. [AC] (4.0)	.045	.095	.047	.041	.018	.059	.304	.027	-.016	.241	.031	-.104	-.158	.681
8.1 The company satisfied all the expectations and needs of the Consumer Market. [CM] (20.0)	.700	-.001	-.006	.026	-.004	-.012	.035	.084	.021	-.071	.029	.388	-.100	-.086
8.2 Profitability. [G] (22.0)	-.022	.071	.077	.063	.545	-.057	-.021	.349	-.117	-.074	-.060	.059	.020	.143
8.3 Physical share (physical sales). [G] (8.0)	.036	.014	.045	.036	-.057	-.018	-.039	.878	-.020	.044	.009	.102	.152	-.016
8.4 Financial share (financial sales). [G] (8.0)	.025	-.002	.021	.099	.292	.016	-.032	.792	.009	.123	-.052	-.030	-.034	.005
8.5 The Company has an excellent organizational climate. [AC] (6.0)	.041	-.022	.134	.748	.008	-.055	-.020	.004	.151	-.026	.029	.061	.060	.208
8.6 Ranking of the Companies that negotiated best with the Supplier Market. [SM] (10.0)	.122	.812	-.045	-.018	.029	.149	-.049	.005	-.008	-.113	-.028	.011	-.128	.059
8.7 Considering all of the items evaluated, what score would you give the Company? [A] (16.0)	.535	.421	.161	.115	.052	.377	-.004	.049	-.067	.105	.135	.049	-.032	.029

• Source: Elaborated by the authors. Legend: [AC] = Audit Committee; [CM] = Consumer Market; [SM] = Supplier Market; [A] = All; and [G] = Outcome of the Game.

Table 3 shows Levene’s test for equality of variances and the t-test for equality of means, in a comparison of the two subgroups (winners and other participants) corresponding to the outcome of the GAME for each of the 14 identified factors. The difference in the variances information is necessary to determine the t-test to be used: under the hypothesis of equality of means or not. The test for the difference in means presented in Table 3 indicates that a significant difference exists between the winners and the other participants for Factor 1 (Focus on the customer) but not for Factor 2. The p-value (two-tailed) combined with the t statistic serves to show the relative importance of the factors. In this case, the most important were factors 5, 8, 4, 3, and 12, followed by factor 1.

Table 3 also presents a summary of this analysis in the first column, along with a description of the factors. Factors followed by (M) had significant differences in means between winners and other participants. Factors followed by (V) had significant differences in variances between winners and other participants. Factors with no indication had no significant differences in means or variances.

Table 3
Levene’s test for equality of variances (α= 5%) and t-test for equality of means

FACTOR	Levene’s test for equality of variances		t-test for equality of means
	F	p-value	t	df	p-value (two-tailed)	Difference in means	Standard deviation of the difference	CI less than 95%	CI greater than 95%
1 - Focus on the customer (M)	0.784	0.236	-3.293	400	0.001	-0.50502	0.15335	-0.80650	-0.20355
2 - Focus on the supplier	0.939	0.733	-0.212	400	0.832	-0.03299	0.15541	-0.33851	0.27252
3 - Considers the competitive environment (M)	1.219	0.416	-3.941	400	0.000	-0.60091	0.15248	-0.90068	-0.30114
4 - Worker satisfaction (VM)	2.672	0.000	-4.253	82,954	0.000	-0.45399	0.10675	-0.66632	-0.24166
5 - Financial indicators of sales (VM)	3.984	0.000	-7.299	103,582	0.000	-0.67403	0.09235	-0.85716	-0.49089
6 - Information for society	0.978	0.877	-1.002	400	0.317	-0.15549	0.15522	-0.46063	0.14966
7 - Vision and mission (M)	1.559	0.067	-2.160	400	0.031	-0.33378	0.15452	-0.63755	-0.03002
8 - Share of sales (VM)	1.760	0.021	-5.094	69,442	0.000	-0.62575	0.12283	-0.87077	-0.38074
9 - Responsibility and ethics	0.725	0.118	-0.636	400	0.525	-0.09876	0.15534	-0.40413	0.20662
10 - Feedback about product quality	1.578	0.060	-1.665	400	0.097	-0.25790	0.15488	-0.56238	0.04657
11 - Production autonomy	1.413	0.152	-1.578	400	0.115	-0.24445	0.15493	-0.54904	0.06013
12 - Lead time (VM)	1.745	0.023	-3.881	69,232	0.000	-0.48230	0.12427	-0.73019	-0.23441
13 - Cost of inputs	1.456	0.120	-1.772	400	0.077	-0.27431	0.15481	-0.57865	0.03003
14 - Selection of suppliers	0.693	0.074	-0.837	400	0.403	-0.12996	0.15528	-0.43523	0.17530

• Source: Elaborated by the authors. Note: Between parentheses after the name of the factor, M indicates the statistical significance of the difference in the means of winners and other participants. V indicates the statistical significance of the variances of winners and other participants.

To complement the analysis, Graph 3 presents a comparison of the histograms of Factor 1 (Focus on the customer) according to the outcome in the GAME. These double histograms have winners on the right side and other participants on the left side. As Graph 3 indicates, the winners have a distribution with values typically higher than those of the other participants, indicating that this is a determining factor for the final success in the GAME. This finding is confirmed using a statistical test. For this factor, a significant difference in the means in accordance with the outcome — but not of the variances — was observed (see Table 3). As Graph 4 indicates, the distribution of the values of Factor 2 (Focus on the supplier) for the winners is very similar to that observed for the other participants, indicating that no differences exist between the means and that this factor does not determine the final success. Visually, the concept is reinforced by the fact that the range for winners (right side of the graph) is also similar to that observed for other participants (left side), which is confirmed by a statistical test (see Table 3). These results corroborate the description of Graph 2 and complement the comments regarding Graph 3 and Graph 4.

Graph 3
Histograms of Factor 1 (Focus on the Customer) according to outcome of the GAME

Graph 4
Histograms of Factor 2 (Focus on the Supplier) according to outcome of the GAME

Generalized linear model

A generalized linear model (GLM) was fitted using the factors identified in the factor analysis as explanatory variables, the logit link function, and a Bernoulli distribution to explain success/failure in the GAME. Of the 402 student teams analyzed in the model, 47 were winners. Table 4 shows the list of factors tested as explanatory variables. Considering the level of confidence, α, of 5%, factors 2, 6, 9, 11, and 14 are not statistically significant. Comparing this list of significant factors with those from Table 3 reveals that the only discrepancies are factors 10 and 13, which appear as significant in Table 4 but not in Table 3 (even though they were significant at 10%). Table 4 also shows the coefficients associated with the factors, as well as the 95% confidence intervals (CI). The Wald chi-square statistic also indicates the relative importance of the factors in the fitted model. In this case, the most important were factors 5, 8, 4, 3, and 12 (p-value equal to 0.000), followed by factor 1 and factor 13.

Table 4
Test of model effects, parameters, and 95% CI

Source	Type III			Parameter	IC 95% of Wald
	Wald chi-square	df	p-value		Lower	Upper
factor 1	8.242	1	.004	4.134	3.208	5.061
factor 2	.318	1	.573	0.625	0.198	1.051
factor 3	15.056	1	.000	-0.114	-0.510	0.282
factor 4	15.700	1	.000	0.873	0.432	1.314
factor 5	21.890	1	.000	1.469	0.742	2.195
factor 6	.303	1	.582	1.844	1.071	2.616
factor 7	7.249	1	.007	0.131	-0.336	0.598
factor 8	17.490	1	.000	0.592	0.161	1.023
factor 9	.120	1	.729	1.214	0.645	1.783
factor 10	4.362	1	.037	0.068	-0.315	0.450
factor 11	3.715	1	.054	0.526	0.032	1.020
factor 12	13.613	1	.000	0.449	-0.008	0.905
factor 13	3.841	1	.050	0.981	0.460	1.502
factor 14	.023	1	.879	0.547	-4.237E-5	1.093

• Source: Elaborated by the authors.

COMMENTS AND CONCLUSIONS

Teaching-learning methods and techniques can focus on the teacher or the student; however, the goal of both is the same: to pass on or reinforce a body of knowledge. Students’ abilities and attitudes are believed to be better developed as they become the main actors of the teaching method, an experiential learning process.

No consensus in the literature exists on the best method for a specific task, which is also the case with business games as a teaching method for integrating knowledge. Importantly, we note that conducting a program using only business games would be difficult; however, the method has been effective as a complement to other more traditional methods, such as expository lessons (VERSIANI and FACHIN, 2007).

The method proposed and analyzed in this article covers a variety of aspects: qualitative (peer-evaluated) and quantitative variables, students’ opinions and feelings, integration of two domains of the learning process, and cognitive and behavioral aspects (soft skills). The connection between reality and the model feeds the learning process and the integration of knowledge. The GAME can be viewed as a method for both integrating knowledge and evaluating learning.

Determining the stage of the teaching-learning process that is responsible for the gap between the maximum achievable score and that actually achieved during the GAME by the teams is, in truth, not possible. The gap may be caused by several reasons: i) the concept was not taught during the previous school/professional life of the students, including in the MBA program they are attending; ii) the concept was taught but not learned; or iii) the concept was learned, but the students had problems applying it integrally. Nevertheless, recognizing the existence of the gap should be the first step in eliminating it.

From the qualitative point of view, based on feedback from the students — even considering that they are not homogeneous — the students playing the GAME (as opposed to those receiving expository lessons) are more participative, more committed to the learning process, more aware of their weaknesses and strengths, more engaged in the teams, and have a more entrepreneurial attitude. Other positive points are the improvement in student morale and increased interest in the topic, reduced absenteeism, creation of an environment conducive to discussions about business ethics, motivation of the participants to further their knowledge, and, lastly, given that the GAME is evaluated by other students, the final outcome is rarely contested.

The decision-making process is something intrinsic in a business game; therefore, students feel responsible for their actions. A wrong decision can affect the team’s result if the team does not notice it in time. However, through its dynamics, the GAME enables a compensatory correction of the error. Students have just one opportunity to win the GAME, unlike electronic games that can be restarted ad infinitum.

The behavioral aspects of the learning process (e.g., acceptance of one’s role), the psychomotor aspects (e.g., handling the pins to create the jewelry), peer evaluation, and the prior knowledge acquired via the program are factors that affect the outcome of the GAME and the students’ view of the gap between theory and practice.

From the quantitative point of view, the analysis shows that some factors do not determine the final outcome of the GAME: the distribution of the winners’ values was similar to that of the other participants. Other factors had values with lower variance for the winners than for the other participants. It is worth noting that these results refer to the study sample. Various alternatives may explain the absence of a significant difference between the winners and other participants for a certain factor: some processes may already have been internalized by all students (e.g., Focus on the supplier); some may be so theoretical-conceptual that they do not constitute a real concern of the students (e.g., Responsibility and ethics); and others involve a degree of value judgment (e.g., Information for society) that may not be possible to fully evaluate in the game. Additionally, some are so controlled by the facilitator of the game (the Government) that the teams’ behavior tends to be ethical and, thus, reasonably uniform (e.g., Responsibility and ethics).

Finally, some factors are determinants for success in the GAME, which was the case of factors 5 (Financial indicators of sales), 8 (Share of sales), 4 (Worker satisfaction), 3 (Considers the competitive environment), 12 (Lead time), 1 (Focus on the customer), and 13 (Cost of inputs).

REFERÊNCIAS

• ALESSI, S. Fidelity in the design of instructional simulations. Journal of Computer-Based Instruction, v. 15, n. 2, p. 40-47, 1988.
• BARÇANTE, L. C.; CORREA, C. J. O jogo master de gestão da qualidade total. In: ENCONTRO NACIONAL DE ENGENHARIA DE PRODUÇÃO, 21., 2001, Salvador. Anais... Salvador: ABEPRO, 2001. Disponível em:<Disponível em:http://www.abepro.org.br/biblioteca/ENEGEP2001_TR65_0866.pdf >. Acesso em: 20 out. 2016.
  » http://www.abepro.org.br/biblioteca/ENEGEP2001_TR65_0866.pdf
• BARÇANTE, L. C.; PITHON, A. J. C.; BROCHADO, M. R. A vez e a hora da administração industrial jogar - Jogai Cefet. In: ENCONTRO NACIONAL DE ENGENHARIA DE PRODUÇÃO, 30., 2010, São Carlos. Anais... São Carlos, SP: ABEPRO, 2010.
• BELTRÃOK. I.; BARÇANTE, L. C. Teaching principles and fundamentals of business excellence to undergraduate students through a game. Total Quality Management and Business Excellence, v. 26, p. 1-18, 2015.
• BIGGS, J. Teaching for quality learning at university. Buckingham: Open University Press, 1999.
• BRAGGE, J.; THAVIKULWAT, P.; TÖYLI, J. Profiling 40 years of research in simulation & gaming. Simulation & Gaming: An Interdisciplinary Journal, v. 41, n. 6, p. 869-897, 2011.
• CONGRESSO BRASILEIRO DE ENGENHARIA - COBENGE. COBENGEs anteriores. [2017]. Disponível em: <Disponível em: http://www.abenge.org.br/cobenge.php >. Acesso em: 28 out. 2017.
  » http://www.abenge.org.br/cobenge.php
• DAVIS, E. S. M. Aligning products with supply chain processes and strategy. The International Journal of Logistics Management, v. 21, n. 1, p. 127-151, 2010.
• ELGOOD, C. Handbook of management games. 5. ed. Aldershot: Gower, 1993.
• ENCONTRO NACIONAL DA ASSOCIAÇÃO NACIONAL DE PÓS-GRADUAÇÃO E PESQUISA EM ADMINISTRAÇÃO - ENANPAD. Outras edições. [2017]. Disponível em:<Disponível em:http://www.anpad.org.br/~anpad/eventos.php?cod_evento=1 >. Acesso em: 28 out. 2017.
  » http://www.anpad.org.br/~anpad/eventos.php?cod_evento=1
• ENCONTRO NACIONAL DE ENGENHARIA DE PRODUÇÃO - ENEGEP. Anais do Encontro Nacional de Engenharia de Produção. [2017]. Disponível em: <Disponível em: http://www.abepro.org.br/publicacoes/ >. Acesso em:28 out. 2017.
  » http://www.abepro.org.br/publicacoes/
• ENCONTRO NACIONAL DOS CURSOS DE GRADUAÇÃO EM ADMINISTRAÇÃO - ENANGRAD. Anais do Encontro Nacional dos Cursos de Graduação em Administração. [2017]. Disponível em:<Disponível em:http://www.angrad.org.br/eventos/enangrad/ >. Acesso em: 28 out. 2017.
  » http://www.angrad.org.br/eventos/enangrad/
• FARIA, A. The changing nature of business simulation/gaming research: A brief history. Simulation & Gaming: An Interdisciplinary Journal, v. 32, n. 1, p. 97-110, 2001.
• FARIA, A. J.; HUTCHINSON, D.; WELLINGTON, W. J. Developments in business gaming: a review of the past 40 years. Simulation & Gaming: An Interdisciplinary Journal, v. 40, n. 4, p. 464-487, 2009.
• FUNDAÇÃO NACIONAL DA QUALIDADE - FNQ. Critérios de excelência: avaliação e diagnóstico da gestão organizacional. São Paulo: FNQ, 2013.
• FUNDAÇÃO NACIONAL DA QUALIDADE - FNQ. Modelo de Excelência da Gestão (MEG): guia de referência de excelência da gestão. São Paulo: FNQ , 2016.
• GENTRY, J. Guide to business gaming and experiential learning. London: Kogan Page, 1990.
• GOSEN, J. The influence of variables easily controlled by the instructor/administrator on simulation outcomes: in particular, the variable reflection. Developments in Business Simulation and Experiential Learning, v. 31, p. 318-324, 2004.
• GOSEN, J.; WASHBUSH, J. A review of scholarship on assessing experiential learning effectiveness. Simulation & Gaming: An Interdisciplinary Journal, v. 35, n, 2, p. 270-293, 2004.
• GROSSLER, A. Don’t let history repeat itself: methodological issues concerning the use of simulators in teaching and experimentation. System Dynamics Review, v. 20, n. 3, p. 263-274, 2004.
• HALL, J. Computer simulation: a design architectronic. Developments in Business Simulation and Experiential Learning, v. 31, p. 166-175, 2004.
• KNOWLES, M. S. The modern practice of adult education: from pedagogy to andragogy. 2. ed.New York: Cambridge Books, 1980.
• KOLB, D. A. Experiential learning: experience as the source of learning and development. Upper Saddle River, NJ: Prentice Hall, 1984.
• LU, D.; BETTS, A.; CROOM, S. Re-investigating business excellence: values, measures and a framework. Total Quality Management & Business Excellence, v. 22, n. 12, p. 1263-1276, 2013.
• MALIK, D.; HOWARD, B. How do we know where we are going if we don’t know where we have been: a review of business simulation research. Developments in Business Simulation and Experiential Learning, v. 23, p. 49-53, 1996.
• MOHAMMAD, M. et al. Business excellence model: an overarching framework for managing and aligning multiple organisational improvement initiatives. Total Quality Management & Business Excellence, v. 22, n. 11, p. 1213-1236, 2011.
• MOTTA, G. S.; QUINTELLA, R. H.; MELO, D. R. A. Jogos de empresas como componente curricular: análise de sua aplicação por meio de planos de ensino. Revista Organizações & Sociedade, v. 19, n. 62, p. 437-452, 2012.
• NORRIS, D. External validity of business games. Developments in Business Simulation and Experiential Learning, v. 13, p. 126-129, 1986.
• OAKSHOTT, L. Business modelling and simulation. Harlow: Pearson, 1997.
• ROSAS, A. R.; SAUAIA, A. C. A. Jogos de empresas na Educação Superior no Brasil: perspectivas para 2010. In: ENCONTRO NACIONAL DA ASSOCIAÇÃO NACIONAL DOS PROGRAMAS DE PÓS-GRADUAÇÃO EM ADMINISTRAÇÃO, 30., 2006, Salvador. Anais... Salvador: Anpad, 2006.
• SEMINÁRIOS EM ADMINISTRAÇÃO - SEMEAD. Anais dos Seminários em Adminsitração. [2017]. Disponível em: <Disponível em: http://semead.com.br/21/edicoes-anteriores/ >. Acesso em: 28 out. 2017.
  » http://semead.com.br/21/edicoes-anteriores/
• SIMPÓSIO DE ENGENHARIA DE PRODUÇÃO - SIMPEP. Edições anteriores. [2017]. Disponível em: <Disponível em: http://www.simpep.feb.unesp.br/index.php >. Acesso em: 28 out. 2017.
  » http://www.simpep.feb.unesp.br/index.php
• SIMULATION & GAMING: AN INTERDISCIPLINARY JOURNAL. SIMULATION & GAMING: AN INTERDISCIPLINARY JOURNAL Los Angeles, CA: Sage, 1970-2017.
• STAINTON, A. J.; JOHNSON, J. E.; BORODZICZ, E. P. Educational validity of business gaming simulation: a research methodology framework. Simulation & Gaming: An Interdisciplinary Journal, v. 5, n. 41, p. 705-723, 2010.
• VERSIANI, Â.; FACHIN, R. C. Avaliando aprendizagem em simulações empresariais. Cadernos EBAPE.BR, Rio de Janeiro, v. 5, n. esp., p. 1-13, 2007.
• WOLFE, J.; KEYS, B. Business simulations, games and experiential learning in international education. New York: Haworth, 1997.
• WOLFE, J. The effectiveness of business games in strategic management course work. Simulation & Gaming: An Interdisciplinary Journal, v. 28, n. 4, p. 360-376, 1997.

• [Translated version] Note: All quotes in English translated by this article’s translator.

Publication Dates

• Publication in this collection
  09 Dec 2019
• Date of issue
  Nov 2019

History

• Received
  20 Nov 2017
• Accepted
  08 Feb 2019