Adoption of business game under the perspective of an excellence model in an MBA course: multicriteria peer evaluation analysis

Resumo Este artigo descreve um jogo de negócios aplicado em cursos do tipo Master of Business Administration (MBA) como uma das últimas disciplinas do currículo e analisa os resultados de sua implementação em 47 turmas, envolvendo 1.410 alunos. O jogo de negócios tem por objetivo integrar as habilidades e competências desenvolvidas ao longo do curso e considera um sistema de avaliação com 39 quesitos baseado em um modelo de excelência que abrange todos os aspectos importantes no processo de gestão. Somente 6 quesitos de avaliação são financeiros e os outros 33 são avaliados pelos próprios alunos, em uma avaliação por pares, e dizem respeito à relação cliente-fornecedor e à gestão de cada empresa. Uma redução de dimensionalidade foi obtida por meio de uma análise fatorial, resultando em 14 fatores. Dentre esses fatores, é possível identificar quais são relevantes para o sucesso no jogo? Esta questão é respondida averiguando por meio de 2 métodos estatísticos distintos (significância dos parâmetros de um modelo logito e teste t para diferença de médias) quais são esses fatores. Ajustou-se um modelo linear generalizado com uma função de ligação logito e uma distribuição Bernouilli, confirmando que a maioria dos fatores com diferenças estatisticamente significativas das médias de vencedores e demais participantes também foram estatisticamente significativas no modelo. A avaliação dos alunos participantes confirmou que o jogo de negócios foi bem recebido: levantou o moral dos alunos; despertou o interesse pelo assunto; fomentou o empreendedorismo; diminuiu o absenteísmo; aumentou a participação em sala, a consciência do trabalho em grupo e o planejamento; e criou um ambiente favorável às discussões sobre ética nos negócios.


INTRODUCTION
A revolution has been witnessed in the value creation process since the end of the 20th century, which can be partly attributed to the confluence of globalization and technological advances in the transition from an industrial economy to an informationbased economy. This context involves much complexity, instability, and uncertainty. Thus, organizational changes have come to be viewed as one of the main ways to structure and explore the new business world. This exploration of new organizational methods requires higher education institutions to strategically "play" with their teaching, research, and outreach abilities to bring the academic world closer to the business world.
In this context, Master of Business Administration (MBA) programs have experienced -especially in recent decades -a progressive and marked expansion, reinforcing their social and economic importance as an instrument for updating, qualifying, and training professionals for the labor market. These programs play an important role in the economic growth of a country, which involves financial aspects and stimuli for new investments, whether they be in infrastructure, the construction of appropriate environments, the acoustics of these environments, the acquisition of multimedia equipment, the payment of teachers and employees, travel and accommodation of teachers, and others. MBA programs are powerful generators of jobs and gatherers of service, commerce, and industry activities. This new concept is based on the idea that MBA students, most of whom are professionals operating in the market, are not simply recipients of information (the basic assumption in traditional classes) but, rather, intangible assets who should be treated as such.
The relevant function that MBA programs have been performing generates a need to use new teaching techniques that aim, on the one hand, for a re-evaluation of the work of the teacher who is now viewed as a facilitator of learning and, on the other hand, for the total involvement of the student with the teaching-learning process -known as experiential learning (KOLB, 1984).
Studies related to experiential learning (WOLFE and KEYS, 1997) have established a connection between it and business simulations and business games, known internationally as simulation and gaming (S&G).
The use of S&G has grown in recent years in Brazil (BARÇANTE, PITHON and BROCHADO, 2010) in both academic and business environments. The prediction of Rosas and Sauaia (2006) has come to fruition in graduate programs: the most important business schools in Brazil have adopted disciplines such as business games, company games, or business simulations in their programs, which have generally been positively evaluated by students.
These disciplines provide a classroom environment that is closer to the environment of a competitive market and represents teaching techniques that stimulate creativity, promote the exercise of communication, enable the exchange of experiences, and stimulate the experience of new roles and decision making in an environment of risk (VERSIANI and FACHIN, 2007). These techniques have experienced a significant expansion at the world level -including in Brazil -over the last 40 years (BRAGGE, THAVIKULWAT and TÖYLI, 2011;FARIA, HUTCHINSON and WELLINGTON, 2009). The international journal Simulation &Gaming, between 1970 (v. 1, n. 1) and2017 (v. 48, n. 5), published 2,360 articles (SIMULATION & GAMING: AN INTERDISCIPLINARY JOURNAL, 1970. The number of publications in Brazilian congresses is much more modest. Table 1 provides the number of articles on business games presented at six annual events -three in Business Administration and three in Engineering. These events were chosen because they are the most frequent forums for publications by researchers on the theme. The information was collected from the online annals of the following events: Brazilian Congress of Engineering (COBENGE, 2017)

Business games
The importance of business simulation is a permanent topic in the scientific world and has been debated exhaustively in a non-univocal manner (GOSEN and WASHBUSH, 2004;GROSSLER, 2004;FARIA, 2001;MALIK and HOWARD, 1996). In discussing the validity of using business games, near unanimity exists over the point that a business game depends heavily on the quality of the model used and its implementation. According to Stainton, Johnson and Borodzicz (2010), three sets of suggestions based on the available literature and the experience of business game designers can be adopted to improve the games. These suggestions concern characteristics of the environment, the content, and the implementation of the game.
The game environment (ELGOOD, 1993;NORRIS, 1986; ALESSI, 1988) must be: 1. verisimilar, that is, close to the real business world environment; 2. complex and flexible to changes, without becoming confusing; and, 3. competitive and strategic.
The content of the game (KNOWLES, 1980) must be: 1. comprehensive and relevant enough to make the exercise stimulating and the participants motivated to get involved; 2. versatile and treated with the appropriate depth to be able to achieve various learning objectives; and, 3. foster the development of the participants' learning.
The game moves (FARIA, 2001;BIGGS, 1999;GENTRY, 1990;KOLB, 1984) must: 1. show that participants' decisions impact the team's performance; 2. show that the team's performance is related to its outcome in the game; 3. develop the cognitive processes through the participatory management theory; 4. show that the learning process can be behavioral through learning by doing; 5. enable cooperation and partnerships and making the exercise more enjoyable and rewarding; 6. be experiential to enable learning through the application of theory in practice; 7. be dynamic and allow the problems that arise to be promptly resolved by the participants; and, 8. integrate knowledge already acquired.
In addition to the mentioned suggestions, others should be taken into account. According to Hall (2004), the duration of each game must be envisaged in such a way that sufficient time exists for an analysis of the results, as well as decision making, debates, and reflection; the game manual should be written clearly and objectively to provide the necessary information to the participants; and the teachers -viewed as facilitators of learning -need to be trained and have enough technical and business knowledge to provide support and, in particular, feedback to the participants (GOSEN, 2004;HALL, 2004;WOLFE, 1997). Finally, the supporting material (physical or computational) must undergo pilot testing to ensure that its use is reliable (OAKSHOTT, 1997).

Business excellence models
At least 94 awards were associated with business excellence models used in 83 countries worldwide (MOHAMMAD, MANN, GRIGG et al., 2011;LU, BETTS and CROOM, 2013). The Business Excellence Model (BEM; known in Portuguese as Modelo de Excelência da Gestão or MEG), created in 1991 by the Brazilian Quality Foundation (Fundação Nacional da Qualidade -FNQ), serves as the basis for the FNQ award. The BEM has been improved to adjust to market changes, and it can be used as a guide to make companies more competitive. At the time of application of the games, the BEM was composed of 13 fundamentals and eight criteria (FNQ, 2013) and was the model used as a reference for the definition of the weights and the elaboration of the evaluation items described in this article. The model has already been used with this objective in undergraduate courses (BELTRÃO and BARÇANTE, 2015). In 2016, the BEM underwent an update (FNQ, 2016), which innovated by being formatted into eight fundamentals that replaced the former eight criteria. The fundamentals break down into themes that, in turn, materialize into processes. Some tools and methodologies are suggested for each theme. It is worth mentioning that the new version of the BEM is still not prescriptive regarding the tools, the structure, or the manner in which the business is managed. It is up to each organization to find the most appropriate way to meet the needs and expectations of its stakeholders and to use the information coming from them to formulate their strategic planning -aligning, integrating, sharing, and directing the entire organization to ensure that it operates with excellence in the value chain and manages results for all stakeholders (FNQ, 2013(FNQ, , 2016.

Description of the game as a discipline
For undergraduate Business Administration programs, Motta, Quintella and Melo (2012, p. 445) identified two different approaches for the Business Games discipline: "the first deals with training for the use of games as a teaching-learning tool, whereas the second deals with the integration of the various specific business disciplines." In MBA programs, given students' experience and prior knowledge, this discipline mainly follows the knowledge integration approach.
Since 1999, MBA programs of a higher education institution operating nationwide have been introducing the Business Games discipline into their course offerings. A business game will be analyzed within this realm (BARÇANTE and CORREA, 2001). The GAME (as the implemented business game is referred to in this article) was adapted to MBA programs. We use the results of the GAME applied to a total of 47 classes and 1,410 students in 21 cities distributed throughout nine Brazilian states.
The GAME lasts 16 class hours, divided into four distinct four-hour encounters. It simulates a pull production chain because the production chain is driven by the customer (DAVIS, 2010). This chain is composed of three types of companies (Mining, Goldsmiths, and Jewelers) -role played by teams of students acting as suppliers and customers, and the Exporter -role played by the teacher, who also assumes the role of the government, thereby ensuring the smooth progress of the GAME. GAME model Figure 1 shows the GAME model, emphasizing that the supplier-customer chain -consisting of three sectors (Mining, Goldsmiths, and Jewelers) -starts with the demand defined by the Exporter, who engages the Jewelers. In turn, the Jewelers order the jewels from the Goldsmiths who, to make them, acquire raw material from the Mines. Buying and selling relationships are represented by the blue arrows. Companies' performance is evaluated (represented by the red arrows) by the stakeholders in accordance with an evaluation form (items listed in the first column of Table 2) whose results determine the winner of the GAME. The teams consist of the following.
• Two to three Mines -they receive and pay (consignment or otherwise) for raw material lots that will be sold as interchangeable parts for the production of consumer goods (jewelry). They sell to the Goldsmiths and evaluate them in the customer-supplier relationship in terms of previously defined items included in the evaluation form presented beforehand to the participants. Each Mine is formed by two students. • Three to four Goldsmiths -they purchase from the Mines interchangeable parts to produce jewelry that is sold to the Jewelers. They evaluate both the Mines and Jewelers in the customer-supplier relationship in terms of the previously defined items. The Jewelers are evaluated as customers and the Mines as suppliers. For each item, the companies typically receive scores from more than one evaluator. The final score of the evaluation items of a company is the mean of the scores given by the different evaluators. Each Goldsmith is formed by at least three students. • Two to four Jewelers -they purchase jewelry from the Goldsmiths according to the demand defined by the Exporter, and they evaluate the Goldsmiths in the customer-supplier relationship in terms of the previously defined items. Each Jeweler is formed by two students. • Exporter -played by the teacher. Negotiates for high volumes of jewelry (demand from the GAME) with each Jeweler without determining the color or design of the jewels but only the amount of raw material used to make each jewel. The Exporter first determines the jewel lots for the Jewelers. As the GAME proceeds, if the Jewelers make counterproposals, the Exporter negotiates the number of jewels and the prices on a case-by-case basis. • Government -also played by the teacher. Takes measures of a general nature to ensure that the GAME favors the production and consequent management of each business. • Audit Committee -formed by a group of students. Its functions include evaluating the companies of the GAME as stakeholders regarding specific items, representing other stakeholders (e.g., government bodies and segments of society), and assisting the teacher with feedback.

The play
Magic Pins (Elka Plásticos Ltda, São Paulo, Brazil) were used as raw material to be handled in the GAME. These pins are produced in five colors and four designs and were chosen because it is an affordable national product that is available in the market.
The colors are white, yellow, red, green, and blue. In the game, they represent precious metals and gems -platinum, gold, rubellite, tourmaline, and aquamarine, respectively. The pins are also classified in terms of the design: 4-hole pins, 3-hole pins, 1-hole pins, and half-wheels. It is worth mentioning that no green half-wheel pins are produced. Figure 2 shows jewelry made from pins of various colors and designs.

Figure 2 Jewelry made with magic pins of various designs and colors
Source: Personal collection of the authors.
In addition to selling, buying, and handling the pins and manufacturing jewelry, the companies have to manage their business as a whole. This effort includes paying the payroll, taxes, and fees, managing storage costs and variable costs, investing in other companies' stocks, and determining the remuneration of the board, among other things. All of this information is contained in the GAME Manual distributed to the students on the first day of class.

The four GAME encounters
Regarding MBA programs (graduate programs of at least 360 hours), the basic assumption is that the students already have the theoretical knowledge and the skills and competencies necessary for a sound execution of the GAME -from the disciplines already studied, the knowledge acquired in undergraduate courses, or the training administered by companies or acquired via professional activities. Thus, during the first encounter, the manual is distributed, the rules of the GAME are presented (including the form used to evaluate the companies), the teams are formed, and the spreadsheets for collecting data are installed on the students' computers. The second encounter is a general rehearsal at which all activities concerning purchases, production, sales, and evaluations are performed simultaneously by the students and the teacher. During this encounter, the students become familiar with their team, the instruments (pins and spreadsheets), and the decisions and actions they need to take to develop the production chain. The results of this round are not considered. In the third encounter, the actual game occurs in which the teams' performances are also evaluated -but this time, to define the final rank and the winner. These evaluations are the object of the analysis conducted herein. The feedback directed by the teacher occurs in the fourth and last encounter, in which the students are distributed into groups (not coinciding with the teams) and are urged to answer some questions with individual and collective responses and discuss their and their teams' performances. The answers are used in this process, which also includes the final outcome of the GAME.

Data structure (form)
As already mentioned, the weights and the items of the GAME evaluation form were based on the excellence criteria of the BEM (FNQ, 2013). The eight excellence criteria at the time were as follows: Leadership, Strategy and Plans, Customers, Society, Information and Knowledge, People, Processes, and Results. In the evaluation for the award, these eight criteria were subdivided into 23 evaluation items (all with some internal breakdown), 17 of which were related to managerial processes and six to organizational results. In the GAME evaluation form (the items are listed in the first column of Table 2 for which the first digit refers to one of the eight criteria and the second digit refers to their breakdown). Some BEM criteria are evaluated with more than one item, totaling 39 -33 related to managerial processes and six to economic-financial results. Despite the new nomenclature, the BEM continues to be based on the same "internalized values and principles in world-class organizations," and all of the questionnaire items are consistent and included in the new version.
Each team receives a score of 1 to 5 for each of the 33 items related to management processes. In the six economic-financial items, the value of the score is given by ordering the results within each sector, except for item 8.2 Profitability, which receives a score of zero if it is negative ( Table 2). The items have different weights, consistent with the BEM score (see Table 2 for the weights). The team with the highest weighted mean is the winner of the GAME.
The scores assigned by the AC to each item were standardized by the median of each item in the class to control for the possible subjectivity of the evaluators, particularly from the Audit Committee (AC).

Analysis
This section presents the results of a factor analysis and, as an example, the distribution of the values obtained for the first two factors. A generalized linear model is subsequently fitted to the data using the factors identified to explain the likelihood of success in the GAME.

Factor analysis
A factor analysis was conducted with 39 items. Fourteen factors were extracted that explained 72% of the variance of the data. The information about the factor loading associated with this decomposition is shown in Table 2. Graph 1 indicates the factor loadings of the items as a function of the first two factors. The items that served as references for naming the factors are indicated by red ellipses. For Factor 1 (Focus on the customer), the items were as follows: 2.1 -The company conducts customer satisfaction surveys; 2.2 -Measures are taken regarding customer satisfaction and complaint surveys; 3.1 -The company knows the expectations and the current and future needs of the consumer markets; 3.2 -The company evaluates and improves its product in accordance with the information from consumer markets; 3.3 -The company has a product line differentiated by quality; 5.3 -Customer information is used by the company; and 8.1 -The company has met all the expectations and needs of the consumer market. For Factor 2 (Focus on the supplier), the items were as follows: 2.7 -The company seeks partnerships with supplier markets; 2.8 -Measures are taken regarding the search for partnerships with the supplier markets; and 8.6 -Ranking of companies that negotiated best with the supplier market.

Graph 1 Factor loadings of items as a function of the first two factors
Source: Elaborated by the authors.
Graph 2 shows Factor 1 (Focus on the customer) versus Factor 2 (Focus on the supplier) for the 402 participating companies, with an indication of the outcomes (winner/others). The points corresponding to the winners are mainly concentrated in the first and fourth quadrants, indicating that Factor 1 seems to be relevant for success in the GAME, whereas Factor 2 is not. These findings are corroborated in the comparative histograms of the factors (see Graphs 3 and 4) and in the tests presented in Table 3.    Table 3 shows Levene's test for equality of variances and the t-test for equality of means, in a comparison of the two subgroups (winners and other participants) corresponding to the outcome of the GAME for each of the 14 identified factors. The difference in the variances information is necessary to determine the t-test to be used: under the hypothesis of equality of means or not. The test for the difference in means presented in Table 3 indicates that a significant difference exists between the winners and the other participants for Factor 1 (Focus on the customer) but not for Factor 2. The p-value (two-tailed) combined with the t statistic serves to show the relative importance of the factors. In this case, the most important were factors 5, 8, 4, 3, and 12, followed by factor 1. Table 3 also presents a summary of this analysis in the first column, along with a description of the factors. Factors followed by (M) had significant differences in means between winners and other participants. Factors followed by (V) had significant differences in variances between winners and other participants. Factors with no indication had no significant differences in means or variances.

Graph 2 Factor 1 (Focus on the customer) versus Factor 2 (Focus on the supplier), with indication of the outcome
continue To complement the analysis, Graph 3 presents a comparison of the histograms of Factor 1 (Focus on the customer) according to the outcome in the GAME. These double histograms have winners on the right side and other participants on the left side. As Graph 3 indicates, the winners have a distribution with values typically higher than those of the other participants, indicating that this is a determining factor for the final success in the GAME. This finding is confirmed using a statistical test. For this factor, a significant difference in the means in accordance with the outcome -but not of the variances -was observed (see Table 3). As Graph 4 indicates, the distribution of the values of Factor 2 (Focus on the supplier) for the winners is very similar to that observed for the other participants, indicating that no differences exist between the means and that this factor does not determine the final success. Visually, the concept is reinforced by the fact that the range for winners (right side of the graph) is also similar to that observed for other participants (left side), which is confirmed by a statistical test (see Table 3). These results corroborate the description of Graph 2 and complement the comments regarding Graph 3 and Graph 4.

Generalized linear model
A generalized linear model (GLM) was fitted using the factors identified in the factor analysis as explanatory variables, the logit link function, and a Bernoulli distribution to explain success/failure in the GAME. Of the 402 student teams analyzed in the model, 47 were winners. Table 4 shows the list of factors tested as explanatory variables. Considering the level of confidence, α, of 5%, factors 2, 6, 9, 11, and 14 are not statistically significant. Comparing this list of significant factors with those from Table 3 reveals that the only discrepancies are factors 10 and 13, which appear as significant in Table 4 but not in Table 3 (even though they were significant at 10%). Table 4 also shows the coefficients associated with the factors, as well as the 95% confidence intervals (CI). The Wald chi-square statistic also indicates the relative importance of the factors in the fitted model. In this case, the most important were factors 5, 8, 4, 3, and 12 (p-value equal to 0.000), followed by factor 1 and factor 13.

COMMENTS AND CONCLUSIONS
Teaching-learning methods and techniques can focus on the teacher or the student; however, the goal of both is the same: to pass on or reinforce a body of knowledge. Students' abilities and attitudes are believed to be better developed as they become the main actors of the teaching method, an experiential learning process.
No consensus in the literature exists on the best method for a specific task, which is also the case with business games as a teaching method for integrating knowledge. Importantly, we note that conducting a program using only business games would be difficult; however, the method has been effective as a complement to other more traditional methods, such as expository lessons (VERSIANI and FACHIN, 2007).
The method proposed and analyzed in this article covers a variety of aspects: qualitative (peer-evaluated) and quantitative variables, students' opinions and feelings, integration of two domains of the learning process, and cognitive and behavioral aspects (soft skills). The connection between reality and the model feeds the learning process and the integration of knowledge. The GAME can be viewed as a method for both integrating knowledge and evaluating learning.
Determining the stage of the teaching-learning process that is responsible for the gap between the maximum achievable score and that actually achieved during the GAME by the teams is, in truth, not possible. The gap may be caused by several reasons: i) the concept was not taught during the previous school/professional life of the students, including in the MBA program they are attending; ii) the concept was taught but not learned; or iii) the concept was learned, but the students had problems applying it integrally. Nevertheless, recognizing the existence of the gap should be the first step in eliminating it.
From the qualitative point of view, based on feedback from the students -even considering that they are not homogeneous -the students playing the GAME (as opposed to those receiving expository lessons) are more participative, more committed to the learning process, more aware of their weaknesses and strengths, more engaged in the teams, and have a more entrepreneurial attitude. Other positive points are the improvement in student morale and increased interest in the topic, reduced absenteeism, creation of an environment conducive to discussions about business ethics, motivation of the participants to further their knowledge, and, lastly, given that the GAME is evaluated by other students, the final outcome is rarely contested.
The decision-making process is something intrinsic in a business game; therefore, students feel responsible for their actions. A wrong decision can affect the team's result if the team does not notice it in time. However, through its dynamics, the GAME enables a compensatory correction of the error. Students have just one opportunity to win the GAME, unlike electronic games that can be restarted ad infinitum.
The behavioral aspects of the learning process (e.g., acceptance of one's role), the psychomotor aspects (e.g., handling the pins to create the jewelry), peer evaluation, and the prior knowledge acquired via the program are factors that affect the outcome of the GAME and the students' view of the gap between theory and practice.
From the quantitative point of view, the analysis shows that some factors do not determine the final outcome of the GAME: the distribution of the winners' values was similar to that of the other participants. Other factors had values with lower variance for the winners than for the other participants. It is worth noting that these results refer to the study sample. Various alternatives may explain the absence of a significant difference between the winners and other participants for a certain factor: some processes may already have been internalized by all students (e.g., Focus on the supplier); some may be so theoreticalconceptual that they do not constitute a real concern of the students (e.g., Responsibility and ethics); and others involve a degree of value judgment (e.g., Information for society) that may not be possible to fully evaluate in the game. Additionally, some are so controlled by the facilitator of the game (the Government) that the teams' behavior tends to be ethical and, thus, reasonably uniform (e.g., Responsibility and ethics).
Finally, some factors are determinants for success in the GAME, which was the case of factors 5 (Financial indicators of sales), 8 (Share of sales), 4 (Worker satisfaction), 3 (Considers the competitive environment), 12 (Lead time), 1 (Focus on the customer), and 13 (Cost of inputs).