Avaliação da implementação em escolas brasileiras de um programa preventivo para crianças

pcp Psicologia: Ciência e Profissão Psicol., Ciênc. Prof. (Impr.) 1414-9893 1982-3703 Conselho Federal de Psicologia Resumo O artigo discute a implementação piloto do programa preventivo baseado em evidência Good Behavior Game (GBG) em escolas públicas de quatro cidades brasileiras. O GBG é considerado um método para o professor de manejo de comportamentos em sala de aula, visando a construção de sociabilidade entre os estudantes, direcionado para turmas do ensino fundamental, com crianças entre seis e 10 anos. O objetivo deste estudo foi a avaliação do processo de implementação com foco na aceitabilidade e percepção de resultados pelos profissionais envolvidos. Foram utilizados métodos mistos na investigação e triangulação de dados na análise, com base na aplicação de questionários e entrevistas semiestruturadas com 28 professores, nove diretores e seis multiplicadoras do programa. Os dados foram analisados através de estatística descritiva e inferencial ou análise de conteúdo, conforme a natureza do dado. O GBG obteve uma alta aceitação entre professores e diretores. Destacou-se a importância do estímulo ao trabalho em grupo, o uso de elogios como reforçadores, a objetividade das regras assumidas coletivamente. A aceitabilidade foi atribuída, em boa parte, à eficácia de sua estratégia de manejo de sala de aula. Houve indicações da necessidade de adaptações que reflitam o contexto social e econômico brasileiro. Introduction The school system is one of the main environments for the integral development of children and adolescents, as a significant amount of their time is spent in school settings. The psychosocial vulnerabilities that children face in environments such as family, community or school could be decisive in the development of socially maladaptive behaviors such as aggression, disruption, shyness, and social isolation. These early maladaptive behaviors are risk factors for future antisocial behavior at a later age, such as substance abuse, risky sexual behavior, and suicidal ideation (Embry, 2002; Kellam et al., 2011; Kellam et al., 2014; Poduska et al., 2008). A school that is not able to manage and help students overcome early risk behaviors can further pathologize those behaviors (Cord, Gesser, Nunes, & Storti, 2015). For example, faculty and staff who produce interactions based on coercion and punishment can intensify students’ maladaptive behaviors (Poduska et al., 2008) and produce the stigmatization of children who are already in the most vulnerable situation. Therefore, schools can play a key role in promoting the mental health of students by implementing universal prevention activities (Embry 2002; Poduska et al., 2008; Sloboda, & Petras, 2014). The school environment allows for early intervention, which may lead to increased effectiveness in combating maladaptive behaviors before they become more serious psychosocial problems. In addition, schools that train their staff in behavior management could help create a stable and secure environment that promotes academic and emotional development of children (Becker, Souza, Oliveira, & Paraguay, 2014). The Good Behavior Game (GBG) was developed in the USA aiming at promoting mental health among children and at intervening in risk situations for future antisocial behavior. GBG was developed by a fourth-grade teacher in 1967. In her first year of teaching, Muriel Saunders teamed up with Montrose Wolf and Harriet Barrish at the University of Kansas to find a way to reduce the disruptive behaviors of her students. They developed a landmark procedure called Good Behavior Game, and in 1969, Barrish, Saunders, and Wolf (1969) published the first study on the Good Behavior Game. GBG has become one of the programs with most extended evidence of short-, mid-, and long-term effectiveness for student outcomes including substance abuse, delinquency, youth violence, and other behavioral disorders (Bayer et al., 2009; Embry, 2002; Foxcroft, & Tsertsvadze, 2011). GBG is a classroom behavior management strategy, geared particularly to students who present aggressive or disruptive behavior, or are shy or socially isolated, which are considered risk factors for future antisocial behaviors (Kellam et al., 2014; Sloboda, & Petras, 2014; Tingstrom, Sterling-Turner, & Wilczynski, 2006). The program helps teachers create collective and collaborative activities among colleagues and reinforce appropriate classroom behaviors consistently, which makes it easier for children to self-regulate and become active members in the classroom community (Kellam et al., 2011; Poduska et al., 2008). GBG is designed to foster supportive behavioral management and positive youth behavior (Halgunseth et al., 2012). Therefore, GBG’s main focus is on early prevention, addressing risk factors that longitudinal studies have connected to, for example, greater instances of drug use. GBG decreases risk behaviors such as aggressive and disruptive behavior and strengthens protective factors, such as positive peer relationships, more commitment to activities collectively agreed upon, increase in personal realization rates, and higher chances of continuity in academic life (Flay, 2009; Poduska et al., 2008). Thus, GBG is a preventive intervention addressing early risk factors, a “behavioral vaccine” against later and more costly risk behaviors (Embry, 2002). The Brazilian government seeks to develop policies to address the socioeconomic reality in the country, developing inclusive educational practices that focus on social and economic inequalities and inequities. It is in this context that the Mental Health Coordination at the Brazilian Ministry of Health, in a partnership with the United Nations Office on Drugs and Crimes (UNODC), launched a project based on well-established evidence-based drug abuse prevention programs from other countries. The goal was to adapt these programs to the Brazilian context and to evaluate their effectiveness as programs to be scaled up and implemented as public policies in Brazil. The GBG was among the suggestions of prevention programs to be tested and adapted to the Brazilian context. It was recommended by the UNODC because of its evidence base and the rigorous research conducted with it in several countries, including the United States, England, Holland, Belgium and Australia (Bayer et al., 2009; Kellam et al., 2011; Tingstrom et al., 2006). The program was chosen for a pilot study in four cities in two Brazilian states during the 2013 school year. The objective of this article is to evaluate the program implementation process, focusing on the acceptability and perceived results by the professionals involved. The Brazilian Pilot Intervention After GBG was chosen as one of the programs for the Brazilian Ministry of Health’s pilot project on drug abuse prevention, the American Institutes for Research (AIR), a provider of GBG, was contacted to train the Brazilian GBG trainers and coaches. The coaches participated in a four-day training –“the trainer event”- and received ongoing support from AIR’s GBG-certified trainers through conference calls during the 2013 school year. Teachers participating in the project attended a two-day training in September 2013, facilitated by the Brazilian coaches, and also received ongoing in-person support by the coaches twice a month. Initial adaptation of GBG materials to the Brazilian context took place before the teacher trainings, and GBG was renamed Jogo Elos —which refers to connections and was a name chosen by the coaches and the program coordinators. Teachers started the pre-implementation activities in classrooms in October 2013. These included: creating teams of students to play GBG; teaching students the expectations for classroom behavior (“classroom rules”); identifying appropriate rewards; and explaining the rules of GBG. On the basis of their initial observation and assessment of each student’s behavior in the classroom, teachers created heterogeneous teams with a balanced number of boys and girls and diversity in terms of behaviors: Each team should include children who, for example, were aggressive, hyperactive, shy, and socially isolated. Teachers explicitly taught and discussed with their students the four basic rules of the game that determine the expectations for appropriate classroom behavior (work quietly, be polite to others, ask for permission to get out of their seats, and follow directions). Together with their students, teachers also determined the appropriate rewards that would be given to winning teams, in other words, teams that met the behavioral expectation of not breaking the classroom rules more than four times in the duration of a game, which could range from 10 to 30 minutes. During the implementation period, once the game had started, teams worked on their typical academic activities, while the team’s behavior was monitored by their teammates and the teacher. If a member of the team broke a classroom rule, the team would receive a checkmark. Groups received positive feedback and rewards when they did not accumulate more than four checkmarks, following the model proposed in GBG (Flower, McKenna, Muething, Bryant, & Bryant, 2014; Kellam et al., 2014; Tingstrom et al., 2006). Having consequences for rule-breaking and positive reinforcement for following the behavioral expectations created for the classroom increased sociability and better relationships among peers, who started to self-regulate their behaviors. Teachers were supported by coaches during the GBG planning and implementation phases, which lasted three months, between October and December 2013. In order to monitor the fidelity of the implementation, they used a few monitoring tools recommended by AIR trainers, such as the Implementation Fidelity Checklist and the Scoreboard Report. The evaluation process for the pilot project was conducted by research teams from the Federal University of São Paulo and the Federal University of Santa Catarina, the results of which are partially reported in this article. This article discusses piloting GBG in Brazilian schools, in order to evaluate its acceptability, the perceived effectiveness, and suitability for the Brazilian school system. Methods Study Design The evaluation study was longitudinal and lasted four months, starting with the training for coaches and teachers and lasting until the end of the school year (September–December 2013). A mixed-methods approach was adopted for the evaluation, with both quantitative and qualitative data sources, as proposed by Creswell (2009). The data analysis was conducted through triangulation of several data sources. Participants The participants in this study were: 28 teachers, nine school administrators (principals or pedagogical coordinators), and six coaches in charge of training and overseeing the implementation of GBG in the schools, totaling 43 participants. A total of 736 children participated in the GBG prevention program in this pilot study in 2013. The project was conducted with students enrolled in the early grades at six public elementary schools. Three of the schools were in the state of São Paulo (two in the city of São Bernardo do Campo and one in São Paulo city), and the other three were in the state of Santa Catarina (two in the city of Tubarão, and one in Florianópolis). A total of 37 classrooms participated in the pilot study between October and November 2013. The participating schools were chosen by representatives of the Departments of Education in these four cities. The selection criterion was the school’s willingness to try out and implement the intervention, including the implementation support and implementation fidelity monitoring conducted by the Brazilian GBG coaches. Procedure Quantitative Data All 28 teachers who implemented GBG, the principals of the six participating schools, three members of the pedagogical team, and the six coaches filled out a questionnaire about school routines and their GBG implementation experiences. Qualitative Data Semi-structured interviews were conducted with (a) 12 teachers who used the program, (b) six principals and one member of a pedagogical team in the participating schools, and (c) six coaches in charge of training and overseeing the implementation of GBG in the schools. Based on a semi-structured interview protocol, data about the GBG implementation process were collected, focusing on its acceptability, feasibility, and perceived effectiveness in changing teacher and student behavior. Instruments and Variables The data collection instruments used in the evaluation process were: a) Teacher Questionnaire, with 22 items analyzed in a 6-point Likert scale; b) Administrator questionnaire, with 15 items, in a 6-point Likert scale; c) A guide for semi-structured interviews with teachers, with 41 items; d) A guide for semi-structured interviews with administrators, with 14 items; e) A guide for semi-structured interviews with coaches, with 10 items. The first three instruments were developed by AIR and were used in previous studies; the last two were developed by the researchers involved in the study, and focused mainly on the process evaluation. Data Analysis Descriptive analysis was used to analyze the quantitative data derived from the questionnaires answered by administrators and teachers. The quantitative data were summarized in absolute and relative frequencies. Due to the small number of participants and in order to facilitate the data analysis, the six points in the Likert scale were transformed into three points: low (grouping “not at all”, “almost nothing”, and “to a minimal extent”), “medium” (“to a moderate extent”), and “high” (grouping “to great extent” and “to a very great extent”). In order to verify the differences between answers from teachers and from principals, the Pearson chi-square test was used. Content analysis methodology was used for the qualitative data, which is based on the one proposed by ground theory (Strauss, & Corbin, 2008), using NVivo 10 software. The qualitative data in the study were based on the experiences of teachers, principals, and pedagogical coordinators who participated in the GBG implementation, representing the schools’ pedagogical teams, as well as on information and reflections provided by the coaches in the program. The program acceptability parameters were analyzed and separated into two categories: (1) acceptability, perception of change, and indications of adaptation by teachers and pedagogical teams; and (2) acceptability and perception of change by the coaches. Ethics/Institutional Review Board Approval This study was approved by the Ethics in Research Committee at Federal University of São Paulo (#473.498) and all stages of the project were complaint with the Declaration of Helsinki. Informed consent to participate in the study was obtained from the participants. Results Quantitative Results In general, teachers and administrators found GBG to be a suitable intervention program. They found GBG easy to be included in the daily classroom routine and commented on it being helpful in classroom and student behavior management. Both groups intend to continue to use GBG and would recommend it to other teachers and administrators (see Table 1). Table 1 Perceptions of teachers and school administrators about the implementation of GBG. Variables Perceptions of teachers (n = 28) Perceptions of administrators (n = 9) Low n (%) Medium n (%) High n (%) Low n (%) Medium n (%) High n (%) p-value Easy to implement 6 (21.4) 4 (14.3) 18 (64.3) 2 (22.2) 5 (55,6) 2 (22.2) 0,028 Easy to include the game in the daily classroom routine 6 (21.4) 7 (25.0) 15 (53.6) 2 (22.2) 2 (22.2) 5 (55.6) 1,000 Fits in with your style of teaching 5 (17.9) 8 (28.6) 15 (53.6) 1 (11.1) 4 (44.4) 4 (44.4) 0,774 Are happy with GBG 4 (14.3) 6 (21.4) 18 (64.3) 1 (11.1) 5 (55.6) 3 (33.3) 0,158 Motivation to use GBG in the classroom 5 (17.9) 8 (28.6) 15 (53.6) 1 (11.1) 4 (44.4) 4 (44.4) 0,774 Useful for managing student behavior in the classroom 3 (10.7) 8 (28.6) 17 (60.7) 2 (22.2) 2 (22.2) 5 (55.6) 0,738 Culturally appropriate for students 3 (10.7) 13(46.4) 12 (42.9) 0 (0) 5 (55.6) 4 (44.4) 0,866 Have the intention of promoting the game in the future 3 (10.7) 9 (32.1) 16 (57.1) 1 (11.1) 3 (33.3) 5 (55.6) 1,000 Their students need a program like GBG 2 (7.1) 11 (39.3) 15 (53.6) 2 (22.2) 3 (33.3) 4 (44.4) 0,487 The amount of time, effort and resources required to implement GBG is reasonable 2 (7.1) 11 (39.3) 15 (53.6) 3 (33.3) 4 (44.4) 2 (22.2) 0,089 Would recommend it to other teachers and administrators 3 (10.7) 8 (28.6) 18 (64.3) 2 (22.2) 1 (11.1) 6 (66.7) 0,476 GBG: Good Behavior Game. However, teachers questioned whether GBG was culturally appropriate for Brazilian students, which is indicated by the number of “low” and “medium” answers (1–2 and 3–4 on a scale of 1–6) exceeding 56 percent on this item, showing a need for a cultural adaptation of the game. Furthermore, there was a significant difference in the perception of teachers and administration on how easy it was to implement GBG: Teachers’ perception was more favorable than administration’s (see Table 1). Regarding teachers’ perception of the implementation process and GBG’s impact on the students, the majority of teachers indicated a medium to high impact in some aspects of students’ behavior, particularly teamwork, attention, positive relationships with peers, and self-control. Such results were confirmed by the qualitative data. Results were not as positive regarding teachers’ perceptions of changes in the level of distraction in the classroom and in engagement in learning. Teachers perceived the level of principal’s support and promotion of GBG and their offering of resources for implementation as low as principals did. From the 28 teachers, 50.0 percent found the resources for implementation to be low, and 53.6 percent found the level of support and promotion of GBG by principals to be low (see Table 2). Table 2 Perception of teachers on the implementation and impact of GBG on their students. Teacher opinion Perceptions of teachers (n = 28) Low n (%) Medium n (%) High n (%) Implementation of the program Managed to implement well GBG 4 (14.3) 8 (28.6) 16 (57.1) The school administrator has provided the resources needed to implement GBG 14 (50.0) 8 (28.6) 6 (21.4) The school administrator has promoted the implementation of GBG 15 (53.6) 10 (35.7) 3 (10.7) GBG impact on students with regard to: Self-control 4 (14.3) 11 (39.3) 12 (42.9) Teamwork 5 (17.9) 10 (35.7) 13 (46.4) Reducing distractions in the classroom 3 (10.7) 14 (50.0) 10 (35.7) Support to others 4 (14.3) 15 (53.6) 9 (32.1) Attention 3 (10.7) 11 (39.3) 13 (46.4) Positive relationships with peers 4 (14.3) 11 (39.3) 13 (46.4) Engagement in learning 4 (14.3) 14 (50.0) 10 (35.7) Respect to others 5 (17.9) 12 (42.9) 11 (39.3) GBG: Good Behavior Game. Principals believed that they had offered the resources needed for the implementation, and none of the administrators rated the level of resources as low. Only one administrator, however, considered himself to be highly supportive in encouraging the use of GBG, confirming teachers’ perception of low levels of support and promotion for GBG from the administrators. Administrators also reported that, despite not being fully supportive of GBG, teachers managed to implement the program well in the classrooms: 2 out of 9 administrators (22.2%) reported a low level of implementation of GBG (see Table 3). Table 3 Perceptions of administrators on the implementation of GBG. Variables Perceptions of administrators (n = 9) Baixa n (%) Média n (%) Alta n (%) Teachers managed to implement the program well in the classrooms. 2 (22.2) 3 (33.3) 4 (44.4) Resources that teachers needed were provided to implement GBG 0 (0) 4 (44.4) 5 (55.6) School administrators encouraged teachers to use GBG 4 (44.4) 4 (44.4) 1 (11.1) School administrators promoted the implementation of GBG 1 (11.1) 5 (55.6) 3 (33.3) GBG: Good Behavior Qualitative Results Principals, pedagogical teams, and teachers who participated in the pilot study were surprised by the positive changes in students’ behavior and in the pedagogical practices related to the use of GBG. At the beginning of the pilot study, when the program was presented to schools and during the training, there were doubts about whether the program would work and whether it could be adapted to the pedagogical context of Brazilian schools. This initial negative perception changed, little by little, as the game was implemented in the classroom. Based on the conversation I had with teachers, I see that my first impression has changed a bit. Students liked it, I received that feedback, and they participated actively in the game. The teacher observed some progress with them. All of this, considering the short time that we used the game. This means that there is actually something positive in the program (Principal). I recommend the game because of the positive gain I had, because of this view that it is not about competitiveness, but it is a cooperative game. My initial opinion has changed… It is not the rigid model that it looked like in the beginning (Teacher). Teachers said that students enjoyed playing GBG and described students as engaging in activities quietly, working in groups, and playing with other students in the group: “They loved it. A student who used to be one of the least quiet mentioned how nice it was to do the activity in silence” (Teacher). The members of the pedagogical teams were initially concerned that GBG would stimulate competition in the classroom by connecting success in the game to a reward. But as the implementation of the program developed, it became clear from teachers’ reports that teams were collaborating among themselves so that everybody would win and nobody would lose. The members of the team supported one another and helped to monitor their peers’ behavior because they wanted everyone to be rewarded–simultaneously developing a cooperative attitude. This confirmed the possibility of using the program within a pedagogical framework based on the Cultural Historical Activity Theory of Vygotsky (1978), which is commonly used in Brazilian schools. In general, teachers highlighted teamwork, the systematic use of collective praise, and the systematic objective use of classroom rules as very positive characteristics of GBG. To teach the rules of the game, teachers followed the guidelines in the manual as much as possible and adapted GBG to more appropriately fit their classroom, in other words, they used examples of the classroom culture to better explain the rules. According to most teachers, students did not have any difficulties in understanding the rules. Teachers considered the rules clear, objective, and covering what is usually necessary for a learning environment: working quietly or with appropriate noise level, being polite to others, following teacher’s instructions, and asking for permission to move around in the classroom. Nevertheless, the way the rules were phrased gave an impression of rigidity, of imposition–a component of GBG that made some teachers uncomfortable, because the rules were not perceived as being in accordance with the constructivist pedagogical project. Therefore, they emphasized the need to rephrase the rules. Teachers also pointed out the need for some changes in the student booklets, to adapt them to the local cultural context. They also asked for more time to discuss the positive rewards and for more suggestions for culturally relevant rewards to be included in the teacher manual. Teachers described the pedagogical and behavioral gains made possible by the game. They emphasized, for example, the importance of dividing the students in heterogeneous teams, because these teams caused important changes in the relationship dynamics in the classroom and helped “rescue those who are quieter and who we, sometimes, end up leaving behind. I think this was the best part” (Teacher). The composition of teams also allowed students with special needs to receive more support from their peers, making social inclusion easier. Some teachers mentioned more general changes in classroom behavior, indicating that during GBG, students were calmer and more concentrated and productive. Collaborative teamwork in heterogeneous teams was an important factor in changing student’s behavior. Other teachers mentioned changes they observed in individual students. Some students who had no concept of boundaries or limits and difficulty staying in their seats improved their behavior, making an effort to stay calmer. Moreover, teachers did not feel they had to ask students as often as before to lower their voice levels when the classroom got too noisy. Other students who had difficulty with interpersonal relationships and who were aggressive with their peers tried to control themselves, respected their peers, and were aware that they would keep their teams from winning with inadequate behaviors. Some students committed to the rules and even asked their peers to do the same. Finally, when shy students were chosen as group leaders, they felt valued and started to feel more secure and participated more actively. According to most testimonials, however, the behavioral changes had not been generalized yet due to the short time of the game implementation. That is, the gains achieved during the game were not being maintained in other class periods or in extra-class spaces, such as recreational spaces, arrival and departure from school, and others pedagogical contexts outside classrooms. Participants in the pilot study indicated interest in continuing the project to see whether the results would generalize beyond the times GBG is played. Teachers also identified negative events related to implementation of GBG. They mentioned the excess of documents to fill out, some of them with repeated information. Another issue was the lack of time in teachers’ busy work routines to dedicate to the implementation of GBG. Teachers’ schedules often did not allow them to discuss and share the results of GBG with their colleagues. Nevertheless, the majority of the respondents would recommend GBG to other teachers and said that they wanted to continue using the program in the following year. Yes, the game is interesting. I intend to keep using it regardless of whether the school will continue [in the project] or not. I will use the experience I had in the classroom, because it was relevant to the entire class. I would recommend it because I saw the results in my classroom (Teacher). I intend to continue. We only received positive information from the teachers. So it wouldn’t make sense for us to stop a project that has been positively evaluated by teachers (Pedagogical coordinator). The Role of the Coaches Teachers interviewed had a positive evaluation of the continuous support provided by the coaches. Some teachers found the coaching support to be necessary and very helpful. Other teachers mentioned the benefits of partnership and exchanging knowledge. The testimonials highlight the importance of the support given to teachers, answering questions and monitoring the implementation of GBG, which made teachers feel more secure: “The coach played a key role in guiding me on what to do in the classroom, in answering questions, in evaluating what was done, and in pointing out what needed to be improved” (Teacher). According to the coaches, one of the main gains from GBG was the changes in teachers’ pedagogical practices. Coaches observed how, little by little, teachers started to recognize student progress instead of mistakes, shifting their focus from problem to progress. Teachers moved from general negativity to seeking explicit and individualized solutions related to their students’ difficulties. GBG and the Pedagogical Project of Brazilian Schools All members of the pedagogical teams indicated that the GBG pilot project was aligned with the Pedagogical Project of the schools, in the measure that the projects included activities to develop good interpersonal relationships and cooperative attitudes within the school community. According to principals and pedagogical teams, GBG did not add excessive requirements to the school routine. It was the dedication of the teachers, however, that made implementation strong and effective in the schools. The fact that teachers spent time outside regular contract hours to prepare for GBG or for coaching was a concern for administration and pedagogical teams, because these teachers were already overloaded with academic activities. From the coaches’ perspective, schools must be more involved in the implementation of the game, so that it is truly incorporated into the pedagogical framework of the school and does not depend solely on teachers’ personal engagement. During the pilot, schools were not always appropriately committed, according to coaches, confirming teachers’ perception of schools support for GBG implementation. Discussion The aim of the pilot study was to investigate the suitability of the GBG prevention program for Brazilian schools. The pilot study collected implementation and student behavior–related data. The implementation results are based on data collected from teachers, school administrators, and coaches involved in the implementation of GBG. The integration of qualitative and quantitative data, using a mixed-methods approach and data triangulation, allowed a more consistent evaluation of the program. The data showed that school staff considered GBG a suitable intervention program for Brazilian schools. Moreover, teachers’ perception of GBG’s effect on student behavior was quite favorable, even with the short, three-month implementation period. At the beginning of the pilot study, GBG was not well accepted and well understood by some Brazilian teachers and administrators because it is based on behavioral theory, which is a different paradigm from the pedagogical paradigm predominant in Brazil (in other words, the Socio-Historical Theory based on Vygotsky). GBG was initially perceived as a competitive game. After the experience of implementing GBG in the classroom, however, teachers’ and administrators’ opinions changed, and they saw the cooperative nature of the game. Such change can be associated with several factors, including the training and support provided by the coaches during the implementation (Becker, Bradshaw, Domitrovich, & Ialongo, 2013). In addition, one of the key components of GBG is team membership (in other words, teamwork): cooperation within and among students is essential for the teams to be successful. The teams are by design heterogeneous, including both boys and girls, and students with different behavioral styles. The success of the heterogeneous teams requires building positive peer relationships and acceptance of differences, which in turn helps reduce classroom conflicts and opens up new possibilities for socialization and positive peer and adult relationships. There was a difference in teachers’ and administrators’ perceptions on how easy the implementation of the program in classrooms was and the support provided by the school administration. Teachers perceived the support by the school administration for the implementation of GBG to be low, although teachers considered GBG easy to implement. According to Donaldson, Vollmer, Krous, Downs, & Berard (2011), GBG is a strategy that teachers consider easy to implement because of the simplicity of the concept and its execution. These data emphasize the relevance of a well-established commitment and readiness by school administration to implement GBG. Although GBG is a prevention strategy that does not require many resources or organizational changes such as changes in school schedules (and therefore it is easy to implement), support from school staff is central to create a safe and supportive environment for teachers to implement it. Most teachers considered GBG to be a useful tool for managing the classroom routine and to support children’s relationship in the classroom environment, a difficult task to accomplish with large class sizes and students coming from different socioeconomic backgrounds, as is the reality in Brazilian schools. As a result, teachers and administrators gave positive feedback on GBG and on its potential as a prevention program for Brazil. It is important that the school staff is convinced that the intervention is useful and appropriate for the school context so it can be implemented with commitment and quality (Baker-Henningham, & Walker, 2009), which is what the teachers who participated in this pilot experience observed. Teachers perceived the effectiveness of GBG in changing student behavior to be aligned with findings from previous research conducted in other countries, emphasizing that GBG is a strategy for classroom intervention that can promote positive changes in students’ behavior, as well as provide support for a more effective pedagogical practice (Donaldson, Vollmer, Krous, Downs, & Berard, 2011; Flower et al., 2014; Poduska et al., 2008). Although it must be taken into consideration that this was the first study of the implementation of GBG in Brazil, a few limitations were observed: the results of the evaluation based on interview and questionnaire data are fairly favorable. Because the pilot study is government funded and led, it is possible that teachers and administrators overstated the positive aspects and minimized the difficulties associated with implementing GBG. In addition, it should be noted that the game was only implemented for three months, which is a too short period to evaluate the real impact of the implementation. The results were important to highlight the acceptability of the program in Brazil and to support the idea that GBG can be transculturally adapted. It is necessary, therefore, to evaluate GBG during a longer implementation period in order to produce a more robust evaluation. Thus, in the following year (2014) a new evaluation of the implementation process was needed, based on cultural adaptations to the pilot study described here. A study on the effectiveness of Jogo Elos–the Brazilian version of GBG–based on a randomized controlled study will also be necessary and the Ministry of Health is planning to conduct it in 2016. Conclusions — Implications for Research and Practice The pilot study of GBG was seen as highly acceptable by school staff, including administrators, pedagogical teams, and teachers. GBG was found to be a simple and effective strategy for classroom management, and teachers reported that it had a positive impact on student behavior. At the end of the pilot study, the program was considered as appropriate by most teachers and administrators to be implemented in Brazilian schools. Nevertheless, participants in the study emphasized the need for changes in the translated materials in terms of language and terminology, especially the phrasing of the classroom rules, and the definition of rewards, to make GBG even more suitable for the Brazilian education system. References Baker-Henningham, H., & Walker, S. (2009). A qualitative study of teacher’s perceptions of an intervention to prevent conduct problems in Jamaican pre-schools. Child: Care Health and Development, 35(5), 632-42. doi:10.1111/j.1365-2214.2009.00996.x Baker-Henningham H. Walker S. 2009 A qualitative study of teacher’s perceptions of an intervention to prevent conduct problems in Jamaican pre-schools Child: Care Health and Development 35 5 632 642 10.1111/j.1365-2214.2009.00996.x Barrish, H. H., Saunders, M., & Wolf, M. M. (1969). Good behavior game: effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis, 2(2), 119-124. doi:10.1901/jaba.1969.2-119 Barrish H. H. Saunders M. Wolf M. M. 1969 Good behavior game: effects of individual contingencies for group consequences on disruptive behavior in a classroom Journal of Applied Behavior Analysis 2 2 119 124 10.1901/jaba.1969.2-119 Bayer, J., Hiscock, H., Scalzo, K., Mathers, M., McDonald, M., Morris, A.,… Wake, M. (2009). Systematic review of preventive interventions for children’s mental health: what would work in Australian contexts? Australian New Zealand Jounrnal of Psychiatry, 43(8), 695-710. doi:10.1080/00048670903001893 Bayer J. Hiscock H. Scalzo K. Mathers M. McDonald M. Morris A. Wake M. 2009 Systematic review of preventive interventions for children’s mental health: what would work in Australian contexts? Australian New Zealand Jounrnal of Psychiatry 43 8 695 710 10.1080/00048670903001893 Becker, A. L. M. M., Souza, P. H., Oliveira, M. M., & Paraguay, N. L. (2014). Child protection network and the intersector implementation of the circle of security as alternatives to medication. Revista Paulista de Pediatria, 32(3), 247-251. doi:10.1590/0103-0582201432316 Becker A. L. M. M. Souza P. H. Oliveira M. M. Paraguay N. L. 2014 Child protection network and the intersector implementation of the circle of security as alternatives to medication Revista Paulista de Pediatria 32 3 247 251 10.1590/0103-0582201432316 Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013). Coaching teachers to improve implementation of the good behavior game. Administration and Policy in Mental Health, 40(6), 482-493. doi:10.1007/s10488-013-0482-8 Becker K. D. Bradshaw C. P. Domitrovich C. Ialongo N. S. 2013 Coaching teachers to improve implementation of the good behavior game Administration and Policy in Mental Health 40 6 482 493 10.1007/s10488-013-0482-8 Cord, D., Gesser, M., Nunes, A. S. B., & Storti, M. M. T. (2015). As significações de profissionais que atuam no programa saúde na escola (PSE) acerca das dificuldades de aprendizagem: patologização e medicalização do fracasso escolar. Psicologia: Ciência e Profissão, 35(1), 40-53. doi:10.1590/1982-3703000952013 Cord D. Gesser M. Nunes A. S. B. Storti M. M. T. 2015 As significações de profissionais que atuam no programa saúde na escola (PSE) acerca das dificuldades de aprendizagem: patologização e medicalização do fracasso escolar Psicologia: Ciência e Profissão 35 1 40 53 10.1590/1982-3703000952013 Creswell, J. (2009). Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE. Creswell J. 2009 Research design: qualitative, quantitative, and mixed methods approaches Thousand Oaks, CA SAGE Donaldson, J. M., Vollmer, T. R., Krous, T., Downs, S., & Berard, K. P. (2011). An evaluation of the good behavior game in kindergarten classrooms. Journal of Applied Behavior Analysis, 44(3), 605-609. doi:10.1901/jaba.2011.44-605 Donaldson J. M. Vollmer T. R. Krous T. Downs S. Berard K. P. 2011 An evaluation of the good behavior game in kindergarten classrooms Journal of Applied Behavior Analysis 44 3 605 609 10.1901/jaba.2011.44-605 Embry, D. D. (2002). The Good Behavior Game: a best practice candidate as a universal behavioral vaccine. Clinical Child and Family Psychology Review, 5(4), 273-297. doi:10.1023/A:1020977107086 Embry D. D. 2002 The Good Behavior Game: a best practice candidate as a universal behavioral vaccine Clinical Child and Family Psychology Review 5 4 273 297 10.1023/A:1020977107086 Flay, B. R. (2009). School-based smoking prevention programs with the promise of long-term effects. Tobacco Induced Diseases, 5(6), 18. doi:10.1186/1617-9625-5-6 Flay B. R. 2009 School-based smoking prevention programs with the promise of long-term effects Tobacco Induced Diseases 5 6 18 10.1186/1617-9625-5-6 Flower, A., McKenna, J., Muething, C. S., Bryant, D. P., & Bryant, B. R. (2014). Effects of the good behavior game on classwide off-task behavior in a high school basic algebra resource classroom. Behavior Modification, 38(1), 45-68. doi:10.1177/0145445513507574 Flower A. McKenna J. Muething C. S. Bryant D. P. Bryant B. R. 2014 Effects of the good behavior game on classwide off-task behavior in a high school basic algebra resource classroom Behavior Modification 38 1 45 68 10.1177/0145445513507574 Foxcroft, D. R., & Tsertsvadze, A. (2011). Universal school-based prevention programs for alcohol misuse in young people (Review). Cochrane Database of Systematic Reviews, 11(5):CD009113. doi:10.1002/14651858.CD009113 Foxcroft D. R. Tsertsvadze A. 2011 Universal school-based prevention programs for alcohol misuse in young people (Review) Cochrane Database of Systematic Reviews 11 5 CD009113 10.1002/14651858.CD009113 Halgunseth, L., Carmack, C., Childs, S., Caldwell, L., Craig, A, & Smith, E. (2012). Using the interactive systems framework in understanding the relation between program capacity and implementation in afterschool settings. American Journal of Community Psychology, 50(0): 311-320. doi:10.1007/s10464-012-9500-3 Halgunseth L. Carmack C. Childs S. Caldwell L. Craig A Smith E. 2012 Using the interactive systems framework in understanding the relation between program capacity and implementation in afterschool settings American Journal of Community Psychology 50 0 311 320 10.1007/s10464-012-9500-3 Kellam, S. G., Mackenzie, A. C., Brown, C. H., Poduska, J. M., Wang, W., Petras, H., & Wilcox, H. C. (2011). The good behavior game and the future of prevention and treatment. Addict Sci Clin Pract, 6(1), 73-84. Kellam S. G. Mackenzie A. C. Brown C. H. Poduska J. M. Wang W. Petras H. Wilcox H. C. 2011 The good behavior game and the future of prevention and treatment Addict Sci Clin Pract 6 1 73 84 Kellam, S. G., Wang, W., Mackenzie, A. C., Brown, C. H., Ompad, D. C., Or, F., & Windham, A. (2014). The impact of the good behavior game, a universal classroom-based preventive intervention in first and second grades, on high-risk sexual behaviors and drug abuse and dependence disorders into young adulthood. Prevention Science, 15(1), 6-18. doi:10.1007/s11121-012-0296-z Kellam S. G. Wang W. Mackenzie A. C. Brown C. H. Ompad D. C. Or F. Windham A. 2014 The impact of the good behavior game, a universal classroom-based preventive intervention in first and second grades, on high-risk sexual behaviors and drug abuse and dependence disorders into young adulthood Prevention Science 15 1 6 18 10.1007/s11121-012-0296-z Poduska, J. M., Kellam, S. G., Wang, W., Brown, C. H., Ialongo, N. S., & Toyinbo, P. (2008). Impact of the Good Behavior Game, a universal classroom-based behavior intervention, on young adult service use for problems with emotions, behavior, or drugs or alcohol. Drug and Alcohol Dependence, 95 Suppl 1, S29-44. doi:10.1016/j.drugalcdep.2007.10.009 Poduska J. M. Kellam S. G. Wang W. Brown C. H. Ialongo N. S. Toyinbo P. 2008 Impact of the Good Behavior Game, a universal classroom-based behavior intervention, on young adult service use for problems with emotions, behavior, or drugs or alcohol Drug and Alcohol Dependence 95 Suppl 1 S29 S44 10.1016/j.drugalcdep.2007.10.009 Sloboda, Z., & Petras, H. (Eds.). (2014). Defining prevention science. New York, NY: Springer. Sloboda Z. Petras H. (Eds.) 2014 Defining prevention science New York, NY Springer Strauss, A. L., Corbin, J., (2008). Pesquisa qualitativa: técnicas e procedimentos para o desenvolvimento de teoria fundamentada. 2ª edição. Porto Alegre, RS: Artmed. Strauss A. L. Corbin J. 2008 Pesquisa qualitativa: técnicas e procedimentos para o desenvolvimento de teoria fundamentada 2ª edição Porto Alegre, RS Artmed Tingstrom, D. H., Sterling-Turner, H. E., & Wilczynski, S. M. (2006). The Good Behavior Game: 1969-2002. Behavior Modification, 30(2), 225-253. doi:10.1177/0145445503261165 Tingstrom D. H. Sterling-Turner H. E. Wilczynski S. M. 2006 The Good Behavior Game: 1969-2002 Behavior Modification 30 2 225 253 10.1177/0145445503261165 Vygotsky, L. S. (1978). Mind in society: the development of higher mental process. Cambridge, MA: Harvard University Press. Vygotsky L. S. 1978 Mind in society: the development of higher mental process Cambridge, MA Harvard University Press