Effectiveness of a serious game for medical education on insulin therapy: a pilot study

Leandro A. Diehl Pedro A. Gordan Roberto Z. Esteves Izabel C. M. M. Coelho About the authors

Abstract

Objective

We report the preliminary assessment of InsuOnline©, a serious game designed for medical education on insulin therapy.

Materials and methods

We conducted a pilot study with 41 undergraduate medical students and Internal Medicine residents to assess the educational effectiveness of InsuOnline©, as compared to a traditional educational activity (lecture, cases discussion). Knowledge, skills and beliefs on insulin therapy were evaluated by a questionnaire applied before, immediately after, and 3 months after both interventions.

Results

Mean knowledge/skills score was improved from 68% to 89% in traditional education group (n = 23; p < 0.001), and from 61% to 90% in game group (n = 18; p < 0.001). After 3 months, mean score decreased (to 80% in traditional education group, and to 78% in game group; p < 0.001 for both) but remained significantly higher than at baseline in both groups (p < 0.001 for both). Although mean score was lower in game group than in traditional education group at baseline (p = 0.04), no difference remained between groups either immediately or 3 months post-intervention. Score increment was better with the game (29%) than with traditional education (21%; p = 0.04). Beliefs improved in the game group only.

Conclusions

InsuOnline© is at least as effective as a traditional educational activity for medical education on insulin therapy, and it can a good option for large-scale continuing medical education on diabetes. Arch Endocrinol Metab. 2015;59(5):470-3

Video games; medical education; insulin; diabetes mellitus; computer-assisted instruction


INTRODUCTION

Primary care physicians (PCPs) are the main healthcare providers for most diabetic patients worldwide (1. DeWitt DE, Hirsch IB. Outpatient insulin therapy in type 1 and type 2 diabetes mellitus. JAMA. 2003;289:2254-64.), but these professionals usually lack knowledge and confidence on several aspects of DM management (2. Rubin D, Moshang J, Jabbour S. Diabetes knowledge: are resident physicians and nurses adequately prepared to manage diabetes? Endocr Pract. 2006;12:17-21.), especially regarding insulin use (3. Hayes RP, Fitzgerald JT, Jacober SJ. Primary care physician beliefs about insulin initiation in patients with type 2 diabetes. Int J Clin Pract. 2008;62:860-8.), which contributes to clinical inertia (“the failure to advance therapy when indicated”) (4. Peyrot M, Rubin RR, Lauritzen T, Skovlund SE, Snoek FJ, Matthews DR, et al.; on behalf of the International DAWN Advisory Panel. Resistance to insulin therapy among patients and providers: results of the cross-national Diabetes Attitudes, Wishes, and Needs (DAWN) study. Diabetes Care. 2005;28:2673-9.), underuse of insulin (5. Riddle MC. The underuse of insulin therapy in North America. Diabetes Metab Res Rev. 2002;18:S42-9.), and poor glycemic control. Continuing medical education (CME), often advocated to optimize PCPs knowledge and practice (6. Peyrot M, Rubin RR, Khunti K. Addressing barriers to initiation of insulin in patients with type 2 diabetes. Prim Care Diabetes. 2010;4(Suppl.1):S11-8.), have small and short-lasting efficacy (7. Sharp LK, Lipsky MS. Continuing medical education and attitudes of health care providers toward treating diabetes. J Contin Educ Health Prof. 2002;22:103-12.), demanding new educational methods. Digital games are promising tools for medical education (8. The New Media Consortium. NMC Horizon Report: 2014 Higher education edition. 2014. Available from: http://cdn.nmc.org/media/2014-nmc-horizon-report-he-EN-SC.pdf. Accessed on: Jan 2, 2015.
http://cdn.nmc.org/media/2014-nmc-horizo...
), since they are well-acepted (9. Kron FW, Gjerde CL, Sen A, Fetters MD. Medical student attitudes toward video games and related new media technologies in medical education. BMC Med Educ. 2010;10:50.,1010 . Diehl LA, Souza RM, Gordan PA, Esteves RZ, Coelho ICM. Gaming habits and opinions of Brazilian medical school faculty and students: what’s next? Games Health. 2014;3:79-85.), promote experiential (active) learning (1111 . Gee JP. What video games have to teach us about learning and literacy. New York: Macmillan; 2003.), and are compatible with principles of Adult Education (1212 . Bryan RL, Kreuter MW, Brownson RC. Integrating adult learning principles into training for public health practice. Health Promot Pract. 2009;10:557-63.), but evidence on their effectiveness is still scarce (1313 . Akl EA, Sackett KM, Pretorius R, Bhoopathi PSS, Mustafa R, Schünemann H, et al. Educational games for health professionals. Cochrane Database of Systematic Reviews. In: The Cochrane Library. 2011;09, Art. No. CD006411.).

MATERIALS AND METHODS

InsuOnLine© is a serious game developed by a multidisciplinary team, composed by endocrinologists, experts in medical education, and game designers, as described elsewhere (1414 . Diehl LA, Souza RM, Alves JB, Gordan PA, Esteves RZ, Jorge MLSG, et al. InsuOnline, a serious game to teach insulin therapy to primary care physicians: design of the game and a randomized controlled trial for educational validation. JMIR Res Protoc. 2013;2(1):e5.). In the game, the player takes on the role of a medical doctor in a primary healthcare unit, who must evaluate an increasing-complexity series of patients with diabetes and choose the best therapeutic option to improve their glycemic control, usually requiring insulin initiation or adjustment (1414 . Diehl LA, Souza RM, Alves JB, Gordan PA, Esteves RZ, Jorge MLSG, et al. InsuOnline, a serious game to teach insulin therapy to primary care physicians: design of the game and a randomized controlled trial for educational validation. JMIR Res Protoc. 2013;2(1):e5.).

To preliminarly assess the educational effectiveness of this game, we conducted a pilot study with a convenience sample of 41 undergraduate medical students (from third to sixth year) and Internal Medicine residents at Londrina State University. Allocation was made by subject preference. In the game group (13 students and 5 residents, 78% female, mean age 24 ± 4), subjects played a Web version of InsuOnLine© from a browser, in their own computers and free time. In the traditional education group (11 students and 12 residents, 56% female, mean age 26 ± 2), subjects attended a presential lecture and clinical cases discussion. Duration (3-4h) and contents were similar for both activities. Clinical problems were designed to depict common situations in primary care. Recommendations were drawn from Brazilian (SBD), (1515 . Brazilian Diabetes Society (SBD). Guidelines from the Brazilian Diabetes Society 2013-2014. [In Portuguese.] São Paulo SP; AC Farmacêutica; 2014.) American (ADA), (1616 . American Diabetes Association. Standards of medical care in diabetes – 2014. Diabetes Care. 2014;37(Suppl 1):S14-80.) and European (EASD) (1717 . Inzucchi SE, Bergenstal RM, Buse JB, Diamant M, Ferrannini E, Nauck M, et al. Management of hyperglycemia in type 2 diabetes: a patient-centered approach: position statement of the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care. 2012;35:1364-79.) diabetes guidelines, and adapted to be applicable in Brazilian primary health care.

Knowledge and skills on insulin therapy were measured by the score on a questionnaire with 32 multiple-choice items, applied at 3 time points: baseline, immediately after interventions, and 3 months after interventions (to assess content retention). Beliefs about insulin were assessed, at the 3 time points, by 13 Likert-scale questions, freely adapted from Lakkis and cols. (1818 . Lakkis NA, Maalouf GJ, Mahmassani DM, Hamadeh GN. Insulin therapy attitudes and beliefs of physicians in Middle Eastern Arab countries. Fam Pract. 2013:30:560-7.). In addition, some group-specific questions (15 Likert-scale questions about user satisfaction and perceived usefulness of the traditional activity, and 27 Likert-scale questions to assess game playability) were asked immediately after the corresponding interventions.

Mean knowledge/skills scores were compared within each group by ANOVA, with Bonferroni correction, and between groups at each time point by Students’ t test, using Epi-Info 7 (CDC, Atlanta), with significance level p < 0.05. Beliefs were compared within each group by chi-square test, and between groups by Fisher exact test. Internal consistency (reliability) of the subscales on knowledge/skills on insulin, beliefs regarding diabetes/insulin, game playability, and impressions on the traditional activity was measured by estimation of Cronbach’s alpha coefficient, using SPSS 14.0 (SPSS Inc., New York).

Research procedures were conducted in accordance with the Declaration of Helsinki. Research protocol was previously approved by Londrina State University review board (#15/2014).

RESULTS AND DISCUSSION

Knowledge and skills on insulin therapy, as measured by mean score on the questionnaire, were significantly improved immediately after both interventions (p < 0.0001 as compared to baseline, in both groups). Three months later, mean scores presented a small but significant decrease, compared to immediately after intervention (p = 0.0008 for both groups), but the 3-month scores were still significantly higher than at baseline (p = 0.0003 for both groups) (Figure 1), which demonstrates similar content retention with both activities.

Figure 1
Mean scores on insulin knowledge/skills (in percentage of correct answers) at baseline, immediate post-intervention, and 3 months after intervention (traditional instruction or game).

In the comparison between groups, we observed a lower baseline score in the game group (p = 0.04 compared to traditional education group), that could be at least partly explained by the larger proportion of residents in the traditional education group. However, no difference between groups remained at immediate or 3-month posttests (Figure 1). This finding suggests that the game was at least as effective as the traditional activity, regarding knowledge/skills acquisition, or slightly more effective. In fact, mean absolute increment from baseline to immediate post-intervention score was significantly better with the game (29% versus 21% with traditional instruction; p = 0.04).

Beliefs about insulin were significantly improved only in the game group, where subjects abandoned the wrong belief that insulin initiation should be delayed until it is absolutely essential, and started believing (correctly) that insulin will be necessary at some point for most patients with diabetes. Data on the most relevant questions on insulin beliefs is presented in table 1.

Table 1
Insulin beliefs at baseline, immediately after, and 3 months after educational interventions (traditional instruction or game)

All subjects, in both groups, said that the intervention has increased their knowledge, and that it would have impact on their professional practice. The traditional educational activity was considered “pleasant” by all subjects in that group, and the game was considered “fun” by all but one (95%) of subjects in the game group. All subjects in game group considered playing the game more effective for learning than seeing a lecture.

Reliability of the subscales used in this pilot study was estimated as “good” for the 32-item insulin knowledge/skills subscale (Cronbach’s alpha = 0.823), “good” for the 15-item traditional activity impressions subscale (alpha = 0.815), “acceptable” for the 27-item game playability subscale (alpha = 0.778), and “poor” for the 13-item beliefs subscale (alpha = 0.280). Aiming to refine those instruments for a further randomized controlled trial, we found that deleting the worst items would generate shorter and improved subscales, with internal consistency considered “good” for a new 20-item knowledge/skills subscale (alpha = 0.839), “good” for a new 10-item traditional activity impressions subscale (alpha = 0.862), and “good” for a new 16-item playability subscale (alpha = 0.871). Beliefs subscale reliability was not much improved by the deleting worst items (maximum alpha = 0.517 with 6 items), but the authors preferred keeping those questions anyway, because they evaluate personal opinions on insulin, deemed relevant, and because those questions were very alike the ones used in previous studies on PCPs beliefs (3. Hayes RP, Fitzgerald JT, Jacober SJ. Primary care physician beliefs about insulin initiation in patients with type 2 diabetes. Int J Clin Pract. 2008;62:860-8.,1818 . Lakkis NA, Maalouf GJ, Mahmassani DM, Hamadeh GN. Insulin therapy attitudes and beliefs of physicians in Middle Eastern Arab countries. Fam Pract. 2013:30:560-7.), allowing comparison. In fact, despite the difference of subjects characteristics (the surveys from Middle East (1818 . Lakkis NA, Maalouf GJ, Mahmassani DM, Hamadeh GN. Insulin therapy attitudes and beliefs of physicians in Middle Eastern Arab countries. Fam Pract. 2013:30:560-7.) and United States (3. Hayes RP, Fitzgerald JT, Jacober SJ. Primary care physician beliefs about insulin initiation in patients with type 2 diabetes. Int J Clin Pract. 2008;62:860-8.) were made with primary care physicians), insulin beliefs of our subjects at baseline were strikingly similar to those two other studies, except that insulin initiation was more often considered difficult in our study.

CONCLUSIONS

This is the first report on the effectiveness of a digital game for medical education on insulin therapy. Our preliminary results suggest that a well-designed game can be at least as effective as traditional instruction to improve knowledge, skills and beliefs on insulin. Games also present other advantages, such as easy dissemination, customizable content, and flexibility of use, that make them excellent tools for large-scale continuing medical education. In order to evaluate InsuOnLine© actual effectiveness, an adequately powered randomized controlled trial with PCPs, using refined questionnaires, is already in progress. We hope our game can contribute to reduce clinical inertia and improve quality of care for patients with diabetes worldwide.

REFERENCES

  • 1
    DeWitt DE, Hirsch IB. Outpatient insulin therapy in type 1 and type 2 diabetes mellitus. JAMA. 2003;289:2254-64.
  • 2
    Rubin D, Moshang J, Jabbour S. Diabetes knowledge: are resident physicians and nurses adequately prepared to manage diabetes? Endocr Pract. 2006;12:17-21.
  • 3
    Hayes RP, Fitzgerald JT, Jacober SJ. Primary care physician beliefs about insulin initiation in patients with type 2 diabetes. Int J Clin Pract. 2008;62:860-8.
  • 4
    Peyrot M, Rubin RR, Lauritzen T, Skovlund SE, Snoek FJ, Matthews DR, et al.; on behalf of the International DAWN Advisory Panel. Resistance to insulin therapy among patients and providers: results of the cross-national Diabetes Attitudes, Wishes, and Needs (DAWN) study. Diabetes Care. 2005;28:2673-9.
  • 5
    Riddle MC. The underuse of insulin therapy in North America. Diabetes Metab Res Rev. 2002;18:S42-9.
  • 6
    Peyrot M, Rubin RR, Khunti K. Addressing barriers to initiation of insulin in patients with type 2 diabetes. Prim Care Diabetes. 2010;4(Suppl.1):S11-8.
  • 7
    Sharp LK, Lipsky MS. Continuing medical education and attitudes of health care providers toward treating diabetes. J Contin Educ Health Prof. 2002;22:103-12.
  • 8
    The New Media Consortium. NMC Horizon Report: 2014 Higher education edition. 2014. Available from: http://cdn.nmc.org/media/2014-nmc-horizon-report-he-EN-SC.pdf. Accessed on: Jan 2, 2015.
    » http://cdn.nmc.org/media/2014-nmc-horizon-report-he-EN-SC.pdf
  • 9
    Kron FW, Gjerde CL, Sen A, Fetters MD. Medical student attitudes toward video games and related new media technologies in medical education. BMC Med Educ. 2010;10:50.
  • 10
    Diehl LA, Souza RM, Gordan PA, Esteves RZ, Coelho ICM. Gaming habits and opinions of Brazilian medical school faculty and students: what’s next? Games Health. 2014;3:79-85.
  • 11
    Gee JP. What video games have to teach us about learning and literacy. New York: Macmillan; 2003.
  • 12
    Bryan RL, Kreuter MW, Brownson RC. Integrating adult learning principles into training for public health practice. Health Promot Pract. 2009;10:557-63.
  • 13
    Akl EA, Sackett KM, Pretorius R, Bhoopathi PSS, Mustafa R, Schünemann H, et al. Educational games for health professionals. Cochrane Database of Systematic Reviews. In: The Cochrane Library. 2011;09, Art. No. CD006411.
  • 14
    Diehl LA, Souza RM, Alves JB, Gordan PA, Esteves RZ, Jorge MLSG, et al. InsuOnline, a serious game to teach insulin therapy to primary care physicians: design of the game and a randomized controlled trial for educational validation. JMIR Res Protoc. 2013;2(1):e5.
  • 15
    Brazilian Diabetes Society (SBD). Guidelines from the Brazilian Diabetes Society 2013-2014. [In Portuguese.] São Paulo SP; AC Farmacêutica; 2014.
  • 16
    American Diabetes Association. Standards of medical care in diabetes – 2014. Diabetes Care. 2014;37(Suppl 1):S14-80.
  • 17
    Inzucchi SE, Bergenstal RM, Buse JB, Diamant M, Ferrannini E, Nauck M, et al. Management of hyperglycemia in type 2 diabetes: a patient-centered approach: position statement of the American Diabetes Association (ADA) and the European Association for the Study of Diabetes (EASD). Diabetes Care. 2012;35:1364-79.
  • 18
    Lakkis NA, Maalouf GJ, Mahmassani DM, Hamadeh GN. Insulin therapy attitudes and beliefs of physicians in Middle Eastern Arab countries. Fam Pract. 2013:30:560-7.

Publication Dates

  • Publication in this collection
    Oct 2015

History

  • Received
    31 Jan 2015
  • Accepted
    31 Aug 2015
Sociedade Brasileira de Endocrinologia e Metabologia Rua Botucatu, 572 - Conjuntos 81/83, 04023-061 São Paulo SP Brasil, Tel: (55 11) 5575-0311 - São Paulo - SP - Brazil
E-mail: aem.editorial.office@endocrino.org.br