Primary care physicians (PCPs) are the main healthcare providers for most diabetic patients worldwide (1), but these professionals usually lack knowledge and confidence on several aspects of DM management (2), especially regarding insulin use (3), which contributes to clinical inertia (“the failure to advance therapy when indicated”) (4), underuse of insulin (5), and poor glycemic control. Continuing medical education (CME), often advocated to optimize PCPs knowledge and practice (6), have small and short-lasting efficacy (7), demanding new educational methods. Digital games are promising tools for medical education (8), since they are well-acepted (9,10), promote experiential (active) learning (11), and are compatible with principles of Adult Education (12), but evidence on their effectiveness is still scarce (13).
MATERIALS AND METHODS
InsuOnLine© is a serious game developed by a multidisciplinary team, composed by endocrinologists, experts in medical education, and game designers, as described elsewhere (14). In the game, the player takes on the role of a medical doctor in a primary healthcare unit, who must evaluate an increasing-complexity series of patients with diabetes and choose the best therapeutic option to improve their glycemic control, usually requiring insulin initiation or adjustment (14).
To preliminarly assess the educational effectiveness of this game, we conducted a pilot study with a convenience sample of 41 undergraduate medical students (from third to sixth year) and Internal Medicine residents at Londrina State University. Allocation was made by subject preference. In the game group (13 students and 5 residents, 78% female, mean age 24 ± 4), subjects played a Web version of InsuOnLine© from a browser, in their own computers and free time. In the traditional education group (11 students and 12 residents, 56% female, mean age 26 ± 2), subjects attended a presential lecture and clinical cases discussion. Duration (3-4h) and contents were similar for both activities. Clinical problems were designed to depict common situations in primary care. Recommendations were drawn from Brazilian (SBD), (15) American (ADA), (16) and European (EASD) (17) diabetes guidelines, and adapted to be applicable in Brazilian primary health care.
Knowledge and skills on insulin therapy were measured by the score on a questionnaire with 32 multiple-choice items, applied at 3 time points: baseline, immediately after interventions, and 3 months after interventions (to assess content retention). Beliefs about insulin were assessed, at the 3 time points, by 13 Likert-scale questions, freely adapted from Lakkis and cols. (18). In addition, some group-specific questions (15 Likert-scale questions about user satisfaction and perceived usefulness of the traditional activity, and 27 Likert-scale questions to assess game playability) were asked immediately after the corresponding interventions.
Mean knowledge/skills scores were compared within each group by ANOVA, with Bonferroni correction, and between groups at each time point by Students’ t test, using Epi-Info 7 (CDC, Atlanta), with significance level p < 0.05. Beliefs were compared within each group by chi-square test, and between groups by Fisher exact test. Internal consistency (reliability) of the subscales on knowledge/skills on insulin, beliefs regarding diabetes/insulin, game playability, and impressions on the traditional activity was measured by estimation of Cronbach’s alpha coefficient, using SPSS 14.0 (SPSS Inc., New York).
Research procedures were conducted in accordance with the Declaration of Helsinki. Research protocol was previously approved by Londrina State University review board (#15/2014).
RESULTS AND DISCUSSION
Knowledge and skills on insulin therapy, as measured by mean score on the questionnaire, were significantly improved immediately after both interventions (p < 0.0001 as compared to baseline, in both groups). Three months later, mean scores presented a small but significant decrease, compared to immediately after intervention (p = 0.0008 for both groups), but the 3-month scores were still significantly higher than at baseline (p = 0.0003 for both groups) (Figure 1), which demonstrates similar content retention with both activities.
In the comparison between groups, we observed a lower baseline score in the game group (p = 0.04 compared to traditional education group), that could be at least partly explained by the larger proportion of residents in the traditional education group. However, no difference between groups remained at immediate or 3-month posttests (Figure 1). This finding suggests that the game was at least as effective as the traditional activity, regarding knowledge/skills acquisition, or slightly more effective. In fact, mean absolute increment from baseline to immediate post-intervention score was significantly better with the game (29% versus 21% with traditional instruction; p = 0.04).
Beliefs about insulin were significantly improved only in the game group, where subjects abandoned the wrong belief that insulin initiation should be delayed until it is absolutely essential, and started believing (correctly) that insulin will be necessary at some point for most patients with diabetes. Data on the most relevant questions on insulin beliefs is presented in table 1.
|Baseline (% agree)||Immediate (% agree)||3 Months (% agree)|
|I prefer to delay the initiation of insulin until it is absolutely essential|
|I believe that insulin therapy should be initiated by an endocrinologist|
|I believe that the initiation of insulin therapy is one of the most difficult aspects of managing diabetes|
|I believe that most patients with type 2 diabetes will eventually need insulin, regardless of their adherence to treatment|
|I believe that for most patients, the benefits of insulin therapy outweigh the risks|
|I believe that most patients would benefit from insulin therapy prior to developing diabetes complications|
|I believe that for most patients, training on the proper usage of insulin is not complicated|
* p < 0.05 compared to traditional instruction. † p < 0.05 compared to baseline.
All subjects, in both groups, said that the intervention has increased their knowledge, and that it would have impact on their professional practice. The traditional educational activity was considered “pleasant” by all subjects in that group, and the game was considered “fun” by all but one (95%) of subjects in the game group. All subjects in game group considered playing the game more effective for learning than seeing a lecture.
Reliability of the subscales used in this pilot study was estimated as “good” for the 32-item insulin knowledge/skills subscale (Cronbach’s alpha = 0.823), “good” for the 15-item traditional activity impressions subscale (alpha = 0.815), “acceptable” for the 27-item game playability subscale (alpha = 0.778), and “poor” for the 13-item beliefs subscale (alpha = 0.280). Aiming to refine those instruments for a further randomized controlled trial, we found that deleting the worst items would generate shorter and improved subscales, with internal consistency considered “good” for a new 20-item knowledge/skills subscale (alpha = 0.839), “good” for a new 10-item traditional activity impressions subscale (alpha = 0.862), and “good” for a new 16-item playability subscale (alpha = 0.871). Beliefs subscale reliability was not much improved by the deleting worst items (maximum alpha = 0.517 with 6 items), but the authors preferred keeping those questions anyway, because they evaluate personal opinions on insulin, deemed relevant, and because those questions were very alike the ones used in previous studies on PCPs beliefs (3,18), allowing comparison. In fact, despite the difference of subjects characteristics (the surveys from Middle East (18) and United States (3) were made with primary care physicians), insulin beliefs of our subjects at baseline were strikingly similar to those two other studies, except that insulin initiation was more often considered difficult in our study.
This is the first report on the effectiveness of a digital game for medical education on insulin therapy. Our preliminary results suggest that a well-designed game can be at least as effective as traditional instruction to improve knowledge, skills and beliefs on insulin. Games also present other advantages, such as easy dissemination, customizable content, and flexibility of use, that make them excellent tools for large-scale continuing medical education. In order to evaluate InsuOnLine© actual effectiveness, an adequately powered randomized controlled trial with PCPs, using refined questionnaires, is already in progress. We hope our game can contribute to reduce clinical inertia and improve quality of care for patients with diabetes worldwide.