Audience response system smartphone application as an adjunct to tuberculosis teaching for medical students during the coronavirus disease 2019 pandemic

SUMMARY

OBJECTIVE:

The growing availability of devices for mobile learning has created new opportunities for teaching. With the development of smartphone apps based on audience response systems, there is a possibility to quickly assess student knowledge. The education of health professionals, including medical students, is an essential strategy for tuberculosis control. In the context of the coronavirus disease 2019 pandemic, audience response systems are very useful as online assessment tools. The aim of this study was to use the audience response systems Socrative to assess medical students during a class on tuberculosis.

METHODS:

This is a quasi-experimental before-and-after study, with pre- and post-tests carried out through the Socrative app, respectively, before and after a lecture on tuberculosis for medical students. Also, a cross-sectional study was carried out after the course to evaluate the participant's satisfaction through an electronic, structured questionnaire with a Likert-type scale.

RESULTS:

A total of 126 students were included in the study. The overall mean pre- and post-test scores were 5.98±1.59 and 8.37±1.36, respectively, with a statistically significant difference (p<0.0001). Almost all students were totally satisfied with the use of Socrative on pre- and post-tests.

CONCLUSION:

This study describes how the use of Socrative in a tuberculosis class was well received by students. In addition, the baseline knowledge on tuberculosis was low in some topics, with some improvement after the lecture. These findings emphasize the need to further improve the students’ knowledge on tuberculosis and help instructors customize the lecture based on the gaps identified in the Socrative assessment.

KEYWORDS:
Tuberculosis; Distance education; Information technology; Distance learning; Health education

INTRODUCTION

The growing availability of devices for mobile learning (m-learning) has created new opportunities for teaching and assessment. With the development of smartphone apps based on audience response systems (ARSs), there is the possibility to quickly assess student knowledge. ARSs are common tools that teachers can implement in classes, which can identify knowledge gaps, guide teaching, and enhance education. ARSs allow you to collect real-time responses from an audience during a lesson. For this, both hardware and software, such as electronic devices called clickers and applications for smartphones, can be used. Socrative (MasteryConnect, Salt Lake City, UT, USA) is a convenient, free ARS app that can be downloaded to personal handheld devices and used by teachers and students. It can also be accessed directly online at socrative.com. The anonymity and simplicity of the process may be of interest to more introverted students and may possibly reduce participation anxiety¹,². In the context of the coronavirus disease 2019 (COVID-19) pandemic, it is very useful as an online assessment tool. In fact, due to the COVID-19 pandemic, teaching practices were very limited, with classes and courses shifted to online platforms³-⁵.

The education of health professionals, including medical students, is an essential strategy for the control of tuberculosis (TB), enabling the early detection and adequate treatment of TB cases⁶. During the COVID-19 pandemic, TB teaching was impaired, and alternatives had to be sought. No studies to date have evaluated the use of an online assessment tool to evaluate medical students’ knowledge of TB. Therefore, the aim of this study is to use Socrative to assess medical students during a class on TB during the COVID-19 pandemic.

METHODS

Study design and location

We conducted a quasi-experimental before-and-after study, with pre- and post-tests carried out through the Socrative app, respectively, before and after a lecture on TB. Also, a cross-sectional study was carried out after the course to evaluate the participant's satisfaction through an electronic, structured questionnaire with a Likert-type scale.

Medical students were recruited at the Hospital de Clinicas de Porto Alegre (HCPA), linked to the Faculty of Medicine of the Federal University of Rio Grande do Sul (UFRGS). The study was approved by the Ethics Committee at the Hospital de Clínicas de Porto Alegre (number 20190051). All participants signed an informed consent form prior to inclusion in the study.

Participants

Third-year medical students from the Faculty of Medicine of the Federal University of Rio Grande do Sul were included in the study. During the mandatory internship of these students in pulmonology, they have several lectures in the schedule of activities, including one on TB.

Lecture on tuberculosis and data collection

Organizational and methodological aspects

It was carried out through a lecture of about 30 min, including a pre-test to assess the participant's current knowledge and a post-test with the repetition of the pre-test questions immediately after the class to assess what participants learned in class. The tests contained nine questions. The objectives of the class were to update participants on the main concepts in TB patient care and to guide students about the actions taken with TB patients.

Content

The content to be covered included (1) TB: Concepts and Epidemiology; (2) Pathogenesis and Pathophysiology of TB; (3) Diagnosis of TB; and (4) Treatment of TB.

Technological aspects

To carry out the pre- and post-tests, the Socrative smartphone app (MasteryConnect, Salt Lake City, UT, USA) was used. To answer the questions, students accessed Socrative directly online or via the app.

Statistical analysis

Data analysis was performed using IBM SPSS Statistics for Windows, version 22.0 (Armonk, NY, IBM Corp.). Data were presented as the number of cases, mean±standard deviation (SD), or median with interquartile range (IQR). We measured learners’ baseline knowledge (pre-test) and knowledge upon completion of the learning modules (post-test). A score for the pre-test and a score for the post-test were calculated. The Wilcoxon test was used to assess if the learners increased their knowledge to a statistically significant degree. To compare the number of correct answers in pre- and post-test questions, we used the chi-square test, using Yates's correction if indicated, or Fisher's exact test. A value of p<0.05 was considered statistically significant.

To calculate the sample size, we estimated a percentage of correct answers of 60% (before intervention) and 80% after intervention. Thus, considering a confidence level of 95% and a beta error of 0.20, at least 79 participants should be included.

RESULTS

During the study period (January 2021 to December 2022), 126 students who met the inclusion criteria were invited and accepted to participate in the study. The mean age of students was 20.5±0.7 years, and 73 (57.9%) were females. The overall mean pre- and post-test scores were 5.98±1.59 and 8.37±1.36, respectively, with a statistically significant difference (p<0.0001). Table 1 shows the distribution of the frequency of correct answers in the pre- and post-tests.

Table 1
Distribution of frequency of correct answers in pre- and post-tests.

	Pre-test n (%)	Post-test n (%)	p-value
Association between TB and HIV	95 (75.4)	107 (84.9)	<0.0001
Characteristics of LTBI	96 (76.2)	108 (85.7)	<0.0001
BCG vaccine	65 (51.6)	96 (76.2)	<0.0001
Primary TB	45 (35.7)	72 (57.1)	<0.0001
Post-primary TB	95 (75.4)	105 (83.3)	0.001
Association between smoking and TB (increased risk of TB infection and death)	92 (73.0)	115 (91.3)	<0.0001
Characteristics of TB in smokers	12 (9.5)	93 (73.8)	<0.0001
Characteristics of TB pleural effusion	54 (42.9)	61 (48.4)	0.114
TB diagnostic tests	96 (76.2)	106 (84.1)	0.001

• TB: tuberculosis; HIV: human immunodeficiency virus; LTBI: latent tuberculosis infection; BCG: bacillus Calmette-Guérin.

At baseline, the frequency of correct answers was very low in some questions: BCG vaccine (51.6%), primary TB (35.7%), characteristics of TB in smokers (9.5%), and characteristics of TB pleural effusion (42.9%). The questions on primary TB and characteristics of TB pleural effusion remained with a low percentage of correct answers (57.1 and 48.4%, respectively).

Table 2 shows the distribution of frequencies and percentages regarding the participants’ satisfaction with class on TB. The majority of students said that they were satisfied or totally satisfied with methodology (84.1%), class content (88.1%), content update (92.1%), and that learning focuses on subjects of interest (92.1%). Almost all students were totally satisfied with the use of Socrative on pre- and post-tests (97.6%).

Table 2
Participants’ satisfaction with class on tuberculosis.

	Totally dissatisfied (%)	Dissatisfied (%)	Satisfied (%)	Totally satisfied (%)
Methodology	0 (0)	1 (0.8)	19 (15.1)	106 (84.1)
Class content: relevance, suitability of content, and organization	0 (0)	0 (0)	15 (11.9)	111 (88.1)
Use of Socrative in pre- and post-tests	0 (0)	0 (0)	3 (2.4)	123 (97.6)
Content update	0 (0)	1 (0.8)	9 (7.1)	116 (92.1)
The learning focuses on subjects of interest	0 (0)	0 (0)	10 (7.9)	116 (92.1)

• Data are presented as number (%). TB: tuberculosis.

DISCUSSION

In the present study, we demonstrated that almost all students were totally satisfied with the use of Socrative in class. In addition, students’ baseline knowledge was low regarding the BCG vaccine, primary TB, characteristics of TB in smokers, and characteristics of TB pleural effusion. After a class on TB, a significant improvement in knowledge was detected, although some questions remained with a low percentage of correct answers.

Education and training on TB infection and disease are an important part of a TB infection control program⁷. Available evidence suggests that the quality of training has a strong impact on the quality of care provided to patients. A minimum TB training should be standardized and incorporated into medical schools’ curricula, especially in setting with medium-high TB burden⁸.

During the COVID-19 pandemic, classes were online and, given the impossibility of conducting face-to-face assessments, online assessment tools became especially useful. In this context, Socrative stimulates student engagement during distant online teaching and allows teachers to conduct real-time assessments³-⁵. In addition, instant feedback on correct answers can be given to students, and teachers can identify gaps in knowledge and modify future lectures⁹. This is the case for some questions that remained with a low percentage of correct answers in the present study. Providing feedback helps to establish an interactive learning environment¹⁰.

We showed that almost all students were totally satisfied with the use of Socrative in class. In fact, students found that ARS promotes a better environment for interaction and participation in class¹¹. Asking and receiving answers anonymously, without embarrassment, helps them participate more actively in class²,¹²,¹³.

Utilization of ARS during lectures has been shown to enhance learning and increase exam performance¹⁴,¹⁵. A prospective, randomized controlled trial compared one group of residents who received ARS lectures with the other group who received traditional lectures. The authors found that there was 21% improvement between pre- and post-test scores in the ARS lecture group and 2% improvement in the traditional lecture group (p=0.018)¹⁶. In another randomized trial, radiology residents who used ARS integrated into the lecture had significantly higher knowledge in a test performed on the day of the lecture and long-term retention 3 months later¹⁷.

This study has some methodological limitations. First, the study was conducted in a single center; however, we believe that these results may apply to other settings. Second, we did not include a control group, so we cannot presume that the changes in participants’ knowledge were attributable only to TB class and the use of Socrative. Finally, we did not formally assess the validity and reliability of the questions, since this requires a considerable number of stages and is time-consuming and costly, and it is possible that we would not have been able to carry out the study at the height of the pandemic. Despite these limitations, the findings of the present study highlight the importance of mobile technology and ARS in promoting more interactive lectures.

In conclusion, this study describes how the use of Socrative in a TB class was well received by students. In addition, the baseline knowledge on TB was low in some topics, with some improvement after the lecture. These findings emphasize the need to further improve the students’ knowledge on TB and help instructors customize the lecture based on the gaps identified in the Socrative assessment.

• Funding: none.

REFERENCES

• ¹
  Chung H, Kallay T, Anas N, Bruno D, Decamps J, Evans D, et al. Using an audience response system smartphone app to improve resident education in the pediatric intensive care unit. J Med Educ Curric Dev. 2018;5:2382120518770674. https://doi.org/10.1177/2382120518770674
  » https://doi.org/10.1177/2382120518770674
• ²
  Guarascio AJ, Nemecek BD, Zimmerman DE. Evaluation of students’ perceptions of the Socrative application versus a traditional student response system and its impact on classroom engagement. Curr Pharm Teach Learn. 2017;9(5):808-12. https://doi.org/10.1016/j.cptl.2017.05.011
  » https://doi.org/10.1016/j.cptl.2017.05.011
• ³
  Darras KE, Spouge RJ, Bruin ABH, Sedlic A, Hague C, Forster BB. Undergraduate radiology education during the COVID-19 pandemic: a review of teaching and learning strategies [formula: see text]. Can Assoc Radiol J. 2021;72(2):194-200. https://doi.org/10.1177/0846537120944821
  » https://doi.org/10.1177/0846537120944821
• ⁴
  Srinivasan DK. Medical students’ perceptions and an anatomy teacher's personal experience using an e-learning platform for tutorials during the Covid-19 crisis. Anat Sci Educ. 2020;13(3):318-9. https://doi.org/10.1002/ase.1970
  » https://doi.org/10.1002/ase.1970
• ⁵
  Schwartz AM, Wilson JM, Boden SD, Moore TJ, Bradbury TL, Fletcher ND. Managing resident workforce and education during the COVID-19 pandemic: evolving strategies and lessons learned. JB JS Open Access. 2020;5(2):e0045. https://doi.org/10.2106/JBJS.OA.20.00045
  » https://doi.org/10.2106/JBJS.OA.20.00045
• ⁶
  Cabral VK, Valentini DF, Rocha MVV, Almeida CPB, Cazella SC, Silva DR. Distance learning course for healthcare professionals: continuing education in tuberculosis. Telemed J E Health. 2017;23(12):996-1001. https://doi.org/10.1089/tmj.2017.0033
  » https://doi.org/10.1089/tmj.2017.0033
• ⁷
  Centers for Disease Control and Prevention. Tuberculosis infection control. [Internet]. 2019. Available from: https://www.cdc.gov/tb/topic/infectioncontrol/TBhealthCareSettings.htm
  » https://www.cdc.gov/tb/topic/infectioncontrol/TBhealthCareSettings.htm
• ⁸
  Figueroa-Munoz J, Palmer K, Poz MR, Blanc L, Bergström K, Raviglione M. The health workforce crisis in TB control: a report from high-burden countries. Hum Resour Health. 2005;3(1):2. https://doi.org/10.1186/1478-4491-3-2
  » https://doi.org/10.1186/1478-4491-3-2
• ⁹
  Socrative. Socrative by mastery connect. [Internet]. 2023. Available from: www.socrative.com
  » www.socrative.com
• ¹⁰
  Kim KJ. Enhancing students’ active learning and self-efficacy using mobile technology in medical English classes. Korean J Med Educ. 2019;31(1):51-60. https://doi.org/10.3946/kjme.2019.118
  » https://doi.org/10.3946/kjme.2019.118
• ¹¹
  Robson N, Popat H, Richmond S, Farnell DJ. Effectiveness of an audience response system on orthodontic knowledge retention of undergraduate dental students--a randomised control trial. J Orthod. 2015;42(4):307-14. https://doi.org/10.1179/1465313315Y.0000000012
  » https://doi.org/10.1179/1465313315Y.0000000012
• ¹²
  Holmes RG, Blalock JS, Parker MH, Haywood VB. Student accuracy and evaluation of a computer-based audience response system. J Dent Educ. 2006;70(12):1355-61. PMID: 17170327
• ¹³
  Landry MA, Lafrenaye S, Roy MC, Cyr C. A randomized, controlled trial of bedside versus conference-room case presentation in a pediatric intensive care unit. Pediatrics. 2007;120(2):275-80. https://doi.org/10.1542/peds.2007-0107
  » https://doi.org/10.1542/peds.2007-0107
• ¹⁴
  Copeland HL, Longworth DL, Hewson MG, Stoller JK. Successful lecturing: a prospective study to validate attributes of the effective medical lecture. J Gen Intern Med. 2000;15(6):366-71. https://doi.org/10.1046/j.1525-1497.2000.06439.x
  » https://doi.org/10.1046/j.1525-1497.2000.06439.x
• ¹⁵
  Preszler RW, Dawe A, Shuster CB, Shuster M. Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses. CBE Life Sci Educ. 2007;6(1):29-41. https://doi.org/10.1187/cbe.06-09-0190
  » https://doi.org/10.1187/cbe.06-09-0190
• ¹⁶
  Pradhan A, Sparano D, Ananth CV. The influence of an audience response system on knowledge retention: an application to resident education. Am J Obstet Gynecol. 2005;193(5):1827-30. https://doi.org/10.1016/j.ajog.2005.07.075
  » https://doi.org/10.1016/j.ajog.2005.07.075
• ¹⁷
  Rubio EI, Bassignani MJ, White MA, Brant WE. Effect of an audience response system on resident learning and retention of lecture material. AJR Am J Roentgenol. 2008;190(6):319-22. https://doi.org/10.2214/AJR.07.3038
  » https://doi.org/10.2214/AJR.07.3038

Publication Dates

• Publication in this collection
  18 Dec 2023
• Date of issue
  2024

History

• Received
  24 Aug 2023
• Accepted
  27 Aug 2023