In-person and telemedicine course models for disaster preparedness: a comparative analysis

ABSTRACT

Objective:

to compare the students’ performance in face-to-face and telemedicine courses for the training and necessary action in disasters, using telemedicine as an effective training tool.

Methods:

online research conducted after the end of the course of preparation in disasters, carried out in-person, as well as by videoconference. We compared the performance of students in the in-person course and through telemedicine.

Results:

in the comparison of the results obtained with the pre- and post-test data between the students who attended via telemedicine and in-person, we observed that in the two modalities there was an increase in knowledge (p<0.001). We also observed no statistically significant differences in the posterior evaluation between the in-person and telemedicine courses (p=1.0), however, there was a significant difference at the pre-test evaluative moment (p<0.001).

Conclusion:

videoconferencing can be effectively used to train health professionals in disaster management, being able to provide adequate knowledge and become an important tool to distance reaching in continuing education.

Keywords:
Telemedicine; Education, Medical; Videoconferencing; Multiple Trauma; Trauma Centers; Disaster Planning.

RESUMO

Objetivo:

comparar o desempenho dos alunos nos cursos presenciais e via telemedicina para a capacitação e atuação necessária em desastres, se valendo da telemedicina como uma ferramenta efetiva de treinamento.

Métodos:

pesquisa online realizada após o término do curso de preparação em desastres, realizado presencialmente, bem como, por videoconferência. Comparou-se o desempenho dos alunos do curso presencial e via telemedicina.

Resultados:

na comparação dos resultados obtidos com os dados pré e pós-teste entre os alunos que cursaram via telemedicina e presencialmente, observou-se que nas duas modalidades do curso houve aumento do conhecimento (p<0,001). Constatou-se ainda que não houve diferenças estatisticamente significativas na avaliação posterior entre os cursos presenciais e via telemedicina (p=1,0), no entanto, houve diferença com significância no momento avaliativo pré-teste (p<0,001).

Conclusão:

as videoconferências podem ser utilizadas de forma efetiva para a capacitação de profissionais da área de saúde na gestão de desastres, sendo capaz de prover o conhecimento de forma adequada e ser ferramenta importante para alcance à distância em educação continuada.

Descritores:
Telemedicina; Educação Médica; Videoconferência; Traumatismo Múltiplo; Centros de Traumatologia; Planejamento em Desastres

INTRODUCTION

Disaster management requires continuous training and rapid response. Thus, international systems for emergency medical activation have been developed, as well as specific protocols such as courses for medical preparation, aimed at optimizing the response to disasters and at the best performance in real scenarios. It is worth mentioning that in a country of continental proportions such as Brazil, with a territorial extension of 8,514,877km² and a population of over 200 million, natural disasters and catastrophes are a real possibility^1,2.

Brazil has been chosen to host two major sporting events in recent years: the FIFA World Cup, in 2014, and the Summer Olympics, in 2016³. For this reason, the training of teams of prehospital and in-hospital care institutions, as well as the preparation of hospitals for disasters and catastrophes, was critical and needed to be re-analyzed.

Courses like the Advanced Disaster Medical Response (ADRM®), developed by the International Trauma and Disaster Institute (ITDI®) of the Massachusetts Hospital in Boston, sponsored by the Pan American Trauma Society (SPT) and supported by the Brazilian Society of Integrated Attention to Trauma (SBAIT), have been carried out and widely disseminated in Brazil. ADMR® is an eight-hour course designed to educate medical professionals so they are aware of the appropriate and effective response to disaster situations. However, due to the large territorial extension of Brazil, large-scale training of emergency teams is a difficult task. Telemedicine can be a useful tool in the training of teams and, for this reason, is becoming more frequent in several areas of health education and training⁴.

This study aimed to validate the use of teleconferencing and telemedicine as effective teaching tools. To date, there are no published studies comparing the effectiveness in-person courses and via telemedicine in disaster preparedness.

METHODS

We compared two types of courses: 1) the in-person course, which consists of a traditional model of education, with a teacher and students in the same physical space, carried out over one day, with a total of eight hours; and 2) the course via telemedicine, with live videoconferencing broadcasting, also known as distance learning, consisting of a weekly lesson lasting one hour over nine weeks. The place used to carry out the course in the distance mode was the Clinics Hospital of the State University of Campinas (Unicamp), a university hospital with a specialized amphitheater for telemedicine, and which integrates the Telemedicine University Network (RUTE).

The enrollment in the course via telemedicine was linked to the completion of an online questionnaire via Survey Monkey, an internet platform for creating and executing questionnaires, which in turn provides the answers to the administrator. Thus, at the time of enrollment, students were asked to respond to a test before the beginning the course, with ten multiple choice questions, each with a one point value. At the end of the course, we reapplied the same questionnaire, in order to verify learning retention. Each student received a copy of the book “Advanced Disaster Medical Response Manual”, from Briggs and Coimbra, used as a bibliographical reference for the course^5,6. In the questionnaires there was the corresponding identification of the student, due to the necessity of carrying out a comparative study of individual progress.

We applied the same questionnaire in all in-person courses organized in Brazil, before and after classes and the programmatic conclusion of the course, which, in turn, had the same content and presented the same course load of the one via telemedicine. We collected data from questionnaires applied in 23 in-person courses conducted between 2011 and 2014. The material and didactic content did not change during the course period, and we used the same materials in both models.

In the telemedicine courses, nine classes lasting one hour each took place in the period between February and April 2014. The classes had the following titles: “ABC of Disaster Response”, “Management and Command System in Disaster Situations”, “Decontamination”, “Explosion Injuries”, “Crush Injuries”, “Intra-hospital Command and Disaster Simulations”, “Experience with Disaster Response in Santa Maria”, “Telemedicine in Disaster Situations”, and “The SBAIT Plan for Coordination of Disaster Response”. We held the same program in the in-person model.

We tabulated and analyzed the results with Microsoft Excel®. We included only students who completed all the questionnaires in the analysis of the results. We excluded blank responses. We compared the performance results in the pre and post-test periods of the courses through telemedicine with the average results of the in-person courses conducted in the country. We summarized the descriptive variables in frequencies and percentages and presented the continuous variables as mean or median and standard deviation, depending on the distribution. For statistical analysis, we used the Mann-Whitney test with the SPSS 13.0 program. We considered p<0.05 as statistically significant.

This project was submitted to, and approved by, the Ethics in Research Committee of Unicamp, via protocol 19486013.1.0000.5404.

RESULTS

The telemedicine courses had 157 students from eight states in Brazil and 13 different cities. Most of the participants in the course were physicians (34.4%), followed by nurses (27.3%), medical students (13.4%) and firefighters (8.9%). Among participants in the course via telemedicine, 58 (36.9%) had never participated in a disaster response course; another 58 (36.9%) had attended theoretical classes of two to six hours and 41 (26.2%) had participated in training lasting more than six hours. Thirty-five (64.8%) participating physicians had already been involved in multi-victim scenarios. Only 44 students (28%) completed the course with a 100% attendance. For the analyses and comparisons, we used only the grades of the participants who completed the entire course. Among those who completed the course, 18.5% were physicians, 37.2% were nurses and 42.9% were firefighters (Figure 1).

Figure 1
Comparison between the number of students enrolled and those who completed the course.

With regard to in-person courses, there were variations of five to seven courses per year in 14 cities in nine different Brazilian states, with 1,398 students. Of these, 968 (69.2%) answered the questionnaires and 914 completed the pre and post-test questionnaires integrally.

In the telemedicine course, the mean pre-test score was 6.950 ± 1.599, with a median of 7.000 ± 1.599 on a scale of 0 to 10, and the mean post-test increased to 8.200 ± 1.340, with a median of 8.000 ± 1.340. We observed that 93.2% of participants considered the opportunity to interact via videoconference as “good” or “excellent”, and 88.6% considered the overall quality of the course to be “good” or “excellent”, conferring an average quality of 8.3 on a scale of 0 to 10. As for in-person courses, the pre-test average score was 6.510 ± 1.679, with a median of 7.000 ± 1.679 on a scale of 0 to 10, increasing to a mean of 8.690 ± 1.224 with a median of 9.000 ± 1.224 in the post-test. Table 1 shows the comparison of the pre and post-test scores between the two courses. The gain obtained in the students’ grades with the in-person course was 2.175 ± 1.624, while the gain in the grades of the course via telemedicine was 1.250 ± 1.672 (p<0.05).

Table 1
Comparison between the pre and post-tests.

Parameters evaluated	Pre-course (Median ± SD)	Post-course (Median ± SD)	p-value
Telemedicine (N=44)	6.95 ± 1.599	8.20 ± 1.340	<0.001 *
In-person (N=914)	6.51 ± 1.679	8.69 ± 1.224	<0.001 *
Value of p	<0.001 *	1.0

• * Statistical significance (p<0.05).

DISCUSSION

The course via telemedicine was effective in providing knowledge to professionals in several cities of different Brazilian states, validating distance education as a useful teaching tool, independent of the geographical aspects of the country, which had already been demonstrated in other studies^7,8. Both courses showed significant results in the comparison between pre- and post-test scores (p<0.001), confirming that both models are effective in teaching disaster response. Goldberg et al.⁹ demonstrated a statistically significant increase in the knowledge of fourth-year medical students after participating in the ADMR® course, thus demonstrating the importance of the course in the acquisition of knowledge related to medical response to disaster situations. The present study corroborates the data found in the literature.

The groups of students who participated in the in-person and telemedicine courses showed a significant difference (p<0.001) in the scores obtained in the pre and post-test periods. This fact could be attributed to prior experience in disaster management. When comparing the post-test scores, however, the groups achieved statistically equivalent results (p=1.0). The comparison of the gains in the grades between the pre and post-test revealed that in the in-person course there was a larger increase (p<0.05). However, we identified no superiority among the course models and both proved to be effective in teaching disaster response.

Most articles available in the literature refer only to pilot projects or studies with short-term results, with few papers addressing the routine or long-term use of telemedicine^8,10. We should emphasize once again that telemedicine technology is capable of reducing borders, and can be used in geographically remote areas, in various medical situations, including emergencies and disaster preparedness, as well as applied medicine in the military context^7,11.

In this study, there was a high number of student dropouts, since the completion rate was only 28% of the total course participants via telemedicine, which unfortunately limits results validation. Dropouts were possibly due to the long duration of the course and occurred to a lesser extent in one-day courses. On the other hand, we found that many dropouts were attributed to work commitments, which prevented participants from completing the course. This number of dropouts does not present statistical significance according to the occupation of the participants (p=0.144). It is important to emphasize that the course through telemedicine happened during nine weeks, and the post-test applied at the end of this period, while in the in-person course, which lasts only one day, the pre-test was applied at the end of the same day. The different moments of questionnaire application may be a bias in our study, but our objective was not to compare the retention of information between different teaching methods, but rather to analyze if the models are effective as teaching tools.

We should considered some other limitations of the study in the analysis, since the course models have different teaching methodologies. Thus, the comparison of the results of the eight one-hour modules via telemedicine with the eight-hour, in-person model can be considered a bias in course analysis. The ideal situation would be to compare in-person and telemedicine courses that had one eight-hour session or eight one-hour sessions. Another limitation of the study was the questionnaire formulated in ten questions, which may not be enough to test knowledge retention after a single day or eight one-hour sessions. A new questionnaire could be created and applied before and after each course, in addition to other predetermined moments (for example, 60, 90 and 180 days) after completion of the course, in order to validate and demonstrate the retention of knowledge of long-term courses.

One aspect to consider was the choice of the ADMR® course. The Pan American Trauma Society (PTS) has decided to adopt the ADMR® course as the institution’s model for disaster response training. As in the countries of North America, Central America and South America, SBAIT, affiliated to PTS, chooses to adopt the decisions made by supranational organizations, with the goal of standardizing education in the Americas. We believe that, in the future, there should be a model course for disaster education. We are currently following the recommendations of the PTS, although it is not possible to state whether ADMR® will be the reference in the future.

With the FIFA World Cup and the Olympics in Brazil, the disaster preparedness course via telemedicine emerged as one of the solutions to overcome the logistical and cost obstacles involved in the rapid and decentralized training of teams in a country with a great territorial extension. This study showed gain in knowledge by participants in a distance-learning course.

REFERENCES

• ¹
  Reichenbheim ME, Souza ER, Moraes CL, Jorge MH, Silva CM, Minayo MC. Violence and injuries in Brazil: the effect, progress made, and challenges ahead. Lancet. 2011;377(9781):1962-75.
• ²
  Pereira BM, Morales W, Cardoso RG, Fiorelli R, Fraga GP, Briggs SM. Lessons learned from a landslide catastrophe in Rio de Janeiro, Brazil. Am J Disaster Med. 2013;8(4):253-8.
• ³
  Shoaf K, Osorio de Castro CG, Miranda ES. Hospital preparedness in advance of the 2014 FIFA World Cup in Brazil. Prehosp Disaster Med. 2014;29(4):409-12.
• ⁴
  Alverson DC, Edison K, Flournoy L, Korte B, Magruder C, Miller C. Telehealth tools for public health, emergency, or disaster preparedness, and response: a summary report. Telemed J E Health. 2010;16(1):112-3.
• ⁵
  Briggs SM, Coimbra R, editores. Manual de resposta médica avançada em desastres. Bogotá: Distribuna Editorial y Librería Médica; 2009.
• ⁶
  Simões RL, Dorigatti AE, Pereira BM, Calderan TRA, Briggs S, Fraga GP. Education on Advanced Disaster Medical Response: initial experience in Brazil. Panam J Trauma Crit Care Emerg Surg. 2016;5(2):76-82.
• ⁷
  García Jordá E. Telemedicine: shortening distances. Clin Transl Oncol. 2010;12(10):650-1.
• ⁸
  Dorigatti AE, Novaes FN, Pereira BMT, Solino MP, Silva II AC, Godoy ACF, et al. Telemedicina como ferramenta de ensino no cuidado ao paciente queimado. J Bras Tele. 2014;3(1):21-6.
• ⁹
  Goldberg S, Aboutanos M, Mata L, Franzen D, Ivatury R, Briggs S. Educational Benefit of the Advanced Disaster Medical Response Course in a Medical School Curriculum. Panam J Trauma Critical Care Emerg Surg. 2013;2(1):14-6.
• ¹⁰
  Hailey D, Roine R, Ohinmaa A. Systematic review of evidence for the benefits of telemedicine. J Telemed Telecare. 2002;8 Suppl 1:1-30.
• ¹¹
  Pereira BM, Calderan TR, Silva MT, Silva AC, Marttos AC Jr, Fraga GP. Initial experience at a university teaching hospital from using telemedicine to promote education through video conferencing. Sao Paulo Med J. 2012;130(1):32-6.

Publication Dates

• Publication in this collection
  2018

History

• Received
  07 Jan 2018
• Accepted
  29 Mar 2018