Can course format influence the performance of students in an advanced cardiac life support (ACLS) program?

Braz J Med Biol Res, January 2011, Volume 44(1) 23-28

Can course format influence the performance of students in an advanced cardiac life support (ACLS) program?

F.D. Garrido¹, M.M.D. Romano², A. Schmidt¹ and A. Pazin-Filho¹

¹Departamento de Clínica Médica, Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, SP, Brasil

²Divisão de Cardiologia do Hospital das Clínicas, Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo, Ribeirão Preto, SP, Brasil

^{Correspondence and Footnotes}

Abstract

Advanced cardiac life support (ACLS) is a problem-based course that employs simulation techniques to teach the standard management techniques of cardiovascular emergencies. Its structure is periodically revised according to new versions of the American Heart Association guidelines. Since it was introduced in Brazil in 1996, the ACLS has been through two conceptual and structural changes. Detailed documented reports on the effect of these changes on student performance are limited. The objective of the present study was to evaluate the effect of conceptual and structural changes of the course on student ACLS performance at a Brazilian training center. This was a retrospective study of 3266 students divided into two groups according to the teaching model: Model 1 (N = 1181; 1999-2003) and Model 2 (N = 2085; 2003-2007). Model 2 increased practical skill activities to 75% of the total versus 60% in Model 1. Furthermore, the teaching material provided to the students before the course was more objective than that used for Model 1. Scores greater than 85% in the theoretical evaluation and approval in the evaluation of practice by the instructor were considered to be a positive outcome. Multiple logistic regression was used to adjust for potential confounders (specialty, residency, study time, opportunity to enhance practical skills during the course and location where the course was given). Compared to Model 1, Model 2 presented odds ratios (OR) indicating better performance in the theoretical (OR = 1.34; 95%CI = 1.10-1.64), practical (OR = 1.19; 95%CI = 0.90-1.57), and combined (OR = 1.38; 95%CI = 1.13-1.68) outcomes. Increasing the time devoted to practical skills did not improve the performance of ACLS students.

Key words: Advanced cardiac life support; Cardiac arrest; Simulation

Introduction

Advanced cardiac life support (ACLS) is a problem-based course, which employs simulation techniques to teach standard cardiovascular emergency management. Its structure is periodically revised according to new versions of the American Heart Association (AHA) guidelines (1). Since it was introduced in Brazil in 1996, the course has been through two conceptual and structural changes (in 2000 and 2005). The course is becoming more case-based with well-defined objectives and practice activities are emphasized (2).

The Training Center of the Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo (TC-FMRP-USP), established in 1999, has trained approximately 3650 professionals in the 146 courses conducted until the end of 2007. Half of these students took the course before the 2000 AHA guidelines, which implemented a new teaching model for the ACLS courses. Information regarding course (place, date, etc.) and students’ characteristics (age, gender, profession, etc.) have been documented for almost all courses (3). However, little information is available about the impact of these changes on approval of students, particularly regarding the Brazilian population (4).

Material and Methods

To evaluate the impact of the changes in the ACLS course, made in the year 2000 and effectively implemented in 2003 in Brazil, on student outcome, we performed a retrospective study on 3720 health professionals (physicians, medical residents, undergraduate medical students, and nurses) who had taken the ACLS course provided by TC-FMRP-USP from 1999 to 2007. We limited the study to July 2007, when a new course change was effectively implemented (2005 guidelines). We gathered data from the form completed by the students before the course, the theoretical evaluation, the instructors’ log, and the student evaluation form of the course for the 3266 students of the 146 courses conducted. The study was approved by the Ethics Committee of the Faculdade de Medicina de Ribeirão Preto, USP.

Course models

Model 1, employed from 1999 to 2003, consisted of 4 h of lectures (20%), 12 h of practice (60%) and 4 h of administrative activities (20%), including course presentation and student evaluation. Model 2, employed from 2003 to 2007, consisted of 1 h of lecture (5%), 15 h of practice (75%), and 4 h of administrative activities (20%). In addition to an increase in practice at the expense of lectures, the structure of practice sessions was also changed (Figure 1).

In both models, complete instructions about the course and a book were sent to the students 30 days before the course. The teaching material (book) was drastically changed from Model 1 to Model 2. In Model 1, chapter 1 consisted of the essentials and all other chapters consisted of further reading and future reference (5). In Model 2, each chapter referred to a specific case problem. Detailed objectives and questions for reviewing the content at the end were provided in each chapter (6). The student interested in the theoretical rationale for the information provided in the book was referred to complete guidelines published elsewhere (1).

Outcome - Approval in the course evaluation

Both models included a theoretical and a practical evaluation. The theoretical evaluation consisted of multiple-choice tests based on the contents of the book provided. In Model 1, at the end of the theoretical evaluation, the questions answered incorrectly by more than 70% of the students were discussed by the group, while in Model 2 a file with the questions commented upon was distributed to the students for review.

An individual practical evaluation was applied to the students in Model 1. The student was supposed to solve a case proposed by the instructor, responsible for judging the student according to predetermined criteria. In Model 2, the practical evaluation was applied to a group of students who alternately participated in the various roles of the resuscitation team, being evaluated by an instructor according to predetermined criteria.

General course approval was considered for the students who obtained a score of more than 85% in the theoretical evaluation and who were considered apt in the evaluation of practical skills by the instructor. The data were analyzed for this compound outcome as well as for the theoretical and practical outcome separately.

Confounders

The following variables were obtained for each student from different forms filled out during the course: gender, age (years), profession, specialty, place of work and time spent in reading the book prior to the course, and course evaluation by the students: organization, program, time, dynamics, content, structure, lectures, practice, instructors, simulators, teaching material, and opportunity to practice. The identification of the student was optional for course evaluation, which limited our ability to link the information to a subset of the students.

Since the practice grid changed between models, we kept in the analysis only the stations common to both models. In the course evaluation, every station was evaluated using a numerical scale ranging from 1 (poor) to 5 (excellent). We generated binary variables coded as one for values greater than or equal to 4 and zero for lower grades (all raw variables were collected in a scale ranging from 1 (bad) to 5 (excellent) for analytical purposes. There were also two different versions of the course evaluation for Model 1 and Model 2, since course structure was changed, but some skill practice sections were kept unchanged (Figure 1). For analytical purposes, only those variables in both versions were considered.

Variables derived from the raw data were generated for logistic regression analysis. Professional activity was coded as 0 (nurse or physiotherapist), 1 (medical student), 2 (non-internal medicine physician), 3 (internal medicine physician), or 4 (cardiologist, emergency or critical care medicine). Participating in a residency program was coded as a binary variable. Time spent studying prior to the course was coded as 0 (<3 h), 1 (3-5 h) or 2 (>5 h). Opportunity for practice was coded as 0 (<4) or 1 (5). The place where the course was held was classified as 0 (outside the State of São Paulo), 1 (in the State of São Paulo, but outside Ribeirão Preto) or 2 (in Ribeirão Preto).

All data were stored in a Microsoft Access database. Forms were considered to be eligible for extracting data if at least one question was answered. Blank answers in valid forms were not considered for analytical purposes.

Data analysis

Categorical variables are reported as percentages and continuous variables as means and standard deviations. The Fisher exact test or the Student t-test was used for group comparison as applicable. Multiple logistic regression was used to compare the outcome between Models while adjusting for potential confounders. Incremental multiple logistic models starting from a model including only the outcome and the exposure were constructed until the fully adjusted model was obtained, which included professional activity, participation in a residency program, time spent studying, opportunity for practice, and place where the course was held. The combined and separate outcomes (theoretical and practical) were studied using the same multiple logistic models. The level of significance was set at P < 0.05 in all tests. Data were analyzed with the Intercool Stata 9 software (7).

Results

There was a change in the students’ profile from Model 1 to Model 2 (Table 1). The TC-FMRP-USP has constructed a place for offering the courses, reducing the number of courses conducted outside the State of São Paulo and the city of Ribeirão Preto. The students were encouraged to come and take the course in Ribeirão Preto.

Model 1 was better evaluated in the overall evaluation (Table 2), with opposite findings for the practice evaluation regarding the practice station common to both Models (Table 3).

Multivariate logistic regression performed to evaluate better performance showed that Model 2 increased the compound and theoretical approval compared to Model 1 even after adjusting for potential confounders. For evaluation of practical skills, there was no significant difference between models (Table 4).

Figure 1.
Schematic representation of the course models. The dotted lines represent administrative activities, the full-line blank boxes represent practice sessions and the black boxes theoretical sessions. VF/VT, PEA, Bradycardia/Asystole and ACS and BLS were kept from Model 1 to Model 2. VF/VT = ventricular fibrillation/pulseless ventricular tachycardia; PEA = pulseless electrical activity; BLS = basic life support; ACS = acute coronary syndrome.

Table 1.
Students’ characteristics according to course model.

Table 2.
Overall course evaluation according to course model.

Table 3.
Evaluation of practice station, with number of students who received a score of 5 on a scale from 1 (poor) to 5 (excellent).

Table 4.
Odds ratio (95% confidence interval) for theoretical, practice or compound outcome for incremental models of multiple logistic regression.

Discussion

Since they were first introduced in Brazil, ACLS courses have gone through several modifications and are currently undergoing the implementation of the last review guidelines (1). These modifications include not only updating the evidence-based theoretical rationale, but implementing new teaching orientation for the course format (5,6). Even though there is evidence of the positive impact of ACLS training on the survival of patients, there is no evidence for the course modifications implemented either regarding patient survival or better student course approval (8). Recent experimental evidence suggests that these modifications do not have the desired effect (9,10).

Course model evaluation is complex, since it can be influenced by several factors (10,11). Factors dependent on the course program, on how it is conducted, on the instructor’s preparation, and on the students’ preparation prior to the course should be taken into account (4). The 2000 course modification expanded the time dedicated to practice during the course at the expense of the lectures (6). The improvement of the teaching material (book) distributed prior to the course should compensate for this lack of theoretical review. These modifications raised concerns among Brazilian students and instructors, who were used to theoretical review at the beginning of the course and who judged it very important for the Brazilian population, especially for older students or for those with lack of time to study prior the course. Even though this argument could be counterintuitive, it is a cultural issue in Brazil for other course modalities and it is not dealt with in the information letter sent before the course. In spite of these concerns, reducing the theoretical component to 5% in Model 2 had a positive impact on the theoretical and compound outcome. It is very probable that the content simplification and better book structure were responsible for these positive findings. The present study has limitations in terms of determining which component (simplification, better didactic content of the book, or both) was responsible for the positive impact.

Even though Model 1 was better evaluated in general, this finding is of no practical importance (very small effect size), probably reflecting the large sample size. Of note, regarding the specific evaluation of the practice sessions, reducing the theoretical time resulted in more time for practice, a larger number of cases per practice session and more opportunity for the students to play the leading role in the practice session.

Multivariate analysis showed better theoretical and compound outcomes for Model 2, but no benefit related to practice outcome. These findings could raise the hypothesis that the new book provided better conditions for studying prior to the course. No improvement in the practice outcome is a disappointing result but could be related to several biases, which could not be addressed with the retrospective design of our study. First, while in Model 1 the student was evaluated individually in terms of overall performance, in Model 2 the evaluation was performed on a group basis and the student was evaluated for every role he played in the resuscitation team. If we consider the evaluation with Model 2 to be stricter and more complete, this could dilute the better results that the new model could bring with more time to practice. Second, even though the instructors’ capacity was not perceived as a differential between models by the students and their performance was monitored by senior instructors, we could not address inter-instructor variability in our study. As a matter of fact, the 2005 AHA modification of the course was strongly concerned with the objectivity of the evaluation of practice, with specific directions and a checklist for study performance, reducing variability. These changes could lead to a better evaluation of course models in terms of practice outcome in the future.

Limitations

We constructed incremental models for multivariate logistic regression. Even though we included several potential confounding variables, other variables pointed out in the literature as playing a confounding role could not be addressed. In particular, subsidized payment of course tuition was not addressed in our models, since this variable showed co-linearity with the place where the course was held and with being enrolled in a residency program (4). There is a strong association between being a resident and receiving a subsidy for course tuition, since it is current practice for the TC-FMRP-USP to partially pay the course tuition for its residents. It is also very common for the TC-FMRP-USP to be hired to provide "closed" courses for medical associations outside TC-FMRP-USP. These medical associations also pay the subscription fees of their associates in part or in full. Since the profile of TC-FMRP-USP students may be very different from that of the previous study, which pointed out subsidy as a confounding variable for course approval by the students, we could not further explore this subject.

Other limitations that should be pointed out regard instructor preparation, which was not taken into consideration in our evaluation. Since it was founded, the TC-FMRP-USP had several instructors and it was impossible to include this factor in multivariate analysis (3). Nevertheless, since in Model 2 coordinators and instructors were supposed to be retrained and the scenarios reviewed and restructured, we could not exclude the possibility of bias in this quasi-experimental design. This bias could explain a better result regarding theoretical evaluation. Implementing a new course model could also raise the expectations of better outcomes among the coordinators and instructors participating in both course models, which would imply better outcomes.

Another limitation of the present study is the loss of 12.2% of students’ data. These data refer to the first courses provided by the TC-FMRP-USP when all data were kept by the TC-FMRP-USP, responsible for structuring our center. These data could not be recovered for the purposes of the present study.

In summary, increasing the time devoted to practice did not increase the performance of ACLS students.

References

1. ECC Committee SaTFotAHA. 2005 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation 2005; 112: IV1-IV203.

2. Hazinski MF, Nadkarni VM, Hickey RW, O’Connor R, Becker LB, Zaritsky A. Major changes in the 2005 AHA Guidelines for CPR and ECC: reaching the tipping point for change. Circulation 2005; 112: IV206-IV211.

3. Pazin-Filho A, Schmidt A, Filipini C, Castro RBP, Rosa RM, Rosa MAF, et al. Simulação de pacientes - Cursos de suporte de vida - ACLS, BLS e PALS na FMRP-USP. Medicina 2007; 40: 204-212.

4. Miotto HC, Goulart EM, Amaral CF, Moreira MC. Influence of financial subsidy and venue on advanced cardiac life support courses, in the learning of cardiovascular emergency. Arq Bras Cardiol 2008; 90: 172-175.

5. Cummins RO. Suporte avançado de vida em cardiologia. 1st edn. Dallas: American Heart Association; 1997.

6. Cummins RO. Suporte avançado de vida em cardiologia - manual para provedores. 1st edn. Dallas: American Heart Association; 1997.

7. Stata Statistical Software. [Computer program]. College Station: StataCorp.; 2005.

8. Moretti MA, Cesar LA, Nusbacher A, Kern KB, Timerman S, Ramires JA. Advanced cardiac life support training improves long-term survival from in-hospital cardiac arrest. Resuscitation 2007; 72: 458-465.

9. Xanthos T, Tsirikos-Karapanos N, Papadimitriou D, Vlachos IS, Tsiftsi K, Ekmektzoglou KA, et al. Resuscitation outcomes comparing year 2000 with year 2005 ALS guidelines in a pig model of cardiac arrest. Resuscitation 2007; 73: 459-466.

10. Beckers S, Mueller M. Comparing apples and oranges. Crit Care Med 2006; 34: 2263-2264.

11. Pazin-Filho A. Características do aprendizado do adulto. Medicina 2007; 40: 7-16.

Acknowledgments

F.D. Garrido is supported by FAPESP (#2008/00883-1).

Address for correspondence: A. Pazin-Filho, R. Bernardino de Campos, 1000, 14015-100 Ribeirão Preto, SP, Brasil. E-mail: [e-mail]

Received April 21, 2010. Accepted October 21, 2010. Available online November 19, 2010. Published January 17, 2011.

The Brazilian Journal of Medical and Biological Research is partially financed by

Correspondence and Footnotes

Publication Dates

• Publication in this collection
  17 Jan 2011
• Date of issue
  Jan 2011

History

• Accepted
  21 Oct 2010
• Received
  21 Apr 2010