Anesthesiology teaching during undergraduation through an academic league: what is the impact in students' learning?

Abstracts

JUSTIFICATIVA E OBJETIVOS: As ligas acadêmicas vêm se firmando como instrumentos de ensino médico e inserção dos estudantes de medicina na prática das especialidades, inclusive na anestesiologia. Como o papel das ligas no processo de desenvolvimento de competências e aprendizado de seus alunos não é bem conhecido, avaliou-se o aprendizado dos integrantes de uma liga acadêmica de anestesiologia após um ano de participação nas suas atividades. MÉTODO: Os alunos de uma liga acadêmica de anestesiologia foram acompanhados de março a dezembro de 2010 e avaliados por meio de testes cognitivos objetivos de múltipla escolha aplicados antes do início das atividades e após sua conclusão. A frequência nas atividades e o perfil epidemiológico dos alunos foram correlacionados aos resultados dos testes. RESULTADOS: Foram analisados 20 acadêmicos do 3º ao 6º ano, com média de idade de 22,8 (21-26) anos. A frequência média nas atividades propostas foi 10,4/13 (80%). A média de acertos no primeiro teste foi 8,1/17 (47,6%), sendo que os alunos do 3º ano apresentam notas menores (p < 0,02) em relação aos demais. No teste pós-liga, a média de acertos foi 11,9/17 (70%), mostrando melhora no desempenho dos alunos (p < 0,05), e não houve diferença entre as notas dos diferentes anos da graduação. Foi encontrada relação forte entre frequência nas atividades e melhora nas notas (r = 0,719; p < 0,001). CONCLUSÕES: Os alunos que participaram da liga apresentaram melhora nos testes de avaliação de conhecimento, sugerindo que a liga é útil instrumento de ensino e promove ganho de aprendizado em anestesiologia. As atividades desenvolvidas nas ligas podem ter papel positivo na formação acadêmica dos graduandos, destacando-se, neste artigo, a anestesiologia

ANESTESIOLOGIA; Avaliação Educacional; Educação de Graduação em Medicina

JUSTIFICATIVA Y OBJETIVOS: Las ligas académicas han venido afianzándose como instrumentos de enseñanza médica e inserción de los estudiantes de medicina en la práctica de las especialidades, inclusive en la anestesiología. Como el papel de las ligas en el proceso de desarrollo de competencias y de aprendizaje de sus alumnos no es algo que se conoce muy bien, se evaluó el aprendizaje de los integrantes de una liga académica de anestesiología después de un año de participación en sus actividades. MÉTODO: Los alumnos de una liga académica de anestesiología tuvieron un seguimiento desde marzo a diciembre de 2010, y fueron evaluados por medio de test cognitivos objetivos de múltiple elección, aplicados antes del inicio de las actividades y después de su conclusión. La frecuencia en las actividades y el perfil epidemiológico de los alumnos fueron correlacionados con los resultados de los test. RESULTADOS: Se analizaron 20 académicos del 3º al 6º año, con un promedio de edad de 22,8 (21-26) años. La frecuencia promedio en las actividades propuestas fue 10,4/13 (80%). El promedio de aciertos en el primer test fue de 8,1/17 (47,6%), siendo que los alumnos del 3º año tenían notas más bajas (p < 0,02) con relación a los demás. En el test post-liga, el promedio de aciertos fue de 11,9/17 (70%), mostrando una mejoría en el desempeño de los alumnos (p < 0,05), sin diferencias entre las notas de los diferentes años del pregrado. Se encontró una fuerte relación entre la frecuencia en las actividades y la mejoría en las notas (r = 0,719; p < 0,001). CONCLUSIONES: Los alumnos que participaron en la liga obtuvieron una mejoría en los test de evaluación de conocimiento, lo que indica que la liga es un instrumento útil de enseñanza y que genera progresos en el aprendizaje en anestesiología. Las actividades desarrolladas en las ligas pueden tener un rol positivo en la formación académica de los graduandos, destacándose, en este artículo, la propia anestesiología

ANESTESIOLOGÍA; Evaluación Educacional; Educación de Pregrado en Medicina

SCIENTIFIC ARTICLE

Alan Saito Ramalho^I; Felipe Duarte Silva^I; Tatiana Barboza Kronemberger^I; Regina Albanese Pose^II; Marcelo Luis Abramides Torres, TSA^III; Maria José Carvalho Carmona, TSA^IV; José Otávio Costa Auler Jr, TSA^V

^IMedical Student, Faculdade de Medicina, Universidade de São Paulo (USP)

^IIMathematician, Center for Development in Medical Education Prof. Eduardo Marcondes, Faculdade de Medicina, USP

^IIIMD, Anesthesiology, Department of Surgery, Faculdade de Medicina, USP

^IVMD, Associate Professor of Anesthesiology, Department of Surgery, Faculdade de Medicina, USP

^VMD, Anesthesiology Professor; Department of Surgery, Faculdade de Medicina, USP

Correspondence to

SUMMARY

BACKGROUND AND OBJECTIVES: Academic leagues have been consolidated as instruments of medical teaching and for the introducing of medical students to practice of specialties, including anesthesiology. As the role of leagues in the development process of competencies and learning of their students is not well known, the learning of members of an anesthesiology academic league was evaluated after participating in its activities for one year.

METHOD: Students of an anesthesiology academic league were followed up from March to December 2010 and evaluated through objective cognitive tests of multiple choice applied before the beginning of activities and after their conclusion. Attendance in activities and epidemiologic profile of students were correlated with the tests results.

RESULTS: Twenty medical students from 3^rd to 6^th year were analyzed, with an average age of 22.8 (21-26) years. The average participation in the proposed activities was 10.4/13 (80%). The average of right answers on the first test was 8.1/17 (47.6%), and 3^rd year students had lower grades (p < 0.02) compared with other students. In the post league test, the average of right answers was 11.9/17 (70%), showing an improvement in performance (p < 0.05), and there was no difference between grades of different years of the medical undergraduation. A strong relationship between participation in activities and improved grades was found (r = 0.719; p < 0.001).

CONCLUSIONS: Students who participated in the league had improvement in knowledge evaluation tests, suggesting that the league is a useful teaching instrument that can provide improved learning of anesthesiology. Participation in activities was connected with improved performance. Activities developed in leagues may have a positive role in students' academic education, more specifically in this article, in anesthesiology.

Keywords: Teaching; Anesthesiology; Educational Measurement; Peer Group; Education, Medical, Undergraduation.

INTRODUCTION

The growth of academic leagues in Brazil and their process of regional and national organization in the last years are highlighted. The Brazilian Association of Medical Academic Leagues was created in 2006¹. In 2010, the Faculdade de Medicina da Universidade de São Paulo registered 62 leagues in activity².

Academic leagues are associations of students from different years of medical school that are oriented and supervised by professors and professionals connected to an educational institution or hospital to carry out educational, assistential, research and science outreach activities in a certain medical specialty or area. Among different teaching techniques in medicine, the leagues have become more important in students' education as extracurricular activity due to its potential to contribute with medical teaching³.

The contact with anesthesiology within the core curriculum of medical undergraduation varies depending on the institution, but it is usually limited, not only in Brazil, but also in other countries^4,5. Nowadays, our institution provides a regular course to 4^th year undergraduate students. There is no internship in anesthesiology during the two years of internship⁶.

The main mission of Anesthesiology, Pain and Intensive Therapy League of Faculdade de Medicina da Universidade de São Paulo (LADTI from Portuguese) is to provide students with the opportunity to get closer to anesthesiology, to reinforce basic knowledge in the specialty and to develop some basic skills.

The evaluation of students' learning is a constitutive element of all teaching-learning process and development of competencies⁷. Competence, in medicine, was defined by Epstein and Hundert⁸ as "customary and wise use of communication, knowledge, technical skills, clinical reasoning, emotions, values and reflexions in daily practice for benefiting both individuals and communities".

To evaluate the evolution of knowledge of member students of an academic league, such as LADTI, it is essential to determine its value as a teaching technique of medical competences; in anesthesiology in this case. The evaluation has a fundamental role so that people and institutions involved in medical education identify and respond to their own learning needs⁹.

Even though more modern evaluation models, like simulations, are more appropriate to evaluate diagnostic reasoning and application of knowledge, it was evidenced that objective tests of multiple choice can meet the five criteria for a proper evaluation^7,10: reliability (data consistency), validity to measure what is proposed, impact in learning and future practice, acceptability and cost¹¹. Multiple choice tests are useful to evaluate knowledge and problem resolution, especially after a course. They also have the merit of evaluating different areas of knowledge in a relatively short time, at low cost and with high reliability, in addition to enabling an easy analysis of the outcomes. Their main limitations are: difficulty to prepare questions and the possiblity of being corretly answered even if there is no real knowledge⁷.

The purpose of this study was to evaluate the learning of members of an anesthesiology academic league after a one-year participation in the actitivies.

METHOD

A prospective population study was carried out between March and December 2010 at LADTI. All students who participated in LADTI's activities in 2010 were included. Medical students who stopped participating in LADTI during the year or refused to participate in the study were excluded from the analysis.

The evaluation instruments used in the study were two objective cognitive tests of multiple choice, each one consisting of 17 questions based on three levels of complexity according to Palmer et al.¹⁰ (Table I). The questions addressed basic anesthesiology topics mentioned during LADTI's activities and were classified according to the level of difficulty in: easy (5 questions), intermediate (6 questions) and difficult (6 questions).

A test was given to the LADTI students on the first day of activities (pre LADTI test). During the nine months of the study, LADTI provided their students with monthly lectures, practical activities instructed by anesthesiology residents, simulation of airway handling and control of oncologic pain, participation in 57^th Brazilian Congress of Anesthesiology and preparation of a review about basic anesthesiology subjects supervised by a professor or an assistant physician. The students' participation data were collected in those activities and 13 attendances were the maximum possible in the considered period. In the last day of activities a new test (post LADTI test) was applied, which kept the same distribution of characteristics of evaluated skills and number of questions by level of difficulty as in the pre LADTI test.

The outcomes of pre and post LADTI tests, the participation in the suggested activities, as well as gender, age and year of course data were organized in Microsoft Excel^® (Redmond, WA, USA) spreadsheets.

Based on the obtained data, the following hypothesis were tested: (a) was there an improvement in academic performance (higher grades)?; (b) is there a correlation between performance in post LADTI test and participationin the activities?; (c) is there a difference in students' performance according to the year coursed (3^rd, 4^th, 5^th, 6^th years)?

A descriptive analysis and verification of formulated hypotheses were conducted with non-parametric tests considering a 95% reliability interval, with SPSS v.15 for Windows and Minitab v.14 for Windows softwares.

RESULTS

There were 25 students in LADTI at the beginning of study.

There was a follow-up loss of 4 students, who stopped participating in LADTI. One student refused to participate the study.

Student's profile and attendance to activities (descriptive population study)

Study's population was formed by 20 students, aged between 21-26 years, enrolled on 3^rd, 4^th, 5^th and 6^th years of the medical course of the institution (Table II). The average age of participants at the end of LADTI was 22.8 years (21-26), with median and mode of 23.0 years and standard deviation of 1.2 years.

Right answers in pre and post ladti tests and participation in activities (Table III)

The average of right answers in the first test (pre LADTI) was 8.1/17 (47.6%), and in the second test (post LADTI) was 11.9/17 (70%). The mode of right answers went from 9/17 in the pre LADTI test to 11/17 in the post LADTI, which was the same value of maximum grade obtained in the pre LADTI evaluation. The average attendance of students in the activities was 10.4/13 (80%), ranging from 3/13 (23.1%) to 13/13 (100%). Median and mode of participation were 11.0 with standard deviation of 2.7 (Table III). A box graph was plotted to evaluate the relationship between participation in activities and the number of right answers in the tests taken¹² (Figure 1).

For the 3^rd year group, the average of right answers in the pre LADTI test was 6/17 (35.3%) and in the post LADTI test was 11.4/17 (67.1%). Among 4^th year students, these percentages of right answers in the pre and post LADTI tests were 8.8/17 (51.8%) and 12.5 (73.5%), respectively. In the 5^th and ₆th year groups (internship), average grades were 8.5/17 (50.0%) in the pre and 11/17 (64.7%) in the post LADTI tests (Figure 2).

The Kolmogorov-Smirnov normality test was conducted for inference analysis with the purpose of evaluating the distribution of number of right answers in pre and post LADTI tests with regard to participation in the suggested activities. That indicated the absence of normal distribution to all three variables (p-values of 0.929; 0.804; 0.192, respectively). Thus, linear association between the participation in the suggested activities and the number of right answers in the post LADTI test was calculated by Pearson's test, which showed a statistically significant positive correlation between variables (r = 0.719; p < 0.001) (Figure 3). That showed a strong relationship between participation in the activities and improvement of grades between tests.

As normality assumptions were not met, non-parametric analysis techniques (Mann-Whitney and Kruskal-Wallis tests) were applied to locate the differences between groups, according to the hypotheses suggested by this study. A significant difference between the average of right answers in pre LADTI test (8.1/17) and post LADTI (11.9/17) was found (p < 0.05).

The Kruskal-Wallis test was used in order to compare the difference between the performance in tests and the participation in the suggested activities according to the course year and the age of the students in this study. It was observed that the performance of 3^rdyear students was significantly inferior in the pre LADTI tests when compared with grades of other years (p < 0.02). There was no difference between right answers of 4^th, 5^th and 6^th year (p > 0.05). On the other hand, in the post LADTI test all years had similar scores, namely, there was no statistical difference between grades of 3^rd, 4^th, 5^th and 6^th year in the last test.

When 3^rd, 4^th, 5^th and 6^th years were compared in relation to participation in the activities suggested by the league and to the age, no statistically significat difference was found (p > 0.05).

The Kruskal-Wallis test was conducted to check if there was any difference in students' performance in the post LADTI test according to the presence in activities suggested by LADTI and to the number of right answers in the pre LADTI test. Difference was found between students who participated in the league activities and their performance in the post LADTI test (p < 0.036). There was no association between right answers in the pre and post LADTI tests, indicating that the first grade obtained in the first test did not influence the number of right answers in the last one (p > 0.05). There was no statistically significant difference (p > 0.05) between grades in the pre LADTI and post LADTI tests or between attendances according to gender.

DISCUSSION

In this study, the main observed outcome was the increased performance of students in cognitive tests applied after participating in LADTI's activities. This growth showed linear correlation with participation in the suggested activities.

Together with the increased grade average from pre to post LADTI test, the permanence of variability between minimum and maximum grades in both leads to the conclusion that there was a relatively uniform improvement in students' performance. The maximum number of right answers in the pre LADTI test is the same mode value in the post LADTI test (11/17), and the average number of right answers on the first test (8.1/17) is equivalent to the minimum value on the second test (8/17).

The limited size of the analyzed sample was determined by the restricted number of LADTI participants. It did not allow a factorial analysis of the data obtained and has hindered more statistical inferences, since it is a group of volunteers and not a randomized sample. The results apply to the own group and cannot be generalized yet. Therefore, more studies with larger sample size are necessary.

However, this is a pioneer study for objectively evaluating the students' learning in an academic league. Notwithstanding the great number of leagues and their growth, few studies on the subject describe activities developed by the leagues^13-15, evaluate students' satisfaction, describe the reasons for the student's interest in the leagues, or even their scientific production^13,16. Improvement in learning among students participating in academic leagues had not yet been demonstrated, as highlighted in this article.

This study shows that, in addition to the higher number of right answers in the post LADTI test in comparison to the pre LADTI test, there was a strong positive association between performance improvement and participation in the suggested activities. This study indicates that students with higher attendance to the activities tend to have increased growth in performance in relation to the grade of the first test and when compared to the population. On the other hand, the applied tests detected deficiency in the personal performance of students who had smaller participation.

A limitation of this study was the absence of a group of students with similar characteristics not undergoing LADTI's activities as a control group to study if the evolution in the test performance could be explained solely by the participation in the league or if there are other factors impacting in the increased performance.

Comparing the different years of medical undergraduation, the 3^rd year had worse grades when compared with other years on the first test. This was expected as in our institution, students only begin the formal contact with the practice of surgical specialties in the clinical cycle of the course (3^rd and 4^th year)⁶.

The highest increase of grades was observed in the 3^rdyear and no difference was found in the post LADTI grades among different years of undergraduation. That means that for LADTI students who had no contact with anesthesiology yet, the league impact is even greater than for those who had already studied this subject at university. However, at the end of LADTI, all groups showed improvement in performance. In other words, the activities suggested by league fostered the development of the knowledge of anesthesiology among students regardless of the year studied. Again, the small sample limits generalization of these findings.

The learning and improved cognitive performance evidenced in this study reinforce the importance of academic leagues in the medical students' education and can mean competence gain in the specialty^3,13.

In addition to this aspect, as students are dealing with a subject of their interest in an environment built and managed by themselves and are under the guidance of a professor, it is possible to acquire knowledge and experience, to develop clinical-scientific reasoning and to foster team work. Such skills may be transferred to different areas in the practice of medicine ¹⁶; for instance, anesthesiology training centers.

Characteristics like the constant search for learning, communication and interpersonal relationships that may be developed among students in an academic league environment are within the ones identified by Smith et al. ¹⁷ as important for the excellence in the anesthetic practice.

Although there is criticism about the fact that academic leagues may induce early specialization of undergraduate students, distorting the moment of their general education^3,16, academic leagues are useful instruments to bring medical students closer to the reality of specialties, including anesthesiology. They also stimulate the interest in teaching, research and science outreach fundaments^13,18.

Observing medical schools in Brazil and abroad, there is a great variability in the way anesthesiology is taught at undergraduation courses. Its space is usually limited when compared with other subjects and is sometimes restricted to sparse training^4,5. In teaching hospitals where this specialty is well-structured, anesthesiologists deal not only with intraoperative anesthetic handling, but also with perioperative medicine, pain medicine and critical patient care^5,19.

Undergraduation is the ideal moment to expose students to the challenges in anesthesiology and to identify who is interested in persuing a career in the specialty⁵. The contact with anesthesiology is important not only due to the fundamental knowledge of the subject to the physician (regardless of his future practice); it is also clear the importance of the contact with professionals and professors when students are to choose their specialty. Positive models in anesthesiology teaching are usually identified among those who choose this specialty^20,21. The little contact in undergraduation may estrange students from certain specialties²².

Academic leagues may contribute to this choice process and stand out as a feasible alternative by providing contact with anesthesiology when the space in the basic curriculum of medical undergraduation is limited. They are also a way of promoting acquisition of knowledge in the specialty as demonstrated in this study, thus contributing to the academic education of students.

Subsequent, controlled studies and with a greater number of students are required to increase evaluation consistency of the impact of leagues on students' learning, perhaps leading to the inclusion of this teaching process as a suggestion for teaching centers of the Brazilian Society of Anesthesiology.

CONCLUSION

Students who participated in the activities of Anesthesiology, Pain and Intensive Therapy League had an improved outcome in applied knowledge evaluation tests (increase in the average of right answers from 8.1 to 11.9/17). Participation in the suggested activities had a strong linear association with improved performance.

Studies with more students are necessary to confirm the positive impact of academic leagues on learning.

REFERENCES

• ¹
  Associação Brasileira de Ligas Acadêmicas de Medicina. Disponível em: http://www.ablam.org.br/institucional.html (Acessado em 20/03/2011).
  » link
• 2. Departamento Científico da Faculdade de Medicina da Universidade de São Paulo - Cadastro das Ligas Acadêmicas da Faculdade de Medicina da Universidade de São Paulo 2010. Disponível em http:// www.dcfmusp.com.br/ligas.htm (Acessado em 23/03/2011).
• 3. Pego-Fernandes P, Mariani A - Medical teaching beyond graduation: undergraduate study groups. Sao Paulo Med J, 2010;128:257-258.
• 4. Cheung V, Critchley LA, Hazlett C et al. - A survey of undergraduate teaching in anaesthesia. Anaesthesia, 1999;54:4-12.
• 5. Longnecker DE - Education in anesthesiology for the twenty-first century. J Anesth, 2002;16:65-69.
• 6. Faculdade de Medicina da Universidade de São Paulo - Grade Curricular do Curso de Medicina. Disponível em http://sistemas2.usp.br/jupiterweb/listarGradeCurricular?codcg=5&codcur=5042&codhab=0 &tipo=N (Acessado em 22/03/2011).
• 7. Epstein RM - Assessment in medical education. N Engl J Med, 2007;356:387-396.
• 8. Epstein RM, Hundert EM - Defining and assessing professional competence. JAMA, 2002;287:226-235.
• 9. Fraser SW, Greenhalgh T - Coping with complexity: educating for capability. BMJ, 2001;323:799-803.
• 10. Palmer EJ, Devitt PG - Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? Research paper. BMC Med Educ, 2007;7:49.
• 11. Van Der Vleuten C - The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ, 1996;1:41-67.
• 12. Bussab WO, Morettin PA - Estatística Básica, 5Ş Ed, São Paulo, Editora Saraiva; 2006.
• 13. Fernandes FG, Hortencio LOS, Unterpertinger FV et al. - Cardiothoracic Surgery League from University of São Paulo Medical School: twelve years in medical education experience. Rev Bras Cir Cardiovasc, 2010;25:552-558.
• 14. Kara José AC, Passos LB, Kara José FC et al. - Ensino extracurricular em Oftalmologia: grupos de estudos / ligas de alunos de graduação. Rev Bras Educ Med, 2007;31:166-172.
• 15. Neves F, Vieira P, Cravo E et al. - Inquérito nacional sobre as Ligas Acadêmicas de Medicina Intensiva. Rev Bras Ter Intensiva, 2008;20:43-48.
• 16. Monteiro LLF, Cunha MS, Oliveira WL et al. - Ligas acadêmicas: o que há de positivo? Experiência de implantação da Liga Baiana de Cirurgia Plástica. Rev Bras Cir Plást, 2008;23:158-161.
• 17. Smith AF, Glavin R, Greaves JD - Defining excellence in anaesthesia: the role of personal qualities and practice environment. Br J Anaesth, 2011;106:38-43.
• 18. Torres A, Oliveira G, Yamamoto F et al. - Academic Leagues and medical education: contributions and challenges. Interface - Comunic Saúde Educ, 2008;12:713-720.
• 19. Rohan D, Ahern S, Walsh K - Defining an anaesthetic curriculum for medical undergraduates. A Delphi study. Med Teach, 2009;31:e1-5.
• 20. Akinyemi OO, Soyannwo AO - The choice of anaesthesia as a career by undergraduates in a developing country. Anaesthesia, 1980;35:712-715.
• 21. Watts RW, Marley J, Worley P - Undergraduate education in anaesthesia: the influence of role models on skills learnt and career choice. Anaesth Intensive Care, 1998;26:201-203.
• 22. Ek EW, Ek ET, Mackay SD - Undergraduate experience of surgical teaching and its influence and its influence on career choice. ANZ J Surg, 2005;75:713-718.

Publication Dates

• Publication in this collection
  02 Feb 2012
• Date of issue
  Feb 2012

History

• Received
  28 Mar 2011
• Accepted
  19 May 2011