Curriculum integration in the formative assessment of distance continuing medical education: the use of integrative activities

Abstract

Introduction:

The article describes the development and implementation of integrative activities as a formative evaluation of a distance-learning course of preceptorship specialization in Family and Community Medicine and discusses how the use of this activity is capable of integrating and assessing different knowledges, practices and performance of students in this modality of education. The course curriculum involves units with modules in three areas: family and community medicine, preceptorship and clinical practice, allowing the approach of essential topics for the practice of the specialty preceptor, which go beyond the study of pedagogical aspects.

Experience report:

The student was encouraged to carry out a textual production based on contextualized problems in the Primary Health Care preceptorship, guided by the tutor, aiming to critically reflect on the contents of each teaching unit. Illustrated situations were used in the activity statement, shown in comic strip format, which qualified the interpretation of the context addressed in the activity and promoted greater approximation with the reality experienced by the student. The contextualization of the activity and the exercise of putting oneself in the preceptor’s place were considered positive points by the tutors.

Discussion:

The student-tutor interaction is a fundamental part of the activity, as it allows the construction of knowledge from an interactionist perspective of education, which is the basis of the pedagogical proposal of the course. The activity allowed the identification of gaps and potentialities in the students’ learning and the implementation of particular pedagogical interventions.

Conclusions:

This type of activity ensures important dimensions of evaluative work in long-term courses: the procedural and interdisciplinary features. The integrative activity model can be used in distance courses with different disciplines, aiming at integrating them, providing a more in-depth approach and bringing the educational contents closer to the student’s reality.

Keywords:
Continuing medical education; Distance Education; formative feedback; Curriculum

Resumo:

Introdução:

O artigo descreve o desenvolvimento e a implementação de atividades integradoras como avaliação formativa de um curso a distância de especialização de preceptoria em medicina de família e comunidade (MFC) e discute como a utilização dessa atividade é capaz de integrar e avaliar diferentes conhecimentos, práticas e o desempenho dos alunos nessa modalidade de ensino. O currículo do curso envolve unidades com módulos em três áreas: MFC, preceptoria e clínica, permitindo que se abordem os temas essenciais para a prática do preceptor da especialidade, que vão além do estudo dos aspectos pedagógicos.

Relato da experiência:

O aluno era instigado a realizar uma produção textual baseada em problemas contextualizados na preceptoria na atenção primária à saúde, orientada pelo tutor, de forma a refletir criticamente sobre os conteúdos de cada unidade de ensino. Foram utilizadas situações ilustradas no enunciado da atividade, na forma de história em quadrinhos, o que qualificava a interpretação do contexto abordado na atividade e promovia maior aproximação com a realidade vivenciada pelo aluno. A contextualização da atividade e o exercício de se colocar no lugar do preceptor foram considerados pontos positivos pelos tutores.

Discussão:

A interação aluno-tutor é parte fundamental da atividade, pois permite a construção do conhecimento a partir de uma perspectiva interacionista da educação, que é a base da proposta pedagógica do curso. A atividade permitiu a identificação de lacunas e potencialidades na aprendizagem dos alunos e a realização de intervenções pedagógicas singulares.

Conclusão:

Esse tipo de atividade garante dimensões importantes do trabalho avaliativo em cursos de longa duração: o caráter processual e interdisciplinar. O modelo de atividade integradora pode ser utilizado em cursos a distância com diferentes disciplinas, com o objetivo de integrá-las, proporcionando uma abordagem mais aprofundada e aproximando os conteúdos educacionais da realidade do aluno.

Palavras-chave:
Educação Médica Continuada; Educação a Distância; Feedback Formativo; Currículo

INTRODUCTION

With the increase in residency vacancies in Family and Community Medicine (FCM) in Brazil, some strategies were developed to expand the training of preceptors in this specialty. One of these strategies was the creation of a tutored, nationwide distance-learning specialization course in preceptorship in FCM, with a workload of 550 hours distributed throughout two years. At the beginning of its third offer, the course had 709 enrolled students, reaching 70.94% approval at the end. The course was implemented by Hospital Moinhos de Vento (HMV), in partnership with the Federal University of Health Sciences of Porto Alegre (UFCSPA). The initiative was supported by the Brazilian Society of Family and Community Medicine (SBMFC) and was part of the National Plan for the Training of Preceptors of the education axis of the Mais Médicos Pelo Brasil Program.

The course curriculum involved eight teaching units, each with modules in three curricular axes: FCM; preceptorship; and clinical practice, through the study of complex cases. The organization of the curriculum allowed addressing essential topics for the specialty preceptor’s practice, which go beyond the study of pedagogical aspects and include technical and ethical training. This organization brought with it the need to develop an assessment tool that would integrate the covered topics, while being related to the practice and reality of each student in the context of a distance course. Curricular integration has been studied in the context of undergraduate teaching, in the in-person modality, focused on vertical and horizontal integration, involving basic scientific disciplines and clinical ones¹; however, there is a lack of data on experiences with curricular integration in the assessment of learning in the context of distance education (DE), in the present article also known as ‘e-learning’.

DE is the result of the joint work of different areas of knowledge, such as pedagogy, information technology and communication, and, in current times, it has been even more present in educational systems, including medical training. According to the expanded concept of Sangrà et al²:

“E-learning is an approach to teaching and learning, representing all or part of the educational model applied that is based on the use of electronic media and devices as tools for improving access to training, communication and interaction and that facilitates the adoption of new ways of understanding and developing learning”.

The evaluation has a vital effect on student learning and has a number of purposes, including providing feedback in the follow-up of the students’ learning path and improving student motivation and commitment³. Among the challenges of the distance education course was the choice of evaluation methods that could have pedagogical potential, by problematizing the student’s routine situations as a family physician and as a preceptor. The problematization brings the possibility of a continuous observation of the students in their own spaces and times; such observation aims at updating and complementing the student’s knowledge construction process⁴. The formative assessment is carried out during the teaching and learning process, being an integral part of this process, both in conventional and distance education contexts^5),(6. This type of assessment also manifests a change of paradigm in pedagogical strategies and in the use of technology for the effective provision of teaching in distance education and is based on providing the students the feedbacks to improve performance in subsequent tasks⁶.

The assessment planning in DE, like any assessment process, includes careful consideration of the appropriate types of assessment and how they correlate with learning objectives and the curriculum as a whole, also considering that they will be carried out in electronic format⁷. The most common assessment strategy in DE courses is based on knowledge, for instance, with the use of multiple-choice questions that can provide immediate feedback, which is fundamental, considering the formative aspect of the activity. However, from a cognitive point of view, DE assessment can offer a wider range of questions and interaction possibilities than the assessment based on multiple-choice questions⁷. Assessment in the virtual environment can include other types of evaluation, such as based on performance (e.g., using patient cases), on practice (e.g., using portfolios) or on behavior/attitude (e.g., based on contributions to discussion forums)⁷.

In this article, we describe how integrative activities (IAs) were developed and implemented as formative assessment in the distance education course of specialization in FCM preceptorship, and on how the use of this activity is capable of integrating and evaluating different knowledges, practices and the construction of the students’ learning in this learning modality.

EXPERIENCE REPORT

The IA have been developed since the first edition of the course, which started in 2016, and has since undergone a process of pedagogical improvement. Its objective has remained the same: to record reflections and learning, integrating the studied theories with the practices experienced during the FCM and preceptorship actions. In order to further qualify the educational actions, the course was evaluated by the students, tutors and the pedagogical team at each edition, which resulted in structural and methodological modifications. The IA has kept up with these changes, so in this report we will describe the structure of how it was applied in its third edition.

For each teaching unit, the team consisting of two professionals trained in pedagogy and a physician who was a specialist in FCM planned the IA, prepared a guide for the tutors and held a web conference with them. The proposal of the IA was made available to the 28 tutors before the web conference. The meeting was divided into two moments: one, which evaluated the integrative activity of the previous unit, to analyze how the tutors perceived the students’ performance and whether the activity reached the proposed objective; and another, aimed at the next integrative activity, with a more formative characteristic, to solve doubts, analyze clarity, degree of difficulty and evaluation criteria for the construction of feedbacks, with the objective of evaluating the proposed activity, aligning the feedbacks and improving tutor mediation. This process was also followed by the course coordinators, including the general, pedagogical and tutoring coordination.

The students, at the end of each teaching unit, were encouraged to carry out a textual production, based on the studied literature, which contemplated an orientation presented in topics, based on the contents of the teaching unit, indicating the step-by-step process to be followed to create the activity. The pedagogical proposal of the activity should be based on problems contextualized in the Primary Health Care preceptorship and provide the student with a critical reflection about the studied contents. Table 1 shows, as an example, the topics studied in teaching unit 6, whose integrative activity will be concisely shown in the Figure 1 model.

Table 1
Topics studied on in the three curricular axes in teaching unit 6.

Unit 6	FCM axis	Preceptorship axis	Clinical practice axis (complex cases)	Integrative activity
Family Approach	Tools for the family approach	Teaching and learning process II - focusing on the family approach	Maria’s case: - Ischemic heart disease - Dizziness and vertigo - Osteoarthrosis - Quaternary prevention	Based on the clinical case of Maria, the student should indicate family approach tools for case management, to analyze a context of preceptorship performance, carry out the teaching planning for the resident, and indicate changes in their planning due to the pandemic scenario
			Case of the Line of Immigrants: - Adult immunizations - Shoulder pain - Erysipelas and cellulitis - Vulnerability

• Source: adapted from the Preceptorship Specialization Course in Family and Community Medicine.

Figure 1
Model of integrative activity.

Illustrated situations were used in the activity’s statement, in the comic book (CB) format, which qualified the interpretation of the context addressed in the activity and promoted greater approximation with the reality experienced by the student. The comics were created with the same characters from a fictional residency program in FCM, which contextualized different teaching and learning situations present in the course modules.

According to Sandars and Patel⁸, e-learning experiences can be very useful for medical educators in other contexts, but it is essential that the reports contain information that allows these experiences to be reproduced. Using structured templates to show the intervention can ensure that the report contains enough information so that this intervention can be used by other medical educators. Figure 1 shows a model of the general structure of the activity, together with the main points of an activity developed to integrate a complex case of approaching an elderly woman with multiple comorbidities and fragile family relationships, with the topic of the family approach and its teaching in preceptorship, in the context of the beginning of the COVID-19 pandemic in Brazil - a situation that was effectively being experienced by the students.

The orientation asked the student to put themselves in the place of the residency preceptor and describe a decision-making in the presented situation. In addition to analyzing the most appropriate family approach strategies for the complex case and building a plan for teaching skills in family approach, the student should reflect on necessary adaptations in care and teaching activities as a preceptor, in a situation of sudden change in context due to the COVID-19 pandemic.

The student had four weeks to develop the preliminary version of the activity and send it to the tutor, who provided feedback with questions related to the content, prompting the articulation of knowledge from the three curricular axes, advising on ways to qualify the activity and indicating whether the expected learning objectives have been achieved. The student then had the opportunity to make changes and resubmit the activity in its final version within a week. This version was once again analyzed by the tutor, who returned with new feedback within a week, briefly indicating whether the student reached the learning objectives this time. This collaborative development provided a guided and in-depth approach to the construction of knowledge about the integrated topics.

The next step consisted in the creation of the integrative activities in a reflective portfolio, which was evaluated as the course’s final paper (TCC - Trabalho de Conclusão de Curso). The student should integrate the discussions triggered by a final conclusive text, reflecting on their own learning process. As with each IA, the portfolio was also delivered in a preliminary version and, in this last opportunity for interaction, the tutor provided guidance on all the IAs, in addition to the introductory and conclusion chapters and aspects of TCC formatting. Table 2 shows a summarized version of the process of IA creation and portfolio delivery at the end of the course.

Table 2
Stages of IA creation and evaluation.

Phases	Description	Deadline	In charge
Creation of the activity orientation	- Pedagogical aspects	1 week	Pedagogues
	- Aspects of the content		FCM specialist physician
Web conference with tutors	- Activity qualification		Course coordination (general, tutoring, and pedagogical)
	- Before the start of the teaching unit		Tutors
Preliminary IA delivery	- First version of the activity	4 weeks	Student
Formative feedback	- Questions and suggestions aiming to qualify the activity	2 weeks	Tutor
	- Encouraging content articulation
Final IA delivery	- Second version of the activity	1 week	Student
Final feedback	- Achieved the learning objectives, or	1 week	Tutor
	- Need to adjust
Portfolio - preliminary version	- TCC with all IAs, introduction and conclusion		Student
	- End of the course
Portfolio feedback	- Orientation for all IAs	4 weeks	Tutor
	- Orientation on formatting and introduction and conclusion chapters of the TCC
Portfolio - final version	- Evaluative	10 days	Student

• Source: the authors.

The virtual learning environments have several interactive and evaluation tools that can be customized for the implementation of the IA (especially the discussion forum, for providing collaborative learning and asynchronous participation); however, the choice of the file sending tool was made according to the intention of carrying out an approach focused on the student-tutor dual relationship and the proposal to develop activities in more targeted stages, since they would constitute the student’s portfolio.

During the meetings held by web conference with the tutors, their reports were registered by recording the meetings and by writing minutes. We found through these records that they perceived a gradual advance in the quality of the activities delivered by the students throughout the course, especially with the improvement of learning about the articulation of the different stages of teaching planning creation. This perception is important, in the sense that the activity contributed to the students’ development of knowledge about the fundamentals of planning in education and teaching skills in medical preceptorship. The contextualization and the exercise of putting oneself in the preceptor’s place were considered positive points by the tutors, making the activity better understood by the students and allowing the acknowledgement that pedagogical planning cannot be based on common sense knowledge, but on the competences to be developed. As a negative point, it is possible that, in some cases, the collaborative process between tutor and student was insufficient and did not result in actual changes in the final version of the activity.

DISCUSSION

Health training, at different levels and modalities of education, including distance learning, has aroused in some institutions the need to discuss the creation and implementation of innovative curricula. From this perspective of changing the curricular paradigm, we can find initiatives that move towards the integration of contents, which also includes the integration of the proposed activities. According to Alonso apud Barboza and Felício⁹:

“This multidisciplinary characteristic of the curriculum becomes viable insofar as it is possible to develop networks of content from different areas of knowledge and objectives in terms of transversal competencies and attitudes to be worked on, in a globalizing and integrating perspective of knowledge and experience (...) The integrative activities that constitute it privilege investigative, reflective, collaborative and active methodologies, and defend a meaningful and constructive perspective of knowledge and experience, so that students can find meaning and relevance in what they learn (...)”.

In the experience of this course, considering its general objective of qualifying medical residents so they can work as preceptors in Primary Health Care and FCM, the way the topics are mobilized in the IA allows the student to put themselves in the preceptor’s place, to carry out the planning of educational programs focused on competency-based learning and reflect on how to act to solve issues in the specific topics of FCM performance and clinical situations. In the IA shown as an example, the students first analyzed the skills and strategies of family approach to be worked by the preceptor with the fictional resident of the comic strip, and created the planning of the pedagogical action, with the teaching and evaluation methods to be used both in the office setting and home setting; in a second moment, when reflecting on possible solutions for teaching in the workplace during the pandemic, the students brought up ideas that had not been foreseen during the creation of the activity. One example is that many have transformed the in-person teaching methods into online activities, such as performing family interviews, a tool for the family approach in FCM, with the presence of the preceptor by videoconference, with subsequent feedback through the use of the same technological resource. According to Amarilla Filho⁴:

“Problematization cannot be thought of as a temporary or sporadic strategy, but it must be considered in Distance Education as a fundamental element in the didactic process, in which it is possible to observe and analyze the paths that students take and what they can do with the knowledge when they are out of the virtual environments”.

In its pedagogical conception in the experience of this course, we can highlight two main characteristics of IA: curricular integration and collaborative learning through formative feedbacks. It is known that the formative assessment promotes the development of complex cognitive processes, such as self-regulation, and that formative feedback has the potential to stimulate student commitment, increase their enthusiasm for learning and lead to better academic performance¹⁰. The development of the self-regulatory skill helps the student’s learning and academic performance and also their professional practice, as some principles that guide the self-regulatory process, such as monitoring, control and reflection, can be applied to several areas of their life^11)-(13. The main objective of feedback is to provide tools to improve student performance, identifying their weaknesses and helping them to create alternatives to overcome them. Therefore, the feedback can be understood as one of the central elements of formative assessment and presupposes that evaluators have mastery of the knowledge to be taught, empathy with the student’s learning process and capacity to evaluate¹⁴. Regarding the frequency, good practices regarding formative assessment recommend that feedback be offered regularly, in order to provide opportunities for students to reflect on and review their practices throughout the educational experience¹⁴.

In this sense, it is noteworthy a brief discussion about the interaction in DE, which interferes with the student’s learning and satisfaction with the course^15)-(17. The Ministry of Education establishes quality benchmarks for the distance modality that reinforce the need for interaction between tutors and students, and this interaction must be privileged and guaranteed¹⁸. Although the interaction between the actors in the virtual learning environment is different from in-person teaching, there are potentially greater opportunities for learning in DE, as it allows extending the time normally used in face-to-face classes¹⁹. However, considering that communication mediated by technology is not inherently interactive and depends on several factors, including the nature and punctuality of the interaction¹⁵, the qualification of this interaction between tutor and student is essential. IA emerges as an opportunity for qualified interaction, in the sense of providing feedback in a timely and previously defined period, individualized manner and with appropriate guidelines for the development of competencies for the real context of the student’s performance.

It is important to note that the course was tutored by FCM specialists with experience in preceptorship, who participated in a training workshop to carry out the tutoring at distance and who were followed in systematic meetings. They interacted with the same group of students throughout the course, through activity feedback, messages and forums. The students were oriented in the construction of integrative activities, and this interaction was a fundamental part of the activity, as it allowed the construction of knowledge from an interactionist perspective of education, which values ​​dialogue and the student’s previous knowledge and represents the basis of the course pedagogical proposal. From this perspective, the students’ needs must be identified and they must be supported and exposed to different methodologies that include problem solving, and tutors, in turn, must interact with students and offer advice to them²⁰. Formative assessment and feedback provide students with an opportunity to better understand the gap between current and desired performance²⁰.

Throughout the course, an important heterogeneity was verified in the students’ levels of knowledge regarding FCM and preceptorship. In this sense, IA is characterized as an effective training methodology, as it allows an integrated approach to curricular content, the identification by the tutors of gaps and potentials in student learning and the implementation of particular pedagogical interventions through formative feedback. Moreover, it contributes to the experience of a collaborative, reflective, procedural and functional process, which might qualify the teaching and learning experiences of the future preceptor.

CONCLUSIONS

Distance education in continuing medical education has been increasingly used in Brazil and in the world, which highlights the need for educational institutions to be prepared for the use of new teaching and evaluation methods in this modality, or even for a reformulation of these methods. In this context of innovations in medical education, it is essential to discuss methods of assessment that deal with the daily life of a wide range of students and that manage to integrate the different contents of a course.

As it is developed in each teaching unit during the course learning process, this type of activity guarantees important dimensions of the evaluative work in long-term courses: the procedural and interdisciplinary features. The integrative activity model can be used for in-person, hybrid or distance courses with different subjects, providing a more in-depth approach and bringing the educational contents closer to the student’s reality.

ACKNOWLEDGMENTS

The authors are grateful to Denise Macedo de Miranda, José Fialho de Oliveira Junior and Leonardo Slaviero Martins for their contribution in the preparation of this article.

REFERENCES

• ¹
  Brueckner JK, Gould DJ. Health science faculty members’ perceptions of curricular integration: insights and obstacles. J Int Assoc Med Sci Educ. 2006;16(1):31-4.
• ²
  Sangrà A, Vlachopoulos D, Cabrera N. Building an inclusive definition of e-learning: an approach to the conceptual framework. Int Rev Res Open Distance Learn. 2012;13(2):145-59.
• ³
  Marriott P, Lau A. The use of on-line summative assessment in an undergraduate financial accounting course. J Account Educ. 2008;26(2):73-90.
• ⁴
  Amarilla Filho P. Educação a distância: uma abordagem metodológica e didática a partir dos ambientes virtuais. Educ Rev. 2011;27(2):41-72.
• ⁵
  Patronis M. Summative and formative online assessments. Proceedings of 7th International Conference. The Future of Education; 2017 Jun 8-9; Florence, Italy. Florence: Pixel International Conferences; 2017.
• ⁶
  Ogange B, Agak J, Okelo K, Kiprotich P. Student perceptions of the effectiveness of formative assessment in an online learning environment. Open Praxis. 2018;10(1):29-39.
• ⁷
  Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education Part 1: learning, teaching and assessment. Med Teach. 2008;30(5):455-73.
• ⁸
  Sandars J, Patel R. The challenge of online learning for medical education during the Covid-19 pandemic. Int J Med Educ. 2020;11:169-70.
• ⁹
  Barboza JS, Felício, HMS. Integração curricular a partir da análise de uma disciplina de um curso de medicina. Rev Bras Educ Med. 2018;42(3):27-35.
• ¹⁰
  McLaughlin T, Yan Z. Diverse delivery methods and strong psychological benefits: a review of online formative assessment. J Comput Assist Learn. 2017;33(6):562-74.
• ¹¹
  Bembenutty H, White MC. Academic performance and satisfaction with homework completion among college students. Learn Individ Differ. 2013;24:83-8.
• ¹²
  Schunk DH, Zimmerman BJ. Motivation and self-regulated learning: theory, research, and applications. New York: Lawrence Erlbaum Associates; 2008.
• ¹³
  Wolters CA. Regulation of motivation: contextual and social aspects. Teach Coll Rec. 2011;113(2):265-83.
• ¹⁴
  Borges MC, Miranda CH, Santana RC, Bollela VR. Avaliação formativa e feedback como ferramenta de aprendizado na formação de profissionais da saúde. Medicina (Ribeirão Preto). 2014;47(3):324-31.
• ¹⁵
  Swan K. Virtual interaction: design factors affecting student satisfaction and perceived learning in asynchronous online courses. Distance Educ. 2001;22(2):306-31.
• ¹⁶
  Roblyer MD, Wiencke WR. Exploring the interaction equation: validating a rubric to assess and encourage interaction in distance courses. J Async Learn Network. 2004;8(4):25-37.
• ¹⁷
  Mattar J. Tutoria e interação em educação à distância. São Paulo: Cengage Learning; 2012.
• ¹⁸
  Ministério da Educação. Referenciais de qualidade para a educação superior a distância. Brasília: Ministério da Educação, Secretaria de Educação a Distância; 2007.
• ¹⁹
  García Cabrero B, Márquez L, Bustos A, Miranda G, Mabel S. Análisis de los patrones de interacción y construcción del conocimiento en ambientes de aprendizaje en línea: una estrategia metodológica. Rev Electron Investig Educ. 2008;10(1):1-19
• ²⁰
  Amoako I. Formative assessment practices among distance education tutors in Ghana. Afr J Teach Educ. 2018;7(3):22-36.

• 2
  Evaluated by double blind review process.

• SOURCES OF FUNDING

  The data presented herein were obtained in partnership with the Ministry of Health of Brazil, through the Support Program for Institutional Development of the Unified Health System (PROADI-SUS, Programa de Apoio ao Desenvolvimento Institucional do Sistema Único de Saúde).

Edited by

Chief Editor: Rosiane Viana Zuza Diniz. Associate Editor: Danilo Borges Paulino.

Publication Dates

• Publication in this collection
  01 Apr 2022
• Date of issue
  2022

History

• Received
  02 Dec 2021
• Accepted
  11 Jan 2022