Breaking barriers: a competency-based framework for promoting the integration of the pediatrician's education

Abstracts

OBJETIVO: Descrever o processo de revisão e de integração curricular de um programa de pediatria por meio da criação de uma matriz de competências referenciada nas Diretrizes Curriculares Nacionais. MÉTODOS: Estudo quali-quantitativo de intervenção que avaliou a percepção de estudantes e docentes em relação ao currículo existente (grupos focais e entrevistas semiestruturadas). Discutiram-se os resultados em oficinas de desenvolvimento docente, o que propôs uma matriz baseada em competências para todo o programa de pediatria do 3º ao 6º ano. O novo currículo foi aprovado, implementado e reavaliado após 6 meses. RESULTADOS: Doze estudantes (12%) do 3º ao 6º ano participaram dos grupos focais, e 11 dos 14 professores (78,5%) responderam ao questionário. A maioria referiu falta de integração entre as disciplinas, desconhecimento dos objetivos de aprendizagem dos estágios, poucas oportunidades de práticas e avaliação predominantemente teórica. Nas oficinas de capacitação, foi criada uma matriz curricular integrada por competências após a pactuação entre professores da pediatria e da saúde coletiva. A matriz destacava a competência geral, os objetivos de aprendizagem, oportunidades disponíveis para aprendê-los e o sistema de avaliação. Após 6 meses, 93% (104/112) dos alunos e 79% (11/14) dos professores relataram que percebiam maior integração do programa e destacaram a incorporação da avaliação de desempenho clínico. CONCLUSÃO: A construção coletiva da matriz curricular por competências levou à maior satisfação de docentes e discentes com a nova proposta que, após a implementação, foi percebida como integradora de conteúdos e práticas de ensino da pediatria, tendo qualificado a avaliação de desempenho clínico.

Currículo; educação baseada em competências

ORIGINAL ARTICLE

Breaking barriers: a competency-based framework for promoting the integration of the pediatrician's education

Alessandra V. Naghettini^I; Valdes R. Bollela^II; Nilce M. S. C. Costa^III; Luciana M. R. Salgado^IV

^IDoutora. Professora, Faculdade de Medicina, Universidade Federal de Goiás (UFG), Goiânia, GO, Brazil

^IIDoutor. Professor, Faculdade de Medicina de Ribeirão Preto (FMRP), Universidade de São Paulo (USP). Ribeirão Preto, SP, Brazil

^IIIDoutora. Professora, Faculdade de Nutrição, UFG, Goiânia, GO, Brazil

^IVAcadêmica de Medicina, UFG, Goiânia, GO, Brazil

^{Correspondence}

ABSTRACT

OBJECTIVE: To describe the process of integration and revision of a pediatric program curriculum which resulted in the creation of a competency-based framework recommended in the Brazilian National Curricular Guidelines.

METHODS: Quali-quantitative analysis of an intervention evaluating the students and professors' perception of the pediatric program curriculum (focus groups and semi-structured interviews). Results were discussed during teaching development workshops. A competency-based framework was suggested for the pediatric program from the 3rd to the 6th year. The new curriculum was approved, implemented, and reevaluated six months later.

RESULTS: Twelve students (12%) from the 3rd to the 6th year participated in the focus groups, and 11 professors (78.5%) answered the questionnaire. Most participants reported lack of integration among the courses, lack of knowledge about the learning goals of the internships, few opportunities of practice, and predominance of theoretical evaluation. In the training workshops, a competency-based curriculum was created after pediatrics and collective health professors reached an agreement. The new curriculum was focused on general competency, learning goals, opportunities available to learn these goals, and evaluation system. After six months, 93% (104/112) of students and 79% (11/14) of professors reported greater integration of the program and highlighted the inclusion of the clinical performance evaluation.

CONCLUSION: The collective creation of a competency-based curriculum promoted higher satisfaction of students and professors. After being implemented, the new curriculum was considered to integrate the teaching practices and contents, improving the quality of the clinical performance evaluation.

Keywords: Curriculum, competency-based education.

Introduction

The challenge of medical education in the turn of the 20th century was faced by the Flexnerian revolution. In that moment, the crisis in the medical education received attention from the public and authorities. This crisis precipitated a series of events leading to a drastic reform in the United States, with consequences in all Latin America. In the beginning of the 21st century, the fundamental issues in medical education were related to the school's social responsibility of training competent physicians for the professional practice.¹ In the last decades, there have been increasing worldwide efforts to determine the meaning of professional competency among the healthcare professions.²

In Brazil, this movement has gained strength and visibility after the publication of the National Curricular Guidelines (Diretrizes Curriculares Nacionais, DCN) for medical undergraduate programs. These guidelines were published by the Ministry of Education in November 2001. A change in the curricular models of the medical programs was expected. The programs were organized in the format of curriculum grids and were usually characterized by excessive inflexibility and fragmentation at the expense of more integrated and flexible proposals as the ones suggested in the DCN.³ Almeida⁴ reflects on the challenges faced by schools that intend to make curricular changes based on the publication of the DCN. According to this author, the DCN describes in detail what should be done in relation to the path or direction of the development of the national medical education. However, how these changes could be put into practice has not been defined yet.

It would be reasonable to expect that the managers of medical schools should try to revise their curricula in the light of the new proposals, considering the creation and implementation of a competency-based curriculum (CBC), since a CBC explains the competencies expected from medical school graduates. Practical experiences in this area are still relatively scarce in Brazil. Nevertheless, many international experiences have produced much knowledge on which a CBC could be built, such as the one suggested by Bollela in 2008,⁵ and by Bollela & Machado in 2010,⁶ for the 2 years of medical internship.

Competency-based education (CBE) is centered on the physician's training for medical practice according to the society's and patient's needs. Thus, CBE is aimed at reducing the focus on time- and content-based training, making room for a more flexible learning process, which is centered on the student.⁷ In this context, competency is a combination of personal qualities used in specific contexts to reach certain results. The focus is on the development of the professional practice in different contexts. This extended concept of competency is also described as capability, or professionals' ability to adapt to changes, produce new knowledge, and continuously improve their performance while becoming competent.⁸

A CBC should provide an in-depth description of the competencies/capabilities expected from the learner; such description should be based on learning goals. Each goal should be related to a plan describing "how" the student can reach it and "how" this acquisition will be measured (evaluated).⁹

Pediatrics, among the major areas of medicine, is the one that mostly contributes to the general training of physicians because it is focused on training a professional who will be able to approach and manage the main issues related to the child health care in a competent manner. Nevertheless, fragmented programs that are oriented towards early specialization can still be found in the curricula of medical undergraduate programs and residency.¹⁰ Within this context, revising the courses of a pediatrics program with a focus on a greater integration is both a challenge and a means of valuing the training in consonance with the DCN and society's needs.

The objective of the present study was to describe the process of curricular revision of the pediatrics undergraduate medical program at the Universidade Federal de Goiás (UFG), Brazil. Such revision included the identification of the changes necessary for the creation of a competency-based framework for the pediatrics and collective health courses from the 3rd to the 6th year of the program based on the logics of an integrated curriculum.

Methods

A longitudinal intervention and quali-quantitative study was conducted with the professors and students enrolled in courses of the department of pediatrics during the study period. The study was divided into three phases:

Phase one: evaluation of the needs of change

The focus group technique was used to investigate the students' perception of the object of study.^11,12 The groups included students from the 3rd and 4th years (pre-internship) and the 5th and 6th years (internship) of the pediatrics program.

The thematic guide was designed with focus on the items of interest: current teaching-learning process, objectives of the courses, satisfaction with the teaching method, learning evaluation methods, and suggestions for the improvement of the course.

The activity of the focus group began with the introduction of the moderator, observer, and participants. Participants were informed about the objective of the meeting and, after signing the informed consent, they were asked to provide permission for the meeting to be videotaped.

Semi-structured interviews were conducted with the professors.¹³ The interview questionnaire was designed considering the assumptions based on the object of the study. The questions were classified and grouped in the following thematic units: current teaching-learning process, objectives of the course, possibilities of integration, learning evaluation, and suggestions for improvement.

The objectives of the study were explained, and we tried to motivate the professors by assuring them that their collaboration would be indispensable for the success of the study. Finally, participants were reassured that the interview would be anonymous and were asked to authorize videotaping it.

The interviews were videotaped and lasted between 30 and 60 minutes. Later, the interviews were transcribed and listened again to make sure the transcription was correct; whenever necessary, we changed the punctuation to make the text easier to read and understand.

We analyzed the content of the interview and the focus group according to Bardin's technique.¹⁴ Professors were identified by the letter "P" and students by the letter "E" followed by the number of insertion in the transcript texts.

Phase two: creation of the competency-based curriculum

Two 16-hour-long workshops were prepared based on the participants' perceptions revealed in the interviews. Pediatrics and collective health professors participated in these workshops, which were intended to define and describe a consensual profile of the graduates according to the general and specific competencies of the DCN. Professors used a competency-based curricular model for the medical internship. This model was the basis for the definition of the learning goals of the courses within each group of general competencies, and it was also used to organize the students' assessment in the different stages of the program.^5,6

Phase three: implementation of the new proposal and initial assessment based on the students and professors' perception

Six months after the implementation of the new curriculum, a questionnaire was administered to professors and students in order to investigate their perception of the change.

The present study was approved by the Research Ethics Committee at Hospital das Clínicas of the UFG under the number 025/2009. The Statistical Package for the Social Sciences (version 17.0, SPSS, Chicago, IL, U.S.) was used to create the database and perform the statistical analyses.

Results

In 2009, there were 100 students enrolled in the pediatrics program and 14 professors were teaching the courses offered by the department of pediatrics from the 3rd to the 6th year. During phase one, 12% (12/100) of these students participated in the focus group, and 78.5% (11/14) of the professors answered the interviews. Although the percentage of the students may seem small, we decided to use the technique of focus groups and a small number of participants in order to achieve a deep understanding of the perception of the students enrolled in the pediatrics internships. The decision of interviewing the professors was made with the purpose of increasing the participation of those responsible for planning and implementing the pediatrics program in the medical school at the UFG.

The main topics mentioned by professors and students were: lack of integration among the courses; students' lack of knowledge about the learning goals of the internships; lack of opportunity for supervised practices; and the fact that the current students' evaluation method was much more focused on theoretical content than on practice.

Next, there are some fragments of the participants' comments stressing one of the main difficulties we found: lack of integration.

"... we should have more integration." (P2)

"... each group (of professors) which is responsible for a year, for example, 4th year, the 3rd year is the semiology, the two years of internship, works in totally dissociated ways." (E1)

There was also lack of knowledge about the objectives of each course.

"... the practical objectives of semiology need improvement. So, you see, I think that this part was... it really needs improvement." (E2)

"... I don't have any classes reinforcing the objectives (of the course)..." (P4)

Few opportunities of practical supervised work were reported.

"... I felt it lacked more practice." (E7)

"... really begin to employ the techniques in practice." (P8)

We also found that the main focus of the evaluation was on the theoretical content.

"... most evaluations are still theoretical exams." (E7)

"... evaluation, it includes, you know, knowledge, responsibility, ethics, effort, punctuality, and even appearance." (P5)

The participants reinforced the need for increasing the integration with other courses.

"... I think that would be great if we could always be working together with the other courses we take." (P5)

"... what is also missing in the pediatrics program is a... a real integration with the other courses." (E1)

The results of the initial phase of the study were presented during the workshops and they serve as the basis to justify the revision of the curriculum. Also, a competency-based framework was proposed for the undergraduate program and the internship in pediatrics. This new curriculum was the result of a consensual decision made by the pediatrics and collective health professors. The model described by Bollela & Machado in 2010⁶ was used as a reference. Competencies, goals, and learning opportunities were defined. Finally, the evaluation methods in accordance with the curricular proposal and the capabilities expected from the medical school graduates from UFG, including the evaluation of clinical performance, were established.

The competency-based framework was concluded with the connection of the evaluation system to the competencies and learning goals. The evaluation system included strategies to evaluate knowledge (essay exams and multiple choice tests), behavior (revision of the global score for internship), and clinical competence. All the strategies were performed using the Objective Structured Clinical Examination (OSCE)^15,16 and the Mini-Clinical Examination Exercise (Mini-CEX).¹⁷

During phase three, six months after the implementation, 98% (104/106) of students and 80% (8/10) of professors were interviewed to evaluate the initial moment of the change. Data were codified and analyzed in a descriptive manner.

The credit hours of the courses were considered satisfactory by 79.8% (83/104) of students. According to 90.4% (94/104) of students, the objectives of the course/internship were clearly presented, and about 61.5% (64/104) considered that the objectives were reached. In addition, 69.2% (72/104) of students approved the teaching method of the courses. Also, according to 82.7% (86/104) of students, the practical activities were stimulating.

The pedagogical strategies were considered adequate for the content/objectives by at least 37.5% (39/104) of students. With regards to performance assessment (OSCE), 64.4% (67/104) of students considered it adequate.

According to 81.7% (85/104) of students, the courses offered a linkage between theoretical knowledge, cultural aspects, reflection about inequality, organization, and logical reasoning. The integration with other courses, especially collective health, was reported by only 46.2% (48/104) of students.

In the opinion of the professors, 80% (8/10) considered that the number of credit hours was adequate, 70% (7/10) reported having handed out the syllabus of the course in the beginning of the program, and 80% (8/10) reported having presented the objectives of the course/internship in a satisfactory manner. About 40% (4/10) of professors reported having offered the students total possibilities for them to reach all the goals of the course, while 40% (4/10) believed that they did it just partially.

All professors reported being satisfied (totally or partially) with the teaching method used in the course, and 80% (8/10) considered that, after the change of the curriculum, there was an increase in the practical activities during the course.

Regarding the didactic methods, 30% (3/10) of professors reported having encouraged more participation of the students, while 20% (2/10) thought that the didactic methods were adequate regarding the content and objectives. Most professors, 70% (7/10), considered that the classes encouraged the logical reasoning and the critical sense of the students.

When asked about the evaluation of capabilities, 80% (8/10) of professors reported that it was adequate, and 60% believed that the evaluation was representative of the program. Regarding the integration of the curricular content and practice, 70% (7/10) considered that the integration was moderate, despite a good connection between the topics.

Participants also stressed the need for further integration between the courses of the medical program. They also made several suggestions for the improvement of practical activities.

Discussion

The interest in the need of defining the professional competencies required for medical undergraduate programs and residency has greatly increased over the past three decades. Experiences in U.S. medical schools have shown the importance of the professors' participation in the creation and implementation of a competency-based curriculum.¹ In 1972, The American Board of Pediatrics (ABP) published one of the first documents related to this topic, which was entitled "A method for defining competency in pediatrics." This document included the competencies expected for the education of pediatricians during medical residency and the best strategies to evaluate such competencies.¹⁸

In the present study, we describe the experience of creating a competency-based framework for an undergraduate pediatrics program. As mentioned above, the active participation of the pediatrics and collective health professors was essential for the successful completion of all the phases.

An evaluation of pediatrics programs in schools of medicine of Rio de Janeiro, Brazil, in 2006 demonstrated the need for revision of the traditional teaching methodologies, increased number of practice settings, and integration of theoretical contents that used to be presented in a fragmented manner.¹⁹ The diagnosis phase of this study included many reports of students and professors about the excessive theoretical load and fragmentation of the courses/internships of the pediatrics program. This situation is similar to the one found in many Brazilian medical schools.⁴

There has been continuous discussion on curricular reform in the medical programs and it has been gaining importance and evidence since the publication of the DCN and, more recently, as a consequence of the promotion of the Ministry of Education and the Ministry of Health during the last 10 years: Program for the Encouragement of Curricular Changes in Medical Programs (Programa de Incentivo às Mudanças Curriculares nos Cursos de Medicina, PROMED); National Program of Reorientation of Health Professionals' Education (Programa Nacional de Reorientação da Formação Profissional em Saúde, PRÓ-SAÚDE 1 and 2); and Program of Education through Work for Health (Programa de Educação pelo Trabalho para a Saúde, PET SAÚDE). These programs encourage curricular changes and reinforce the involvement of undergraduate students in the basic care provided by the Brazilian public Unified Health Systems (Sistema Único de Saúde, SUS).

For the implementation of the curricular changes proposed, it is essential to take actions that surpass the formal sphere of pedagogical documents and projects. It is a key point that the superior instances of the program support the process of change, which must involve the largest possible number of participants: professors, students, health system managers, and academic managers.²⁰ The joint creation and approval of the competency-based framework by the program board enabled the update and integration of the courses. The model of the Accreditation Council for Graduate Medical Education (ACGME)⁹ adapted for the DCN³ was very useful and practical because the definition of learning goals and competencies led the professors to think about the learning opportunities and the need for organizing an adequate evaluation system of the students.

The definition of an evaluation system that is coherent and compatible with the curricular proposal is a critical stage when dealing with a competency-based framework. During the workshops, the existing evaluation system (written exams and grades by letters) was not considered able to guarantee the evaluation of essential competencies such as: interpersonal communication, ability to obtain medical history, physical examination, and counseling about a care plan. This situation raised the need and the motivation among the professors to better know the evaluation methods of clinical performance in simulated situations and in real practice settings. Furthermore, it demanded specific training, as it is usual to occur when new methods of evaluation are defined.^21-23 In the present study, we observed that the evaluation of the OSCE and the Mini-CEX was considered adequate by 80% of professors and 64% of students.

The joint discussion involving pediatrics and collective health professors during the creation of a competency-based framework made it possible for the professors to broaden the horizons of the family health internship. The discussion of the competencies prompted the comprehension of the complementariness of the programs so much that the collective health and pediatrics professors reached a consensus about a framework with focus on child care, family, and community in the context of SUS.

Despite including all the phases of a curricular revision (diagnosis of the needs, planning, intervention, and evaluation), the present study has some limitations that should be mentioned. If, on one hand, it has great internal validity for the medical program at UFG, on the other hand, its external validity should be evaluated, since the design and implementation of the curriculum are specific of a context and may vary a lot depending on the characteristics and conditions of each institution. The change was assessed only by means of a survey about the students and professors' level of satisfaction because of the short time of implementation. However, we should go further and also evaluate the impact of new teaching and evaluation practices on the quality of the graduates' education and on the institutional culture.²⁴

Conclusion

The joint creation of the competency-based curriculum framework led to more satisfied professors and students regarding the new proposal. After being implemented, this proposal was perceived as an integrator of contents and teaching practices of pediatrics. The continuous evaluation of the curriculum is essential and should consider the satisfaction of those involved, the impact on the teaching/evaluation methods, and, last but not least, the capability of training competent and prepared physicians to meet society's needs.

Acknowledgements

The authors thank the Instituto Regional FAIMER Brasil.

References

1. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med. 2002;77:361-7.

2. Reeves S, Fox A, Hodges BD. The competency movement in the health professions: ensuring consistent standards or reproducing conventional domains of practice? Adv Health Sci Educ Theory Pract. 2009;14:451-3.

3. Brasil, Ministério da Educação. Conselho Nacional de Educação. Câmara de Educação Superior. Diretrizes Curriculares do Curso de Graduação em Medicina. Resolução CNE/CES Nº 4, de 07/11/2001. 2001. Disponível em: http://portal.mec.gov.br/cne/arquivos/pdf/CES04.pdf. Acesso: 19/06/2011.

4. Almeida, MA. Gestão da escola médica: crítica e autocrítica. Rev Bras Educ Med. 2008;32:202-9.

5. Bollela VR. Mind the gap! Med Educ. 2008;42:519.

6. Bollela VR, Machado JL. Internato Baseado em Competências: Bridging the Gaps. Belo Horizonte: Medvance; 2010.

7. Frank JR, Mungroo R, Ahmad Y, Wang M, De Rossi S, Horsley T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach. 2010;32:631-7.

8. Fraser SW, Greenhalgh T. Coping with complexity: educating for capability. BMJ. 2001;323:799-803.

9. Joyce B. Developing a competency-based curriculum. Facilitator's guide. ACGME Outcome Project. Disponível em: http://www.acgme.org/outcome/e-learn/FacManual_module4.pdf. Acesso: 19/06/2011.

10. Costa NM. Docência no ensino médico: por que é tão difícil mudar? Rev Bras Educ Med. 2007;31:21-30.

11. Gondim SM. Grupos focais como técnica de investigação qualitativa: desafios metodológicos. Paidéia. 2002;12:149-61.

12. Mazza VA, Melo NS, Chiesa AM. O grupo focal como técnica de coleta de dados na pesquisa qualitativa: relato de experiência. Cogitare Enferm. 2009;14:183-8.

13. Bauer MW, Gaskell G. Pesquisa qualitativa com texto, imagem e som: um manual prático. Petrópolis: Vozes; 2008. p. 64-89.

14. Bardin L. Análise de conteúdo. 4. ed. Lisboa: Edições 70; 2009.

15. Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ. 1979;13:41-54.

16. Sandoval GE, Valenzuela PM, Monge MM, Toso PA, Triviño XC, Wright AC, et al. Analysis of a learning assessment system for pediatric internship based upon objective structured clinical examination, clinical practice observation and written examination. J Pediatr (Rio J). 2010;86:131-6.

17. Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann Intern Med. 2003;138:476-81.

18. Burg FD, Brownlee RC, Wright FH, Levine H, Daeschner CW, Vaughan VC 3rd, et al. A method for defining competency in pediatrics. J Med Educ. 1976;51:824-8.

19. Veiga EQ, Batista, NA. Undergraduate teaching of pediatrics in medical schools of the state of Rio de Janeiro. J. Pediatr (Rio J). 2006;82:1150-20.

20. Ronzani, TM. A reforma curricular nos cursos de saúde: qual o papel das crenças? Rev Bras Educ Med. 2007;31:38-43.

21. Norcini JJ, MCKinley DW. Assessment methods in medical education. Teaching and Teacher Education 2007;23:239-50.

22. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32:676-82.

23. Hicks PJ, Schumacher DJ, Benson BJ, Burke AE, Englander R, Guralnick S et al. The pediatrics milestones: conceptual framework, guiding principles, and approach to development. J Grad Med Educ. 2010;2:410-8.

24. Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, et al. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teach. 2006;28:497-526.

Manuscript submitted Feb 10 2011, accepted for publication Jul 13 2011.

Financial support: Luciana M. R. Salgado received an undergraduate research scholarship from Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq).

No conflicts of interest declared concerning the publication of this article.

Suggested citation: Naghettini AV, Bollela VR, Costa NM, Salgado LM, et al. Breaking barriers: a competency-based framework for promoting the integration of the pediatrician's education. J Pediatr (Rio J). 2011;87(6):529-34.

Publication Dates

• Publication in this collection
  01 Feb 2012
• Date of issue
  Dec 2011

History

• Accepted
  13 July 2011
• Received
  10 Feb 2011