Competency-based anesthesiology teaching: comparison of programs in Brazil, Canada and the United States

Abstract

In 2017, the Brazilian Society of Anesthesiology (SBA) and the National Medical Residency Committee (CNRM) presented a joint competence matrix to train and evaluate physicians specializing in Anesthesiology, which was enforced in 2019. The competency-based curriculum aims to train residents in relation to certain results, in that residents are considered capable when they are able to act in an appropriate and effective manner within certain standards of performance. Canada and the United States (US) also use competency-based curriculum to train their professionals. In Canada, the format is the basis for using an evaluation method known as Entrustable Professional Activities (EPA), in which the mentor assesses residents’ capacity to perform certain tasks, classified in 5 levels. The US, in turn, uses Milestones as evaluation, in which competencies and sub-competencies are assessed according to residents’ progress during training. The present article aims to describe and compare the different competency-based curriculum and the evaluation methods used in the three countries, and proposes a reflection on future paths for medical education in Anesthesiology in Brazil.

KEYWORDS
Competency-based curriculum; Anesthesiology; Medical residency; Medical education; Entrustable Professional Activities (EPA); Milestones

Introduction

In 2017, the Brazilian Society of Anesthesiology (SBA) and the National Medical Residency Committee (CNRM) chose to unify the competence matrix used to build the curriculum and evaluate physicians specializing in Anesthesiology. The aim was standardize teaching in Anesthesiology and develop evaluations throughout the country. The new competence matrix was enforced in 2019.¹

Competencies

The characterization of the skills of physicians training as specialists to consider them competent depends on the evaluator’s judgement. As suggested by Mulder, professionals are competent when they act in a responsible and effective way according to certain standards of performance.² However, unlike the assessment of a physician trainee’s performance, which is observable and measurable, competency assessment cannot be specified accurately. This is so because it involves characteristics inherent to each individual, such as their capacity and skills, making an accurate description difficult.³ Given those characteristics are not accessible to evaluators, they will infer in order to assess physicians training as specialists. Therefore, assessment describes physicians’ performance during specialization, and not their competence. Performance is, therefore, the behavior observed and assessed as competent.

Competency-based Medical Education emphasizes program assessment in two distinct functions – assessment of development (formative) and decision making (summative). Assessment performed in a low-risk workplace aims to provide significant feedback to the apprentice in order to orient learning, while decisions on high-risk competencies should be performed based on aggregated assessment data, collected throughout time and on several points in time through individual assessments.⁴

Countries such as Canada and the United States (US) also use a competency-based curriculum similar to the Brazilian matrix of competency for evaluating and assuring the quality of education of specialty training. In Brazil, the matrix presented by SBA and CNRM guides the evaluation of physician trainees’ learning throughout the years, in that they acquire appropriate objectives and competences in each year of specialization.

In Canada, competencies are the basis for another evaluation method, called Entrustable Professional Activities (EPA). EPA emerged as an evaluation tool to assess performance at the workplace. Besides Canada, other countries, such as The Netherlands, Australia, and New Zealand also have a consolidated system based on EPA for some specialties, such as Gynecology and Obstetrics, and Psychiatry.⁵ The Netherlands also have taken the first step toward this more contemporary format of competency-based curriculum in Anesthesiology.⁶

The US uses a competency- and sub-competency- based curriculum as a means of a standardized evaluation called milestones. The system offers a framework to assess the development of specialty trainees regarding essential physician competencies. The competencies are Patient Care, Professionalism, Practice Based Learning and Improvement, Systems-Based Practice, Interpersonal and Communication Skills, and Medical Knowledge.⁷ Unlike other teaching methods, the competency-based curriculum aims toward the development and assessment of physicians undergoing specialization in regard to certain results, adding concepts from other methods of teaching, such as the Miller pyramid to assess progression of the apprentice, and style of education for each level of the Dreyfus model.⁸

Objective

The present study aims to present and compare three distinct competency-based curriculum and assess them in terms of training in Anesthesiology specialization in three different countries, Brazil, Canada, and the United States, and proposes a reflection on the future paths for teaching Anesthesiology in Brazil.

Competence-matrixes of residency programs

Brazil

Specialization in Anesthesiology in Brazil has evolved, and in the past year began to be used as a joint competency matrix between the Brazilian Society of Anesthesiology and the National Medical Residency Committee. Such measure aims to train and build skills of physicians in Anesthesiology specialization with the objective of acquiring the competencies required to perform appropriate anesthesia for diagnostic, surgical, and therapeutical procedures.¹ The competency matrix is divided by year, and each year of specialization describes the competencies required. There are a total of 10 specific objectives and 49 competencies divided among the 3-years of specialization. Assessment of competencies varies among services, and there is no single national format on how each specialty trainee should be assessed, or a timetable to be followed to assure that everyone reaches a certain level until the end of a specialization program.

In the first year of specialization (R1), the physician should improve knowledge in theory and practices in medicine, and be capable of relating them to Anesthesiology. Toward that end, the R1 should present skills for lower complexity procedures with supervision on elective patients submitted to minor and medium size surgeries. Examples of competences to be reached at the end of the first year are: (1) gather accurate and essential information on patients and their complaints, along with complete, general, and specific physical examination during pre-anesthesia assessment; (2) set up and interpret basic monitoring, and do whatever is required to maintain clinical stability of patients; (3) assess and perform tracheal intubation and extubation; and (4) assess and perform neuraxial anesthesia.¹

In the second year of specialization (R2), more responsibility is granted to physicians undergoing specialization, and they are in charge of pre-anesthetic evaluation and anesthesia plan for medium size and major surgeries, and for anesthesia for higher complex procedures, with the help of other tools or not, such as ultrasonography. Knowledge on managing acute pain and patients in intensive care, whether in the ICU or post-anesthesia recovery room, is also explored. Examples of competencies to be reached in this stage of learning are: (1) difficult airway assessment and domain of the control algorithm; (2) assessment and performance of anesthesia blockades and ultrasound-guided vascular access; and (3) analysis, diagnosis and treatment of intraoperative and postoperative anesthesia complications in the post-anesthesia recovery room.¹

During the third and last year of specialization (R3), physicians should have a comprehensive view of the patient, including, but not restricted to, preoperative evaluation of patients to be submitted to any kind of surgical procedure, intraoperative management of anesthesia for major surgeries (e.g., heart surgeries, transplantations, vascular radiology, among others), and intensive postoperative management when required. R3 are expected to present professionalism and commitment to their training, including theory, practice and scientific learning, and should deliver, in a specified timeframe, a course conclusion paper. Examples of competencies to be the reached by R3 in order to successfully conclude specialization are: (1) making decisions under adverse conditions, showing emotional control and equilibrium, using leadership to minimize any possible complications, and being aware of their limitations; (2) communicating effectively with physicians, other health professionals, and health-related services, especially with surgeons, during surgery, as to changes in physiological parameters capable of interfering unfavorably in the immediate result of anesthesia or surgery; (3) deciding, during anesthesia, the need to use scientifically acceptable technical variants, in order to overcome unexpected difficulties.¹

Canada

In Canada, the competencies described in the Canadian Medical Education Directives for Specialists (CanMEDS), developed by the Royal College of Physicians and Surgeons of Canada, initially in 1996 and reviewed most recently in 2015, is the foundation for using Entrustable Professional Activities (EPA). EPAs were introduced by Ten Cate in 2005 as a new model for evaluation in the workplace, aimed at helping supervisors to determine if required competences were reached by a physician undergoing specialization.⁹

CanMEDS is a matrix that identifies and describes skills that physicians training as specialists should acquire to effectively respond to patient needs. It includes seven competencies: Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar and Professional.

Medical Expert is described as the core competence, the intersection point among all competences, representing the central role of the physician. Competencies approach physicians’ knowledge, such as clinical skills and professional values aimed at providing high quality patient-centered care. Toward that end, patient status, best practices and scientific evidence are taken into account. The Communicator competency represents physician-patient/family relations, which must be well established to facilitate attainment of and sharing information effectively for patient care, always aimed at a patient-centered approach. Collaborator represents work in a multidisciplinary team, reminding physicians of their duty to work along with other health professionals, family members and the community, based on respect and trust. Physicians must understand the roles of other health professionals, the responsibility and need to learn from others who partake in patient care. Physicians take over the role of leaders in the health care system, whether at the local, regional, national or global level. They join other health professionals to contribute to and guarantee a high-quality health system, and are responsible for providing excellence in services to patients through their clinical teaching activities and administrative responsibilities. A physician’s work is not only treating conditions diagnosed, but also includes disease prevention, health promotion, protection of and equity in health, which characterizes the Health Defensor competency. Equity in health should be viewed as a way to promote integral health without discriminating the population in view of color, race, gender, sexual orientation, ethnicity, religion, social class, economic status or schooling, for example. The Scholar competency remits to lifelong commitment to excellence through continuous learning, conveying knowledge to others, assessing evidence and contributing to research. The seventh competence is Professionalism, which reflects what society expects of physicians, promotion of public well-being, integrity, altruism, humbleness and respect to diversity, the basis of the implicit contract established between society and the professional, the physician.

The seven competencies described in CanMEDS also include key-competencies, enabling competencies and key-concepts, so as to divide and explain in more detail what is expected of physicians undergoing specialization. The Medical Expert competency has five key-competences, for example, “Practice medicine within the scope defined for each one’s practice and experience”. Each key-competence has enabling competencies to qualify each key-competence. Enabling competencies related to the previous example include “demonstrate commitment to high quality care of patients”, “apply clinical knowledge and of relevant biological sciences to the discipline” and “acknowledge and respond to the complexity, uncertainty and ambiguity inherent to medical practice”, among others. Key-concepts are comprised of enabling competencies, which may fit into more than one key-concept. Enabling competencies mentioned above would be respectively related to the following key-concepts: “compassion” and “duty to care for”; “application of clinical knowledge and from biomedical sciences”, “clinical reasoning” and “working with a healthcare team”; and “clinical decision making” and “complexity, uncertainty and ambiguity in clinical decision making” (Table 1).

Table 1
Competence and its subclassifications. Example of how competences, key-competencies, enabling competencies and key-concepts are described. Each key-competency is listed and associated with key-concepts to which they apply.

Competency: specialist physician
Key-competency	Enabling competencies	Key concepts
Practice medicine within the scope defined by practice and experience	(1) Demonstrate commitment to high quality care to patients;	Apply clinical knowledge and from biomedical sciences: 2
	(2) Apply clinical knowledge and from biomedical sciences relevant to the discipline;	Compassion: 1
		Complexity, uncertainty and ambiguity in clinical decision making: 3
	(3) Acknowledge and respond to complexity, uncertainty and ambiguity inherent to medical practice.	Duty to care: 1
		Clinical reasoning: 2
		Work with healthcare team: 2
		Clinical decision making: 3

Evaluation, however, is not only based on the competencies described above. They are the foundation for the evaluation system known as Entrustable Professional Activities (EPA). Each EPA comprises several competencies, and the same competencyy can be present in more than one EPA (Table 2). EPA can be defined as a professional practice activity that can be totally entrusted to a physician undergoing specialization as soon as the required skilled to execute such activity without supervision is demonstrated.¹¹ In Canada, EPAs are divided in Transition to Discipline, Foundations of Discipline, Core of Discipline and Transition to Practice.¹²

Table 2
Constitution of EPAs. Example of EPA matrix using competencies. Note that a certain Competency can be present in more than one EPA, and that EPA encompasses several and different numbers of competences.

	EPA 1	EPA 2	EPA 3	EPA 4	EPA 5
Competency1	●		●	●	●
Competency2	●	●		●
Competency3		●		●	●
Competency4		●	●		●
Competency5	●		●	●	●
Competency6		●	●	●
Competency7		●	●		●

• Adapted from Holmboe, E., Durning, S. and Hawkins, R., Practical Guide To The Evaluation Of Clinical Competence, 2018).¹⁰

The first EPA group, Transition to Discipline represents the transition of a medical undergraduate student to the beginning of specialization in Anesthesiology, and represents simpler and less complex activities. Examples of this group of EPA are: (1) perform preoperative evaluation of patients ASA I or II to be submitted to low complexity elective surgical procedures; (2) preparation of the Operating Room for low complexity elective surgical procedures for patients ASA I or II; (3) perform postoperative transfer of adult ASA I or II patients after low complexity elective surgical procedures, including postoperative prescriptions.

The second EPA group, Foundations of Discipline, encompasses basic-fundamental activities of Anesthesiology, more complex than the previous one, but still with restriction as to complexity. Examples of EPA to be attained in this group are: (1) identify patients with potential difficult airway and prepare initial management options; (2) anticipate, prevent and conduct adequately expected or common intraoperative events and physiological changes during low and medium risk surgical procedures; (3) manage pediatric patients with common postoperative complications at the post anesthesia recovery room or ward.

After accomplishing these activities, physician trainees should then attain the EPA related to the Core of Discipline, the group that concentrates the higher number of activities, therefore considered the core of teaching Anesthesiology. Examples of activities classified in this group are: (1) use ultrasound to help diagnostics and to manage hemodynamically unstable or critical patients; (2) provide multimodal management to patients with acute pain or episode of exacerbation during chronic pain; (3) establish and manage difficult venous access and invasive monitoring for pediatric patients over 1 year of age.

The last EPA group that trainee physicians in Anesthesiology should attain is called Transition to Practice. This group proposes to assure that physicians in their last year of specialization are ready to practice anesthesia management in the largest variety of complex patients and surgical procedures, so as to be apt to work without supervision. Examples of activities of this group are: (1) management of all aspects of care of patients admitted, referred for opinion by the Anesthesiology service; (2) management of all aspects of care of the obstetric patient, including obstetric ward management-related organizational aspects; (3) development of a learning and personal career plan.

Considering what has been exposed, we may conclude that EPA do not directly evaluate the competencies to be reached, but the “doing” skills of a physician specializing. The key question for evaluation of EPA is: “Can we trust this physician undergoing specialization to perform this activity?” Additionally, to evaluate the capacity of a physician training as a specialist to perform the activity in a trustworthy way, 5 levels of supervision and trust reliability are considered.^13–15 They are:

1. Physicians undergoing specialization may be present and observe, but are not able to execute the EPA. In the initial stages, physician trainees are expected to be present and observe what they will do in the following stage of learning. Gradually, they will be trusted to perform some parts of the activity.

2. Allowed to perform the activity with direct and proactive supervision of the mentor, who should be present in the operating room. During this stage, physician trainees can perform the activity fully and independently. The mentor, who is present in the room can interfere or take over the activity at any time believed required.

3. Allowed to perform activity with indirect and reactive supervision, in which the supervisor should be ready to be promptly available to enter the operating room. In this stage, physician trainees are fully independent to perform activity, without a supervisor present in the room, albeit available within minutes.

4. Allowed to act without qualified supervision in the proximities; with at-distance supervision, basically performing without supervision. In this stage, physician trainees can perform activity fully without nearby supervision. Physician trainees report to supervisor on the same day or following day.

5. Allowed to supervise physician trainees in years junior to them performing an EPA. This level is assured for last year physicians undergoing specialization, when they can supervise other, less experienced, trainees.

The United States (US)

Currently, evaluation of the performance of physicians undergoing specialization or fellowships in Anesthesiology in the US is done using evaluation of sub-competencies and competencies, known as The Anesthesiology Milestone Project, established in December 2013. Milestones are a framework to evaluate physician trainees’ performance in the main competence and sub-competence elements that they should have in a specialty or subspecialty. Every six months, all medical residency programs in Anesthesiology should submit to the regulating agency, the Accreditation Council for Graduate Medical Education (ACGME), the quality of physician teaching in the US, the results indicating in which milestone each one of its physician trainees are at, to assess progress. Beside ACGME, the American Board of Anesthesiology (ABA) also requires that programs send evaluation of clinical competencies of all physicians in specialization and fellows in Anesthesiology electronically, annually in January and July. In order to complete such evaluations, global marks should be selected, classified as “satisfactory”, when a physician training as specialist/fellow constantly meets expectations, or “unsatisfactory”, when below expectations. If a physician training as specialist/fellow does not pass a test on basic concepts twice or more, this is also considered unsatisfactory. These competencies are considered the initial step toward Anesthesiology certification.¹⁶

Milestones are organized from levels 1 to 5 and, by narrative of each sub competence, represent an assessment tool of the development of a physician undergoing specialization during training in Anesthesiology. Level 1 represents the internship year, before beginning training in Anesthesiology, when the physician trainee should present the milestones expected of someone who has finished the first year after graduating Medical school. Level 2 is appropriate to those who present milestones expected of a physician trainee during Anesthesiology training, but that still has not been exposed in a significant manner to Anesthesiology subspecialties. At level 3, physician trainees present the milestones expected of someone after having significant contact and experience in anesthesiology subspecialties. Level 4 substantially shows the milestones expected for specialization in Anesthesiology, and that one is ready for the transition to independent practice as an Anesthesiologist. Level 4 is the graduation target – end of R3 – however, according to ACGME, the decision if a physician trainee is ready to conclude residency belongs to the residency coordinator, and reaching level 4 is not required to conclude specialization. Level 5 is reserved only to some exceptional physicians undergoing specialization. At this level, having advanced beyond the objectives defined for the residency has to be demonstrated, with advanced goals and skills, which may describe the performance of someone already practicing Anesthesiology for some years. For example, participating in the organization of tests to attain the title of Anesthesiology or provide ideal pre-anesthesia consulting to other Anesthesiologists. In addition to these levels, there is a field to check “did not reach level 1”. For each level, there is the option of checking an option at the level and one between levels. The option to check the level represents that the physician undergoing specialization is at the level checked, and the option between levels represents that the physician undergoing specialization shows milestones of the previous level and some of the levels above (Table 3).

Table 3
Example of table containing Competency, sub-Competency and milestone to be completed by medical residency programs in Anesthesiology.

Patient care: management of perioperative pain
Did not reach level 1	Competency:		(Sub competency
	Level 1	Level 2	Level 3	Level 4	Level 5
	Knowledge of an individual prior to first year of Anesthesiology (Milestone)	Knowledge of a physician undergoing specialization with little exposure to Anesthesiology subspecialties (Milestone)	Knowledge of a physician undergoing specialization with significant contact with Anesthesiology subspecialties (Milestone)	Appropriate knowledge for independent practice of Anesthesiology (Milestone)	Objective knowledge and objectives beyond those defined by residency; presenting performance similar to Anesthesiologist with some years of experience (Milestone)

• Adapted from The Anesthesiology Milestones Project, 2015.⁷

Each milestone should be checked according to each sub-competency and competency. There are six competencies. They are Patient Care, Medical Knowledge, Systems-based Practice, Practiced-based Learning and Improvement, Professionalism and Interpersonal and Communication Skills. Each one of the competencies has sub competencies specifying characteristics. For example: (1) Interpersonal and Communication Skills – communication with patients and family; (2) Interpersonal and Communication Skills – leadership time and skills; (3) Professionalism – provide and receive feedback; (4) Professionalism – responsibility with patients, family and society; (5) Practiced-based Learning and Improvement – self-learning; (6) Patient Care – perioperative pain management.

Milestone evaluation is performed at different levels. During each rotation, there is an assessment for specialty training physicians, such as in Neuroanesthesia, Anesthesia for Obstetrics, or Anesthesia for Pediatrics. Each rotation is responsible for the evaluation method, and there can be more than one evaluator, generally someone who has worked with the physician trainee, or the rotation coordinator. These assessments are then submitted to the clinical competency committee, that is responsible for the final evaluation of the physician trainee each semester, taking into account the assessment made in each rotation, and each committee member’s own experience with a certain physician trainee. Then, the evaluation is submitted to the ACGME to analyze each one’s progress.

Discussion: similarities and singularities

The three countries have competency-based curriculum that guide the training and consequent evaluation of physicians undergoing specialization. However, they are used in significantly different ways. In the North American countries, there are competency-based evaluation formats, in which a concept is assigned to the physician trainee in a standardized manner for all medical residency programs in both countries. Medical residency in Anesthesiology in Brazil lasts 3 years; in Canada, 5 years; in the US it lasts 4 years, with the first year as internship.

In the US, submission every semester of the evaluation of physician trainees by the clinical competency committee, comprised by the Anesthesiologists of the programs, assures that the programs assess trainees systematically. Due to ACGME requirements, physician trainees are assured evaluation according to the nationally proposed guidelines. Such format allows monitoring progression and addressing deficiencies of each physician trainee more easily and promptly, and in a customized way. However, the pressure for documents to be filled out may mean extra red tape to the program. This may lead to Anesthesiologists being shifted and having extra work in order to be part of the evaluation committee, making the program cope with possible costs and overtime or the need to have a larger Anesthesiology staff. In Canada, on the other hand, despite the unified evaluation format, programs do not need to submit the performance of their specialty training physicians to a central agency.

Currently in Brazil, competencies are the basis for each program to guide its teaching and individual performance evaluation. Despite medical residency programs being required to be accredited by the Ministry of Education (MEC), by the Brazilian Society of Anesthesiology (SBA) and have the endorsement of the Brazilian Medical Association (AMB), there is no standardized evaluation system for practice in the workplace that controls the technical quality of future Anesthesiologists. The National Medical Residency Committee (CNRM) demands quarterly evaluations of physicians undergoing specialization, including knowledge of theory of the curriculum matrixes and behavior and professionalism, which each program is in charge of implementing. In Canada and in the US, evaluation of behavior and professionalism are also explained by evaluation criteria in EPA and Milestones, respectively. The evaluation of theory learning is standardized by the Brazilian Society of Anesthesiology (SBA), by means of an annual national test for physicians undergoing specialization. The evaluation is similar to the In-Training Examination (ITE) of the US, and also is carried out by most of the Anesthesiology residency programs in Canada.

In both North American countries, competencies are classified in major groups and in turn classified in sub competencies. In Brazil, competencies are listed in a general way, and described according to what physician trainees should attain each year. The classification and subclassification format of Canada and the US can facilitate the analysis and judgement of assessment as to competencies attained, making them less vague or subjective, generating a better guide for evaluation. However, excess of subclassifications can be more confusing to understand by the evaluating reader, and may require a lot of time for understanding or classifying and assessing physician trainees, leading to possibly abandoning or undervaluing filling out the evaluation. This can result in a less specific and detailed evaluation done with less care, and with impact on the way each one gets feedback to improve learning.

The Brazilian Society of Anesthesiology requires that certain rotations be performed by physicians undergoing specialization during their training period. According to the 2020 SBA Regulation of the Training and Teaching Centers (Centros de Ensino e Treinamento – CET)¹⁷: I – Pre- and postoperative: minimum of 10% of annual time load, for pre-anesthesia evaluation (preoperative evaluation office and pre-anesthesia visit), post-anesthesia visit, treatment of postoperative pain and acute and chronic pain syndromes; II – Intensive Care Unit and anesthesia for urgency and emergency: minimum of 15% of annual time load; III – Operating Room, diagnostic and therapeutic services: minimum of 45% of annual time load; IV – Obstetric Center: minimum of 10% of annual time load; V – Optional rotations: cardiology, pneumology, neurology, clinical laboratory, physiology laboratory, pharmacology laboratory, experimental surgery and hemotherapy, or other, at the discretion of the Institution. In Canada, the Royal College of Physicians and Surgeons of Canada determines a minimum of requirements for physician training during the 5 years of specialization. Programs should include a minimum of 12 months of anesthesia in adult patients; 3-months of pediatric anesthesia; 2 months of anesthesia for obstetrics; 1 month of chronic pain management; 6 months of internal medicine; and 3 months in intensive care unit, not exceeding 6 months.¹⁸ A similar control is required by ACGME in the US, in that there is minimum amount of rotations required by ACGME that physician trainees should complete.¹⁹ The program should provide during the internship year experience with pain management; at least 1-month, but not more than 2, in intensive care unit and emergency medicine. During the 3 years of Clinical Anesthesiology, physician trainees should complete at least: 2 weeks in preoperative medicine; 2 weeks in postanesthesia recovery; 4 months in intensive care unit (including the month(s) during the pre-anesthesiology year – internship); 2 months in anesthesia for pediatric surgery, obstetrics anesthesia, neuroanesthesia and anesthesia for cardiothoracic surgery; 3 months in pain medicine; 2 weeks in anesthetic care for diagnostic or therapeutic procedures outside the surgical suite. All specialty training physicians should obtain a certificate in Advanced Cardiovascular Life Support (ACLS) at least once during training, and should take part in at least one clinical experience drill a year.

Given the competency-based curriculum is based on clinical training, registration of cases done by physicians during specialization can reflect each one’s competence. There are currently in the US minimum requirements of number of cases to be done in each subspecialty. In Brazil and Canada there is no regulation as to the number of cases required. In Brazil, physicians during specialization are required to do at least 440 acts and 900 annual hours of practical training, but without specification of the number of cases in each subspecialty. Although the US requires a minimum number of cases for each physician during specialization, a study has shown that within the same residency program in the US where everyone attained the number of required cases, there was a variability of 2 to 3 times in the number of cases each one did.²⁰

Table 4 summarizes and explains the evaluation methods used, in addition to comparing other aspects, with time load, years of training in the specialty, number of cases each physician trainee should do throughout training, and mandatory rotations during residency.

Table 4
Comparison of main characterístics of each country for evaluating physician training in Anesthesiology.

	Brazil	US	Canada
Competency-based curriculum	Yes	Yes	Yes
Milestone-based evaluation	No	Yes	No
EPA-based evaluation	No	No	Yes
Central agency for submission of performance of physicians undergoing specialization	No	Yes	No
Common competence evaluation system in the workplace for the country	No	Yes	Yes
Years of training in anesthesiology	3	4a	5
Limit of working hours per week 19	60 h/week; 6-h rest after night duty; 1 day off per week	80 h/week, average of 4 weeks; 1 day off per week; 10-h rest between duty shifts	Maximum 24 hours per duty shift and not more than seven duty days in 28 days. Variable among provinces. In Alberta, duty can last up to 26 hours, while in Quebec, they are limited to 16 hours.
Rotations required by programs	Yes	Yes	Yes
Number of cases required	Yesb	Yesc	No

• ^a
  4-years, with the first year of internship.
• ^b
  According to the2020 Regulation of the Centros de Ensino e Treinamento (CET) of the Brazilian Society of Anesthesiology (SBA): Provide minimum of 440 anesthesia acts and 900 annual hours of practical training in anesthesia for each medical specialist, comprising, mandatory anesthesia procedures for General Surgery, Obstetrics, children 0 to 12 years and urgency and emergency, and also, for at least the following surgical specialties: Proctology, Peripheral Vascular Surgery, Orthopedics and Trauma, Gynecology, ENT, Ophthalmology, Urology, Diagnostic Tests, Cardiothoracic Surgery and Neurosurgery.
• ^c
  Minimum number of procedures and cases required by ACGME: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/040_Anesthesiology_2019_TCC.pdf?ver=2019-03-21-161242-837.

Possible paths to be taken and conclusions

The competency-based curriculum approaches the responsibility of training programs in Anesthesiology to train capable specialists, able to practice independently. Such competencies are structured according to the needs of society and patients.

There are currently, several medical residency programs in the US, not only in Anesthesiology, studying and trying to implement EPA as the evaluation method of physicians undergoing specialization, such as Pathology, Radiology and Pediatriacs.^8,21,22 Breckwoldt J et al. describe EPA as the progress of the competency-based curriculum concept in medical education, presenting a more holistic vision of physicians being trained.²³ One core evaluation point in EPA is professionalism of physician trainees in regard to patient, family, society and work colleagues, beyond practical skills for procedures. This part can be considered difficult to evaluate, and its teaching and evaluation is the role of more experienced mentor physicians. Several studies have shown the versatility of EPA and application in providing feedback and evaluation to the apprentice and development of the curriculum.

One way of evaluating the development in the workplace of a physician undergoing specialization is through EPA. This form of evaluation generates valuable feedback for development, which should be provided adequately by mentor physicians. Ideally, they should be trained to provide constructive feedback and point out their purpose without damaging the physician trainee-mentor relationship. Simultaneously, there should be a change in feedback culture, that should stop being seen, both by the physician trainee and the mentor, as something negative, but as a trigger for learning.

Feedback, therefore, can be defined as a catalyzer that transforms evaluation into learning,²⁴ in which information is given to improve the performance of physician trainees according to observation in comparison to expected standards.²⁵ The development of an electronic portfolio (e-portfolio) in a website format or mobile app can help specialty training physicians and the program to monitor feedback and progression throughout the years. Fast and easy access would be useful, in addition to the assessment of procedures or specific activities.

Good-quality feedback has several benefits for the apprentice, such as improvement in interpersonal relations, growth in self-confidence, motivation, competency and team work, improving the quality of patient care.²⁶ Although mentors tend to think that physician trainees do not want to get feedback, the study by Wolpaw, J et al. showed that they do want to get more feedback than they are getting, and that possible retaliation to the staff responsible for negative feedback is not likely to happen.²⁷

A model characterized by an approach in which routine information on the competencies of apprentices is collected and their progress is continuously analyzed is called Programmatic Assessment. When required, it is complemented by additional assessments to collect information purposefully, aimed at informing the maximum to students and mentors, and to allow for high-risk decisions at the end of the training stage.²⁸ In this model, each approach provides constructive feedback to physician trainees.

Regarding EPA, the longitudinal analysis of evaluation leads to a line of development, and, therefore, makes it possible to analyze in which activities and moments physicians undergoing specialization showed gains during learning. In turn, progression related to Milestones has only been described recently in Internal Medicine, showing that, in general, more advanced physician trainees have higher marks than those during initial training.²⁹

A standard evaluation in the learning workplace in Anesthesiology throughout the country may help assure the desired quality in the training of future Anesthesiologists in Brazil. The requirement of sending evaluations to a central committee, like SBA, can make evaluation of physicians undergoing specialization more effective, also facilitating the analysis of progress by trainees themselves and by their programs.

References

• ¹
  Melo RL, Azevedo RA. SBA e CNRM unificam programa para MEs. Anest Rev. 2018;68:35-7.
• ²
  Mulder M. Conceptions of Professional Competence. In: Billet S, Harteis C, Gruber H, editors. International Handbook of Research in Professional and Practice-based Learning. Dor-drecht: Springer; 2014. p. 107-37.
• ³
  Hager P. The competence affair, or why vocational education and training urgently needs a new understanding of learning. J Vocat Educ Train. 2004;56:409-33.
• ⁴
  Martin L, Sibbald M, Brandt Vegas D, Russell D, Govaerts M. The impact of entrustment assessments on feedback and learning: Trainee perspectives. Med Educ. 2020;54:328-36.
• ⁵
  Englander R, Carraccio C. From theory to practice: making entrustable professional activities come to life in the context of milestones. Acad Med. 2014;89:1321-3.
• ⁶
  Wisman-Zwarter N, van der Schaaf M, Ten Cate O, Jonker G, van Klei WA, Hoff RG. Transforming the learning outcomes of anaesthesiology training into entrustable professional activities: a Delphi study. Eur J Anaesthesiol. 2016;33:559-67.
• ⁷
  The anesthesiology milestone project. J Grad Med Educ. 2014;6:15-28.
• ⁸
  McCloskey CB, Domen RE, Conran RM, et al. Entrustable professional activities for pathology: recommendations from the College of American Pathologists Graduate Medical Education Committee. Acad Pathol. 2017;4:2374289517714283.
• ⁹
  ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39:1176-7.
• ¹⁰
  Holmboe ES, Ten Cate O, Durning SJ, Hawkins RE. Assessment Challenges in the Era of Outcomes-Based Education. In: Holmboe ES, Durning SJ, Hawkins RE, editors. Practical Guide th toe Evaluation of Clinical Competence. Philadelphia: Elsevier; 2018. p. 24-78.
• ¹¹
  Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach. 2015;37:983-1002.
• ¹²
  Anesthesiology Specialty Committee. EPA Guide: Anesthesiology. Ottawa: Royal College of Physicians and Surgeons of Canada; 2017.
• ¹³
  ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice?. Acad Med. 2007;82:542-7.
• ¹⁴
  Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013;5:157-8.
• ¹⁵
  Ten Cate O, Hart D, Ankel F, et al. Entrustment decision making in clinical training. Acad Med. 2016;91:191-8.
• ¹⁶
  The American Board of Anesthesiology. Gauging Performance, https://theaba.org/training%20info.html Acesso em 12 de Dezembro de 2020.
  » https://theaba.org/training%20info.html
• ¹⁷
  Sociedade Brasileira de Anestesiologia. https://www.sbahq.org/resources/pdf/arquivos/estatuto/sba/2020/REGULA-MENTO-DOS-CET.pdf, 2020.
  » https://www.sbahq.org/resources/pdf/arquivos/estatuto/sba/2020/REGULAMENTO-DOS-CET.pdf
• ¹⁸
  Royal College of Physicians and Surgeons of Canada. https://www.med.mun.ca/getattachment/65f8411c-aeb4-474d-9923-5a2946900960/Specialty-Training-Requirements-in-Anesthesiol-(1).aspx, 2006,. Acesso em 15 de Agosto de 2020.
  » https://www.med.mun.ca/getattachment/65f8411c-aeb4-474d-9923-5a2946900960/Specialty-Training-Requirements-in-Anesthesiol-(1).aspx
• ¹⁹
  Accreditation Council for Graduate Medical Education (ACGME). https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/040Anesthesiology2019TCC.pdf?ver=2019-03-21-161242-837, 2019.
  » https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/040_Anesthesiology_2019_TCC.pdf?ver=2019-03-21-161242-837
• ²⁰
  Yamamoto S, Tanaka P, Madsen MV, Macario A. Analysis of resident case logs in an anesthesiology residency program. A A Case Rep. 2016;6:257-62.
• ²¹
  Wagner JP, Lewis CE, Tillou A, et al. Use of entrustable professional activities in the assessment of surgical resident competency. JAMA Surg. 2018;153:335-43.
• ²²
  Schumacher DJ, Schwartz A, Zenel JA, et al. Narrative performance level assignments at initial entrustment and graduation: integrating epas and milestones to improve learner assessment. Acad Med. 2020, http://dx.doi.org/10.1097/ACM.0000000000003300
  » http://dx.doi.org/10.1097/ACM.0000000000003300
• ²³
  Breckwoldt J, Beckers SK, Breuer G, Marty A. Entrustable professional activities : Promising concept in postgraduate medical education. Anaesthesist. 2018;67:452-7.
• ²⁴
  Konopasek L, Norcini J, Krupat E. Focusing on the formative: building an assessment system aimed at student growth and development. Acad Med. 2016;91:1492-7.
• ²⁵
  van de Ridder JM, Stokking KM, McGaghie WC, ten Cate OT. What is feedback in clinical education?. Med Educ. 2008;42:189-97.
• ²⁶
  Duffy K. Providing constructive feedback to students during mentoring. Nurs Stand. 2013;27:50-6.
• ²⁷
  Wolpaw J, Saddawi-Konefka D, Dwivedi P, Toy S. Faculty underestimate resident desire for constructive feedback and overestimate retaliation. J Educ Perioper Med. 2019;21:E634.
• ²⁸
  Schuwirth L, van der Vleuten C, Durning SJ. What programmatic assessment in medical education can learn from healthcare. Perspect Med Educ. 2017;6:211-5.
• ²⁹
  Hauer KE, Vandergrift J, Lipner RS, Holmboe ES, Hood S, McDonald FS. National internal medicine milestone ratings: validity evidence from longitudinal three-year follow-up. Acad Med. 2018;93:1189-204.

Publication Dates

• Publication in this collection
  24 May 2021
• Date of issue
  Mar-Apr 2021

History

• Received
  6 Sept 2020
• Accepted
  24 Dec 2020