length Of inteRnship influences peRfORMance On Medical Residency exaM

744 Rev Assoc Med Bras 2009; 55(6): 55(6): 744-8 *Correspondência: Av. Dr. Arnaldo, no 455 sala 2342 São Paulo SP CEP: 01246-903 Tel/Fax: (11) 3082 4076 Email: joaquimev@usp.br Summary Objectives. Medical education encompasses globally diverse context and conditions. The Brazilian scenario seemed a natural environment to study the influence of medical education programs and internship duration on the entrance exam for medical residency. This investigation evaluates some methods used during the entrance exam for medical residency as a means to make a distinction between candidates with longer clerkships. MethOds. Candidates selected for a residency program performed a multiple-choice (MC), an open question (OQ) and OSCE-like tests, an interview and a curriculum analysis for participation in scientific meetings, papers published and voluntary activities. Groups were compared for gender, year of graduation, tests and OSCE scores. Results. Participants were distributed into two groups based on clerkship duration: 2 years or less than 2 years. There was no difference for the MCT score among groups or any of the activities from interview and curriculum analysis. The 2 years clerkship group showed significantly higher OQ (p=0.009) and OSCE-like affective (p=0.025) and knowledge (p=0.002) scores. cOnclusiOn. The OSCE test identified some aspects related to competence acquisition and assessed basic skills and attitudes essential to the supervised practice of medicine during residency. OSCE discriminated aspects not perceived by the sole use of knowledge tests


intROductiOn
To a large extent, medical education is often perceived as globally comparable..However, context and conditions are usually diverse 1 .Programs have included from one to two or even three years of internship.
During undergraduate years, students undergo assessment processes by a wide variety of tests, from multiple-choice (MCT) to short answers, essay questions (EQ), and oral examinations.On the other hand, assessment of clinical competence is becoming increasingly complex as well as demanding since it is patient centered and student driven 2 .Non-cognitive goals of medical education that include values, attitudes and skills continue do be difficult to define and even more complicated to measure.
Teamwork and interprofessional skills; duty and responsibility; communication and interpersonal skills; professionalism and values; and trustworthiness and ethical behavior were all considered important non-cognitive goals in a recent survey and could be designated as Professionalism 3 .The OSCE (Objective Structured Clinical Examination) has been proposed as a better way to evaluate competences and perhaps even one of the best 4 .
The current Brazilian scenario for medical education includes a large number of new medical schools.length of internship among traditional and new institutions may vary.There is evidence in literature as well as some opinion from educators recommending longer clerkships as a method to improve skills for the practice of medicine 5,6 .The background of medical education in Brazil seems a natural environment to study the influence of a range of programs and the internship duration on performance of the entrance exam for medical residency at the University of São Paulo Medical School.
The purpose of this observational study was to evaluate the capacity of some tests used in the residency entrance exam to identify candidates with longer clerkships during graduation.

MethOds
A multiple-choice test (MCT) and short-answer open questions (OQ) exam selected candidates for an Internal Medicine residency program at University of Sao Paulo Medical School.Performance of these candidates is described in this investigation.All candidates received written information for the exam.Their anonymity was ensured by a serial number on the application form.
Candidates selected were submitted to an OSCE-like test, an interview and curriculum analysis.The OSCE-like test consisted of five station sections contemplating the core areas (Internal Medicine, Obstetrics&Gynecology, Pediatrics, Public Health, Surgery): 1 -an upper digestive bleeding presentation, 2 -a supraventricular tachycardia, 3 -a community-acquired pneumonia, 4 -a N. meningitidis meningitis clinical case documented in a school and 5 -a pregnancy follow-up.In sections 2 to 5, candidates interacted with a professional actor/actress.Evaluation of the candidates encompassed two competences: cognitive associated with psychomotor abilities and affective abilities desirable to approach clinical situations (eye-to-eye contact, calming an anxious mother, explaining an electrical cardioversion procedure).
The interview with candidates sought evidences of participation in activities such as scientific meetings and paper/abstract presentations and published papers, as well as voluntary activities in patient care.
All test results were transformed into a score of 0 to 1000 points.The OSCE-like test score was subdivided.Those based on cognitive associated with psychomotor abilities were labeled OSCE knowledge and those based on affective abilities, OSCE affective scores.

Statistical analysis
Groups were compared for gender, year of graduation, MCT, OQ, OSCE knowledge and OSCE affective scores.Student t test, Chi-square and Mann-Whitney U tests were used accordingly from a statistical package (SPSS 10.0 for Windows).Candidates were arranged into quartiles for a post-hoc statistical analysis.

Results
Five hundred and seventy eight candidates applied to the residency entrance exam that selected 111 candidates (19.2%) by means of results from MCT and OQ.These candidates (45% male) had an undergraduate education with clerkships of 1 year (8.1%), 1.5 years (29.7%) or 2 years (62.2%).For statistical purposes, they were distributed into two groups: with clerkship undergraduate programs of 2 years or less than 2 years.Most applicants finished their medical education just before taking the exam (76.6%) (Table 1).
There was no difference for the MCT score among groups.The 2 years clerkship group showed a significantly higher OQ score (740 vs. 720 points, p=0.009,Mann-Whitney U test).The OSCE-like results revealed a higher score for the 2 year clerkship group, for the affective (850 vs. 793 points, p=0.025) and the knowledge scores (808 vs. 736 points, p=0.002) (Table 2).
Performance during the OSCE-like exam could not be predicted from results in the MCT or OQ, nor from the previous clerkship duration.The OSCE scores were higher when comparing the two clerkships groups (Table 3).These groups did not differ for any of the activities assessed during the interview and curriculum analysis (Table 4).

discussiOn
This investigation suggests that OSCE is reliable to differentiate aspects difficult to detect in multiple choice tests.It is an exam technique capable of assessing basic skills and attitudes essential to the practice of supervised medicine that characterizes medical residency.
Literature on residency selection criteria appears to be limited, and perhaps very specific according to national educational systems.General tests, USMLE or related tests' scores, ranking in medical school and interview performance may be reported.Notwithstanding, there are different perspectives between applicants and program directors about what might be more important or valued for residency admission 7 .Also, assessment exercises and interview performance could

Clerkship duration p
All (n=111) be related to selecting doctors for postgraduate training in pediatric medicine, but erformance provided a greater breadth and depth of information about candidates than the structured interview 8 .Results obtained in this sample showed no difference in the knowledge exam with MCT, but there was a better performance with open questions and the OSCE, among applicants who had had a longer internship.The non-cognitive attributes, although considered equally important for admission to residency programs, are also the subject of debate on recruitment and selection.There is an ongoing debate regarding ways to promote student recruitment and selection, aiming for a better match between the medical student population and the healthcare requirements of the population.This includes greater access for applicants from underprivileged areas and lower socio-economic groups 9 .
A review of pertinent literature pointed to the low discrimination capacity of grade point average and Medical College Admission Test scores because these offer no measurement of important non-cognitive attributes 10 .The OSCE provided a method of examining the skill acquisition of medical students 11 .However, effects of such assessment on performance of in-training physicians could still be under evaluated 12 .Brazilian medical education has had different conditions for offering internship among medical schools.Nowadays, this education system faces a national turmoil about implementation of new medical schools without appropriate evaluation of their need and even their quality, according to some professional associations. 13Some schools are offering internships that vary from 1, to 1.5 and 2 years.This scenario, as previously stated, seemed to be a natural environment for study of the influence of such a variety of internship durations upon the performance in the entrance exam for medical residency.
Development of practical methods for assessment of selection for residency programs could identify candidates with extensive skills and better attitudes, The OSCE has been related to these possibilities, although it is not yet a complete test 14 .Results presented here point to a reasonable understanding that OSCE-based admission exams, despite being time-consuming and more expensive, could focus on the objective of encouraging students to participate more intensely in daily practical activities during internship or even before 15 .
In addition, this model of exam may also influence medical teachers to review teaching plans, directing more energy to practical activities and attitude counseling.Contrary to such efforts, a previous report stressed that cultural aspects and lack of a favorable educational environment could delay expected improvements in traditional medical schools that adopt new assessment plans 16 .On the other hand, those teaching efforts may be easily recognized by resident physicians who pointed to quality of teaching as the best indicator when evaluating a learning environment 17 .
This investigation advocates that exams for residency selection need to emphasize and enhance importance of competence acquisition, besides knowledge content but distinct from scientific production or received honors.In addition, some authors have found data suggesting that performance during residency training was not predicted by scientific production.There are evidences revealing that personal qualities, performance during medical studies and clinical skills are better for predicting higher achievement during residency, in addition to a mentoring/coaching program as a powerful factor for career success, supporting the quality of teaching 4,17,18 .
The present study has some limitations.Sample size was not adequate to perform the proposed post-hoc analysis, resulting in non-significant tendencies.Also, it only focused on candidates for the internal medicine residency program, from whom full data could be obtained.

cOnclusiOn
In conclusion, this investigation suggests that OSCE is a reliable instrument to distinguish some aspects difficult to perceive when using tests that measure knowledge or even from open questions.The OSCE can be an exam technique to assess basic skills and attitudes that are essential for the practice of medicine under supervision that characterizes medical residency.

Table 3 -OSCE-like test performance according to grades in open-question test and clerkship duration (CD).
Percentile range is related to the first exam result (MCT+OQ); IQ -interquartile; 2y -two-years clerkship; <2y -less than two-years clerkship.*p-values for comparison between 2-year and less than 2-year CD groups.