Evaluation of Residency Admission Exams

Avaliação de Questões de Prova de Concursos de Residência Médica

ABSTRACT

Introduction:

Residency admission exams, although not intended to evaluate medical training, do so in an indirect way. The evaluation of the quality of the medical residency tests allows, among other things, to re-evaluate the training process itself and the skills expected of the candidates.

Objective:

To evaluate first phase exam tests of different medical residency programs in the largest Brazilian urban centers.

Method:

We evaluated 500 questions of residency admission exams in the states of São Paulo, Rio de Janeiro and Minas Gerais. The items were evaluated in terms of their origin, geographical location, area of knowledge, contextualization, context scenarios and complexity by Bloom's taxonomy.

Results:

Most of the questions presented contextualization (64.4%, n = 322), with predominant scenarios of high complexity and in hospital environment. The predominant taxonomic category was identified as recognition (41.60%, n = 208), the second most frequent was judgment, in 26% of the questions (n = 130), followed by synthesis (15%, n = 75), analysis (7.60%, n = 38), comprehension (6%, n = 30) and application (3.8%, n = 19). Considering the dichotomization between questions of theoretical and clinical reasoning, we found a balance between both (clinical reasoning: 48.9%, n = 243; theoretical reasoning: 51.4%, n = 257). The association of contextualization with clinical reasoning was high, with the relative risk of an item requiring clinical reasoning in the presence of contextualization of 26.31 (CI 11.06 – 62.59).

Final considerations:

The scenario outlined by the present research demonstrates that the different selective processes for medical residency in Brazil differ greatly in relation to the selection profile, with hospital-centered focus, favoring scenarios of high complexity in a hospital environment. Although much has been done and discussed in order to promote changes in medical education in Brazil, the selection process for Medical Residency still fails to reflect the changes advocated since the end of the last century and consolidated in the public policies of the beginning of this century. If we consider that the selected professionals are likely to remain at that institution after the end of their undergraduate studies, then we can have some understanding of the feedback cycle that is created in the programs.

KEY WORDS
Internship and Residency; Educational Measurement; Classification; Examination Questions

RESUMO

Introdução:

A prova de residência, apesar de não ter o objetivo de avaliar a formação médica, o faz indiretamente. A avaliação da qualidade das provas de residência médica permite, entre outras coisas, reavaliar o próprio processo de formação e as competências esperadas para os profissionais.

Objetivo:

Avaliar provas de primeira fase de diferentes programas de residência médica dos maiores centros urbanos brasileiros.

Método:

Foram avaliadas 500 questões de provas de residência dos estados de São Paulo, Rio de Janeiro e Minas Gerais. As questões foram avaliadas considerando sua origem, localização geográfica, área de conhecimento, contextualização, cenários do contexto e complexidade pela taxonomía de Bloom.

Resultados:

A maioria das questões apresentava contextualização (64,4%, n = 322), sendo os cenários predominantes de alta complexidade e em ambiente hospitalar. Identificou-se que a categoria taxonômica predominante foi o reconhecimento (41,60%, n = 208), sendo a segunda categoria mais frequente o julgamento em 26% das questões (n = 130), seguidas de síntese (15%, n = 75), análise (7,60%, n = 38), compreensão (6%, n = 30) e aplicação (3,8%, n = 19). Considerando a dicotomização entre questões de raciocínio teórico e clínico, encontramos uma situação de equilíbrio entre ambas (raciocínio clínico: 48,9%, n = 243; raciocínio teórico: 51,4%, n = 257). A associação de contextualização com raciocínio clínico foi alta, com risco relativo de uma questão solicitar raciocínio clínico na presença de contextualização de 26,31 (IC 11,06 – 62,59).

Considerações finais:

O quadro delineado pela presente pesquisa demonstra que os diferentes processos seletivos para residência médica no Brasil diferem muito entre si quanto ao perfil de seleção, com provas de caráter hospitalocêntrico, privilegiando cenários de alta complexidade em ambiente hospitalar. Embora muito se tenha feito e falado no sentido de promover mudanças na educação médica do Brasil, o processo seletivo para residência médica ainda não reflete as mudanças preconizadas desde o fim do século passado e consolidadas nas políticas públicas do início deste século. Se pensarmos que estamos selecionando profissionais que muito provavelmente se fixarão naquela instituição após o fim de sua pós-graduação, podemos ter uma ideia do círculo de retroalimentação que se cria nos programas.

PALAVRAS-CHAVE
Internato e Residência; Avaliação Educacional; Classificação; Questões de Exames

INTRODUCTION

According to the Brazilian Ministry of Education (MEC), the medical school graduate must have a generalist, humanistic, critical and reflexive education and be capable of acting in the health and disease process, with the aim of promotion, prevention, recovery and rehabilitation of health^1,2.

Medical schools have organized curricula to prepare professionals who will provide answers to health needs and problems, seeking primarily to change the diagnosis and treatment generated under the currently accepted disease-centered paradigm³. Moreover, the essential training aspect of general practitioners is that they have an analytical and creative ability to use knowledge in problem solving. The main objective is the training of professionals with autonomy and discernment, in order to ensure comprehensive attention as well as high quality and humanized care for individuals, families and communities^4-6.

Medical residency is a modality of postgraduate education for physicians, in the form of a specialization course, recognized in Brazil as the best form of training specialists⁷. Medical residency programs are regulated and operate within health institutions under the guidance of medical professionals, being considered an effective way of developing graduates abilities and inserting them into a professional setting^8,9. For some authors and institutions, it is considered the cornerstone of specialization (p. 447).

The medical residency, considered the gold standard of medical specialization, is defined as a postgraduate education modality in the form of a specialization course in health institutions under the guidance of highly qualified professionals. The Medical Residency Program, fully accomplished within a specific specialty, gives the resident doctor the title of specialist. ¹⁰

Most medical graduates seek to enter a medical residency, however the number of positions offered in different areas does not always meet the demand, leading to a concern (throughout the course and, more intensely, during the final years of the course) regarding access to this further stage of professional qualification. This process includes everything from preparing for the residency test to choosing an area of specialization (of which there are more than one hundred) and the programs to which a candidate can apply^11,12.

Considering medical residency as a complementary process to graduation, it is expected that its selection process should be aligned to the medical training objectives, requesting situations in which the newly graduated professional can demonstrate knowledge acquired during the course, being capable of clinical reasoning, use medical techniques and have the ability to identify and solve morbid conditions with the adequate application of correct diagnostic methods and appropriate therapy^13,14.

Current Brazilian legislation determines the that applicants must perform a written test - restricted exclusively to the cognitive component of the training - with a minimum weight of 50%, followed by an optional second phase, composed of a practical test, worth 40% to 50%, and an interview with weight of 10% of the overall grade¹⁵.

The majority of Brazilian institutions prioritize the written test in residency selection processes. This exam might indicate a process centered on memorization and using multiple choice items. As noted by Hamamoto Filho and Zeferino¹⁶ (p. 551):

This new way of “learning medicine” – or rather of memorizing answers to a test – certainly does not meet the needs of a critical and reflective education, threatening medical training.

The overlapping of undergraduate training and preparing for medical residency admission, the impact of the residency program on one's medical career, and the widespread, hegemonic adoption of this model nationwide are factors that have an ominous effect on the student and, consequently, on undergraduate medical training^10,16.

Residency admission exams, although not aimed at evaluating medical education, do so in an indirect form, as they intend to measure the knowledge, skills and abilities of medical graduates¹⁷. From this perspective, evaluating the quality of medical residency admission tests allow us, among other things, to re-evaluate the training process itself and the skills expected of the professionals.

The purpose of this study is to evaluate the first phase questions of different medical residency programs in the largest Brazilian urban centers, aiming to assess their adequacy to the expected competencies of the profession and public health policies.

METHOD

In Brazil, hundreds of institutions offer medical residency vacancies, approximately half of which are concentrated in the southeast region¹⁸. In order to consider relevant programs in that region, direct access tests (those applied to candidates for first year residency) were selected from three states in the Southeast region of Brazil (São Paulo, Rio de Janeiro and Minas Gerais) due to the highly concentrated number of residencies in these areas¹⁹. Institutions were chosen that offered the highest number of available positions and had published their exam papers. The programs chosen were: Faculdade de Medicina da USP de Ribeirão Preto (USP Medical School of Ribeirão Preto), Sistema Único de Saúde de São Paulo (Unified Health System of São Paulo), Fundação Hospitalar do Estado de Minas Gerais (Hospital Foundation of the State of Minas Gerais), Secretaria Estadual de Saúde do Rio de Janeiro (State Health Department of Rio de Janeiro) e Universidade Federal do Rio de Janeiro (Federal University of Rio de Janeiro). The tests used were those of the 2011 selection process²⁰.

The year 2011 was chosen due to the presumed remoteness of the candidates from the events related to the selection process, whether they were successful or not. Such distance allows the critical analysis of the evaluation processes to be carried out without compromising the institutions or medical residency programs. On the other hand, in 2014, the Brazilian Ministry of Education released new Curricular Guidelines that imposed changes on medical education, mostly in internship and core curricular areas. The authors believe that this change in the guidelines holds little effect on the residency selection process, but this historical fact must be noted as a drawback in the present study.

The questions were evaluated considering their origin, geographical location, area of knowledge, contextualization, context scenarios and complexity. The evaluation was performed in pairs, after which the analysis was re-grouped and the database was checked for discrepancies with the participation of the entire research team.

Complexity relates to the degree of reasoning and abstraction needed to answer the question and was defined in Bloom's Taxonomy²¹, being categorized as Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation.

Considering Bloom's Taxonomy levels, the questions whose objective is to evaluate the ability to remember information and contents are classified as knowledge; comprehension items involve the ability to understand and give meaning to content; questions that prioritize the ability to use information and methods in concrete situations are categorized as application; analysis assumes the ability to subdivide the content to understand its structure; aggregation and combination of the parts so as to form a “new whole” sets up the synthesis and finally the evaluation as the ability to judge the concepts and situations from defined criteria²¹.

Besides complexity, another important factor in the quality of issues is contextualization. The presence of contextualization demonstrates the effort to correlate theory with practice, signifying the learning process. Through contextualization, the student is led to reflect on a concrete situation, and is able to understand how knowledge is applied in practice and places him or herself in the position of the professional who must choose a position or solve a problem^22,23. The scenario used for contextualization was stratified into levels of attention based on the principles described in the National Curricular Guidelines and in the current public policies, classified as: Primary Care, Outpatient Specialties, Urgency/Emergency, Hospital or None.

Database modeling dichotomized complexity as theoretical reasoning (knowledge, understanding and application) and clinical reasoning (analysis, synthesis and judgment). The results of the analysis were inserted into a database and submitted to univariate and bivariate analysis of their frequencies and analyzed by the chi-square test. For the dichotomous variables, calculation of relative risk was performed as this measure of association is considered more appropriate to sectional studies than the odds ratio^24,25.

The present study obtained submission clearance from the Ethics Committee on Human Research of the Centro Universitário de Volta Redonda (UniFOA), which is subordinate to the National Commission for Research Ethics, of the Ministry of Health²⁶.

RESULTS AND DISCUSSION

Five exams from the year 2011 used in the selection processes for medical residencies in the state of São Paulo (Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo - FMRP-USP - and Secretaria Estadual de Saúde de São Paulo - SES-SP), Rio de Janeiro (Universidade Federal do Rio de Janeiro - UFRJ - and Secretaria Estadual de Saúde do Rio de Janeiro - SES-RJ) and Minas Gerais (Processo Seletivo de Residência Médica de Minas Gerais - PSU-MG) were assessed. Each exam was composed of 100 questions, totaling 500 questions for analysis.

In terms of areas of knowledge, the highest incidence was Internal Medicine (24.6%, n = 123), followed by Pediatrics (20.2%, n = 101), Gynecology and Obstetrics (19.6%, n = 98) Public Health (15%, n = 75), Surgery (14.6%, n = 73) and Basic Sciences (6%, n = 30) (Figure 1).

Figura 1
Scenario prevalence in contextualized questions

The predominance of internal medicine as the most discussed topic is understandable when considering the comprehensive content of this area, as well as its points of interface with other areas. Likewise, there was very little content exclusively from basic areas found in the evaluated tests. These data are justified as they reflect the knowledge on which the clinical competences of the professionals will be built on.

In analysis of the contextualization aspect, contextualized questions predominated (64.4%, n = 322). When analyzing the scenarios used, the most prevalent was the Hospital (26.03% n = 82), followed by Outpatient Specialties (25.71% n = 81) and Urgency and Emergency unit (24.84% n = 80). Primary Care was the scenario for only 4.97% (n = 16) of the questions with contextualization (Figure 2). Of these, 19.6% (n = 63) present a case or problem situation, without setting a specific scenario.

Figura 2
Scenario prevalence in contextualized questions

It is important to emphasize that the overwhelming majority of the scenarios presented to the candidates are still set in highly complex contexts (Urgency/Emergency and Hospital), making up half the scenarios presented. On the other hand, Primary Care scarcely features. These data show how the hegemony of the hospital-centered model, focused on procedures and curative medicine, remains unchanged in Brazilian medical education, despite all the changes that have taken place in the public health policies over the last decades.

In addition to the hospital-centered trend mentioned above, Table 1 demonstrates the low prevalence of basic care in all areas. Even in Public Health, an area in which primary care would (theoretically) be more important, this scenario is only presented in 25% (n = 10) of the contextualized issues. Furthermore, it would be expected that other areas presented cases and problems of basic attention in a more relevant way, considering the importance of this scenario in current public health policies.

Table 1
Relationship between knowledge area and the scenario contextualization

Scenario	Outpatient Specialties		Primary Care		Hospital		Urgency/Emergency		None		Total
Area	n	%	n	%	n	%	n	%	n	%	n	%
Basic Sciences	0	0.0	0	0.0	0	0.0	0	0.0	13	100.0	13	4.0
Surgery	11	18.6	0	0.0	20	33.9	26	44.1	2	3.4	59	18.3
Internal Medicine	22	27.5	4	5.0	22	27.5	24	30.0	8	10.0	80	24.8
OBGYN	31	48.4	0	0.0	17	26.6	10	15.6	6	9.4	64	19.9
Pediatrics	17	25.8	2	3.0	22	33.3	18	27.3	7	10.6	66	20.5
Public Health	0	0.0	10	25.0	1	2.5	2	5.0	27	67.5	40	12.4
Total	81	25.2	16	5.0	82	25.5	80	24.8	63	19.6	322	100.0

As the taxonomy of the questions was evaluated, Knowledge was identified as the predominant taxonomic category (41.60% n = 208), the second most frequent being Evaluation in 26% of the questions (n = 130) followed by Synthesis (15 % N = 75), Analysis (7.60% n = 38), Comprehension (6% n = 30) and Application (3.8% n = 19). Considering the dichotomization between questions of theoretical and clinical reasoning, we found a balance between both (Clinical reasoning 48.9% n = 243 - Theoretical reasoning 51.4% n = 257). However, the stratified analysis showed a discrepancy between the prevalence of questions of clinical reasoning among the different programs analyzed, with rates varying from 68% (FMUSP-RP) to 28% (SES-SP). Full data on the taxonomy of the programs can be found in Table 2.

Table 2
Type of reasoning required per Medical Residency program

Reasoning	Theoretical		Clinical
Program	N	%	n	%
SES-RJ	62	62.0	38	38.0
UFRJ	45	45.0	55	55.0
SES-SP	72	72.0	28	28.0
FMRP-USP	32	32.0	68	68.0
PSU-MG	46	46.0	54	54.0
Total	257	51.4	243	48.6

Although the general picture shows a certain balance between the requirements of both categories, the stratified analysis per institution shows a wide variation. Based on these data, it is possible to infer that programs may present sensitive differences in the candidate selection profile. Candidates more focused on memorization will be more likely to perform better in exams centered on conceptual aspects. Although one may argue that candidates with clinical reasoning would perform better in both situations, we should bear in mind that conceptual evidence tends to demand content of little clinical relevance, which may imply a worse performance of this type of candidate compared to memorizers.

As expected, the association of contextualization with clinical reasoning was high, with a relative risk of a question demanding clinical reasoning in the presence of contextualization of 26.31 (CI 11.06 – 62.59). On the other hand, the presence of context does not always imply that this type of reasoning is required. Of the 315 contextualized questions, more than a quarter (26.67% n = 84) did not demand clinical reasoning of the candidate. Considering that candidates face long exams (100 questions) and that the “pseudocontext” does not influence the resolution of the question, these situations only consume time and attention in their resolution, functioning as real distractors in the evaluation (Table 3).

Table 3
Relation between context and reasoning

Reasoning	Theoretical		Clinical		Total
Context	n	%	n	%	n	%
Contextualized	84	26.1	238	73.9	322	64.4
Non contextualized	173	97.2	5	2.8	178	35.6
Total	257	51.4	243	48.6	500

The comparison between programs offered by university institutions and non-university institutions reveals a higher concentration of questions of clinical reasoning in the former category (61.5% n = 123 versus 40.0% n = 120). Comparing the two incidences we obtained a relative risk of 1.53 (CI – 1.2 – 1.83) of obtaining this type of question in a test. This fact can be explained by the fact that such institutions are more involved with teaching and more often follow the changes proposed in Brazilian medical education, while non-university institutions can still reproduce old and settled evaluative models.

Another difference identified when comparing the contents of exams from different institutions was related to the scenarios addressed in the contextualized issues in each program, and this difference was especially sensitive in relation to basic care (Table 4). These data clearly demonstrate that some institutions prioritize, even unintentionally, a highly complex professional profile.

Table 4
Distribution of contextualized scenarios in programs

Scenario	Outpatient Specialties		Primary Care		Hospital		Urgency/Emergency		None
Area	n	%	n	%	n	%	n	%	n	%
FMUSP-RP	31	38.3	9	56.3	16	19.5	20	25.0	12	19.0
PSU-MG	20	24.7	7	43.8	12	14.6	19	23.8	0	0.0
SES-RJ	14	17.3	0	0.0	5	6.1	12	15.0	22	34.9
SUS - SP	16	19.8	0	0.0	8	9.8	18	22.5	18	28.6
UFRJ	0	0.0	0	0.0	41	50.0	11	13.8	11	17.5
Total	81	100.0	16	100.0	82	100.0	80	100.0	63	100.0

The presence of Primary Care in contextualized questions reveals a significant discrepancy in only two programs, with merely 16 questions concentrated in two institutions. In contrast to this, we see a profusion of hospital situations, to the detriment of even outpatient settings.

FINAL CONSIDERATIONS

We hope that the present study can stimulate discussions about the evaluation processes practiced in the scope of Brazilian medical education, allowing reflections on the routine of teachers and students in Brazil and supporting modifications to the processes, as well as teacher qualification programs in evaluation.

An obvious limitation of the present study lies in its temporal and geographical characteristics. Future surveys that evaluate the issue in other years and regions may contribute to a more detailed picture of the selection processes across the country. As noted before, one may argue that the new guidelines published in 2014 might have modified the way residency institutions select their candidates, but this does not seem to have happened in relation to those guidelines published in 2001. Furthermore, changes implemented in 2014 will take at least six years (in the best case scenario) to reach the residency programs.

The exams analyzed, despite having the same number of questions, display diverse characteristics regarding the construction of questions, varying in the content addressed, in the presence (or absence) of contextualization, in scenario contexts and in the complexity required. Such diversity makes this process more complicated for candidates, who generally participate in many different processes at several institutions, further strengthening, albeit indirectly, participation in preparatory courses for medical residency exams, a facility widely used by medical students.

The context outlined by the present research demonstrates that the different selective processes for medical residency in Brazil differ greatly from each other regarding the selection profile, especially if we consider the professional profile promoted by Brazilian public policies, which prescribes the critical and reflective physician as the result of a learning process that suits the needs of our community.

The medical residency exam is a process that is independent of the existence of any medical training at some institutions, and is often performed at exclusive hospital institutions, that is to say, without offering a medical course or any other undergraduate qualification in health. Unlike undergraduate institutions, hospital-based institutions have not been driven to change the focus of medical training, but rather represent niches of a traditional practice, focused on content, disciplines and compartmentalized knowledge. This mentality is clearly evident in the medical residency selection tests, which present a collection of decontextualized or pseudocontextualized content that does not require the candidate to exercise clinical reasoning necessary for the good medical professional.

Some institutions - both university and non-university - present hospital-centered evidence, favoring high complexity scenarios in a hospital environment. Although some may argue that the test selects future specialists, whose practice scenario may be much more focused on the hospital, we recall that most of the nosological entities treated by these professionals will be managed as outpatients for a far longer period of time than in the hospital. Moreover, it is necessary to bear in mind that Primary Care represents the priority locus for disease prevention throughout the entire health network. If the system does not recognize it as such and does not value it in formulating a selective process, then we can infer that this devaluation permeates all training practice from undergraduate training to medical residency, reinforcing the paradigm of curative medicine, focused on high complexity and hospital procedures.

Although much has been done and discussed in order to promote changes in Brazilian medical education, the medical residency selection process still fails to reflect the changes advocated since the end of the last century and consolidated in the public policies of the beginning of this century. If we consider that the selected professionals are likely to remain at that institution after the end of their undergraduate studies, then we can have some understanding of the feedback cycle that is created in the programs

REFERENCES

• ¹
  Brasil. Ministério da Educação. Diretrizes curriculares nacionais do curso de graduação em medicina. Resolução CNE/CES n°4, de 7 de novembro de 2001. 2001. p.47.
• ²
  Brasil. Ministério da Educação. Diretrizes curriculares nacionais do curso de graduação em medicina. Resolução n°3, de 20 de junho de 2014. 2014. p.47.
• ³
  Souza PMM, Nunes CA, Silveira CS, Nóbrega-Therrien SM. Integração ensino-pesquisa na educação médica: perfil docente de um colegiado. Revista Brasileira de Educação Médica. 2012;36(1):14-23.
• ⁴
  Queiroz SJ, Andrade M, Casas ALF, Santos GMD, Chiochetti NB. Ligas Acadêmicas e Currículo Médico: relações e relato de experiência. Fragmentos de Cultura. 2015;25(1):47-55.
• ⁵
  Lampert J. Tendências de mudanças na formação médica no Brasil. 2.ed. São Paulo: Hucitec/Associação Brasileira de Educação Médica; 2009.
• ⁶
  Lampert JB, Rego S, Araujo JGC. Educação médica em transformação: instrumentos para a construção de novas realidades. Editora Hucitec; 2004. 390 p.
• ⁷
  Scheffer M, et al. Demografia médica no Brasil 2015 [Internet]. São Paulo: Faculdade de Medicina da USP. Conselho Regional de Medicina do Estado de São Paulo. Conselho Federal de Medicina; 2015 [cited 2015 Dec1]. Available from: http://www.flip3d.com.br/web/temp_site/edicao-c6a01432c8138d46ba39957a8250e027.pdf
  » http://www.flip3d.com.br/web/temp_site/edicao-c6a01432c8138d46ba39957a8250e027.pdf
• ⁸
  Brasil. Ministério da Educação. Lei n° 6.932, de 7 de julho de 1981. [Internet]. 6.932 Jul 7, 1981. Available from: http://www.planalto.gov.br/ccivil_03/Leis/L6932.htm
  » http://www.planalto.gov.br/ccivil_03/Leis/L6932.htm
• ⁹
  Brasil. Ministério daEducação. Decreto-80281de05/09/1977-Publicação: Diário Oficial da União de 06/09/1977 [Internet]. 80281 May 9, 1977. Available from: http://legis.senado.gov.br/legislacao/ListaNormas.action?numero=80281&tipo_norma=DEC&data=19770905&link=s
  » http://legis.senado.gov.br/legislacao/ListaNormas.action?numero=80281&tipo_norma=DEC&data=19770905&link=s
• ¹⁰
  Leite ICG, Teixeira MTB, Neves HS, Oliveira LRS, Garcia LAO, Cunha PHM. Avaliação da efetividade dos cursos preparatórios para residência médica. Rev bras educ med. 2008;32(4):445–51.
• ¹¹
  Ribeiro MAA. Apontamentos sobre residência médica no Brasil. Brasília (DF): Câmara dos Deputados-Consultoria Legislativa [Internet]. 2011 [cited 2016 Oct 25]; Available from: http://www2.camara.leg.br/documentos-e-pesquisa/publicacoes/estnottec/areas-da-conle/tema11/2011_123.pdf
  » http://www2.camara.leg.br/documentos-e-pesquisa/publicacoes/estnottec/areas-da-conle/tema11/2011_123.pdf
• ¹²
  Silva SM, Rosa VF, Brandão PRP, Oliveira AC, Oliveira PG, Sousa JB. Cursos preparatórios para a residência médica: visão dos estudantes de medicina. Revista do Colégio Brasileiro de Cirurgiões. 2011;38(5):349–354.
• ¹³
  Neto JAC, Sirimarco MT, Bicalho TC, Araújo FCC, Resende LO, Balbi GGM. Conhecimento dos estudantes de Medicina sobre a avaliação curricular padronizada no processo seletivo da residência médica. Rev Med Minas Gerais [Internet]. 2012 [cited 2016 Oct 25]; Available from: http://rmmg.org/artigo/detalhes/88
  » http://rmmg.org/artigo/detalhes/88
• ¹⁴
  Perim GL, Abdalla IG, Aguilar-da-Silva RH, Lampert JB, Stella RCR, Costa NMSC. Desenvolvimento docente e a formação de médicos. Revista Brasileira de Educação Médica. 2009;33:70–82.
• ¹⁵
  Brasil. Resolução n°1 de 25 de maio de 2015 [Internet]. 1 May 25, 2015. Available from: http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=20741-res01-25052015-cnrm-regulamenta-requisitos-pdf&category_slug=setembro-2015-pdf&Itemid=30192
  » http://portal.mec.gov.br/index.php?option=com_docman&view=download&alias=20741-res01-25052015-cnrm-regulamenta-requisitos-pdf&category_slug=setembro-2015-pdf&Itemid=30192
• ¹⁶
  Hamamoto Filho PT, Zeferino AMB. Cursinhos preparatorios para residencia médica: reflexões sobre possíveis causas e consequências. Revista Brasileira de Educação Médica [Internet]. 2011 [cited 2016 Oct 27]; Available from: http://unicamp.sibi.usp.br/handle/SBURI/29217
  » http://unicamp.sibi.usp.br/handle/SBURI/29217
• ¹⁷
  Goldwasser R, Fonseca V, Lobo MS, Coelho A, Santos EG, Pereira SMP. Seleção para a residência médica da Universidade Federal do Rio de Janeiro: percepção dos candidatos sobre o modelo da prova. Rev Bras Educ Med. 2009;33(1):115-121.
• ¹⁸
  Nunes MPT. Residência médica no Brasil: situação atual e perspectivas. Cad ABEM. 2004;1:30-2.
• ¹⁹
  Chaves HL, Borges LB, Guimarães DC, Cavalcanti LP de G. Vagas para residência médica no Brasil: Onde estão e o que é avaliado. Rev bras educ méd. 2013;37(4):557-65.
• ²⁰
  Barbosa AGB, Caldeira SD. Sic Provas na Integra Brasil 2011/2012. 1st ed. MEDCEL EDITORA; 2011.
• ²¹
  Ferraz APCM, Belhot RV. Taxonomia de Bloom: revisão teórica e apresentação das adequações do instrumento para definição de objetivos instrucionais. Gestão & amp; Produção. 2010;17(2):421-31.
• ²²
  Pereira LC, Queirós WP, Santos SX. Estado da arte do conceito de contextualização em periódicos da área de ensino de Ciências. Anais do Congresso de Ensino, Pesquisa e Extensão da UEG [Internet]. 2016 Mar 13;2. Available from: http://www.anais.ueg.br/index.php/cepe/article/view/5874
  » http://www.anais.ueg.br/index.php/cepe/article/view/5874
• ²³
  Troncon LEDA. Avaliação do estudante de medicina. Medicina (Ribeirao Preto Online). 1996;29(4):429.
• ²⁴
  Diaz-Quijano FA. A simple method for estimating relative risk using logistic regression. BMC Med Res Methodol. 2012;12:14.
• ²⁵
  Sato T. Risk ratio estimation in case-cohort studies. Environ Health Perspect. 1994;102 (Suppl 8):53-6.
• ²⁶
  Cury SE. Dispensa de submissão. 2016.

Publication Dates

• Publication in this collection
  Apr-Jun 2018

History

• Received
  25 Mar 2017
• Accepted
  27 Apr 2017