Evaluation and classification of residence programs in
                    Otorhinolaryngology

Crespo, Agrício Nubiato

doi:10.5935/1808-8694.2013S001

Acessibilidade / Reportar erro

Brasil

Brazilian Journal of Otorhinolaryngology

Español English

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

• Braz J Otorhinolaryngol. 79 (5 Suppl 1) • Sep-Oct 2013 • https://doi.org/10.5935/1808-8694.2013S001 copy

Evaluation and classification of residence programs in Otorhinolaryngology

Authorship SCIMAGO INSTITUTIONS RANKINGS

Residence training is defined in Brazil as a full time learning in practice process, developed in public health institutions, private hospitals and clinics, under surveillance of qualified medical staff.

OBJECTIVE:

To present the development process of a protocol of evaluation of residence programs in otolaryngology and its classification by quality.

METHOD:

Design a comprehensive protocol of evaluation to cover the broad aspects of medical education in otolaryngology. Classify the training programs by quality. Evaluate board certification performance of its residents. Analyze the correlation between program quality classification status and residents scores at the board certification exam.

RESULTS:

Eighty two residence programs were evaluated across the country in 2004, as follows: level A (12.20%), B+ (7.3%), B (19.5%), C+ (20.7%), C (17.1%), D (20.7%) and E (2.4%) p < 0.005.

CONCLUSION:

The evaluation program was able to discriminate, quality-wise, the training programs. The grades from those who passed the ABORL-CCF Board's Exam have the same ranking trend as that of their training institution. There was an improvement in the ranking of training programs after the program was implemented.

education; education, medical, undergraduate; internship and residency; specialization

Associação Brasileira de Otorrinolaringologia e Cirurgia Cérvico-Facial. Sede da Associação Brasileira de Otorrinolaringologia e Cirurgia Cérvico Facial, Av. Indianópolia, 1287, 04063-002 São Paulo/SP Brasil, Tel.: (0xx11) 5053-7500, Fax: (0xx11) 5053-7512 - São Paulo - SP - Brazil
E-mail: revista@aborlccf.org.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Send correspondence to: Agrício Nubiato Crespo, Av. Heitor Penteado, nº 1541. Taquaral, Campinas - SP. Brazil. CEP: 13087-000

1 - Program Venue	Value
a) Within a Medical School	10*
Within a hospital with internship	6*
Within a general hospital	4*
In a private clinic	1*
		Score
b) Program held together with medical residency programs from other specialties:
Yes	(5*)
	(0*)
		Score
c) Complexity levels:
Primary	2*
Secondary	4*
Tertiary	6*
		Score
d) Region (geographic distribution)
North	10*
Northeast	7.5*
Midwest	5*
South	2.5*
Southeast	0*
		Score
2 - Scope, support services and special programs:
Oral mobility rehabilitation	1*
Neurotology	1*
Program in the field of hearing impairment	1*
Hearing aid fitting	1*
Cochlear implant	1*
Oto-neurosurgery	1*
Neonatal hearing screening	1*
Occupational otology	1*
Vestibular rehabilitation	1*
Language disorders rehabilitation	1*
Endoscopic sinus surgery	1*
Allergy program	1*
Voice rehabilitation program	1*
Facial framework surgery	1*
Stenoses	1*
Nasal plastic surgery	1*
Facial plastic surgery	1*
Ear plastic surgery	1*
Head and neck surgery	1*
Rehabilitation program	1*
Snoring and obstructive sleep apnea	1*
Laser surgery	1*
Traumatology	1*
Stomatology	1*
Skull base surgery	1*
		Grade

1 - Outpatient ward
Number of stalls or offices per resident	1 for 2 resid.	1 for 3	1 for 4	1 for more
	4*	3*	2*	1*	Score
	Yes	No
Treatment room	1*	0*
Nursing station in annex	1*	0*
ENT urgency room	1*	0*
Pathology department in the institution	1*	0*
CT scan in the institution	1*	0*
MRI in the institution	1*	0*
ICU in the institution	1*	0*
Clinical analysis lab in the institution	2*	0*
					Score
2 - Teaching and scientific activities
Number of international journals	More than 5	3 to 5	1 to 3	None
	3*	2*	1*	0*	Score
	Yes	No
Library	1*	0*
Internet access	1*	0*
Video library	1*	0*
Auditory or meeting room	1*	0*
Multimídia capabilities	1*	0*
Dissection lab	2*	0*
					Score
	Regular	Esporadic	No
Promotion of scientific events	2*	1*	0*
					Score
3 - Outpatient ward equipment
	Yes	No
Microscope	1*	0*
Endoscopy system	1*	0*
					Score
4 - Additional tests
	Three	Two	One
Audiometrie tests booth	3*	2*	1*		Score
	Yes	No
Endoscopy system	2*	0*
Voice laboratory	2*	0*
ABR	2*	0*
ECOG	2*	0*
Otoacoustic emissions	2*	0*
Vecto-electronystagmography	2*	0*
Stroboscopy	2*	0*
Image documentation (video, photo)	2*	0*
Polysomnography	2*	0*
					Score
5 - Surgical equipment of teaching usefulness
	Yes	No
Microscope with a camera	1*	0*
Videoendoscopy system	1*	0*
					Score
			Total score in item II

1 - Number of instructors per first-year resident (R1)
	Value 1*	2*	3*	4*	5*
	3:01 ( )	4:01 ( )	5:01 ( )	6:01 ( )	+ 6:1 ( )
	4:02 ( )	5:02 ( )	6:02 ( )	7:02 ( )	+ 7:2 ( )
	5:03 ( )	6:03 ( )	7:03 ( )	8:03 (x)	+ 8:3 ( )
	6:04 ( )	7:04 ( )	8:04 ( )	9:04 ( )	+ 9:4 ( )
	8:05 ( )	9:05 ( )	10:05 ( )	11:05 ( )	+ 11:5 ( )
	10:5 + ( )	12:5+ ( )	14:5 + ( )	16:5 + ( )	+ 16:5 + ( )
						Score
2 - Faculty titling
		N².	Value
	N². of MSc.		1*
	N². of MSc.		3*
	N². of PhDs		4*
	N². of PhDs		8*
Total N². of Assistants						Score
Titling: % MSc x 1 + % MSc x 3 + % PhD x 4 + % PhD x 8 = Score
3 - Participation in MSc and PhD programs' activities
	Yes	Cooperation with Graduate programs			N²
	2*	1*			0*
						Score
				Total score of item III

1 - Regular schedule of lectures
	Yes	No
Otology	1*	0*
Rhinology	1*	0*
Laryngology	1*	0*
Head and neck surgery	1*	0*
Snoring and sleep apnea	1*	0*
Plastic surgery	1*	0*
Traumatology	1*	0*
Skull base surgery	1*	0*
Stomatology	1*	0*
Buccopharyngology	1*	0*
Orthodontic surgery	1*	0*
Alergy	1*	0*
Pediatric ENT	1*	0*
Defense against malpractice	1*	0*
			Score
2 - Regular meetings to discuss journal papers
	Yes	No
Otology	1*	0*
Rhinology	1*	0*
Laryngology	1*	0*
Head and neck surgery	1*	0*
Snoring and sleep apnea	1*	0*
Plastic surgery	1*	0*
Traumatology	1*	0*
Skull base surgery	1*	0*
Stomatology	1*	0*
Buccopharyngology	1*	0*
Orthodontic surgery	1*	0*
Allergy	1*	0*
Pediatric ENT	1*	0*
			Score
3 - Regular discussion of clinical cases
	Yes	No
Otology	1*	0*
Rhinology	1*	0*
Laryngology	1*	0*
Head and neck surgery	1*	0*
Snoring and sleep apnea	1*	0*
Plastic surgery	1*	0*
Traumatology	1*	0*
Skull base surgery	1*	0*
Stomatology	1*	0*
Buccopharyngology	1*	0*
Orthodontic surgery	1*	0*
Allergy	1*	0*
Pediatric ENT	1*	0*
			Score

4 - Volume of outpatient care:
N². of patients/resident/day		( ) < 10
	> 30	( ) 10-30	< 10
		( )>30
Value		10*	30*	10*	Score
Volume of surgeries: N².of surgeries/resident/year	101-200	51-100	< 50
Value	40*	20*	0*		Score
5 - Resident participation mode in the surgeries performed
	Surgeon	Assistant	Observer	No
Oral endoscopy	3*	2*	1*	0*
Abscess drainage	3*	2*	1*	0*
Adenotonsillectomy	3*	2*	1*	0*
Uvulopalatopharyngoplasty	3*	2*	1*	0*
Septoplasty	3*	2*	1*	0*
Turbinectomy and turbinoplasty	3*	2*	1*	0*
Maxillary sinusectomy	3*	2*	1*	0*
Caldwell-Luc procedure	3*	2*	1*	0*
External and intranasal ethmoidectomy	3*	2*	1*	0*
External and intranasal frontal sinusectomy	3*	2*	1*	0*
Sphenoidectomy	3*	2*	1*	0*
Partial and total Maxillectomies	3*	2*	1*	0*
Tympanic paracenthesis	3*	2*	1*	0*
Miryngotomy for ventilation tube placement	3*	2*	1*	0*
Tympanoplasty	3*	2*	1*	0*
Tympanomastoidectomy	3*	2*	1*	0*
Stapes surgery	3*	2*	1*	0*
Partial and total resection of the temporal bone	3*	2*	1*	0*
Rhinoplasty	3*	2*	1*	0*
Otoplasty	3*	2*	1*	0*
Blepharoplasty	3*	2*	1*	0*
Other facial plastic surgeries	3*	2*	1*	0*
Submandibular gland exeresis	3*	2*	1*	0*
Parotidectomy	3*	2*	1*	0*
Exeresis of congenital neck lesions	3*	2*	1*	0*
Exeresis of benign neck tumors	3*	2*	1*	0*
Neck dissection	3*	2*	1*	0*
Mouth and pharynx tumor resection	3*	2*	1*	0*
Partial and total glossectomies	3*	2*	1*	0*
Marginal and segmentary Mandibulectomy	3*	2*	1*	0*
Tracheal intubation	3*	2*	1*	0*
Tracheostomy	3*	2*	1*	0*
Laryngeal microsurgery	3*	2*	1*	0*
Laryngotracheoplasty	3*	2*	1*	0*
Endoscopic cordectomy	3*	2*	1*	0*
Thyroplasty	3*	2*	1*	0*
Partial laryngectomies	3*	2*	1*	0*
Total Laryngectomy	3*	2*	1*	0*
Nasal fracture	3*	2*	1*	0*
Maxillary fractures	3*	2*	1*	0*
Mandible fracture	3*	2*	1*	0*
Zygomatic arch fracture	3*	2*	1*	0*
Facial nerve surgery	3*	2*	1*	0*
Craniofacial dysfunction	3*	2*	1*	0*	Score

6 - Activities in Experimental Surgery
	Regular	Esporadic	No
Otology	10*	5*	0*
Rhinology	10*	5*	0*
Laryngology	10*	5*	0*
					Score
7 - Activities in research (Presentation of free themes and publications -average in the past 3 years)	(+1 p/Res)	(1 p/Res)	(-1 p/Res)	None
5 minute presentation by resident per year in the Triological or Brazilian Meeting	15*	10*	5*	0*
Scientific paper published by resident in journal indexed at the Lilacs or more	30*	20*	10*	0*
					Score
	51 to 100%	1 to 50%	None
Residents who publish papers in scientific journals	10*	5*	0*
OBS: We consider only one resident per paper					Score
8 - Number of residents who participated in meetings	(+ than 50%)	up to 50%	None
Brazilian Meeting of Otorhinolaryngology		5*	2*	0*
Meeting of the Triological Society of Otorhinolaryngology		5*	2*	0*	Score
			Total score of item IV

Institution's scientific production and research activities (in the past 3 years)
	Weight	N	Score
Papers published in Med Line-indexed journals	5*
Papers published in Scielo-indexed journals	3*
Papers published in journals of other indexations	2*
Book chapters	3*
Book authorship (book co-authorship)	5*
Book organization	3*
Abstracts published in meetings's procedures	1*
Thesis dissertations	5*
Weight x the number of occurrences (for the program)
Total score of item V

Aspects evaluated		Weight
I	- General program description and academic environment	1
II	- Infrastructure	2
III	- Student body	2
IV	- Training activities	2
V	- Program's Research and scientific production	1
VI	- Student body	2
VII	- Program's assessment by the student body	0
VIII	- Program's self-assessment	0

Classification	Reclassification
Level A	every 5 years
Level B+	every 4 years
Level B	every 4 years
Level C+	every 2 years
Level C	every 2 years
Level D	Immediate prohibition of receiving new applicants; report to MEC on the shortcomings of the residency program: annual re-registering;
Level E	Immediate prohibition of receiving new applicants; report to MEC on the shortcomings of the residency program; Reassessment in 6 months, if it remains ranked as E in the second assessment the program shall be terminated and a new program and preceptor are prevented from reaccreditation for the next 4 years;

Classification	Grade ranges
A	de 80 to 100
B+	de 75 to 79
B	de 65 to 74
C+	de 55 to 64
C	de 45 to 54
D	de 30 to 44
E	de 0 to 29

Ranking	B+	B	C+	C	D	E
A	0,000347587 Rules out H0*	*	*	*	*	*
B+		0,0000191898 Rules out H0*	*	*	*	*
B			0,0000000039342 Rules out H0*	*	*	*
C+				0,00000097283 Rules out H0*	*	*
C					0,000000017741 Rules out H0*	*
D						0,00503107 Rules out H0*

Ranking	Training programs
A	10; 11; 18; 20; 37; 38; 39; 45; 65; 84
B+	16; 17; 26; 36; 42; 81
B	1; 4; 19; 21; 22; 30; 32; 46; 48; 50; 52; 58; 63; 64; 74; 80
C+	3; 5 14; 27; 47; 53; 54; 56; 68; 71; 72; 73; 75; 76; 79; 82; 83
C	2; 12; 15; 23; 24; 31; 35; 41; 44; 51; 55; 57; 61; 69
D	6; 7; 8; 9; 25; 28; 29; 34; 49; 59; 60; 62; 66; 67; 70; 77; 85
E	13; 78

Ranking
Region	A		B+		B		C+		C		D		E
	%	n	%	n	%	n	%	n	%	n	%	n	%	n
Southeast
São Paulo	60.00	6	50.00	3	37.50	6	52.90	9	42.90	6	17.60	3	50.00	1
Rio de Janeiro	-	-	-	-	18.80	3	5.90	1	35.70	5	23.50	4	-	-
Minas Gerais	-	-	16.70	1	12.50	2	23.50	4	21.40	3	-	-	50.00	1
South
R.G. do Sul	10.00	1	16.70	1	-	-	11.80	2	-	-	-	-	-	-
Paraná	10.00	1	-	-	12.50	2	-	-	-	0	11.80	2	-	-
Northeast
Bahia	-	-	16.70	1	6.30	1	-	-	-	-	17.60	3	-	-
Pernambuco	10.00	1	-	-	-	-	-	-	-	-	-	-	-	-
Ceará	-	-	-	-	6.30	1	5.90	1	-	-	-	-	-	-
R. G. do Norte	-	-	-	-	-	-	-	-	-	-	5.90	1	-	-
Alagoas	-	-	-	-	-	-	-	-	-	-	5.90	1	-	-
Midwest
Distrito Federal	10.00	1	-	-	-	-	-	-	-	-	-	-	-	-
Goiás	-	-	-	-	6.30	1	-	-	-	-	-	-	-	-
North
Amazonas	-	-	-	-	-	-	-	-	-	-	17.60	3	-	-
Brasil	12.20	10	7.3	6	19.50	16	20.70	17	17.10	14	20.70	17	2.40	2

Region	A		B+		B		C+		C		D		E		Total
Region	%	n	%	n	%	n	%	n	%	N	%	n	%	n
Southeast
São Paulo	17.65	6	8.82	3	17.65	6	26.47	9	17.65	6	8.82	3	2.94	1	34
Rio de Janeiro	-	-	-	-	23.08	3	7.69	1	38.46	5	30.77	4		-	13
Minas Gerais	-	-	9.09	1	18.18	2	36.36	4	27.27	3	-	-	9.09	1	11
South
R.G. do Sul	25.00	1	25.00	1	-	-	50.00	2	-	-	-	-	-	-	4
Paraná	20.00	1	-	-	40.00	2	-	-	-	-	40.00	2	-	-	5
Northeast
Bahia	-	-	20.00	1	20.00	1	-	-	-	-	60.00	3	-	-	5
Pernambuco	20.00	1	-	-	-	-	-	-	-	-	-	-	-	-	1
Ceará	-	-	-	-	50.00	1	50.00	1	-	-	-	-	-	-	2
R. G. do Norte	-	-	-	-	-	-	-	-	-	-	-	1	-	-	1
Alagoas	-	-	-	-	-	-	-	-	-	-	100.00	1	-	-	1
Midwest
Distrito Federal	100.00	1	-	-		-	-	-	-	-	-	-	-	-	1
Goiás		-	-	-	100.00	1	-	-	-	-	-	-	-	-	1
North
Amazonas	-	-	-	-	-	-		-	-	-	100.00	3	-	-	3

Ranking	B+	B	C+	C	D
A	0,276151 accepts H0	0,479883 accepts H0	0,143826 accepts H0	0,00056 rejects H0*	0,021284 rejects H0*
B+		0,644867 accepts H0	0,892801 accepts H0	0,119467 accepts H0	0,107445 accepts H0
B			0,492737 accepts H0	0,016604 rejects H0*	0,084488 accepts H0
C+				0,102699 accepts H0	0,16675 accepts H0
C					0,611638 accepts H0
D

Ranking	Mean grades
A	84.8
B+	76.3
B	68.7
C+	58.9
C	54.8
D	38.9
E	28.0

Training program	2004	2006	2007
78	E	D	D
13	E	E	E
85	D	C	C
77	D	C	C
70	D	C	C
67	D	C	C
66	D	C	C
62	D	C	C
60	D	C	C
59	D	D	D
49	D	D	D
34	D	D	D
29	D	D	D
28	D	C	C
25	D	C	C
12	D		Terminated
9	D	C	C
8	D	C+	C+
7	D	C	B
6	D	C	C+
69	C		C
61	C		B
57	C		C
55	C		C+
44	C		C
51	C		C+
41	C		Absent
35	C		C
24	C		Absent
23	C		B+
15	C		Absent
2	C		C+
83	C+		B
82	C+		B
79	C+		D
76	C+		C+
75	C+		C+
73	C+		C+
72	C+		C