Predictive Factors of Graduation Delay in a Medical Program: a Retrospective Cohort Study in Brazil, 2010-2016

Fatores Preditivos de Atraso na Graduação em um Curso De medicina: Estudo de Coorte Retrospectivo no Brasil, 2010-2016

Abstract:

Introduction:

Medical training is a long and expensive process. Admission processes are highly competitive all over the world but being accepted is no guarantee of academic success. Medical school is demanding and stressful, and some students are not able to cope with this new scenario successfully. It is estimated that 10-15% of medical students experience difficulties in adapting to the course, which can lead to academic failure. The identification of predictive factors of failure supports the creation of mechanisms and strategies to avoid course dropout or graduation delay. To identify predictive factors of academic failure in a Brazilian medical program.

Methods:

A retrospective observational study was carried out with all medical students admitted to a private Brazilian medical school in 2010 and 2011. The main outcome was academic success. Academic failure was defined as graduation delay or course dropout (Group 1), and academic success was defined as graduating within 6 years (Group 2). Sociodemographic and academic data were collected, including grades obtained at the admission process and the first-semester courses. Freshman students and students with passing grades in the first semester (passed students) were analyzed separately. Descriptive and comparative analyses, logistic regression and ROC curve analysis were performed. The level of significance was 0.05.

Results:

A total of 312 students were admitted during the study period, but 10 were excluded due to lack of information. Of the 302 students included in the study, 105 were included in Group 1 and 197 were included in Group 2. Thirty-two students failed the first semester. The 270 students with passing grades in the first semester were divided into Group 1 (n=73) and Group 2 (n=197). Among the freshman students, lower admission grades were associated with a higher chance of failure (p_adjusted=0.012). Of the 270 students with passing grades, low academic performance (courses’ mean grades) was associated with graduation delay (p_adjusted<0.001).

Conclusions:

Low grades at the admission process (freshman students) and low academic performance in the first semester (students with passing grades) were predictive factors of academic failure.

Key-words:
Academic Failure; Academic Performance; Brazil; Medical Students; Risk Factors.

Resumo:

Introdução:

a formação do médico é um processo longo e dispendioso. Os processos de seleção são muito competitivos em todo o mundo, mas a aprovação não é garantia de sucesso acadêmico. Os cursos de Medicina são muito exigentes e estressantes, e alguns estudantes não são capazes de lidar adequadamente com esse novo cenário. Estima-se que 10%-15% dos estudantes de Medicina tenham alguma dificuldade em adaptar-se ao curso, o que pode levar ao insucesso acadêmico. A identificação de fatores preditivos de insucesso pode suportar a criação de mecanismos e estratégias para evitar o atraso ou abandono do curso. O objetivo deste estudo é identificar fatores preditivos de insucesso acadêmico em um curso de Medicina no Brasil.

Método:

estudo observacional retrospectivo foi realizado com todos os estudantes de Medicina que foram admitidos em uma escola médica privada em 2010 e 2011. O desfecho principal foi o sucesso acadêmico. Definiu-se o insucesso acadêmico como atraso na graduação ou abandono do curso (Grupo 1) e o sucesso acadêmico como graduação seis anos após o ingresso no curso (Grupo 2). Coletaram-se dados sociodemográficos e acadêmicos, além das notas no processo de admissão e nas disciplinas do primeiro semestre. Os alunos ingressantes no curso e aqueles aprovados no primeiro semestre (aprovados) foram analisados separadamente. Realizou-se análise descritiva e comparativa, regressão logística e análise de curva ROC. O nível de significância foi de 0,05.

Resultados:

312 alunos foram admitidos no período do estudo, mas dez foram excluídos por falta de informação. Dos 302 ingressante, 105 foram incluídos no Grupo 1 e 197 no Grupo 2. Trinta e dois alunos foram reprovados no primeiro semestre. Os 270 alunos aprovados no primeiro semestre foram divididos em Grupo 1 (n = 73) e Grupo 2 (n = 197). Entre os alunos ingressantes, notas baixas no processo de admissão estavam associadas à maior chance de insucesso acadêmico (p_ajustado = 0,012). Dos 270 alunos aprovados no primeiro semestre, o baixo desempenho acadêmico (média das notas do semestre) estava associado ao insucesso acadêmico (p_ajustado = 0,001).

Conclusões:

notas baixas no processo seletivo (alunos ingressantes) e o baixo desempenho acadêmico no primeiro semestre (alunos aprovados no primeiro semestre) foram fatores preditivos de insucesso acadêmico.

Palavras-chave:
Fracasso Acadêmico; Desempenho Acadêmico; Brasil; Estudantes de Medicina; Fatores de Risco

INTRODUCTION

Medical training is a long (8 to 11 years) and expensive process. The undergraduate program alone has an estimated direct cost of US $ 125,000 to US $ 310,000 depending on the country^1-3. In addition, it is a considerable investment of time and money for students and their families. In many countries, there is also a major investment of the government’s money through partial or total subsidies to cover the cost of the program^3,4.

Admission processes for medical programs are highly competitive all over the world, possibly due to the prospects of a satisfying and rewarding career, social recognition, and a solid upper-middle-class lifestyle². However, being accepted to medical school is no guarantee of academic success. Medical programs are demanding and stressful for students. Most students manage this new situation and achieve academic success. However, some are less able to successfully cope with the transition and experience difficulties in adapting to the program routine^3-5. Yates and James observed that 10% to 15% of Nottingham University Medical School students experienced some form of adjustment difficulty^3,6.

The literature describes several predictive factors of academic success in medical programs, such as prior academic performance; performance in the admission process; and psychological, behavioral, demographic, social and economic factors^3,7-10. Most studies evaluate predictive factors of success. The identification of reliable features of students at risk of failure has not been widely studied, and one cannot simply assume that the factors predicting failure are the inverse of those predicting success⁵.

Predictive factors of academic performance are usually divided into three groups: pre-program, admission process and program factors. Academic performance achieved prior to the admission process is one of the main predictive factors of medical program success^4,5,11,12. Some studies have observed an association between worse performance during medical studies and sociodemographic factors such as male gender^3,7, non-white ethnicity^3,7, age over 24 years^7,11 and low socioeconomic background¹¹. Pudey and Mercer also observed an association between a background in humanities in the pre-medical years and academic failure¹¹. High performance in the admission process has also been observed to be an independent factor of success in several studies^{3,4,7,8,11,12}.

Problems related to low academic performance during an educational program can be classified into three types: (1) academic, (2) intrapersonal, and (3) interpersonal problems⁸. The first group refers to students who achieve low grades, fail courses, and have poor attendance, especially in the first two years^3,6,8,13. The second group refers to students who experience personal conflicts or exhibit inappropriate academic behavior or psychiatric disorders, such as excessive anxiety, hypochondria and depression^3,8,10. Students classified in the third group experience difficulties in interacting with teachers, colleagues, supervisors or patients⁸.

A study carried out by Hendren shows that 43% of students who experience academic problems and 29% of students with intrapersonal problems drop out of the program. The dropout rate was found to be much higher among students with interpersonal problems, with only 8% of them completing the program⁸.

Determining the predictive factors of academic success or failure is beneficial for students and their families, society and educational institutions, because it provides for the creation of mechanisms to identify and support students experiencing difficulties and for the onset of early intervention before students drop out for avoidable reasons^3,4.

No study was identified in the literature review on predictive factors of academic performance among medical students in Brazil. The aim of this study is to identify predictive factors of academic failure among students from a Brazilian medical program.

METHOD

Population

The medical program of Universidade José do Rosário Vellano (UNIFENAS) on the Campus of Belo Horizonte, Minas Gerais, Brazil, was founded 17 years ago. It is a 6-year full-time program with pre-clinical years integrated into clinical years and involves a single admission process, occurring at the beginning of the program. Eighty students are admitted each semester, with a total of 160 per year. The university itself conducts the admission process. Low-income students are also admitted through the federal government’s full scholarship program (PROUNI). The government selects these students. The curriculum of the first 4 years is based on problem-based learning approaches and is organized into thematic modules (courses). Due to the curriculum structure, failure in any course during the program delays the student’s graduation.

This study involved four cohorts of students who started the program in 2010 and 2011 with the intention of graduating in 2015 and 2016, respectively. The inclusion criteria were as follows: students entering the program through the institution’s admission process or through PROUNI in 2010 or 2011 and whose academic information was available in the institution’s system. Students admitted through transfers from other institutions or who transferred to other institutions during the study period were excluded.

Data collection

Data were directly collected from the institution’s Academic System using Microsoft Excel® worksheets. The students’ names were removed to keep the study data confidential. The spreadsheets were then integrated into a single database based on each student’s registration number. Data were collected until July 2017.

The following variables were collected: gender, age at admission, marital status, admission type (institution’s process or PROUNI), type of high school (private or public), grade achieved during the admission process (although not for PROUNI students), time interval between finishing high school and admission to the medical program, and grades during and passing grades of first-semester courses. The overall performance during the first semester was calculated from the arithmetic mean of course grades. Students with grades ≥ 60% and frequency ≥ 75% passed each course according to the institution’s criteria. Four courses are offered in the first semester. The first is entitled Introduction to the Study of Medicine, which describes the program, trains the students for problem-based learning tutorial groups, and addresses basic concepts related to homeostasis, physiology, cytohistology, and anatomy. The second course, Hemorrhage and Shock, addresses the physiology, histology, and anatomy of the cardiovascular system. The third course, Oliguria, and the fourth, Dyspnea, address the same aspects regarding the genitourinary and respiratory tracts, respectively.

The study outcome was academic performance. Academic failure was defined as graduation delay (> 6 years) or dropping out of the program (Group 1). Academic success was defined as achieving graduation 6 years after program admission (Group 2).

Statistical analysis

Two separate statistical analyses were performed, because a failure in any course implies graduation delay. The first analysis involved all students entering the program (freshman students), and all variables were considered, except for grades and achieving passing grades at the first-semester courses. For the second analysis, only students who passed the first-semester courses were considered (students with passing grades), and all variables were included, with the exception of course approval.

A descriptive analysis was performed for each variable, based on the frequency distribution of categorical variables and using means and standard deviations for continuous variables. A chi-square test was performed to compare the categorical variables, and Student’s t test was applied to measure the continuous variables. Logistic regression was performed using the stepwise technique for variables with p <0.20. For continuous variables related to academic failure, according to the logistic regression, a ROC curve analysis was performed. The level of significance was set at 0.05. The software SPSS 19 (IBM, USA) and Stata 12.1 (StataCorp, USA) were used for our statistical analysis.

RESULTS

Three hundred and twelve students were admitted in 2010 and 2011, but seven were excluded because they transferred to other programs during the study period. Three other students were removed from the study due to conflicting academic information. The final sample consisted of 302 students (freshman students). Group 1 (academic failure) included 105 students, and Group 2 (academic success) included 197 students (Figure 1). The following causes of academic failure were identified: dropped out of the program (27 students - 25.7%) and graduation delay (78 students - 74.3%). Of these, 58 graduated late, and 20 were still completing the program as of July 2017. Of the 302 students, 32 students failed at least one course in the first semester. The second analysis covered 270 participants (students with passing grades), of whom 73 were in Group 1 and 197 were in Group 2 (Figure 1).

Figure 1
Study flowchart

Table 1 shows sociodemographic and academic data for the freshman students. The majority was young (20.3 years old), female (193 - 63.9%) and single (295 - 97.7%). Most of them (85.5%) had attended high school at private institutions and had finished high school less than 4 years prior to the medical program admission. Most were admitted through the program’s process (274 - 90.7%), with a mean grade of 80.6 points. Only admission grades showed a statistically significant association with academic failure.

Table 1
Comparative analysis of sociodemographic and academic variables of freshman students according to academic failure (Group 1) or success (Group 2)

Categorical variables		Group 1 (n=105)		Group 2 (n = 197)		Total (n = 302)		p
		n	%	n	%	n	%
Gender	Female	60	57.1	133	67.5	193	63.9	0.079
	Male	45	42.9	64	32.5	109	36.1
Marital status	Single	103	98.1	192	97.5	295	97.7	0.763
	Married	2	1.9	4	2.0	6	2.0
	Divorced	0	0	1	5.0	1	3
High school type*	Private	73	84.9	169	85.8	242	85.5	0.843
	Public	13	15.1	28	14.2	41	14.5
Admission type	Institutional process	99	94.3	175	88.8	274	90.7	0.120
	PROUNI	6	5.7	22	11.2	28	9.3
Continuous variables		Mean	SD	Mean	SD	Mean	SD	p
Age		20.2	2.8	20.3	3.6	20.3	3.3	0.636
Time since high school graduation (years)**		3.4	2.7	3.7	3.5	3.6	3.2	0.346
Admission grades***		78.8	8.4	81.7	9.2	80.6	9.0	0.008

• SD = standard deviation; * no information = 19; ** no information = 3; *** - PROUNI students were not subject to institutional selection.

Table 2 shows sociodemographic and academic data for students who achieved passing grades in the first-semester courses. As for freshman students, admission grades showed a statistically significant association with academic failure. A significant association was also observed between academic failure and first-semester course grades and overall performance.

Table 2
Comparative analysis of sociodemographic and academic variables of the students with passing grades according to academic failure (Group 1) or success (Group 2)

Categorical variables		Group 1 (n=73)		Group 2 (n = 197)			Total (n = 270)		p
		n	%	n	%	n	%
Gender	Female	44	60.2	133	67.5	177	65.6	0.266
	Male	29	39.7	64	32.5	93	34.4
Marital status	Single	71	97.2	192	97.5	263	97.4	0.266
	Married	2	2.7	4	2.0	6	2.2
	Divorced	0	0.0	1	0.5	1	0.4
High school type*	Private	50	68.5	169	85.6	219	81.1	0.467
	Public	11	15.0	28	14.2	39	14.4
Admission type	Institutional process	67	91.8	175	88.8	242	89.6	0.481
	PROUNI	6	8.2	22	11.2	28	10.4
Continuous variables		Mean	SD	Mean	SD	Mean	SD	p
Age		20.3	2.9	20.3	3.6	20.3	3.3	0.902
Time since high school graduation (years)**		3.5	2.7	3.7	3.5	3.7	3.3	0.599
Admission grades***		79.0	9.0	81.7	9.2	81.0	9.2	0.043
Course		Mean	SD	Mean	SD	Mean	SD	p
Introduction to the Study of Medicine		77.4	6.4	82.5	5.8	81.1	6.4	<0.001
Hemorrhage and Shock		76.1	6.1	81.0	5.5	79.6	6.1	<0.001
Oliguria		77.0	7.1	82.8	5.7	81.3	6.6	<0.001
Dyspnea		78.4	7.1	83.5	5.9	82.1	6.6	<0.001
Overall performance		77.2	6.1	82.4	5.2	81.0	5.9	<0.001

• SD = standard deviation; * no information = 19; ** no information = 3; *** - PROUNI students were not subject to institutional selection.

The logistic regression results are shown in Table 3. The following variables were included in the regression model for freshman students: gender, admission type and admission grades. Collinearity was observed between the admission type and admission grades, and the admission type was excluded from the analysis because it did not show a statistically significant association with academic performance. After adjustments, admission grades continued to be statistically associated with academic failure among freshman students. For students with passing grades, the regression model included the following: admission grades, course grades and overall performance. Collinearity was observed between course grades and overall performance because overall performance is the arithmetic mean of course grades. Two logistic regression models were analyzed. Model 1 included admission and course grades, whereas model 2 included admission grades and overall performance. After analyzing the two models, model 2 was chosen, and only overall performance was maintained as a predictive factor of failure.

Table 3
Final logistic regression model for predictive factors of academic failure among freshman students and those with passing grades

Freshman students	Raw p		Adjusted p
Gender	0.079		0.084
Admission type*	0.120		---
Admission grades**	0.008		0.012
Passing students	Raw p	Adjusted p - Model 1	Adjusted p - Model 2
Admission grades**	0.043	0.857	0.729
Introduction to the Study of Medicine	<0.001	0.061	---
Hemorrhage and Shock	<0.001	0.951	---
Oliguria	<0.001	0.022	---
Dyspnea	<0.001	0.583	---
Overall performance***	<0.001	----	<0.001

• *collinearity with admission grades; ** except for PROUNI students; *** collinearity with course grades

A ROC curve analysis was performed on the admission grades of the freshman students (Figure 2A) and on the overall performance of those with passing grades in the first semester (Figure 2B). In both cases, no cutoff point with good sensitivity (identifying students more likely to succeed) and specificity (identifying students more likely to fail) was identified, although the two analyses confirmed the statistical association indicated by the logistic regression.

Figure 2
ROC curve of variables statistically associated with academic failure among freshman students (2A)

Ethical approval

The study was approved by UNIFENAS’s Institutional Review Board (Reference Number 2.006.948) and was carried out in accordance with the current version of the Declaration of Helsinki and its amendments and with Resolution 466/2012 of the National Council of Ethics in Human Research of the Brazilian Health Ministry and its amendments.

DISCUSSION

The objective of this study was to identify predictive factors of academic failure during medical school. Based on a literature review, no similar studies carried out in Brazil were identified, making this study an innovative one.

Two factors were found to be statistically associated with academic failure throughout medical program: grades achieved during the admission process among freshman students and overall performance achieved during the first-semester courses among students with passing grades in that semester. These results corroborate the findings of other authors who observed an independent association between grades during the admission process and academic difficulties in the USA⁷, Australia^11,12, the UK³, and Croatia⁴. An association between underperformance in the first year and academic failure has also been reported in the literature^3,6,8,13.

This study showed no association between academic failure and age, as observed by Andriole and Jeffe⁷ and by Pudey and Mercer¹¹. There was also no association between male gender and academic failure, as reported by Yates and James³ and by Andriole and Jeffe⁷. Considering that students from public high schools or students who were admitted through the federal government scholarship program may have low socioeconomic background, no association was observed between these variables and academic failure, as observed by Puddey and Mercer¹¹.

The main limitation of this study lies in its retrospective design and in the use of a secondary database that did not have information on other variables associated with academic performance, such as ethnicity, academic performance prior to entering the medical program and behavioral changes or mental illness. It should be noted that the findings of this study might be specific to the evaluated program, although the study can serve as a model from which other institutions can identify predictive factors of program success or failure.

Despite these limitations, this study has a large sample and, to the best of our knowledge, is the first Brazilian study on academic performance in medical programs. The findings show that it is possible to identify students with a higher risk of experiencing academic failure early in the program. With this information, it is possible to develop intervention strategies to reduce failure and dropout rates throughout the program.

CONCLUSION

Low grades achieved during the admission process and low overall performance in first-semester courses were statistically associated to graduation delay at the UNIFENAS Belo Horizonte medical program. No association was observed between sociodemographic characteristics and academic failure.

ACKNOWLEDGEMENTS

The authors are very grateful to professors Alexandre Moura and Maria Turci for critically reviewing the paper.

REFERENCES

• ¹
  Brazilian Medical Programs Tuitions. 2018. (Accessed 02 mar. 2018., 2018, at , 2018, at http://www.escolasmedicas.com.br/mensalidades.php .)
  » http://www.escolasmedicas.com.br/mensalidades.php
• ²
  Doroghazi RM, Alpert JS. A medical education as an investment: financial food for thought. Am J Med 2014;127:7-11.
• ³
  Yates J, James D. Predicting the “strugglers”: a case-control study of students at Nottingham University Medical School. BMJ 2006;332:1009-13.
• ⁴
  Maslov Kruzicevic S, Barisic KJ, Banozic A, Esteban CD, Sapunar D, Puljak L. Predictors of attrition and academic success of medical students: a 30-year retrospective study. PLoS One 2012;7:e39144.
• ⁵
  Jardine DL, McKenzie JM, Wilkinson TJ. Predicting medical students who will have difficulty during their clinical training. BMC Med Educ 2017;17:43.
• ⁶
  Yates J. Development of a ‘toolkit’ to identify medical students at risk of failure to thrive on the course: an exploratory retrospective case study. BMC Med Educ 2011;11:95.
• ⁷
  Andriole DA, Jeffe DB. Prematriculation variables associated with suboptimal outcomes for the 1994-1999 cohort of US medical school matriculants. JAMA 2010;304:1212-9.
• ⁸
  Hendren RL. Predicting success and failure of medical students at risk for dismissal. J Med Educ 1988;63:596-602.
• ⁹
  Kusurkar RA, Ten Cate TJ, van Asperen M, Croiset G. Motivation as an independent and a dependent variable in medical education: a review of the literature. Med Teach 2011;33:e242-62.
• ¹⁰
  Samouei R, Fooladvand M, Janghorban S, Khorvash F. Predicting the educational performance of Isfahan University students of medical sciences based on their behaviour profile, mental health and demographic characteristic. J Educ Health Promot 2015;4:44.
• ¹¹
  Puddey IB, Mercer A. Predicting academic outcomes in an Australian graduate entry medical programme. BMC Med Educ 2014;14:31.
• ¹²
  Sladek RM, Bond MJ, Frost LK, Prior KN. Predicting success in medical school: a longitudinal study of common Australian student selection tools. BMC Med Educ 2016;16:187.
• ¹³
  Shaban S, McLean M. Predicting performance at medical school: can we identify at-risk students? Adv Med Educ Pract 2011;2:139-48.

Publication Dates

• Publication in this collection
  27 Feb 2020
• Date of issue
  2020

History

• Received
  22 July 2019
• Accepted
  25 Sept 2019