Burnout in dental students: effectiveness of different methods

Objective: This study was proposed to estimate the effectiveness of different screening methods of the Burnout Syndrome among dental students. Material and method: The Burnout Syndrome assessment was performed using the Oldenburg Inventory-Student survey (OLBI-SS) and the Copenhagen Inventory-Student survey (CBI-SS). The Maslach Burnout Inventory-Student survey (MBI-SS) was used as the gold standard. The psychometric properties of the instruments were measured. The second-order hierarchical model was estimated to calculate the overall scores for OLBI-SS and CBI-SS, and ROC curves were constructed and the areas were estimated (AUROC). Result: A total of 235 undergraduate students participated in this study. The instruments showed an adequate reliability and validity; however three questions had to be removed from OLBI-SS and one from CBI-SS. The Exhaustion dimension of OLBI-SS, and Personal Burnout and Study related Burnout of CBI-SS presented a good discriminating capacity. Conclusion: CBI-SS showed higher discriminating capacity, than OLBI-SS, to identify the Burnout Syndrome (∆AUROC=.172 [.103-.240]; p<.05). Descriptors: Burnout professional; methods; dentistry; students dental.


INTRODUCTION
The Burnout syndrome is defined by Maslach and Jackson 1 as a working environment syndrome, characterized by a process of response to the chronicity of the occupational stress when coping methods fail or are insufficient.It is a multidimensional syndrome composed by emotional exhaustion, cynicism, and reduced job satisfaction, resulting from the process of response to chronic occupational stress 1,2 .
The early work on Burnout referred exclusively to caregiving professions, such as social workers, nurses and psychologists 3 .Currently, research on Burnout extends to all occupational groups, including students.Schaufeli, Taris 4 (2005) point out that, although formally the students do not perform work or have a job, from the psychological perspective the core of their activities can be considered as work, since they are involved in a structure with rules and mandatory activities that need to be performed.Burnout can have negative consequences at the individual, professional, social and family levels, which may exert a direct influence on the teaching-learning process.Thus, early screening of the syndrome may be an interesting strategy to prevent and/ or minimize the effects of the syndrome, such as exhaustion, disbelief and a feeling of low professional efficacy, which may extend in the definition of the professional profile.to this Syndrome due to socio-economic pressures, relationships with peers and teachers, evaluations, and the work hours they are exposed to in their school routine, thus it is a risk population that should to be taken into account.The interest in the Burnout Syndrome is increasing, as it is becoming a social problem of great importance.In this way several studies have been developed in different countries over the past few years.
According to the literature 6,[8][9][10][11] the assessment of the Burnout Syndrome has occurred in parallel with the evolution of its concept, being the Maslach Burnout Inventory (MBI) the most used instrument in burnout assessment.
Gil-Monte 12 (2005) points out as the MBI's major advantage the fact that it is highly accepted in all countries of Latin America, the European Union and the United States, presenting stable psychometric properties in different populations.
The student version of this instrument, called MBI-SS, was developed by Schaufeli et al. 5 (2002) in English, its validity has been confirmed in student samples from three European countries (Portugal, Spain and Holland).Recently, Hu, Schaufeli 13 (2009) presented the scale's validity study for Chinese students, and Campos et al. 7 (2011) for Brazilian and Portuguese students.
Although this instrument is stable and presents adequate psychometric values, Demerouti et al. 14 (2001) state that the MBI has limitations due to the fact the Job Satisfaction dimension is written in the opposite direction of the Emotional Exhaustion and Cynicism dimensions, which may bias the responses to the scale.Thus, the authors proposed the Oldenburg Burnout Inventory (OLBI) which consists of two dimensions, Exhaustion and Disengagement.The OLBI was proposed to assess the Burnout Syndrome in the general population, and studies point to its validity [14][15][16] .This instrument was originally proposed in German, and later adapted and validated into English, by Halbesleben, Demerouti 15 (2005).Campos et al. 17 (2012) propose the Portuguese version of the instrument for students (OLBI-SS).Among MBI's issues, Kristensen et al. 18 (2005) consider Job Satisfaction as a dimension independent from the others, and suggest that when the questions are translated into other languages they may have different meaning.In addition, the authors consider that Burnout's core is Exhaustion, and question the importance of the dimensions Cynicism and Job Satisfaction, included in the MBI.Thus, these authors developed Copenhagen Burnout Inventory (CBI) to solve these limitations.The CBI is a questionnaire composed by three dimensions Personal Burnout, Work-related Burnout and Client-related Burnout.The CBI is directed to the general population.
Considering that a specific version of this inventory for students does not yet exist in the literature, Campos et al. 19 (2013) performed an adaptation of the original inventory which includes the following dimensions Personal Burnout (PB), Studies-related Burnout (SRB), Colleague-related Burnout (CRB), and Teacherrelated Burnout (TRB), and presented its Portuguese version called CBI-SS.Despite the differences between the Burnout Syndrome's screening instruments, it is essential that each of them is able to correctly diagnose the Syndrome.Thus, we carried out this study with the objective of verifying the effectiveness of different instruments (OLBI-SS and CBI-SS) in assessing Burnout Syndrome in Dentistry students.

Sample
The sample consisted of Dentistry undergraduate students, from the Faculty of Dentistry -UNESP, who agreed to participate in this study.A total of 150 students participated in the first study (pilot study).To assert the psychometric qualities and perform ROC analysis, all students (n=300) were invited to participate and 235 accepted (compliance rate=78.3%).The average age of the participants was 21.0 years (SD=1.8),and 72.8% were female.Regarding the participant's course year, 18.6% were enrolled in first year, 24.3% in the second, 24.2% in the third, and 32.9% in the fourth year.

Study's Variables
The Burnout Syndrome's assessment was performed using the Maslach Burnout Inventory -student version (MBI-SS) 7 , which is an self-assessment instrument with answers given in Likert-type seven-point scale (0: never to 6: every day).The instrument consists of 15 questions which are divided into three dimensions, Emotional Exhaustion, Cynicism and Professional Efficacy.Validation of the MBI-SS for use in Portuguese-speaking college students was conducted by Campos, Maroco 20 (2012).The Oldenburg Burnout Inventory -Student Version (OLBI-SS) 17 was originally proposed by Demerouti et al. 14 (2001) in German, and consists of 16 statements with Likert-type four-point response scale (1: totally disagree to 4: totally agree).The statements are divided into two dimensions, Exhaustion and Disengagement, being that four items for each dimension are positive statements and four are negative statements.The Portuguese version was validated for use in college students by Campos et al. 17  Given that MBI-SS is the most used instrument worldwide for this Syndrome's screening, that its metric qualities have been considered adequate and stable in several studies [7][8][9][10][11]21 , and given the lack of a 'gold standard' for diagnosis of Burnout, we decided to consider the MBI as the 'gold standard' in this study, as previously recommended by Schutte et al. 22 (2000).

Procedures
The questionnaires were self-completed by the students, in the classrooms, in weeks with no assessments, and at an hour previously established with the professors.The average time of completion of the questionnaires by the students was 15.0 ± 2.5 minutes.The order of presentation of the three instruments was random.

Pilot Study
Prior to the final study, pilot study was conducted to determine the reproducibility of the responses to the questionnaires (MBI-SS, OLBI-SS, CBI-SS), a pilot study was performed, where the researcher applied the tools in duplicate, with a one week interval between the applications 150 students participated in this stage.To assess the agreement of the average score obtained in each instrument's dimensions, the Intraclass Correlation Coefficient (ICC) was estimated by point and by a 95% confidence interval (CI 95% ).

Analyses of the psychometric qualities
To study the construct validity of the scales confirmatory factor analysis was carried out and the convergent validity of the instruments was calculated.As indices of goodness of fit we used the χ 2 /df, CFI, GFI e RMSEA which were considered adequate when χ 2 /df≤2.00,CFI e GFI≥.90 and RMSEA≤.10 23.The convergent validity was estimated through the Average Variance Extracted (AVE) and though the Composite Reliability (CR), and was considered adequate when AVE≥.50 e CR≥.70 23 .Internal consistency was estimated by standardized Cronbach's alpha coefficient (Cronbach, 1951)  24 .
The analysis were performed with SPSS ® 21.0 e AMOS  21.0 (IBM SPSS Inc, Chicago, IL).

Effectiveness study
Initially, the mean scores of the dimensions Exhaustion and Cynicism (MBI-SS) were computed, and then the 66th percentile of the sample for each dimension was calculated.The subjects who had an average score above this value were considered positive.Subsequently, the scores obtained in each dimension of OLBI-SS and CBI-SS were computed.ROC curves were constructed for each dimension of OLBI-SS and of CBI-SS, taking into consideration the theoretical conceptualization involved in each dimension's elaboration 14,18 .Thus, for OLBI-SS' Exhaustion dimension and all CBI-SS' dimensions, the MBI-SS Exhaustion dimension was considered as the 'gold standard'; also for the OLBI-SS Disengagement scale the MBI-SS Cynicism dimension was used.
Then, and given the lack of a theoretical framework to identify the presence of Burnout Syndrome when using OLBI-SS and CBI-SS, a second-order hierarchical model was developed to obtain a loadings' matrix.This matrix was used to calculate the dimensions' scores based on each item, thus obtaining an overall burnout score.The overall scores were compared with the MBI-SS in the identification of Burnout Syndrome, individuals that present both average scores above the 66th percentile in the Exhaustion and Cynicism dimensions, and below the 33th percentile on the Professional Efficacy dimension, are considered to "test positive" for the Syndrome.
To verify the discriminatory capacity of each dimension/ instrument, the area under the ROC curve (AUROC) was calculated by point and by a 95% confidence interval (CI 95% ) 25 .The comparison between curves was performed using the z test.

Ethical aspects
The questionnaires were anonymous and only volunteer students participated in the study.The development of this research was approved by the Ethics in Human Research Comity of the Faculty of Dentistry of Araraquara -UNESP (protocol n.06/09).

RESULT 1. Pilot Study
A total of 150 students participated in this stage.They completed the inventories in two different moments, with a one week interval.The data regarding the test-retest is presented in Table 1.
A good agreement is observed in all the inventories' scales, except for the OLBI-SS' Disengagement dimension, which presented moderate reproducibility.

Analyses of the Psychometric Qualities
In the Confirmatory Factor Analysis, for the Portuguese version of the MBI-SS, an appropriate adjustment of the sample to the three-factor model was observed (χ 2  To obtain an adequate factor adjustment of OLBI-SS, items 8, 9 and 14 were excluded since they presented sub-optimal factor weights (λ<.50).It is noteworthy that items with lower factor weights are those whose answer scale is inverted.After removal of these items the two-factor adjustment was considered adequate (χ 2 /df=2.68;CFI=.88;GFI=.90;RMSEA=.08).We observed a limited validity (Exhaustion: AVE=.35,CR=.79; Distancing: AVE=.40,CR = .79)and good internal consistency for OLBI-SS (Exhaustion: α=.78; Distancing: α=.78).
CBI-SS' four-dimension structure (PB, SRB, CRB and TRB) presented a good fit (χ 2 /df=1.79;CFI=.95;GFI=0.86;RMSEA=.06), for this it was necessary to eliminate item 10.It is, however, important to highlight that this was the only item whose answer scale was inversed, which may have led students to complete it incorrectly.The convergent validity was adequate for all dimensions (PB: AVE=.58,CR=.89; SRB: AVE=.50, CR=.87; CRB: AVE=.62,CR=.90; TRB: AVE=.69,CR=.93) and the internal consistency was excellent (α=.86-.93).Thus, CBI-SS was a reliable and valid instrument for characterization of the factors, in this sample.

OLBI-SS and CBI-SS' Effectiveness
Table 2 presents the data regarding the effectiveness of the instruments.
A high sensitivity and specificity for the OLBI-SS dimensions Exhaustion and PB, and CBI-SS' dimension SRB, were observed.
Figure 1 shows the ROC curves for the dimensions of the Oldenburg Burnout Inventory (OLBI-SS) and the Copenhagen Burnout Inventory (CBI-SS).Furthermore, an adequate discriminating capacity of the OLBI-SS dimension Exhaustion, and of the CBI-SS dimensions PB and SRB, was observed.The CBI-SS presented a discriminating capacity significantly higher than the OLBI-SS in identifying the Burnout Syndrome.

DISCUSSION
The assessment of latent measures is a challenge in the medical area, since they are not directly measurable, and are intrinsically related to the instruments' metric qualities in the different samples 7,26,27 .
Another frequent difficulty is the lack of tools that may be considered the 'gold standard' .To overcome this issue we used the instrument that is referenced, in the literature, as being able to produce data with stable metric quality in different samples.Thus, to evaluate the Burnout Syndrome the Maslach Burnout Inventory (MBI) is considered the standard instrument, as recommended by Schutte et al. 22 (2000).
Before assessing the effectiveness of the instruments, the metric quality of the data collected with those instruments, in a specific sample, must be confirmed.In this study, so that the reliability and validity of OLBI-SS and CBI-SS could be considered appropriate, it was necessary to remove some items.These items were formulated in the opposite direction and intercalated with items formulated positively which; according to Yeh et al. 28 (2007), this may have resulted in completion errors by the students, who may not have noticed the reversal of the scale's response pattern.
After the adjustment of the scales, the best cutoff point for each factor can be defined.Peterson et al. 16 (2008) had already proposed cutoff points for OLBI's Exhaustion (≥2.25) and Disengagement (≥2.10) dimensions, based on MBI-GS.However, these cutoff points were proposed by the authors based on the data collected from a normative population of professionals from different areas.In the sample of students (Table 2) it can be seen that the cutoff points, which maximize sensitivity and specificity, are superior to those proposed by Peterson et al. 16 (2008).Thus, caution should be exercised in the use of cutoff points for the evaluation of Burnout Syndrome in different populations.For CBI, previous studies that defined cutoff points for different factors were not found in the literature.
It is important to note that OLBI-SS' exhaustion dimension and CBI-SS' PB and SRB dimensions presented a greater effectiveness (Table 2, Figure 1), which can be attributed to the fact that the theoretical core of Burnout is Exhaustion 18,29 .The fact that the dimensions CRB and TRB present a lower predictive ability can denote that, for students, inter-personal relationships both with peers and with teachers have less impact on the development of exhaustion.It is noteworthy that these two dimensions were proposed by Campos et al. 19 (2013) in their adaptation of the original factor "Burnout related to the clients" to the student population.
Knowing that the theoretical conceptualization of Burnout predicts the existence of a global latent state, and that factors from different scales are significantly correlated, the inclusion of a second-order factor (SOHM) creates the possibility of estimating an overall score for each scale 23 .Thus, using the global scores, we observed (Figure 1) that CBI-SS was more effective than OLBI-SS in the detection of Burnout Syndrome (OLBI-SS: AUROC=.674(.610-.734);p<.05; CBI-SS: AUROC=.846(.793-.889);p<.05; ∆AUROC=.172(.103-.240);p<.05).This can be explained by the inventories' structure 14,18 , since in OLBI the intercalation of positive and negative items may have hindered its completion by students.Moreover, the OLBI considers aspects related to Exhaustion and Disengagement, while CBI only addresses Exhaustion which represents the core of Burnout.CONCLUSION Thus, it can be concluded that the Copenhagen Burnout Inventory (CBI-SS) was more effective than the Oldenburg Inventory (OLBI-SS) in the detection of Burnout Syndrome in students, when the Maslach Burnout Inventory (MBI-SS) was used as the standard.
(2012).The Portuguese version of the Copenhagen Burnout Inventory -Student Version (CBI-SS) was proposed and validated by Campos et al.19 (2013), and consists of 25 items arranged in four dimensions (Personal Burnout, Studies related Burnout, Colleagues related Burnout, Teachers related burnout).The answers are arranged in Likerttype 5-point scale (1: never to 5: always).

Table 2 .
Effectiveness measures of the Oldenburg Inventory (OLBI-SS) and of the Copenhagen Inventory (CBI-SS)