SciELO - Scientific Electronic Library Online

vol.42 issue5Organization of the demand for a Centre of Dental SpecialtiesEvaluation of the pH, calcium release and antibacterial activity of MTA Fillapex author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand




Related links


Revista de Odontologia da UNESP

On-line version ISSN 1807-2577

Rev. odontol. UNESP vol.42 no.5 Araraquara Sept./Oct. 2013 



Burnout in dental students: effectiveness of different methods


Síndrome de Burnout em estudantes de Odontologia: efetividade de diferentes métodos



Juliana Alvares Duarte Bonini CamposI; Paula Cristina JordaniI; Miriane Lucindo ZucolotoI; Fernanda Salloume Sampaio BonaféI; João MarocoII

IFaculdade de Odontologia, UNESP – Univ Estadual Paulista, Araraquara, SP, Brasil
IIInstituto Superior de Psicologia Aplicada, Lisboa, Portugal





OBJETIVO: Este estudo teve por objetivo estimar a efetividade de diferentes métodos de rastreamento da Síndrome de Burnout em estudantes de Odontologia.
MATERIAL E MÉTODO: Utilizou-se o Inventário de Oldenburg (OLBI‑SS) e o Inventário de Copenhagen (CBI-SS). Como padrão-ouro utilizou-se o Inventário de Burnout de Maslach (MBI-SS). As qualidades psicométricas dos instrumentos foram estimadas. O modelo hierárquico de segunda ordem foi estimado para cálculo do escore global do OLBI-SS e do CBI-SS. As curvas ROC foram construídas e as áreas estimadas (AUROC).
Participaram 235 estudantes de um curso de graduação. Os instrumentos apresentaram adequada confiabilidade e validade e para tanto, foi necessário remover três questões do OLBI-SS e uma do CBI-SS.
CONCLUSÃO: Observou-se boa capacidade discriminante das dimensões Exaustão do OLBI-SS e BP e BRE do CBI-SS. O CBI-SS apresentou capacidade discriminante superior à do OLBI-SS na identificação da Síndrome de Burnout (DAUROC=.172 [.103-.240]; p<.05).

Descritores: Esgotamento profissional; métodos; odontologia; estudantes de odontologia.


OBJECTIVE: This study was proposed to estimate the effectiveness of different screening methods of the Burnout Syndrome among dental students.
MATERIAL AND METHOD: The Burnout Syndrome assessment was performed using the Oldenburg Inventory-Student survey (OLBI-SS) and the Copenhagen Inventory-Student survey (CBI‑SS). The Maslach Burnout Inventory-Student survey (MBI-SS) was used as the gold standard. The psychometric properties of the instruments were measured. The second-order hierarchical model was estimated to calculate the overall scores for OLBI-SS and CBI-SS, and ROC curves were constructed and the areas were estimated (AUROC).
RESULT: A total of 235 undergraduate students participated in this study. The instruments showed an adequate reliability and validity; however three questions had to be removed from OLBI-SS and one from CBI-SS. The Exhaustion dimension of OLBI-SS, and Personal Burnout and Study related Burnout of CBI-SS presented a good discriminating capacity.
CBI-SS showed higher discriminating capacity, than OLBI-SS, to identify the Burnout Syndrome (DAUROC=.172 [.103-.240]; p<.05).

Descriptors: Burnout professional; methods; dentistry; students dental.




The Burnout syndrome is defined by Maslach and Jackson1 as a working environment syndrome, characterized by a process of response to the chronicity of the occupational stress when coping methods fail or are insufficient. It is a multidimensional syndrome composed by emotional exhaustion, cynicism, and reduced job satisfaction, resulting from the process of response to chronic occupational stress1,2.

The early work on Burnout referred exclusively to caregiving professions, such as social workers, nurses and psychologists3. Currently, research on Burnout extends to all occupational groups, including students. Schaufeli, Taris4 (2005) point out that, although formally the students do not perform work or have a job, from the psychological perspective the core of their activities can be considered as work, since they are involved in a structure with rules and mandatory activities that need to be performed.

Burnout can have negative consequences at the individual, professional, social and family levels, which may exert a direct influence on the teaching-learning process. Thus, early screening of the syndrome may be an interesting strategy to prevent and/or minimize the effects of the syndrome, such as exhaustion, disbelief and a feeling of low professional efficacy, which may extend in the definition of the professional profile.

According to Schaufeli et al.5 (2002), Maroco et al.6 (2008) and Campos et al.7 (2011), students in higher education are subject to this Syndrome due to socio-economic pressures, relationships with peers and teachers, evaluations, and the work hours they are exposed to in their school routine, thus it is a risk population that should to be taken into account. The interest in the Burnout Syndrome is increasing, as it is becoming a social problem of great importance. In this way several studies have been developed in different countries over the past few years.

According to the literature6,8-11 the assessment of the Burnout Syndrome has occurred in parallel with the evolution of its concept, being the Maslach Burnout Inventory (MBI) the most used instrument in burnout assessment.

Gil-Monte12 (2005) points out as the MBI’s major advantage the fact that it is highly accepted in all countries of Latin America, the European Union and the United States, presenting stable psychometric properties in different populations.

The student version of this instrument, called MBI-SS, was developed by Schaufeli et al.5 (2002) in English, its validity has been confirmed in student samples from three European countries (Portugal, Spain and Holland). Recently, Hu, Schaufeli13 (2009) presented the scale’s validity study for Chinese students, and Campos et al.7 (2011) for Brazilian and Portuguese students.

Although this instrument is stable and presents adequate psychometric values, Demerouti et al.14 (2001) state that the MBI has limitations due to the fact the Job Satisfaction dimension is written in the opposite direction of the Emotional Exhaustion and Cynicism dimensions, which may bias the responses to the scale. Thus, the authors proposed the Oldenburg Burnout Inventory (OLBI) which consists of two dimensions, Exhaustion and Disengagement. The OLBI was proposed to assess the Burnout Syndrome in the general population, and studies point to its validity14-16. This instrument was originally proposed in German, and later adapted and validated into English, by Halbesleben, Demerouti15 (2005). Campos et al.17 (2012) propose the Portuguese version of the instrument for students (OLBI-SS).

Among MBI’s issues, Kristensen et al.18 (2005) consider Job Satisfaction as a dimension independent from the others, and suggest that when the questions are translated into other languages they may have different meaning. In addition, the authors consider that Burnout’s core is Exhaustion, and question the importance of the dimensions Cynicism and Job Satisfaction, included in the MBI. Thus, these authors developed Copenhagen Burnout Inventory (CBI) to solve these limitations. The CBI is a questionnaire composed by three dimensions Personal Burnout, Work-related Burnout and Client-related Burnout. The CBI is directed to the general population.

Considering that a specific version of this inventory for students does not yet exist in the literature, Campos et al.19 (2013) performed an adaptation of the original inventory which includes the following dimensions Personal Burnout (PB), Studies-related Burnout (SRB), Colleague-related Burnout (CRB), and Teacher-related Burnout (TRB), and presented its Portuguese version called CBI-SS.

Despite the differences between the Burnout Syndrome’s screening instruments, it is essential that each of them is able to correctly diagnose the Syndrome. Thus, we carried out this study with the objective of verifying the effectiveness of different instruments (OLBI-SS and CBI-SS) in assessing Burnout Syndrome in Dentistry students.




The sample consisted of Dentistry undergraduate students, from the Faculty of Dentistry - UNESP, who agreed to participate in this study. A total of 150 students participated in the first study (pilot study). To assert the psychometric qualities and perform ROC analysis, all students (n=300) were invited to participate and 235 accepted (compliance rate=78.3%). The average age of the participants was 21.0 years (SD=1.8), and 72.8% were female. Regarding the participant’s course year, 18.6% were enrolled in first year, 24.3% in the second, 24.2% in the third, and 32.9% in the fourth year.

Study’s Variables

The Burnout Syndrome’s assessment was performed using the Maslach Burnout Inventory - student version (MBI-SS)7, which is an self-assessment instrument with answers given in Likert-type seven-point scale (0: never to 6: every day). The instrument consists of 15 questions which are divided into three dimensions, Emotional Exhaustion, Cynicism and Professional Efficacy. Validation of the MBI-SS for use in Portuguese-speaking college students was conducted by Campos, Maroco20 (2012). The Oldenburg Burnout Inventory - Student Version (OLBI-SS)17 was originally proposed by Demerouti et al.14 (2001) in German, and consists of 16 statements with Likert-type four-point response scale (1: totally disagree to 4: totally agree). The statements are divided into two dimensions, Exhaustion and Disengagement, being that four items for each dimension are positive statements and four are negative statements. The Portuguese version was validated for use in college students by Campos et al.17 (2012). The Portuguese version of the Copenhagen Burnout Inventory - Student Version (CBI-SS) was proposed and validated by Campos et al.19 (2013), and consists of 25 items arranged in four dimensions (Personal Burnout, Studies related Burnout, Colleagues related Burnout, Teachers related burnout). The answers are arranged in Likert-type 5-point scale (1: never to 5: always).

Given that MBI-SS is the most used instrument worldwide for this Syndrome’s screening, that its metric qualities have been considered adequate and stable in several studies7-11,21, and given the lack of a ‘gold standard’ for diagnosis of Burnout, we decided to consider the MBI as the ‘gold standard’ in this study, as previously recommended by Schutte et al.22 (2000).


The questionnaires were self-completed by the students, in the classrooms, in weeks with no assessments, and at an hour previously established with the professors. The average time of completion of the questionnaires by the students was 15.0 ± 2.5 minutes. The order of presentation of the three instruments was random.

Pilot Study

Prior to the final study, pilot study was conducted to determine the reproducibility of the responses to the questionnaires (MBI‑SS, OLBI-SS, CBI-SS), a pilot study was performed, where the researcher applied the tools in duplicate, with a one week interval between the applications 150 students participated in this stage. To assess the agreement of the average score obtained in each instrument’s dimensions, the Intraclass Correlation Coefficient (ICC) was estimated by point and by a 95% confidence interval (CI95%).

Statistical Analysis

Analyses of the psychometric qualities

To study the construct validity of the scales confirmatory factor analysis was carried out and the convergent validity of the instruments was calculated. As indices of goodness of fit we used the c2/df, CFI, GFI e RMSEA which were considered adequate when c2/df≤2.00, CFI e GFI≥.90 and RMSEA≤.1023. The convergent validity was estimated through the Average Variance Extracted (AVE) and though the Composite Reliability (CR), and was considered adequate when AVE≥.50 e CR≥.7023. Internal consistency was estimated by standardized Cronbach’s alpha coefficient (Cronbach, 1951)24.

The analysis were performed with SPSSâ 21.0 e AMOS® 21.0 (IBM SPSS Inc, Chicago, IL).

Effectiveness study

Initially, the mean scores of the dimensions Exhaustion and Cynicism (MBI-SS) were computed, and then the 66th percentile of the sample for each dimension was calculated. The subjects who had an average score above this value were considered positive. Subsequently, the scores obtained in each dimension of OLBI- SS and CBI-SS were computed.

ROC curves were constructed for each dimension of OLBI‑SS and of CBI-SS, taking into consideration the theoretical conceptualization involved in each dimension’s elaboration14,18. Thus, for OLBI-SS’ Exhaustion dimension and all CBI-SS’ dimensions, the MBI-SS Exhaustion dimension was considered as the ‘gold standard’; also for the OLBI-SS Disengagement scale the MBI-SS Cynicism dimension was used.

Then, and given the lack of a theoretical framework to identify the presence of Burnout Syndrome when using OLBI‑SS and CBI-SS, a second-order hierarchical model was developed to obtain a loadings’ matrix. This matrix was used to calculate the dimensions’ scores based on each item, thus obtaining an overall burnout score. The overall scores were compared with the MBI-SS in the identification of Burnout Syndrome, individuals that present both average scores above the 66th percentile in the Exhaustion and Cynicism dimensions, and below the 33th percentile on the Professional Efficacy dimension, are considered to “test positive” for the Syndrome.

To verify the discriminatory capacity of each dimension/instrument, the area under the ROC curve (AUROC) was calculated by point and by a 95% confidence interval (CI95%)25. The comparison between curves was performed using the z test.

Ethical aspects

The questionnaires were anonymous and only volunteer students participated in the study. The development of this research was approved by the Ethics in Human Research Comity of the Faculty of Dentistry of Araraquara – UNESP (protocol n.06/09).



Pilot Study

A total of 150 students participated in this stage. They completed the inventories in two different moments, with a one week interval. The data regarding the test-retest is presented in Table 1.



A good agreement is observed in all the inventories’ scales, except for the OLBI-SS’ Disengagement dimension, which presented moderate reproducibility.

Analyses of the Psychometric Qualities

In the Confirmatory Factor Analysis, for the Portuguese version of the MBI-SS, an appropriate adjustment of the sample to the three-factor model was observed (c2/df=2.30; CFI=.95; GFI=.90; RMSEA=.07). The convergent validity (Exhaustion: AVE=.62; CR=.89; Cynicism: AVE=.72; CR=.91; Professional Efficacy AVE=.50; CR=.85) and internal consistency of MBI‑SS (Exhaustion: a=.89; Cynicism: a=.90; Professional Efficacy: a=.85), were adequate. These results attest to the adequate reliability and validity of MBI-SS.

To obtain an adequate factor adjustment of OLBI-SS, items 8, 9 and 14 were excluded since they presented sub-optimal factor weights (l<.50). It is noteworthy that items with lower factor weights are those whose answer scale is inverted. After removal of these items the two-factor adjustment was considered adequate (c2/df=2.68; CFI=.88; GFI=.90; RMSEA=.08). We observed a limited validity (Exhaustion: AVE=.35, CR=.79; Distancing: AVE=.40, CR = .79) and good internal consistency for OLBI-SS (Exhaustion: a=.78; Distancing: a=.78).

CBI-SS’ four-dimension structure (PB, SRB, CRB and TRB) presented a good fit (c2/df=1.79; CFI=.95; GFI=0.86; RMSEA=.06), for this it was necessary to eliminate item 10. It is, however, important to highlight that this was the only item whose answer scale was inversed, which may have led students to complete it incorrectly. The convergent validity was adequate for all dimensions (PB: AVE=.58, CR=.89; SRB: AVE=.50, CR=.87; CRB: AVE=.62, CR=.90; TRB: AVE=.69, CR=.93) and the internal consistency was excellent (a=.86-.93). Thus, CBI-SS was a reliable and valid instrument for characterization of the factors, in this sample.

OLBI-SS and CBI-SS’ Effectiveness

Table 2 presents the data regarding the effectiveness of the instruments.

A high sensitivity and specificity for the OLBI-SS dimensions Exhaustion and PB, and CBI-SS’ dimension SRB, were observed.

Figure 1 shows the ROC curves for the dimensions of the Oldenburg Burnout Inventory (OLBI-SS) and the Copenhagen Burnout Inventory (CBI-SS).

Furthermore, an adequate discriminating capacity of the OLBI-SS dimension Exhaustion, and of the CBI-SS dimensions PB and SRB, was observed. The CBI-SS presented a discriminating capacity significantly higher than the OLBI-SS in identifying the Burnout Syndrome.



The assessment of latent measures is a challenge in the medical area, since they are not directly measurable, and are intrinsically related to the instruments’ metric qualities in the different samples7,26,27.

Another frequent difficulty is the lack of tools that may be considered the ‘gold standard’. To overcome this issue we used the instrument that is referenced, in the literature, as being able to produce data with stable metric quality in different samples. Thus, to evaluate the Burnout Syndrome the Maslach Burnout Inventory (MBI) is considered the standard instrument, as recommended by Schutte et al.22 (2000).

Before assessing the effectiveness of the instruments, the metric quality of the data collected with those instruments, in a specific sample, must be confirmed. In this study, so that the reliability and validity of OLBI-SS and CBI-SS could be considered appropriate, it was necessary to remove some items. These items were formulated in the opposite direction and intercalated with items formulated positively which; according to Yeh et al.28 (2007), this may have resulted in completion errors by the students, who may not have noticed the reversal of the scale’s response pattern.

After the adjustment of the scales, the best cutoff point for each factor can be defined. Peterson et al.16 (2008) had already proposed cutoff points for OLBI’s Exhaustion (≥2.25) and Disengagement (≥2.10) dimensions, based on MBI-GS. However, these cutoff points were proposed by the authors based on the data collected from a normative population of professionals from different areas. In the sample of students (Table 2) it can be seen that the cutoff points, which maximize sensitivity and specificity, are superior to those proposed by Peterson et al.16 (2008). Thus, caution should be exercised in the use of cutoff points for the evaluation of Burnout Syndrome in different populations. For CBI, previous studies that defined cutoff points for different factors were not found in the literature.

It is important to note that OLBI-SS’ exhaustion dimension and CBI-SS’ PB and SRB dimensions presented a greater effectiveness (Table 2, Figure 1), which can be attributed to the fact that the theoretical core of Burnout is Exhaustion18,29. The fact that the dimensions CRB and TRB present a lower predictive ability can denote that, for students, inter-personal relationships both with peers and with teachers have less impact on the development of exhaustion. It is noteworthy that these two dimensions were proposed by Campos et al.19 (2013) in their adaptation of the original factor “Burnout related to the clients” to the student population.

Knowing that the theoretical conceptualization of Burnout predicts the existence of a global latent state, and that factors from different scales are significantly correlated, the inclusion of a second-order factor (SOHM) creates the possibility of estimating an overall score for each scale23. Thus, using the global scores, we observed (Figure 1) that CBI-SS was more effective than OLBI-SS in the detection of Burnout Syndrome (OLBI-SS: AUROC=.674 (.610-.734); p<.05; CBI-SS: AUROC=.846 (.793-.889); p<.05; DAUROC=.172 (.103-.240); p<.05). This can be explained by the inventories’ structure14,18, since in OLBI the intercalation of positive and negative items may have hindered its completion by students. Moreover, the OLBI considers aspects related to Exhaustion and Disengagement, while CBI only addresses Exhaustion which represents the core of Burnout.



Thus, it can be concluded that the Copenhagen Burnout Inventory (CBI-SS) was more effective than the Oldenburg Inventory (OLBI-SS) in the detection of Burnout Syndrome in students, when the Maslach Burnout Inventory (MBI-SS) was used as the standard.



The authors would like to thank the grant #2010/16399-1, São Paulo Research Foundation (FAPESP).



Maslach C, Jackson SE. The measurement of experienced burnout. J Occup Behav. 1981; 2: 99-113.

Levert T, Lucas M, Ortlepp K. Burnout in psychiatric nurses: Contributions of the work environment and a sense of coherence. S Afr J Psychol. 2000; 30: 36-43.

Chambel MJ, Pinto AM. Burnout e engagement em contexto organizacional: estudos com amostras portuguesas. Lisboa: Livros Horizonte; 2008.

Schaufeli WB, Taris TW. The conceptualization and measurement of burnout: Common ground and worlds apart. Work & Stress. 2005; 19(3): 256-62.

Schaufeli WB, Martinez IM, Pinto AM, Salanova M, Bakker AB. Burnout and Engagement in university students. A cross-national study. J Cross Cult Psychol. 2002; 33(5): 464-81.

Maroco J, Tecedeiro M, Martins P, Meireles A. O Burnout como factor hierárquico de 2ª ordem da Escala de Burnout de Maslach. Anál Psicol. 2008; 4(26): 639-49.

Campos JADB, Zucoloto ML, Bonafé FSS, Jordani PC, Maroco J. Reliability and validity of self-reported burnout in college students: a cross randomized comparison of paper-and-pencil vs. online administration. Comput Human Behav. 2011; 27(5): 1875-83.

Humphris G, Blinkhorn A, Freeman R, Gorter R, Reddick GH, Murtomaa H, et al. Psychological stress in undergraduated dental students: baseline results from seven European dental schools. Comput Human Behav. 2002; 6(1): 22-9.

McManus IC, Keeling A, Paice E. Stress, burnout and doctor´attitudes to work are determined by personality and learnig style: a twelve year longitudinal study of UK medical graduates. BMC Med. 2004; 2: 29. PMid:15317650 PMCid:PMC516448.

Pohlmann K, Jonas I, Ruf S, Harzer W. Stress, burnout and health in the clinic period of dental education. Eur J Dent Educ. 2005; 9(2): 78‑84. PMid:15811155.

Ried LD, Motycka C, Mobley C, Meldrum M. Comparing self-reported Burnout of Pharmacy students on the fouding campus with those at distance campuses. Am J Pharm Educ. 2006; 70(5): 1-14.

Gil-Monte PR. Factorial validity of the Maslach Burnout Inventory (MBI-SS) among Spanish professionals. Rev Saúde Publica. 2005; 39(1): 1-8. PMid:15654454.

Hu Q, Schaufeli WB. The factorial validity of the Maslach Burnout Inventory - student survey in China. Psychol Rep. 2009; 105: 394-408.

Demerouti E, Bakker AB, Nachreiner F, Schaufeli WB. The job demands-resources model of burnout. J Appl Psychol. 2001; 86(3): 499‑512. Epub 2001/06/23.

Halbesleben JRB, Demerouti E. The construct validity of an alternative measure of burnout: Investigating the English translation of the Oldenburg Burnout Inventory. Work Stress. 2005; 19(3): 208-20.

Peterson U, Demerouti E, Bergstrom G, Asberg M, Nygren A. Work characteristics and sickness absence in burnout and nonburnout groups: a study of Swedish health care workers. Int J Stress Manag. 2008; 15(2): 20.

Campos JADB, Carlotto MS, Maroco J. Inventário de Oldenburg para estudantes: adaptação cultural e validação para português. Psicol Refl Crít. 2012; 25(4): 709-18.

Kristensen TS, Borritz M, Villadsen E, Christensen KB. The Copenhagen Burnout Inventory: a new tool for the assessment of burnout. Work & Stress. 2005; 19(3): 192-207.

Campos JADB, Carlotto MS, Maroco J. Copenhagen Burnout Inventory - student version: adaptation and transcultural validation for Portugal and Brazil. Psicol Refl Crít. 2013; 26(2): 87-97.

Campos JADB, Maroco J. Adaptação transcultural Portugal-Brasil do Inventário de Burnout de Maslach para estudantes. Rev Saúde Pública. 2012; 46(5): 816-24.

Qiao H, Schaufeli WB. The convergent validity of four burnout measures in chinese sample: a confirmatory factor-analytic approach. J Appl Psychol. 2011; 60(1): 87-111.

Schutte N, Toppinen S, Kalimo R, Schaufeli W. The factorial validity of the Maslach Burnout Inventory - general survey (MBI-GS) across occupational groups and nations. J Occup Organ Psychol. 2000; 73: 53-66.

Maroco J. Análise de equações estruturais. Lisboa: Report Number; 2010.

Cronbach LJ. Coefficient Alpha and the Internal Structure of Tests. Psychometrika. 1951;16(3): 297-334.

Maroco J. Análise estatística com utilização do SPSS. Lisboa: Reportnumber; 2011.

Honaker LM. The equivalency of computerized and conventional Mmpi administration - a critical-review. Clin Psychol Rev. 1988; 8(6): 561-77.

Suris A, Borman PD, Lind L, Kashner TM. Aggression, impulsivity, and health functioning in a veteran population: equivalency and test-retest reliability of computerized and paper-and-pencil administrations. Comput Human Behav. 2007; 23(1): 97-110.

Yeh WY, Cheng Y, Chen CJ, Hu PY, Kristensen TS. Psychometric properties of the Chinese version of Copenhagen burnout inventory among employees in two companies in Taiwan. Int J Behav Med. 2007; 14(3): 126-33. Epub 2007/12/07.

Schaufeli WB, Greenglass ER. Introdution to special issue on burnout and health. Psychol Health. 2001; 16: 501-10. PMid:22804495.



Juliana Alvares Duarte Bonini Campos
Faculdade de Odontologia, UNESP – Univ Estadual Paulista, Rua Humaitá,1680, Centro, 14801-903 Araraquara - SP, Brasil

Received: April 12, 2013
Accepted: July 24, 2013

CONFLICTS OF INTERESTS: The authors declare no conflicts of interest.

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License