Oldenburg Burnout Inventory – Student Version : Cultural Adaptation and Validation into Portuguese Inventário de Oldenburg para Estudantes : Adaptação Cultural e Validação para o Português

The Oldenburg Burnout Inventory for college students (OLBI-S) was adapted to Brazilian Portuguese and its reliability and validity were measured in a sample of both Brazilian (n=958) and Portuguese (n=602) college students. The confirmatory factor analysis of the OLBI-S showed good fit but two items were removed since they lacked individual reliability (λ<.50). The new structure showed good fit to 2/3 of the test sample and was invariant to the other 1/3 of the total sample. Convergent validity and internal consistency were low, but discriminant, concurrent and divergent validity were good. The OLBI-S was not invariant in the Brazilian and Portuguese samples. The adapted OLBI-S did not show cross-cultural validity.

The burnout syndrome was initially defined as a psychological condition related to work in help professions and that require interaction with others, and is characterized by high levels of emotional exhaustion, high disbelief in the function and low professional accomplishment.Initially, it was believed that the burnout syndrome affected only professionals with extensive human interaction as social workers, nurses, doctors and psychologists.However, currently it is known that the burnout syndrome exceeds care services, since the stressors that can initiate it may be present in any workplace (Demerouti, Bakker, Nachreiner, & Schaufeli, 2001).More recently, higher education students have been identified in the literature as a vulnerable group for the develop-ment of Burnout (Balogun, Helgemoe, Pellegrini, & Hoeberlein, 1995;Marôco & Tecedeiro, 2009;Marôco, Tecedeiro, Martins, & Meireles, 2008;Martinez, Pinto, & Silva, 2000;Schaufeli, Martinez, Pinto, Salanova, & Bakker, 2002) due to socio-economic pressures, concerns about their professional future, relationships with peers and teachers, tests and papers to which they are constantly exposed, and therefore constitute a risk population to be studied.Carlotto and Câmara (2006), Dyrbye et al. (2010), Martinez et al. (2000) and Salanova, Schaufeli, Martinez, and Breso (2010) and emphasize that early detection of symptomatic significant levels of burnout can be an indicator of potential problems, in both school and professional contexts, enabling preventive interventions.Marôco and Tecedeiro (2009) also indicate that the occurrence of Burnout can seriously limit students psychosocial wellbeing and the academic performance.From this perspective, the authors argue that the detection of Burnout and its determinants, in higher education, is an asset for understanding and developing psychological interventions in this area.
Despite the obvious need for studies on the burnout syndrome in students, these are incipient which, according Marôco and Tecedeiro (2009) and Marôco et al. (2008), may be occurring due to a lack of appropriate instruments to assess this syndrome in students.The only instrument proposed in the literature to assess the syndrome in students is the Maslach Burnout Inventory -Student Survey (MBI-SS; Schaufeli et al., 2002) whose psychometric properties have been evaluated and their stability attested in several studies, including university students in Portugal (Marôco & Tecedeiro, 2009;Schaufeli et al., 2002) and Brazil (Carlotto & Câmara, 2006).However, this scale has been the target of criticism leading several authors to propose alternative assessment tools (Demerouti, Bakker, Vardakou, & Kantas, 2003;Kristensen, Borritz, Villadsen, & Christensen, 2005).Demerouti et al. (2001), Demerouti et al. (2003) and Halbesleben and Demerouti (2005) criticize the fact that the MBI dimensions Exhaustion and Cynicism have a positive formulation, while the Professional Efficacy dimension has negative formulation, which can compromise the inventory's sensitivity and its discriminatory power.Another aspect highlighted by the authors is that the MBI only considers the emotional aspects of the exhaustion dimension excluding the physical and cognitive elements.
Given the criticisms made regarding the MBI, Demerouti and Nachreiner (1998) proposed the Oldenburg Inventory (OLBI) which consists of 16 items describing different states of emotional exhaustion and detachment.The degree of agreement with each item is expressed in 4-point ordinal scale (from 1 -totally disagree to 4 -totally agree).Half of the items were formulated positively and half were formulated negatively.This instrument was proposed originally in German, and was based on a theoretical model that assumes that Burnout is a twodimensional syndrome that can occur regardless of the individual's occupation.The validity of OLBI was tested in different population groups (Demerouti et al., 2001;Demerouti et al., 2003;Demerouti, Mostert, & Bakker, 2010;Halbesleben & Demerouti, 2005;Peterson, Demerouti, Bergstrom, Asberg, & Nygren, 2008).
Taking into consideration that OLBI was developed to meet different occupational groups, its use in students can be an interesting alternative to MBI-SS.Thus, we carried out this study with the objective of proposing and assessing the reliability and validity of the Portuguese version of the Oldenburg Burnout Inventory adapted for students (OLBI-S).

Participants
Participants were higher education students, volunteers, enrolled in institutions in Brazil and Portugal.The invitation to participate in the study was done for each institution by researchers, in person or via an e-mail sent to the schools Direction.The instruments were completed by a total of 1052 Brazilian students and 612 Portuguese.However, only 958 Brazilians (Response Rate -RR = 91.1%)and 602 Portuguese (RR = 98.4%) completed all OLBI's items and were included in the study.The mean age of the Brazilian students was 23.3 (SD = 5.4) years and of the Portuguese 22.9 (SD = 5.0) years.The characteristics of the participants are presented in Table 1.
To characterize the sample, information on gender, age, area of undergraduate course attended, type of institution, classes shift, dwelling, studies' financing, use of medication due to studies and thought about quitting the course, were assessed.

Instruments
Given that we did not find in the literature a Portuguese version of OLBI, in this study translation, and back translation, from the original English version by Halbesleben and Demerouti (2005) was performed, for development of the instrument based on the Portuguese language orthographic agreement (Ministry Science, 2008), being adapted for its application in university students.The adapted instrument is called by us the Oldenburg Burnout Inventory -Student Version (OLBI-S).

Procedures
A website was created for the sociodemographic questionnaire and OLBI-S' Portuguese version.The questionnaires were available for completion online for 7 months (May to November).Each web page hosted one instrument, so that the respondent could view all items simultaneously.Non-responses to items were allowed, and the participant could return to verify and/or correct the answer to each inventory before its submission.Before any analysis, all items with negative statements were reversed so that the scores of all items were in the same conceptual direction (lower scores represent disagreement, higher scores correspond to agreement with the statements).

Psychometric Analyzes
Face Validation.The process of face validation involved the participation of a multidisciplinary team (portuguesespeaking psychologists) with eight members.The idiomatic, semantic, cultural and conceptual equivalence of the instrument were analyzed in order to obtain agreement and consensus.Thus, an intermediate version of the instrument was obtained.This version was pre-tested in a group of 20 students to assert the misunderstanding index of each question.Content Validation.A total of 13 Psychology professionals (judges) Participated in this step, to analyze each OLBI-S' items regarding essentiality and classified them into "essential", "useful but non-essential" and "not necessary".For each item, the number of judges categorizing the item as "essential" was computed for calculation of the Content Validity Ratio (CVR).To decide the significance of each item Laewshe's proposal (1975) was used, and a 5% significance level was adopted.

Psychometric Qualities
Sensitivity Analysis of Items.The psychometric sensitivity was investigated with the dual purpose of evaluating both the shape of the responses' distribution to each item and the approximate normality of the items to be used in the confirmatory factor analysis.The psychometric sensitivity was assessed for each item by calculating descriptive statistics, including measures of central tendency and measures of shape.We considered that items with skewness (Sk) and kurtosis (Ku) close to 0 had maximum sensitivity, and that the greater the difference from 0 the lower the items' sensitivity.Note that, in the population, the distribution of the responses to each item must follow an approximately normal distribution to maximize the ability of the item to discriminate structurally different individuals.Moreover, it is expected that the items' scores present a normal distribution if they are good projections of the construct, which is supposed to be normally distributed in the population.We considered that items with a Sk greater than 3, in absolute value, and Ku greater than 7, in absolute value, had sensitivity and normality problems that recommend against their use in factor analysis (Marôco, 2010).

Construct Validity
Factorial Validity.We carried out confirmatory factor analysis (CFA) of the Portuguese version for students (OLBI-S) to verify the two-dimensional structure proposed by Halbesleben and Demerouti (2005).In this sense the following goodness of fit indices were used χ 2 /df (ratio chi-square and degrees of freedom), CFI (confirmatory fit index), GFI (goodness of fit index) and RMSEA (root mean square error of Approximation).The model's adjustment was considered good for CFI and GFI values above .9and RMSEA values below .06(Boomsma, 2000;Byrne, 2001;Marôco, 2010;McDonald & Ho, 2002).To assess the AFC we used AMOS ® 18.0.
Factorial Invariance.Initially, to verify the stability of the factor solution obtained, a cross-validation of the model was made in order to compare the indexes observed in the validation sample with another independent test sample coming from the same population (Hair, Black, Babin, Anderson, & Tatham, 2005;Marôco, 2010).For this the total sample was divided into three equal parts with two parts constituting the "validation sample" and one part the "Test sample" to perform a cross testing of the hypothetical model.The invariance test was conducted imposing equality restrictions to the factorial weights of the two groups, being the statistical test the difference between the χ 2 of the model fixed factor weights and the model with equal factor weights.When the hypothesis of factorial invariance was accepted, the analysis of the invariance of the correlations between factors was performed, and finally the invariance of specific factors (residues; Kaplan, 2000).Then, the same procedure was made to assess the invariance of the factor solution obtained in the Portuguese sample and in the sample Brazilian.
Convergent Validity.To examine whether each dimension's observed variables were strongly related to each other, the convergent validity was evaluated, to this end the Average Variance Extracted (AVE) and the Composite Reliability (CC) were estimated (Fornell & Larcker, 1981;Marôco, 2010).According to Hair et al. (2005) values VEM j >.5 and CC j >.7 are indicative of convergent validity and adequate construct reliability being that, for exploratory research only, values above these cutoff points may be acceptable.
Discriminant Validity.Discriminant validity assesses whether the items that reflect a dimension are not correlated with another dimension, i.e., if the average variance extracted for each factor is greater than the average variation shared between each factor and other factors in the model (Marôco, 2010).The discriminant validity was estimated according to Fornell and Larcker proposal (1981) as described in Marôco (2010) who claim that for two factors i and j, if VME i and VME j > ρ ij 2 (ρ ij 2 : squared correlation between factors i and j) evidence of discriminant validity exists.
Criterion Validity.The criterion-related validity was assessed through the concurrent validity and discriminant validity.For estimating both the concurrent validity of the divergent validity, Pearson's correlation analysis was used.For the former, the mean score of OLBI-S each factor was correlated with those obtained in each dimension of the Maslach Burnout Inventory (MBI-SS).For divergent validity OLBI-S' factors scores were correlated with the average of the scores obtained with the Beck Depression Scale (BDI).
Reliability.The internal consistency was assessed using standard Cronbach's alpha coefficient (α) for each dimension proposed in the inventories, and to calculate the consistency of the total scale stratified alpha Coefficient (α estr ) was estimated.
Ethical Aspects.The present study was approved by the Ethics Committee on Human Research of the Lutheran University of Brazil, Canoas / RS (protocol: 2010-188H).

Results
Table 2 presents the Portuguese version of the Oldenburg Inventory adapted for students (OLBI-S) after face validation.
It should be clarified that the statements in Table 2 were grouped according to their dimension to facilitate the understanding of OLBI's structure (Demerouti et al., 2001).
In the pre-test we observed that no item showed index misunderstanding <.20, thus, the calculation of the content validity ratio was made (CVR; Table 3).In the opinion of the judges only 4 items of the inventory are essential to investigate the Burnout Syndrome in students.
Summary measures of OLBI-S' items to characterize the psychometric sensitivity of the Portuguese and Brazilian sample can be found in Table 4.
All items presented skewness and kurtosis values close to those of a normal distribution (Sk = 0, Ku = 0) in both samples.
Items that had factorial weights (λ) negative and lower than .50were removed.Items whose errors are correlated with the factors or other errors, according to the modifi-cation indices provided by AMOS®, were also removed.The decision on reducing the items was defined with the goal of producing a scale with greater reliability and validity, preserving the original factor structure.
The Confirmatory Factor Analysis pointed to an appropriate adjustment (χ 2 /df = 4.543, CFI = .913,GFI = .944,RMSEA = .058),however, the factorial weights of  items 5 and 12 of the Exhaustion factor, and item 13 of the Disengagement factor were below .5.Thus, there was a refinement of the original model using the modification indices for both obtained with AMOS ®.Item 5 was deleted, since the modification indices indicated a correlation of the measuring error with the Disengagement dimension and item 13 of the Exhaustion dimension.None of the re-moved items, except one, were considered "essential" by the content validity panel, therefore usuring that these items remotion did not affect the content validity of the factors under study.Thus, we propose a reduced scale, with the dimensions Exhaustion and Disengagement consisting of 7 items each (Figure 1).The values shown in the figure are standardized estimates of covariance between factors, factorials weights and explained variance of each item respectively.We can observe that the factor model has appropriate quality of adjustment indices.All items of the scale (OLBI-S), with the exception of item 14, showed factorial weights (λ≥.5) and adequate individual reliability (R 2 ≥.25).The option of maintaining item 14 on the scale was due to the fact that the modification indices did not suggest significant changes in the quality of the adjustment associated with the removal of that item.
In the external validation of the modified structure, we analyzed the invariance of the model in the test sample and the validation sample.The evaluation in both samples, simultaneously, showed very good adjustment indices (χ 2 / df = 3.777, CFI: .911;GFI: .938,RMSEA: .042).
The differences in the goodness of fit of the factor model to the two samples revealed no significant differences.Similarly, neither the covariances between factors nor the residues associated with the items revealed significant differences between the two samples.These observations point to the invariance of the model in the two independent samples confirming the stability of the factor structure proposed in this study of adaptation and validation [λ: χ 2 (12) dif =19.666, p=.074; Cov: The convergent validity for OLBI-S was low (VEM Ex =.358; VEM Dist =.363; CC Ex =.793; CC Dist =.797) while discriminant validity was adequate (r 2 = .067).
The internal consistency of the Exhaustion dimension was below the desired levels (α = .565)and was adequate for the Disengagement dimension (α = .700).The total scale OLBI-S showed adequate consistency (α estr = .952).
The correlational analysis between the reduced Oldenburg Inventory for students (OLBI-S) and the Maslach Burnout Inventory -Student Survey (MBI-SS) and Beck's Depression Inventory (BDI) is presented in Table 5.
The correlation coefficients found between the OLBI-S and the MBI-SS dimensions point to a moderate concurrent validity.Moreover, the low correlation between the dimensions of OLBI-S and BDI denote good divergent validity from a measurement scale for depression distinguishing the two constructs.
A cross-cultural equivalence between countries was not found.In the Brazilian sample items 14, 1, 7 and 11 showed factor weights lower than .50,while in the Portuguese sample this only occurred for item 6.The correlation between the two scales of OLBI-S is also considerably higher in the Portuguese sample when compared to the Brazilian sample (r = .69vs. r = .59).

Discussion
This study presents a version in Portuguese of the Oldenburg Burnout Inventory, adapted for college students, and examines its reliability and validity when ap-plied to a sample of Brazilian and Portuguese students.Despite any discussion, it is worth noting that so far the psychometric properties of OLBI had been tested only on workers from various fields (Demerouti et al., 2001;Demerouti et al., 2003;Demerouti et al., 2010;Demerouti  & Nachreiner, 1998;Halbesleben & Demerouti, 2005, Peterson et al., 2008), to the best of our knowledge, this is the first study to use this inventory and test its properties when applied to students.
The results of this study confirm the two-dimensional structure of OLBI, previously described by Demerouti et al. (2001), Demerouti et al. (2003), Demerouti et al. (2010), Demerouti and Nachreiner (1998) and Halbesleben and Demerouti (2005), however, it should be emphasized that to achieve adequate factorial validity, an item of each dimension of the instrument was excluded (Figure 1).This occurred because the confirmatory factor analysis indicated factorial weights of the items of the original version of the instrument were below desired values.This structure was then confirmed in a second similar sample, but independent from the first sample, where the factor structure was refined.After the refinement of the model, the only quality adjustment index that was inadequate was χ 2 /df, however, this fact is not relevant since this index is extremely sensitive to the number of variables in the model and sample size, as already emphasized by Bentler (1990) and Marôco (2010).
Another aspect to be highlighted is that in the content analysis (Table 2) only four items from OLBI were considered, by the experts, as essential for evaluation of the Burnout syndrome in students.One of these four items, the item O5 belonging to the Exhaustion dimension, was subsequently removed from the inventory, as it presented factorial weight lower than .50.Given that the sample is big and includes students from different areas of study, from different years and two Lusophone countries, it is sustainable in the conclusion that this item is a weak expression of Exhaustion in the student population.It should also be noted that this item also presented weak correlations (r = .3for Exhaustion, r = .2for Cynicism and Effectiveness) with scores on MBI-SS' factors.These observations contradict the assessment of the relevance of the item by the experts.
OLBI's concurrent validity when applied to workers from different professional categories has been ascertained with the Maslach Burnout Inventory -General Survey (MBI-GS).Demerouti et al. (2003) and Halbes-leben and Demerouti (2005) found a high correlation between the dimensions Exhaustion and Disengagement/ Cynicism from OLBI and MBI-GS, pointing to OLBI's high concurrent validity.Because it is a research with students, the concurrent validity of OLBI-S was measured before with the student version of the MBI (MBI-SS; previously validated for the population of Portugal and Brazil by our research team Campos et al., 2011;Carlotto & Câmara, 2006;Marôco et al., 2008) and moderate correlations can be noted between the dimensions of the instruments (Table 5) indicating that they are part of the same conceptual model.In addition, the low convergent validity observed may indicate that the scale is not measuring the intended concept (Hair et al., 2005), in other words, items that should be reflective of a factor are not heavily saturating in this factor (Marôco, 2010) When the refined model is implemented separately for the Brazilian and Portuguese sample there is an absence of cross-cultural equivalence, which reflects mainly the differences found between the factor weights of each item of the instrument in the different samples (Figure 2).Before attributing this disparity to cultural differences between the countries, it should be noted that in this study the characteristics of the samples were not paired (Table 3).This fact is a limitation that must be considered, since the psychometric properties of an instrument are specifically related to an individual population and is not, therefore, an absolute characteristic of each test, as already pointed out by Honaker (1988) and by Suris, Borman, Lind and Kashner (2007).However, the possible influence of the sample characteristics in the factor validity of the model can point to low factor stability of OLBI-S when used in different populations.It is, however, important to highlight that despite significant differences between the factorial weights of the items on the two factors considered (exhaustion and cynicism) in samples from Portugal and Brazil, the same items were retained in the analysis of factorial validity in both the Portuguese sample and the Brazilian sample.This indicates the stability of the factor structure proposed for OLBI-S adapted into Portuguese, not dismissing however other studies in different samples.
Despite confirming the two-dimensional structure of the Portuguese version of OLBI-S, it had limitations regarding the convergent validity and internal consistency.Factor structure was not invariant between the Brazilian and Portuguese sample pointing to the absence of cross-cultural validity

Table 1
Assessment [n (%)] Characteristics of the Participating Students

Table 2
Portuguese Version of the Oldenburg Inventory Adapted for Students (OLBI-S) After work, I tend to need more time than in the past in order to relax and feel better *O4.Depois das aulas/estudo preciso de mais tempo para relaxar e sentir-me melhor do que precisava antigamente O5.I can tolerate the pressure of my work very well O5.Consigo suportar muito bem as pressões dos meus estudos *O8.During my work, I often feel emotionally drained *O8.Durante os meus estudos, sinto-me emocionalmente esgotado O10.After working, I have enough energy for my leisure activities O10.Depois das tarefas escolares, tenho geralmente energia para as minhas atividades de lazer O16.When I work, I usually feel energized O16.Quando estudo, sinto-me geralmente com energia *O12.After my work, I usually feel worn out and weary *O12.Depois dos meus estudos sinto-me cansado e sem energia O14.Usually, I can manage the amount of my work well O14.De uma forma geral, consigo administrar bem a quantidade de trabalho que tenho

Table 3
Content Validity Ratio of the Portuguese Version of Oldenburg Inventory for Students (OLBI-S) a Values below the significant minimum.

Table 4
Analysis of the Items Composing of the Portuguese Version of Oldenburg Inventory for Students (OLBI-S) -Brazilian (BR) and Portuguese (PT)