Psychometric properties of the Burnout Inventory for Referees

Abstracts

The purpose of this study was to assess the psychometric properties of the Burnout Inventory for Referees (BIR) in Portuguese in a sample of professional soccer referees and to explore the nomological validity of the inventory through concurrent validation. The analysis showed a factorial structure of burnout for the referees with one second order factor and three first order factors; physical and emotional exhaustion, reduced sense of accomplishment and sport devaluation. A cross-validation procedure showed that the factor structure was replicable, and the inventory demonstrated internal reliability as well as convergent and discriminant validity. In addition, the study assessed the concurrent validity of the inventory by examining the correlation between the scores of the Burnout Inventory for Referees and the scores of the Profile of Mood States (POMS). The subscales of burnout and total burnout were positively correlated with tension, depression, anger, fatigue and confusion and negatively with vigor, as expected. In conclusion, the Portuguese version of BIR presents good evidence of validity and reliability in the assessment of referees who may be at risk of suffering from burnout.

burnout; psychological assessment; mood states; referee

"Propriedades psicométricas do Inventário de Burnout para Árbitros." O objetivo deste estudo foi avaliar as propriedades psicométricas do Inventario de Burnout para Árbitros (BIR) em português em uma amostra de árbitros de futebol profissional e explorar a validade nomológica do inventário através da validação concorrente. A análise mostrou uma estrutura fatorial de burnout para os árbitros com um segundo fator de ordem e três primeiros fatores de ordem; esgotamento físico e emocional, reduzido senso de realização e desvalorização esportiva. Um procedimento de validação mostrou que a estrutura de fator é replicável e o inventario apresenta uma confiabilidade interna, validade convergente e discriminante. Além do mais, o estudo avaliou a validade concorrente do inventario através da correlação entre os escores do BIR e o Perfil dos Estados de Humor (POMS). As subescalas de burnout e o burnout total se correlacionaram positivamente com a tensão, depressão, raiva, fadiga e confusão e negativamente com vigor, como esperado. Para concluir, o BIR em português mostrou ser confiável e válido na avaliação de árbitros que podem estar em situação de risco para o burnout.

burnout; avaliação psicológica; estados de humor; árbitros

"Propriedades psicométricas del Inventário de Burnout para Árbitros." El objetivo de este estudio fue evaluar las propiedades psicométricas del Inventario de Burnout para Árbitros (BIR) en portugués en una muestra de árbitros de fútbol profesional y explorar la validez nomological del inventario a través de la validación concurrente. El análisis mostró una estructura del factorial de burnout para los árbitros con un segundo factor de orden y tres primeros factores de orden; agotamiento físico y emocional, sentido de realización reducido y devaluación del deporte. Un procedimiento de validación mostró que la estructura del factor era replicable, y el inventario demostró la fiabilidad interna, validez convergente y discriminante. Además, el estudio evaluó la validez concurrente del inventario a través de la correlación entre los escores del BIR y el Perfil de los Estados de Humor (POMS). Las sub-escalas de burnout y burnout total se correlacionaron positivamente con tensión, depresión, rabia, fatiga y confusión y negativamente con el vigor, como esperado. Para concluir, el BIR en portugués presenta el testimonio confiable de validez y fiabilidad en la evaluación de árbitros que pueden estar en peligro del sufrimiento de burnout.

burnout; evaluación psicológica; estados de humor; árbitros

Introduction

Burnout is a syndrome which has been described in the literature in the past 40 years and is defined as a prolonged response to chronic stress which leads to fatigue and emotional exhaustion or loss of physical and mental energy resulting in interpersonal relationship problems, especially at work. It is rooted in three dimensions or subscales including emotional exhaustion, reduced sense of accomplishment and depersonalization (Fejgin, Talmor, & Erlich, 2005; Maslach, 1976; Maslach & Jackson, 1981; Roelofs, Vebraak, Keijsers, de Bruin, & Schmidt, 2005). Burnout is a dysfunctional condition that develops gradually and may even go unnoticed for a long time (Schaufeli & Enzmann, 1998). Burnout is characterized by five main elements: i) predominance of physical symptoms and mental or emotional exhaustion, tiredness and depression; ii) atypical stress physical symptoms may occur; iii) work related symptoms; iv) symptoms appear in individuals who do not suffer from psychopathologies and, v) detrimental to effectiveness of work performance as result of negative attitudes or behaviors (Roelofs et al., 2005). It is an existing phenomenon not only in the workplace, but also in other contexts such as sport. Specifically, there are people whose expectations about their profession have an existential meaning, a sense that their occupation provides significance in their life, and they work for long periods of time in situations of highly emotional demands. These individuals can feel emotionally exhausted, modify their attitudes and develop burnout if these situations become chronic (Maslach, 1998; Maslach & Leiter, 2000; Maslach, Schaufeli, & Leiter, 2001).

Different studies conducted on burnout in sport have focused especially on athletes and coaches (Gould, Dieffenbach, & Moffett, 2002; Goodger, Gorely, Lavallee, & Harwood, 2007; Gustafsson, Hassmén, Kenttä, & Johansson, 2008; Hendrix, Acevedo, & Hebert, 2000, Hodge, Lonsdale, & Ng, 2008; Pires, Brandão, & Silva, 2006; Raedeke, 2004; Raedeke et al., 2002). These researchers believed that participation in sports, although challenging and rewarding, can also be frustrating and emotionally exhausting. Consequently, the burnout syndrome is developed, in part, due to the constant interpersonal relationships frictions with teammates, referees, coaches, friends and relatives. Thus, negative feelings are often experienced through the expectations and criticism from others, resulting in prolonged stress and eventually burnout.

As noted above, as burnout in sport has important implications for athletes and coaches, referees also can be affected by this phenomenon. Actually, this is no surprise, because, officiating in any sport at any level of competition is not an easy task. Guillén and Jiménez (2001) and González Oya (2006) noted that officiating presents high complexity, not only due to the difficulty of making decisions in a short period of time, but also by exposure to criticism and pressure from spectators, athletes, and coaches. Furthermore, positive reinforcements are scarce, and probability of ending a sporting event without making any mistakes is low. Specifically, Louvet, Gaudreau, Menaut, Genty and Deneuve (2009) stated:

Indeed, referees have to deal with interpersonal conflicts and conflicts arising from the criticism of players, coaches, and even leaders. They are almost always attacked physically and verbally by players, coaches and unsatisfied fans, which establish an environment dominated by fear of making mistakes during the game. A great number of referees have to deal with family, professional and social life, what can be a stressful task. On top of that, competition among referees and the poor social appreciation of this profession, so it is no surprise that they live under high levels of stress (p. 125).

According to Balch and Scott (2007), there is no easy relationship between participants and referees in sport. Historically, a referee is often the center of controversies and the focus of anger by the fans who normally believe they tend to "make bad calls" against their team. Weinberg and Richardson (1990) stated that the oral abuse and physical threats from coaches, players and fans, criticism of the press and assessment by officiating peers, are part of the everyday life of referees. As a consequence, burnout and dropout may reflect the failure in dealing with disruptions resulting from interpersonal conflicts, role conflicts, fear of failure and other contextual factors involved in officiating (Taylor, 1991). Furthermore, the context of officiating involves evaluation and social comparison because the individual efforts are witnessed by peers and significant others (Brandão, Serpa, Krebs, Araújo, & Machado, 2011).

Based on the knowledge from theoretical essays about the burnout syndrome, there was a need to develop a reliable and valid measurement of burnout. Garcés de Los Fayos et al. (1999) presented a review of 15 burnout-measuring tools, including the "Maslach Burnout Inventory" (MBI), which is considered the first psychometrically valid measurement tool. As with research involving other professions, in sport the MBI was used to help develop the Athlete Burnout Questionnaire (ABQ) to measure burnout in athletes (Raedeke & Smith, 2001).

There is no doubt that the use of psychological inventories is one of the main pillars of psychology, as emotional competences are often evaluated through psychological instruments. But, in order to produce valid and reliable measures, the psychological tests need to reflect the specific nature of the context, increasing the ecological validity of this instrument. This ecological vision of the inventories makes it possible to analyze individuals in their real world and gives proper attention to the importance of the relationship between the environment and the psychological competences of the individuals.

Along these lines Weinberg and Richardson (1990)developed a measuring tool of burnout adapted for referees, called the Burnout Inventory for Referees. However, the inventory has methodological limitations because no validation studies were carried out. Additionally, although the work by Weinberg and Richardson (1990) made an important contribution to our understanding about the burnout syndrome, the utility of the scale is further limited by language barriers. Furthermore, the validation of the scale in a different language (Portuguese) will enable further testing of the psychometric qualities of the scale and will encourage future cross-cultural research on burnout.

Thus, the purposes of this study were twofold: (1) to assess the psychometric properties of the Burnout Inventory for Referees (BIR) in a sample of professional soccer referees and (2) to explore the nomological validity of the inventory through the concurrent validation of BIR by examining correlations between BIR and Profile of Mood States (POMS) scores.

Study 1

The purpose of the Study 1 was to provide validity evidence for the measures of the BIR in the Portuguese language.

Method

Participants

Two hundred and twenty-four male professional soccer referees (113 referees and 111 assistants; 64 from Portugal and 160 from Brazil) participated in Study 1. An assistant referee was one of several officials who assisted the head referee in controlling a match. He could be a linesman or an official who assisted in administrative or other match related tasks, as directed by the referee. A total of 361 referees received the protocols and 242 sent them back, which corresponded to a 62 % response rate. Excluded from the sample of the present study were referees who have disconnected their association with the referee organizations of both countries.

Procedures

Prior to the main study, permissions were requested from the soccer referee associations in Brazil and Portugal to administer the translated version of the Burnout Inventory for Referees to their associated members. Next, the referees were contacted by email with an explanatory video about the objectives of the study and guidelines about the participation (strictly voluntary). The content of their answers, were treated confidentially. The referees in this study lived in different places in Brazil and Portugal, suggesting that data collection by email was the most appropriate method. Specifically, the use of e-mail (internet) has been shown to increase the response rate of participants (Lonsdale, Hodge, & Rose, 2006).

Referees gave written consent for their voluntary participation in the study. Then they received the Portuguese version of the BIR and a biographical data form (age, gender, educational background, officiate level, and average length of time of officiating). The two instruments were completed in their own homes and answers were forwarded, according to the convenience of each participant and to ensure confidentiality, directly to the email of the principal researcher without going through the referees' supervisors. This investigation was approved by the Ethics Committee in Research at the University of São Judas Tadeu (# 062/2008).

Instruments

The translation of the BIR followed the guidelines for cross-cultural adaptations of measuring tools in the health sciences (Beaton, Bombardier, Guillemin, & Ferraz, 2002) and consisted of seven steps. These included: (1) translation of the English Inventory into Portuguese by two professionals from the respective countries, who were fluent in the English language and were also sport psychologists, generating two versions (T1 and T2) of the instrument; (2) synthesis of the two translations to determine the T3 version of the instrument; (3) reverse translation of T3 by two native English speakers who were fluent in the Portuguese language, generating the T4 and T5 versions; (4) synthesis of the two translations to determine the T6 version of the instrument; (5) evaluation of the inventory by PhD specialists, generating the T7 version of the instrument; (6) sending the T7 version to the author of the original instrument for an expert opinion and, (7) final version of the inventory (T8).

Finally, the translation of Portuguese in both Brazil and Portugal was carefully made in order to use common terms of both languages in such a way that there were no differences in the interpretation of the sentences. Thus, the same instrument could be used by referees from both countries.

Data analysis

The confirmatory analysis of the factorial structure of the BIR to Portuguese version was accomplished with AMOS 18 (SPSS Inc, Chicago, Il) software by using the maximum likelihood method (ML). The assumptions of the confirmatory factorial model, namely, the multivariate normality of the items and the absence of outliers were evaluated by the coefficients of skewness and kurtosis and by Mahalanobis distance, respectively (Arbuckle, 2009). To evaluate the quality of the global adjustment of the model, the indicative values of good adjustment CFI and GFI superior to .9 and PCFI and PGFI superior to .6 were considered. A RMSEA (root mean square error of approximation) value less than .06 was indicative of good fit, while an acceptable fit was assumed for a minimum cut-off of .08 (Byrne, 2000). The adjustment of the model was made from the values of the modification indexes by the Lagrange Multipliers (LM), by considering that trajectories and/or correlations with LM>11 (p< .001) were indicators of significant variation of the quality of the model. Internal consistency was estimated through composite reliability and values above .70 were considered indicative of good internal consistency (Hair, Black, Babin, & Anderson, 2009). Convergent validity was evaluated through the average variance extracted (AVE) and values greater than .50 were considered indicative of good convergent validity (Fornell & Larcker, 1981; Hair et al., 2009). Discriminant validity was accepted when AVE for each construct was greater than the squared multiple correlations between that construct and any other (Fornell & Larcker, 1981). The scores of the factors were achieved by using the multiplication of the scores obtained by the respective weight factors by the regression imputation method built in AMOS 18.

Cross-validity was also verified by a multi-group analysis. In addition, a multi-group analysis was conducted for testing the model's invariance among referees and assistants. To test the difference between two invariance models, we did not rely on the chi-square difference test, as it is judged to be too restrictive. Instead we rely on the change in CFI value (Byrne, 2009), which has to be lower than .01 (Cheung & Rensvold, 2002). This analysis was tested comparing the unconstrained model (structural weights varying freely across both groups) with the constrained models (measurement weights and structural weights fixed to an equal value to both groups) (Loehlin, 2003).

Results

The referees in this study officiated in the National and International Soccer Championships in Brazil and Portugal in 2008/2009. Sixty-two of them had college degrees and 162 were high school graduates. In addition, 21 officiated in international level championships (e.g., FIFA, EUFA), 92 in nationwide championships and 111 in statewide championships. Their average age was, 31.9 years (SD= 5.8) and average length of time of officiating was 9.4 years (SD= 5.8).

To examine the factorial validity of the Portuguese version confirmatory factor analysis was conducted. To evaluate if the items were close enough to the normal distribution and could be used in further factorial analysis, its skewness and kurtosis values were examined. This assumption was accepted for absolute values of skewness and kurtosis lesser than 3 and 10, respectively (Kline, 1998). Initial descriptive statistics analysis showed that the four subscales were normally distributed because all the standard kurtosis and skewness scores were small. A preliminary analysis of the scale was conducted and the initial confirmatory factorial analysis of the factorial structure of the BIR indicated that the original model presented an inadequate adjustment to the sample under study (χ2/gl=3.52; CFI= .642; PCFI= .540; GFI= .816; PGFI= .606; RMSEA= .107; P[rmsea ≤ .05] < .000).

The scale was than refined based on individual and composite reliabilities analyses. In addition, extracted average variance (Table 1), and interconstruct covariances (Table 2) were used to examine convergent and discriminant validity, respectively.

Table 1.
Factor loadings, Z-values, composite reliability, and Average Variance Extracted (AVE) for the first-order constructs

Item	Factor loading	Z-value	Composite Reliability	AVE
Physical and emotional exhaustion
Item 1: I feel emotionally drained from officiating	.81	12.35
Item 2: I feel fatigued when I get up in the morning and have to face another officiating assignment	.51	4.72
Item 14: I am totally spent at the end of officiating an event	.78	8.23	.86	.68
Sporting devaluation
Item 6: Working with players and coaches is really a strain on me	.50	5.63
Item 8: I woory that officiating is hardening me emotionally	.77	10.78
Item 13: I’ve become more callous toward people since I started officiating	.59	11.91	.65	.44
Reduced sense of sports accomplishment
Item 9: I feel frustrated by officiating	.70	8.99
Item 11: I feel burned out from officiating	.79	12.17
Item 15: I feel like I’m at the end of my rope	.59	7.69	.74	.50

Table 2.
Discriminant validity results for the first-order constructs

		EX	DES	RED
AVE	.68	.44	.50
EX	.68	1
DES	.44	.29	1
RED	.50	.17	.22	1

• Note. EX: Physical and emotional exhaustion; DES: Sporting devaluation; RED: Reduced sense of sports accomplishment.

According to the goodness of the fit index, the new global model showed a satisfactory fit to the data (χ2/gl = 2.55; CFI= .932; GFI= .947; RMSEA= .058; [rmsea ≤ .05] = .156). The scale shows, in this context, the factorial structure of burnout for the sports as described by Raedeke and Smith (2001). Figure 1 displays factorial weight and the individual reliability of the items of each factor, as well as the correlations among factors.

Figure 1.
Structural model showing the factorial and individual reliability of items for each factor, as well as correlations among the factors.

The model, considered acceptable, confirms the factorial structure of the initial model and accepts as an indicator of the physical and emotional exhaustion factors in items 1, 2 and 14; the sport devaluation factor included as items 6, 8 and 13 and the reduced sense of accomplishment factor was determined in items 9, 11 and 15. These item numbers came from the original inventory of Weinberg and Richardson (1990), but Table 3 has proposed the final inventory with the items appropriately noted.

Table 3.
Proposed Burnout Inventory for Referees.

Intensity		1		2	3	4	5	6	7
		Not strong							Extremely strong
1			I feel emotionally drained from officiating.
2			I feel fatigued when I get up in the morning and have to face another officiating assignment.
3			I am totally spent at the end of officiating an event.
4			I worry that officiating is hardening me emotionally.
5			Working with players and coaches is really a strain on me.
6			I’ve become more callous toward people since I started officiating.
7			I feel frustrated by officiating.
8			I feel burned out from officiating.
9			I feel like I’m at the end of my rope.

• Instructions for the assessment: Physical and emotional exhaustion = (1+2+3); Sporting devaluation = (4+5+6); Reduced sense of sports accomplishment = (7+8+9)

The model was also submitted to a cross-validation study. In the typical cross-validation, parameters of a model are estimated from one sample and the predictive effectiveness of the model is subsequently determined on a separate independent sample drawn from the same population as the first. So, our sample was divided in two blocks; the calibration and the validation model. The model's invariance in both samples was tested by comparing the unconstrained model with the constrained models (factor loadings fixed and variances/co-variances fixed). Factorial invariance was accepted when the models did not differ significantly (p> .05), according to the χ² statistic (Loehlin, 2003). The first study (i.e., calibration) was carried out with a sample of 162 individuals and the validation study was carried out with a sample of 224 individuals. The calibration and validation demonstrated the existence of an invariance structure of three factors of first order and one factor of second order, as expected by the theory (Table 4).

Table 4.
Multi-group analysis across the unconstrained model and the constrained models of the scale.

Multi-group models	χ²	df	∆ χ²	∆df	p
Model 1	86.73	34	---	---	---
Model 2	87.42	39	.629	5	.98
Model 3	87.91	42	.181	8	.99

• Note: Model 1 = Unsconstrained; Model 2 = Measurement weights; Model 3 = Structural weights

According to Weinberg and Richardson's (1990)initial proposal, it was possible to define burnout as a second order factor correlated with physical and emotional exhaustion (r = .60, p < .001), with sport devaluation (r = .81, p < .001) and with reduced sense of accomplishment (r = .68; p = .001). Finally, in relation to first order factors, sport devaluation is correlated positively with reduced sense of accomplishment (r = .55; p < .001) and with physical and emotional exhaustion (r = .41; p < .001). The correlation between sport devaluation and physical and emotional exhaustion was .48 (p < .000).

Invariance between referees and non-assistants was also tested as the CFI changes between the two models that represented configural and metric invariance was lower than .01. We assumed that there was a factor loading invariance between both groups of referees.

Discussion

The purpose of Study 1 was to investigate the psychometric validity of the BIR. This procedure has shown that the internal consistency as well as the construct validity of the instrument exhibited loadings of strong factors and produced scores for subscales that were internally consistent. Therefore, it is valid to measure the intensity of the burnout experiences in soccer referees. This form has nine items that reflect a multidimensional characteristic with three subscales, physical and emotional exhaustion, sport devaluation and reduced sense of sports accomplishment. These subscales basically reproduced the results of studies on burnout in sports described in literature (Raedeke & Smith, 2001; Raedeke et al., 2002). We found minor psychometric problems with sporting devaluation where a lack of convergent validity fail to meet Hair et al.'s, (2009) rule of thumb in which AVE values should be at least .5. In fact, as Figure 1 shows, there is a correlation (r= .29; p < .001) between the residuals of item 2 from p hysical and emotional exhaustion and item 6 from sport devaluation suggesting that the two items associated with these errors are more related to each other than the original measurement model predicted. Item 6 ("working with players and coaches is really a strain on me") seems to be problematic in the sport evaluation factor and future analysis of the item should be undertaken. Specifically, the rewording of the item should be attempted in other studies to clarify its meaning. Nevertheless, the reexamination of the theory and the definitions of this construct led to the conclusion of its practical significance and the maintenance of it in the factor. Given the strong conceptual arguments for the distinctness of this factor, the size and significance of its standardized loading estimate and the need that all factors to have at least three indicators (Hair et al., 2009), the item was not eliminated at this time. Further research is needed to determine if this item should, in fact, be retained or deleted.

Despite the referees being considered the third dimension of sport, the demands and needs are different from those found in athletes and coaches. Thus, the burnout dimensions for referees are described as follows: (a) the physical and emotional exhaustion factor is associated with exhaustion coming from the intense demands of officiating; (b) the sporting devaluation factor is related to the fact that referees generate a pattern of attitudes and behaviors of insensibility and reduced affect toward coaches and athletes and, (c) the reduced sense of sport accomplishment factor is associated with the feeling of frustration and tiredness, and to the constant loss of enthusiasm in the referee's career.

Study 2

The purpose of the Study 2 was to explore the concurrent validation of BIR in Portuguese by examining correlations between BIR and Profile of Mood States (POMS) scores (Hair, 2009).

Method

Evidence of concurrent validity was obtained by studying the association among construct measures under theoretical foundations. We empirically confirmed relationship between the constructs (Campbell & Fiske, 1959). The POMS profile is designed to measure six mood states: tension (TE), depression (DE), anger (AN), vigor (VI), fatigue (FA) and confusion (CO); and also to measure emotional variations associated with exercise and psychological well-being. Furthermore, research using the POMS has consistently supported the association among changes in the states of mood and stress, overtraining and burnout (Cohn, 1990; Gould et al., 1996; Pires et al., 2006). In the present investigation, we hypothesized that total burnout (TBN), as well as the subscales of physical and emotional exhaustion (PEE), sporting devaluation (DES), and reduced sense of accomplishment (RSA) will be related to the emotional states measured by the POMS. The hypothesized specific relationships between burnout and the POMS subscales include the following: (a) positively related to the states of tension, depression, anger, fatigue and confusion and (b) negatively related to vigor.

Participants

Thirty-six male professional soccer referees (22 referees and 14 assistants), including 18 from each country, participated in the study. This sample was chosen as it represented the worldwide elite of the referees, along with experience officiating in the Soccer International Championships in Brazil and Portugal 2008/2009. Excluded from the sample were two referees who have disconnected their relationship with the referee organization of both countries. The ethics committee documentation and requirements were the same as in Study 1.

Measures

The validated version of the BIR from Study 1 and the adaptation of the reduced version of POMS to Portuguese by Viana, Almeida, and Santos (2001) were herein used. Concurrent validity was analyzed using Pearson's correlation coefficients among the six mood states measured by the POMS, and the value of the total POMS with the three subscales of the BIR and the value of the total burnout (calculated by the sum of the values of the three subscales). The correlations among the scales were interpreted according to the criteria established by Bryman and Cramer (1990).

Procedures

The referees were contacted by email and invited to take part in Study 2. Those who agreed to participate were sent the purposes of the study, guidelines on the participation and instructions for completing the two questionnaires and the consent form. The two instruments were completed by the participants in their own home and were, according to each participant's convenience, returned directly to the responsible researcher's email.

Results

Eighteen referees in this study had college degrees and 18 high school diploma. In addition, six officiated in international championships (e.g., FIFA, UEFA) and 30 officiated at the national level, with average age of 35 years (SD = 4.0) and of 14.7 years (SD = 4.8) officiating.

Correlations were used to examine the relationship among the three BIR subscales, the total burnout, the six mood states and the total POMS (Table 5). The results are presented in Table 5 and reveal that, as hypothesized, the burnout scales were positively correlated to the mood states TE, DE, AN, FA and CO, and negatively correlated with VI. Significant correlations, from weak to high, were observed among the total burnout, the mood states and the total POMS, which suggest a relationship between burnout and the mood states. Burnout subscales were not correlated in the same way as the mood states. The physical and emotional exhaustion subscale was the only one that showed significant correlations with all the others subscales. Depression and confusion appeared as the best predictors of the relationship between burnout and mood states. As hypothesized, BIR scores were significantly correlated with scores on the POMS and provided evidence of concurrent validity for the BIR.

Table 5.
Pearson's correlation of the subscales and the total burnout with six mood states and the total POMS.

	Tension	Depression	Anger	Vigor	Fatigue	Confusion	Total POMS
PEE	.38*	.47**	.27*	.42*	.54**	.69**	.64**
SDE	.37*	.39**	.36*	-.12	.20	.56**	.44**
RSA	.18	.63**	.36*	-.26	.25	.63**	.51**
TBN	.33*	.60**	.39*	-.30*	.37*	.72**	.51**

• PEE: physical and emotional exhaustion; SDE: sports devaluation; RSA: reduced sense of accomplishment; TBN: total burnout. *p < .05. **p < .01.

Discussion

The purpose of Study 2 was to examine the concurrent validity of the BIR through its correlations with mood states (POMS), a theoretically related construct. In addition, these surveys have identified other symptoms associated with burnout such as loss of motivation and pleasure, and changes in mood and apathy towards an activity once pleasant (Goodger, Gorely, Lavallee, & Hardwood, 2007; Raedeke & Smith, 2001). Although the literature related to burnout and emotional states in referees is scarce when compared to those related to athletes and coaches, results of Study 2 indicate that these relationships seen in athletes and coaches are also applicable to referees.

According to Creswell and Eklund (2003), different studies have shown that negative mood states can be altered as a result of physical and emotional exhaustion. Therefore, it is no surprise that, within the burnout subscales, physical and emotional exhaustion were the most consistent dimension and that best characterized the syndrome. These results are in accordance with the literature in which physical and emotional exhaustion are found to support all theoretical approaches developed for the understanding of the syndrome.

For Shirom (2005), depression is the psychological comorbidity most prominent in burnout, (i.e., there is a higher predisposition in the individual with burnout to exhibit symptoms of depression). Accordingly, Iacovides, Fountoulakis and Kaprinis (2003) believe that depression would be predicted from burnout due to high levels of psychological demands, low levels of social support and high levels of stress.

Generally, referees' activity in soccer is associated with different conditions that generate psychological stress. Criticism from coaches, soccer players, media and fans, as well as verbal abuse, physical threats, aggressions (Alonso-Arbiol, Arratibel, & Gómez, 2008; Luz, 2000; Weinberg & Richardson, 1990) are part of the everyday life of referees. Taylor, Daniel, Leith and Burke (1990), Garcés de los Fayos, Elbal and Reyes (1999), and Luz (2000) claimed that, as a consequence, failure to deal with disturbances due to interpersonal conflicts, role conflicts, fear of failure and negative work environment can produce high levels of stress. Similarly, Brandão et al. (2011) found that making mistakes during a match, not being appreciated by the leaders and peers, and the influence of officiating in referees' personal life also produced high levels of stress for referees. Accordingly, the results of Study 2 supported the concurrent validity of the BIR indicating that high levels of burnout are related to negative emotions.

General discussion

The primary research contribution of the two studies is the development of a validated instrument for evaluating referees burnout by following a more rigorous process than predecessors and revealing a wider range of dimensions. Measurement instruments are in a very real sense a research "infrastructure." The development of the measure in itself may promote many research programs going forward. The present investigation has demonstrated the validity of BIR for a population of professional soccer referees. A second contribution is associated with the importance of studying burnout in officiating. Researchers as well as practitioners need to know how burnout influences individuals both in health, psychological well-being, as well as in performance. This is especially important due to the fact that burnout and resignation rates are higher among referees than among athletes and coaches (Kaissidis & Anshel, 1993). Considering the preliminary validation of the adaptation to the officiating context of the Portuguese version of the BIR, we concluded that the factorial structure shows high acceptable indexes of validity and reliability. Thus, it seems reasonable to state that the BIR could be used for burnout evaluation in the context of soccer officiating. Nevertheless, considering sport devaluation, it seems necessary to recommend further studies, since the construct should have higher levels of convergence validity (Cronbach & Meehl, 1955). Indeed, for constructs that were underrepresented, additional items closely related to the remaining items should be generated by the researchers, keeping in mind the conceptual definitions of the construct. Nevertheless, as burnout in referees is a multidimensional syndrome, to better understand the burnout impact on referees, it is necessary that further study investigate burnout in officials by level and category, by age and years of experience, and among referees of different sports.

Furthermore, Schaufeli and Buunk (2001) observed that within the reactions associated with the burnout syndrome is the variation in mood states. Thus, the results found in the present investigation confirmed that burnout presented a close relationship with changes in mood states, especially with a predisposition to a depressive state (Glass, Mcknight, & Valdimarsdottir, 1993; Grobbelaar, Malan, Steyn, & Ellis, 2010; Leiter & Durup, 1994; Martini, Arfken, Churchill, & Balon, 2004). In addition, since burnout also has been shown to have a negative impact on the physical and psychological wellbeing of athletes and coaches (Raedeke & Smith, 2004), demonstrating the link between burnout and mood states in referees is an important extension for future studies.

Furthermore, Gould et al. (1996)refer to the burnout experience in sports as the transformation of a passionate involvement in athletic activity into a behavior characterized by reduced or lack of motivation, lethargy or, even the complete resignation of the modality. The relationship of burnout to negative emotions and moods in referees, as well as decreased vigor, extends Gould et al.'s (1996)observations about referees. Thus, it appears that referees experience occupational stress that is defined as a process in which an individual notices demands of work as stressors. This results about a negative view of the environment, of the person himself/herself and of the future is expressed by exhaustion, depression, pessimism, feelings of resignation, low expectations and low self-confidence (Brandão, Pizzo, & Rubin, 2007).

Continual growth and interest in sport has created a challenging job for sport officials. However, officiating-specific demands are extreme since sport officials need to assess situations as rapidly and accurately as possible and must take into account the many aspects of a game. All of this not only makes the job very complex, but also makes it easy for referees to suffer from burnout.

In summary, the BIR instrument presented good evidence of validity and reliability in the assessment of referees who may be at risk of suffering from burnout. One of the values of the scale, considering the implications for practice is that it can be a first step to identify situations that potentially lead to a progressive professional disillusion with consequent physical and emotional symptoms. Of course, the BIR is not the final word in measurement instruments for referees' burnout. Therefore, a potential avenue for future research is to adapt the scales to different types of sport referees and attempt to replicate the model's structure. Like any measurement instrument for an important research domain, it will certainly be modified, added to, and refined in terms of reliability and validity via future empirical investigations.

References

• Alonso-Arbiol, I., Arratibel, N., & Gómez, E. (2008). La motivación del colectivo arbitral en fútbol: un estudio cualitativo. [Motivation in soccer referees: a qualitative study]. Revista de Psicología del Deporte, 17, 187-203.
• Arbuckle, J.L. (2009). AMOS 18 Reference Guide (Version 18) [Computer Software]. Chicago, IL: SPSS Inc.
• Balch, M.J., & Scott, D. (2007). Contrary to Popular Belief, Refs Are People Too! Personality and Perceptions of Officials. Journal of Sport Behavior, 30, 3-20.
• Beaton, D., Bombardier, C., Guillemin, F., & Ferraz, M.B. (2002). Recommendations for the cross-cultural adaptation of health status measures. Retrieved May 31, 2010 from Institute of Work & Health, American Academy of Orthopaedic Surgeons (AAOS). Web site: http://www.dash.iwh.on.ca.
  » http://www.dash.iwh.on.ca
• Brandão, M.R.F., Serpa, S., Krebs, R., Araújo, D., & Machado, A.A. (2011). El significado del arbitrar: percepción de jueces de fútbol profesional. [Referee personal meaning: perception of football officials]. Revista Psicología del Deporte, 20, 275-286.
• Brandão, M.R.F., Pizzo, F.L., & Rubin, M. (2007). O profissional do futebol e a satisfação laboral. [Work satisfaction perceived by soccer coaches and physical trainers]. Coleção Pesquisa em Educação Física, 5, 439-444.
• Bryman, A., & Cramer, D. (1990). Quantitative Data Analysis for Social Scientists. Routledge: New York.
• Byrne, B.M. (2009). Structural equation modeling with AMOS: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum Associates.
• Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.
• Cheung, G.W., & Rensvold, R.B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation, 9, 233-255.
• Cohn, P.J. (1990). An Exploratory Study on Sources of Stress and Athlete Burnout in Youth Golf. The Sport Psychologist, 4, 95-106.
• Creswell, S.L., & Eklund, R.C. (2003). The athlete burnout syndrome: a practitioner's guide. New Zealand Journal of Sports Medicine, 31, 4-9.
• Cronbach, L.J., & Meehl, P.C. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
• Fejgin, N., Talmor, R., & Erlich, I. (2005). Inclusion and Burnout in Physical Education. European Physical Education Review, 11, 29-50.
• Fornell, C., & Larcker, D.F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18, 39-50.
• Garcés de los Fayos, E.J., Elbal, & Reyes (1999). Burnout en árbitros de fútbol. [Burnout in soccer referees]. In G. Nieto and E.J. Garcés de los Fayos (Eds.), Psicología de la actividad física y del deporte. Áreas de investigación y aplicación (pp. 628-633). Murcia: Sociedad Murciana de la Actividad Física y el Deporte.
• Glass, D.C., Mcknight, J.D., & Valdimarsdottir, H. (1993). Depression, Burnout, and Perceptions of Control in Hospital Nurses. Journal of Consulting and Clinical Psychology, 61, 147-155.
• González-Oya, J. (2006). Psicología aplicada al árbitro de fútbol: Características psicológicas y su entrenamiento. [Psychology applied to football referee: Psychological characteristics and training]. Sevilla: Wanceulen.
• Goodger, K., Gorely, T., Lavallee, D., & Harwood, C. (2007). Burnout in sports: A systematic review. The Sport Psychologist, 21, 127-151.
• Gould, D., Dieffenbach, K., & Moffett, A. (2002). Psychological characteristics and their development in Olympic champions. Journal of Applied Sports Psychology, 14, 177-209.
• Gould, D., Tyffey, S., Udry, E., & Loehr, J. (1996). Burnout in competitive junior tennis players: II. Qualitative analysis. The Sports Psychologist, 10, 341-366.
• Guillén, F., Jiménez, H. (2001). Características deseables en el arbitraje y el juicio deportivo. [Desired characteristics in sports refereeing]. Revista Psicología del Deporte, 10, 23-34.
• Goodger, K., Gorely, T., Lavallee, D., & Harwood, C. (2007). The Sport Psychologist, 21, 127-151.
• Grobbelaar, H.W., Malan, D. Steyn, B. & Ellis, S. (2010). Factors affecting the recovery-stress, burnout and mood state scores of elite student rugby players. South African Journal for Research in Sports, Physical Education and Recreation, 32, 41-54.
• Gustafsson, H., Hassmén, P., Kenttä, G., & Johansson, M. (2008). A qualitative analysis of burnout in elite Swedish athletes. Psychology of Sports and Exercise, 9, 800-816.
• Hair, J.F., Black, W.C., Babin, B., Anderson, R.E (2009). Multivariate data analyses (7th ed.). New York: Prentice Hall
• Hendrix, A.E., Acevedo, E.O., & Hebert, E. (2000). An examination of stress and burnout in Certified Athletic Trainers at Division I-A universities. Journal of Athletic Training, 35, 139-144.
• Hodge, K., Lonsdale, C., & Ng, J.Y. (2008). Burnout in elite rugby: Relationships with basic psychological needs fulfillment. Journal of Sports Sciences, 26, 835-844.
• Iacovides, A., Fountoulakis, K.N., & Kaprinis, S. (2003). The relationship between job stress, burnout and clinical depression. Journal of Affective Disorders, 75, 209-221.
• Kaissidis, A., & Anshel, M. (1993). Sources and intensity of acute stress in adolescent and adult Australian basketball referees: a preliminary study. Australian Journal of Science and Medicine in Sports (AJSMS), 25, 97-103.
• Kline, R.B. (1998). Principles and practice of structural equation modeling. New York: The Guilford Press.
• Leiter, M.P., & Durup, J. (1994). The discriminant validity of burnout and depression: a confirmatory factor analytic study. Anxiety, Stress and Coping, 7, 357-373.
• Loehlin, J.C. (2003). Latent variable models: An introduction to factor, path, and structural equation analysis (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
• Lonesdale, C., Hodge, K., & Rose, E. (2006) Pixels vs. Paper : Comparing online and traditional survey methods in sport psychology. Journal of Sport and Exercise Psychology, 28, 100-108.
• Louvet A., Gaudreau, P., Menaut, A., Genty, J. & Deneuve, P. (2009). Revisiting the changing and stable properties of coping utilization using latent class growth analysis: A longitudinal investigation with soccer referees. Psychology of Sport and Exercise, 10, 124-135.
• Luz, C.J.P.M. (2000). Stress, esgotamento e intenção de abandono entre os árbitros de futebol portugueses. [Stress, burnout and intention to dropout in Portuguese soccer referees]. (Master dissertation, Faculdade de Motricidade Humana), Available from http://thesaurus.reitoria.utl.pt. (M 621).
  » http://thesaurus.reitoria.utl.pt
• Martini, S., Arfken, C., Churchill, A., & Balon, R. (2004). Burnout Comparison among Residents in Different Medical Specialties. Academic Psychiatry, 28, 240-242.
• Maslach, C. (1998). A multidimensional theory of burnout. In C.L. Cooper (Ed.), Theory of Organizational Stress (pp. 68-85). Oxford, UK: Oxford University Press.
• Maslach, C., & Jackson, S.E. (1981). The Measurement of Experienced Burnout. Journal of Occupational Behavior, 2, 99-113.
• Maslach, C., & Leiter, M.P. (2000). Burnout. In G. Fink (Ed.), Encyclopedia of stress. (pp. 358-362). San Diego, CA: Academic Press.
• Maslach, C., Schaufeli, W.B., & Leiter, M.P. (2001). Job burnout. Annual Review of Psychology, 52, 397-422.
• Pires, D.A., Brandão, M.R.F., & Silva, C.B. (2006). Validação do Questionário de Burnout para Atletas. [Validation of the Burnout Inventory for Athletes]. Revista da Educação Física, 17, 17-36.
• Raedeke, T. (1997). Is athlete burnout more than just stress? A sports commitment perspective. Journal of Sport and Exercise Psychology, 19, 396-417.
• Raedeke, T.D. (2004). Coach commitment and burnout: A one-year follow-up. Journal of Applied Sport Psychology, 16, 333-349.
• Raedeke, T.D., Lunney, K., & Venables, K. (2002). Understanding Athlete Burnout: Coach Perspectives. Journal of Sport Behavior, 25, 181-206.
• Raedeke, T.D., & Smith, A.L. (2001). Development and preliminary validation of an athlete burnout measure. Journal of Sport and Exercise Psychology, 23, 281-306.
• Raedeke, T.D., & Smith, A.L. (2004). Coping resources and athlete burnout: An examination of stress mediated and moderation hypotheses. Journal of Sport and Exercise Psychology, 26, 525-541.
• Roelofs, J., Verbraak, M., Keijsers, P.J., de Bruin, M., & Schmidt, A.J.M. (2005). Psychometric properties of a Dutch version of the Maslach Burnout Inventory General Survey (MBI-DV) in individuals with and without clinical burnout. Stress and Health 21, 17-25.
• Schaufeli, W., & Buunk, B. (2001). Affect generated by social comparisons among nurses high and low in burnout. Journal of Applied Social Psychology, 3, 1500-1520.
• Schaufeli W., & Enzmann, D. (1998). The burnout companion for research and practice: a critical analysis of theory, assessment, research, and intervention. London: Taylor & Francis.
• Shirom, A. (2005). Reflections on the study of burnout. Work & Stress, 19, 263-270.
• Taylor, A.H. (1991). Burnout in Sports. In S.J. Bull (Org.), Sports Psychology: A self-help guide (pp. 157-177). Ramsbury: The Crowood Press.
• Taylor, A.H., Daniel, J.V., Leith, L., & Burke, R.J. (1990). Perceived stress, psychological burnout and paths to turnover intentions among officials. Journal of Applied Sport Psychology, 2, 84-97.
• Viana, M.F., Almeida, P.L., & Santos, R.C. (2001). Adaptação portuguesa da versão reduzida do Perfil de Estados de Humor - POMS [Portuguese version of the Reduced POMS Profile]. Análise Psicológica, 1, 77-92.
• Weinberg, R.S., & Richardson, P.A. (1990). Psychology of officiating. Champaign, Il: Leisure Press.

Publication Dates

• Publication in this collection
  Oct-Dec 2014

History

• Received
  07 June 2013
• Accepted
  15 Sept 2014