Burnout Syndrome Assessment Scale in Public Safety: Psychometric Parameters

ABSTRACT The burnout variable has been studied in different professions; however, it presents challenges in its measurement. The present study provides evidence of validity based on the internal structure of the Burnout Syndrome Assessment Scale in Public Security Workers (EASB-SP) and presents the estimation of item parameters through the Item Response Theory (IRT). A total of 10,885 public security agents from all states of Brazil participated in the study. The analyses indicated a three-factor structure, with adequate reliability indices and discrimination of the items. The Professional Fulfillment factor presented a lower theta value to be endorsed. The instrument was shown to be an adequate measure, with evidence of validity for the context of Brazilian public security.

In Brazil, the institutions that make up the public security system must ensure the maintenance of public order (Constituição da República Federativa do Brasil, 1988/2001), which includes duties that range from monitoring of the simplest events (e.g., accompanying a cultural event) to dealing with highly stressful life-threatening situations (e.g., kidnappings and homicides).Specifically, the corporations that compose the Brazilian public security system are the federal, civil, military, federal highway and federal penitentiary police and the military fire brigade (Constituição da República Federativa do Brasil, 1988/2001).
In this context, the stress experienced by workers who are members of these security forces is one of the main risk factors for physical and psychological illness, increasing the probability of absenteeism (Bravo et al., 2016, Rodríguez, 2021) and suicide among these professionals, according to data presented by the Institute for Research, Prevention and Studies on Suicide (Instituto de Pesquisa, Prevenção e Estudos em Suicídio -IPPES, 2021).In these workers, stress can result from organizational factors, such as departmental and administrative culture, low autonomy and interpersonal conflicts between officers (Ascari et al., 2016;Chae & Boyle, 2013); and operational factors, such as participation in events with unpredictable outcomes, exposure to trauma, and potential fear (Ascari et al., 2016;Chae & Boyle, 2013), among other factors.
A consequence of going through these experiences is the development of burnout syndrome, which can be defined as an emotional, mental, and physical reaction due to prolonged exposure to work stressors (World Health Organization -WHO, 2019).According to Maslach et al. (2001), this syndrome is structured in three correlated dimensions that represent emotional exhaustion, depersonalization and low professional fulfillment.Specifically, emotional exhaustion encompasses feelings of tiredness and exhaustion, depersonalization encompasses emotional withdrawal from others, and low professional fulfillment involves lack of job satisfaction coupled with feelings of incompetence and professional unhappiness (Maslach & Jackson, 1981).
Burnout syndrome has been evidenced in public security workers in recent studies that carried out reviews on the presence of this condition (De Carvalho et al., 2020;Dias & Andrade, 2020).However, one of the limitations in these studies is the lack of evidence of validity of the instruments used to assess burnout syndrome in the context of public security, as well as lack of sufficient measures available in Brazil that cover this construct.This aspect was exposed in an integrative review carried out in the PsycINFO, PubMed and Google Scholar databases, covering the period up to June 4, 2020 (Esteves et al., In Press).This review showed the use of five instruments to assess burnout syndrome in public security workers, namely: (1) Maslach Burnout Inventory -General Survey (MBI-GS) (Maslach, Jackson & Leiter, 1996); (2) Human Services Survey (MBI-HSS) (Maslach, Jackson & Leiter, 1996); (3) Burnout Measure -Short (BM-S) (Malach-Pines, 2005); (4) Copenhagen Burnout Inventory (CBI) (Kristensen et al., 2005); and (5) Cuestionario para la Evaluación del Syndrome de Quemarse por el Trabajo (CESQT) (Gil-Monte, 2011) (Esteves et al., in Press).None of these five instruments explicitly presents evidence of validity for the public security context, especially for Brazilian workers, and there are limitations such as the financial cost of using one of the instruments and the absence of adaptation studies for Brazil.
Based on the specific characteristics of the Brazilian Public Security forces, the existence of evidence of validity studies for burnout instruments is not observed in literature.Accordingly, the present study aimed to gather evidence of construct validity and validity based on the internal structure for the Burnout Syndrome Assessment Scale in Public Security Workers (Escala de Avaliação da Síndrome de Burnout para a segurança pública -EASB-SP).It is a newly developed instrument based on Brazilian culture and language.In this way, the cultural specificities of the stressors related to the work of public security agents in Brazil were considered.This is the first study designed to investigate the dimensionality of the EASB-SP, which was built based on Maslach et al.'s (2001) three-factor model.The suitability of the items in this measure for the estimation of the traits was also investigated through the parameters of the Item Response Theory (IRT).

METHOD Participants
Study participants were 10,885 public security agents, from different corporations including the military police, civil police, criminal police, technical-scientific police and military fire brigade, from all states of Brazil and from federal institutions such as the federal highway police and the national penitentiary department.It was a non-probabilistic convenience sample, including those agents who were invited and agreed to participate.The participants had a mean age of 41.16 years (SD = 7.95), most were male 81.6% (⨍ = 8,878), married 74% (⨍ = 8,053), with a family income between R$3,300.00 and R$ 5,500.00(32.2%; ⨍ = 3,517) and with a mean time working in the public security force of 14.41 years (SD = 0.08).

Instruments
The Burnout Syndrome Assessment Scale in Public Security Workers (Escala de Avaliação da Síndrome de Burnout na Segurança Pública -EASB-SP)is an instrument designed specifically for the public security context, by Esteves et al. (in Press).The scale consists of 26 items answered on a response scale from 1 (Never) to 5 (Always), distributed into three dimensions: (i) Emotional Exhaustion, consisting of 10 items; (ii) Depersonalization, comprising 10 items and; (iii) Professional Fulfillment, comprising six items.The score of the factors on this scale is performed through the arithmetic mean of the item scores per factor, with higher values in the Emotional Exhaustion and Depersonalization factors indicating greater signs of psychological distress and burnout.For the Professional Fulfillment factor, lower values indicate greater signs of psychological distress and burnout.
The Sociodemographic Questionnaire was a questionnaire prepared to collect data from the research participants, such as age, sex, marital status, income and length of service in the security force.

Procedures
The participants were invited to take part in the study through a link to access the instruments.When accessing the link, a consent term was encountered first.This research is part of the Capes PrInt Project "Refugees in the Americas: A comparison of refugees' settlement in the global north and global South" and is linked to the project entitled "Assessment of Health and Intervention Propositions in the Public Security Area -A National Study", approved by the research ethics committee (nº 3,965,395), performed throughout Brazil, through a Decentralized Execution Term (TED No. 009/2019/CGPP/DPSP/SENASP) between the University of Brasília (UnB) and the Ministry of Justice/ National Public Security Department (MJ/SENASP).Due to the secrecy, confidentiality and sensitivity of the information, data collection was carried out through an online platform developed specifically for the study, identified as the Public Security Health Assessment System (SASSP), with the data treated as a whole, without identifying the forces.

Data analysis
Initially, descriptive statistics (frequency, percentage, mean and standard deviation) were calculated, using the SPSS version 24 software, to describe the sample profile in terms of demographic data.Subsequently, using the R software, the total sample was divided into two random subsamples.Using this same software and the psych package (Revelle & Revelle, 2015), Exploratory Factor Analysis (EFA) was performed with the first subsample (n = 5,443), using the polychoric correlation matrix, with the Weighted Least Squares (WLS) estimation method (Kiers, 1997).In order to decide on the number of factors to be retained, Parallel Analysis was performed (Horn, 1965) using Promax rotation.Subsequently, with the second subsample (n = 5,442), Confirmatory Factor Analyses (CFA) were performed, seeking to identify the adequacy of the model through the following fit indices: Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI) and Tucker Lewis Index (TLI), with values lower than .08 for the RMSEA and higher than .95for the CFI and TLI being indicators of fit (Bandalos & Gerstner, 2016;Hu & Bentler, 1999).With the total sample, we sought to evaluate the item parameters through the Item Response Theory (IRT), with the R software (v.4.0.2),using the mirt package.Specifically, the Graduated Response Model (Samejima, 1969) was used.The Item Response Theory (IRT) was used to access the parameters of difficulty and discrimination of items and to assess at which level of burnout syndrome the EASB-SP has greater functionality (Kamata & Bauer, 2008).In addition to the clear advances provided by the IRT in relation to the TCT (Hambleton et al., 1991), the IRT performs an item-by-item estimate that can be useful to identify the suitability of the instrument for a given use (Baptista et al., 2018).Finally, also with the total sample (n = 10,885), analyses were carried out to obtain evidence of reliability for the EASB-SP factors.Therefore, the coefficients of internal consistency were calculated using the Cronbach's alpha (α), McDonald's Omega (ω) and Guttman's Lambda (λ Guttman

RESULTS
In order to comprehend to what extent the burnout syndrome was present in the studied sample, descriptive analyses of the scores were performed for each of the factors.The results suggest a profile characterized by Emotional Exhaustion (EE), Depersonalization (DP) and Professional Fulfillment (PF), as presented in Table 1.
Subsequently, the factorability of the matrix was verified using the Kaiser-Meyer-Olkin (KMO) test, which proved to be acceptable (.97), and the Bartlett sphericity test (χ 2 (351) = 94402.19;p < .00).Horn's Parallel Analysis (PA) was used to identify the number of factors in the correlation matrix, which indicated a five-factor model, unlike the theoretical structure composed of three factors in the model.Considering this result, it was decided to follow the structure indicated in the theoretical model and set the number of factors to be extracted at three.The EFA, reported in Table 2, indicated that first factor was composed of a total of ten items (18,17,12,04,03,14,21,09,15,13) with factor loadings ranging from .90 (Item 18) to .45 (Item 13).The items that made up this factor were those designed to represent a feeling of tiredness and exhaustion, referring to Emotional Exhaustion (EE).The second factor grouped another ten items (10,26,08,23,25,20,07,05,11,19), with factor loadings ranging from .86 (item 10) to .41 (item 19).In this factor, the grouped items describe lack of pleasure in relation to work, accompanied by feelings of professional incompetence and failure, characteristic of low Professional Fulfillment (PF).The third factor, which covered items related to depersonalization, indicating detachment and low empathy towards others, grouped six items (06,24,01,02,22,16), with factor loadings ranging from .86 (item 06) to .47 (item 16).In order to test the suitability of the factor structure of the EASB-SP, CFA was performed in another subsample using the Weighted Least Squares adjusted for Mean and Variance (WLSMV) estimation method.The model of three related dimensions presented model fit indicators with evidence of psychometric adequacy: CFI = .99;TLI = .99and RMSEA (90%CI) = .08(.07-.08).Factor loadings ranged from .93 (Items 17 and 18) to .63 (Item 14) (See Figure 1).

ITEM PARAMETERS THROUGH IRT
In line with the evidence that indicated a three-factor structure, the parameters of the EASB-SP items were estimated separately for each factor, using the Graded Response Model (Samejima, 1969).For the Emotional Exhaustion factor items (See Table 3), the mean discrimination was 2.81 (SD = 1.27), with only item 14 presenting moderate discrimination and the rest, very high discrimination, that is, higher than 1.70 (Baker, 2001).Overall, this factor presented 89.46 of information, with item 18 providing the greatest amount of information [I (θ; -3/+3) = 19.25].Regarding the difficulty parameter, the items in the Emotional Exhaustion dimension had a mean theta (θ) value of 1.67 (SD = 0.58) to be answered at the highest point of the response scale (See Table 3).In this factor, item 14 required a higher θ value to be answered at the highest point of the response scale (b 4 = 3.10).Item 19 required the lowest θ value to be answered at the highest point of the response scale (b 4 = 1.11).
Regarding the items of the Professional Fulfillment factor, the mean discrimination was 2.22 (SD = 0.55), with item 20 having high discrimination and the rest presenting very high discrimination (Baker, 2001) (See Table 3).Overall, this factor presented 65.28 of information, with item 26 providing the greatest amount of information [I (θ; -3/+3) = 3.39] (See Table 3).The items in the Professional Fulfillment dimension had a mean θ value for difficulty of 0.84 (SD = 0.61), to be answered at the highest point of the response Note.a -discrimination; b 1-4 -difficulty; I (θ; -3/+3) -information in the range from -3 to +3.
scale (See Table 3).The highest θ value needed for the item to be answered at the highest point of the response scale was presented by item 20 (b 4 = 2.22).Conversely, item 8 required the lowest θ value (b 4 = 0.04).
Finally, the items of the Depersonalization factor showed a mean discrimination of 2.57 (SD = 0.55), with all items having very high discrimination (Baker, 2001) (See Table 3).The factor presented a total of 45.21 information, with item 6 providing the greatest amount of information [I (θ; -3/+3) = 8.59] (See Table 3).Furthermore, the mean θ value for the difficulty of the items that made up the Depersonalization factor was 1.65 (SD = 0.76) when answered at the highest point of the response scale (See Table 3).The item that presented the highest θ value to be answered at the highest point of the response scale was item 23 (b 4 = 2.14) and the one that required the lowest θ value was item 1 (b 4 = 1.22).

DISCUSSION
The present study aimed to gather evidence of construct validity and validity based on the internal structure for the Burnout Syndrome Assessment Scale (EASB-SP).Furthermore, it was also sought to estimate the parameters of the items through Item Response Theory (IRT), specifically using the Graduated Response Model.It is estimated that this aim was achieved, and that evidence related to the validity and characteristics of the items was presented.
Therefore, the EFA indicated that the EASB-SP assesses the burnout syndrome, which can be defined as an emotional, mental, and physical reaction due to prolonged exposure to work stressors (WHO, 2019), by a model composed of three factors , these being emotional exhaustion (ten items), depersonalization (ten items) and professional fulfillment (six items), in line with the theoretically predicted structure for the EASB-SP (Esteves et al., In Press).The EFA also indicated that item 12 did not present a satisfactory factor loading (>.30) in any of the factors and, consequently, it was removed from the instrument.In a more detailed analysis of item 12 ("I do not want to interact with anyone at work"), it is postulated that, in the context of a sickening work environment, the behavior of withdrawing from people at work (or from triggers of psychological distress) could, in fact, be more characteristic of psychological protective features than of emotional exhaustion, as initially hypothesized.Therefore, the theoretical and methodological adequacy of the item in the scale was reassessed and it was discarded.
Overall, the psychometric properties of the EASB-SP found in this study can be considered satisfactory, with good reliability indices for all factors (Hair et al., 2019).In addition to the evidence found in the EFA, more evidence on the internal structure was found by performing AFC that showed adequate fit indices (Bandalos & Gerstner, 2016;Hu & Bentler, 1999).
Considering the item parameters, only item 14 presented moderate discrimination, while all others showed high and very high discrimination (Baker, 2001), indicating that all items are able to properly differentiate public security workers according to the amount of burnout syndrome.The difficulty parameter, on the other hand, indicated that the Emotional Exhaustion items were the ones that required the highest amount of latent trait for the respondent to mark the highest point on the response scale ("Always"), followed by the Professional Fulfillment factor and, then, by the Depersonalization factor.Regarding the information provided by the factors of the EASB-SP, they present an appropriate assessment of low, moderate and high levels of the three factors that make up the instrument.In general, the instrument does not seem to need a high theta value for full agreement of the content.As this is an initial study on the validity evidence of the EASB-SP, it is understood that, although the structure presented and its reliability were adequate, other studies could add greater contributions to the analysis of its internal structure.It can be concluded that the evidence reported here indicates that the EASB-SP scores are able to provide a psychometrically adequate assessment of burnout syndrome in public security professionals.Accordingly, the EASB-SP constitutes an alternative for the assessment of the psychological illness of these professionals which, in comparison with the other available instruments, was developed specifically for the context of public security workers.
However, the effect of the data collection design on the selected sample should be noted.First, computer-mediated data collection was used, remotely and asynchronously.
Recent studies show the effect of this design on the selection of WEIRD (Western, Educated, Industrialized, Rich, and Democratic) samples, that is, with social characteristics compatible with Western, industrialized and democratic cultures and individuals with good education and financial resources (Ghai, 2021).This is especially relevant in a country such as Brazil, with marked social inequalities.
Another aspect that should be mentioned refers to the profile of the public security agents that composed this sample.Most were people who identified with the male gender, with a mean age of approximately 40 years, married and with children, that is, it reveals a profile of young, male, adults, this being typical of the workers in this sector.Therefore, the influence of this profile on the responses obtained and even on the construction of the normative tables arising from this group should be noted.This is especially relevant and highlights the need to construct measures for specific contexts.Studies such as the present one, with a proposal to construct an instrument for a specific context that recognizes and represents specific characteristics of that context, are welcome and would contribute to the realization of more reliable processes for the evaluation of the reality, therefore, providing more reliable results.
Finally, as the main contribution of this study, a scale with psychometric parameters suitable for evaluating the burnout syndrome in public safety is presented.Also it is understood that the results reported here should be interpreted with parsimony, and it is important to obtain evidence of validity through other strategies, such as the relationship between the EASB-SP and other instruments consolidated in the literature.Furthermore, future studies may establish norms for a better interpretation of the results.

Table 1
Mean, Standard Deviation, Minimum and Maximum,

Table 2
Factor Loadings of the EASB-SP Items.
Figure 1.Structure and Factor Loadings of the EASB-SP.

Table 3
Parameters and Amount of Information for the EE,PF and DP Items.

Table 4
Internal Consistency Coefficients of the EASB-SP Factors.