Work Environment Evaluation Instrument (WEEI): development, validation, and association with burnout

Introduction: Physician burnout is considered an epidemic. In 2019, 44% of U.S. physicians reported feeling burned out. The work environment is a central risk factor for this. The aim of this study is to develop and test an instrument to evaluate work environment factors in medical training courses. Method: After focus groups, an initial pool of 14 items was generated and tested in a pilot study (n = 66). Face validity was verified, and small adjustments were made. The resulting version was administered to a sample of 115 psychiatry residents. Eleven items were selected based on the correlations between them, principal component analysis, and theoretical reasons, and then tested for internal and construct validity. Results: The final version had high reliability (Cronbach’s alpha = 0.898) and comprised three dimensions: relations with the institution; with colleagues; and with preceptors. Both total scores and dimensions correlated significantly with burnout scores (p < 0.01). Cutoffs defining the environment as healthy (>32 points); risky (23-31 points); or toxic (<22 points) were suggested and related to the risk of burnout. Conclusion: Several authors have emphasized the importance of approaching institutional factors as an effective strategy for coping with the increased prevalence of burnout. This instrument should contribute to these efforts.


Introduction
Burnout rates are increasing, and many authors are describing the phenomenon as an epidemic. Medical doctors and students are a special concern. In 2014, 54.4% of U.S. physicians reported at least one symptom of burnout, compared with 45.5% in 2011. 1,2 In 2019, 44% of physicians reported feeling burned out. 3 Rates of burnout are rising, despite greater recognition of the problem. Some authors argue that this may be because efforts to reduce burnout are typically focused on increasing resilience and wellness, rather than combating problematic changes in how medicine is practiced by physicians in the current era, or the roots of the problem that involve institutional factors and culture. 2,4,5 With respect to medical education, many studies report rates of burnout and other mental diseases that are much higher than those in the general population. 6,7 The prevalence of burnout in residents is generally about 25-75%, varying by specialty, country, and measurement method. 8,9 These findings point to the fact that the training process and environment may impact on the health of physicians in training. 10 Some risk factors are known, such as heavy workload, female gender, presence of physical illness and or mental disorders, medication use, dissatisfaction with career, high demand for perfection, and others. Nevertheless, institutional factors are as yet poorly investigated and understood. 4,11 Medical students and residents spend a great proportion of their time immersed in the institutional environment and are in constant contact with professors, assistants, colleagues, patients, and the institutional staff and culture. The nature of this contact and also the beliefs and values common to the members of the institution may play a fundamental role in development of burnout, especially if they are not compatible with one's own beliefs. Moreover, problematic interpersonal relations can be highly harmful, for instance, conflict with colleagues and preceptors, competition, abuse of power by superiors, lack of empathy, and lack of support. Furthermore, contemporary physicians are faced with the challenge of delivering increasingly patient-centered care, at the same time that they are continually exposed to ongoing economic, technological, and regulatory changes, posing unrealistic expectations of physician availability. 4 The consequences of burnout can be devastating, for

Development of the instrument and pilot study
Several focus groups were held with experts, professors, medical students, and residents during the first semester of 2017 to identify and define items that could be related to development of burnout symptoms.
Fourteen items were defined and tested in a pilot study that was conducted in July 2017. For this step, all psychiatry residents in the city of Porto Alegre (n = 87) were invited by e-mail to answer the questionnaire and 75% (n = 66) of them participated in the study. 14 Item comprehensibility and face validity were evaluated and discussed, and small adjustments were made.
Additionally, three items were reformulated as negative sentences to detect unreliable answers.

Participants
The sample comprised 115 adult Brazilian participants The preliminary version of the WEEI was a 14 item self-report questionnaire with a Likert response scale on which 0 corresponded to "totally false" and 4 corresponded to "totally true." Six items evaluated relations with teachers/preceptors, 5 with colleagues/ peers, and 3 with the institution (online-only supplementary material S1).

Data analysis
The

Selection of the final items
The

Evaluation of internal consistency and construct validity
According to Burns, at least 10 subjects for each item on a scale are required for proper validation. 16 There were 115 subjects in our sample. The Cronbach's Alpha coefficient (α) for each of the dimensions was 0.79 for "institution", 0.78 for "colleagues", and 0.87 for "preceptors".
Based on the assumption that, in accordance with our conceptualization, work environment factors would be related to burnout, we evaluated associations between WEEI scores and burnout scores to analyze construct validity. Both total WEEI score and its dimension scores (RI, RC, and RP) were correlated with Burnout scores (EE, DP, and PA) (p < 0.01) in our sample (Table 1).
There was a difference in mean WEEI TS between subjects who were positive for EE according to the MBI-HSS and those who were not (22.69±10 versus 31.29±8.4; p < 0.001). Considering that EE is the most consistent dimension of burnout, to the extent that some authors even use it as the only diagnostic criterion, we defined proposed WEEI cutoff points based on mean and SD WEEI scores for these groups, (positive and negative for EE), and performed risk evaluations tests.
The cutoff points tested classified the environment as healthy (> 32 points), risky (between 23 and 31 points), or toxic (< 22 points). In a toxic environment, the odds ratio (OR) for being positive for EE was 6.2 (95%CI: 2.6-15; p < 0.001) and OR for DP was 14.4 (95%CI: 4.2-48.6; p < 0.001). In a healthy environment, the OR for being positive for EE was 0.25 (95%CI 0.11-0.6; p = 0.001) and OR for DP was 0.74 (95%CI 0.64-0.85; p < 0.001). As expected, the results regarding risky environments were not significant, suggesting environments that are nor protective nor consistently associated to burnout risk.
We suggest that the scores should be used as a continuous variable. The cutoff points for toxic and/or healthy environment can be used when appropriate to the objective of the study (online-only supplementary material S4). Nevertheless, the robust association between characterization of the environment as healthy or toxic with EE and DP burnout dimensions is also indicative of the validity of the construct measured by the instrument.

Discussion
We were able to develop a rapid, self-administered instrument that adequately evaluates three work environment dimensions (institution, preceptors, and peers) with high reliability (Cronbach's alpha = 0.898).
This version of the instrument was developed specifically for the medical training community. To our knowledge there is no other instrument like this reported in the literature to date.
Burnout has a huge impact on personal life, work capacity, and the economy and efforts that have been implemented to cope with the problem have not been effective in mitigating the phenomenon. It is also worth noting that other psychiatric conditions for which burnout can be an important risk factor are also more prevalent among doctors, specifically, depression and suicide.
As pointed out by several authors, institutional factors must be addressed and the instrument developed in this study can contribute to further research investigating these factors. 2,4,5 Limitations of this study include the fact that it was conducted in a specific population (psychiatry residents from Rio Grande do Sul, Brazil). Therefore, studies

Disclosure
No conflicts of interest declared concerning publication of this article.