Latent structure evidence of the Depression, Anxiety and Stress Scales – Short Form Evidências da estrutura latente da Depression, Anxiety and Stress Scales – Short Form

1  0000-0002-9670-8999 Abstract This study aimed to investigate the psychometric properties of the Depression, Anxiety and Stress Scales – Short Form in a Brazilian sample. The instrument was answered online by 250 university students. The following models were tested through Confirmatory Factor Analysis: one-dimensional, three oblique factors, hierarchical, and bifactor. The estimated indices showed a better adjustment for a bifactor model composed of three specific factors and one global factor. Additional statistical analysis, such as explained common variance and omega hierarchical estimates, indicated that the measure is predominantly one-dimensional. The results also indicated evidence of convergent validity (Average Extracted Variance between 0.48 and 0.60), internal consistency (Cronbach’s alpha between 0.87 and 0.94) and temporal reliability of the instrument (Intraclass Correlation Coefficient between

In the field of Psychopathology, some authors highlight the high rates of comorbidity among emotional disorders and point out similarities in their etiology, suggesting that they emerge from shared psychosocial and biogenetic diatheses (Barlow, Allen, & Choate, 2016;Falcone & Gonçalves, 2019), which would justify the presence of a global factor for the symptoms measured by the DASS-21. For Lovibond and Lovibond (1995), a possible vulnerability factor common to depressive, anxious, and stressful states would be neuroticism, a personality trait characterized by emotional instability and propensity to psychological stress (American Psychological Association, 2010). Individuals with high neuroticism scores have a tendency to experience intense emotional responses and a difficulty in returning to a normal state after emotional excitement, being anxious and temperamental people (Feist, Feist, & Roberts, 2014).
In Brazil, some questions remain about the DASS-21 latent structure with native samples. Martins et al. (2019) verified the adequacy of two DASS-21 latent structure models (three correlated factors and hierarchical) in a sample of Brazilian university students through CFA. Both models showed good adjustment indicators similar to each other. However, there was an absence of evidence of discriminant validity between depression, anxiety, and stress, which would justify bifactor analysis in order to investigate whether the items should be considered as multidimensional and not specific. These analyses were not performed in that study.
Subsequently, Zanon et al. (2020) investigated, through CFA, the adequacy of the four models tested by Osman et al. (2012) to the DASS-21 data collected from participants from eight countries, including Brazil. As found by Osman et al. (2012), the models with three correlated factors and hierarchical showed indicators of good adjustments and identical to each other, however, the bifactor model with three specific factors and a global one showed a better adjustment to the data. Additional analysis, such as explained common variance and hierarchical omega, indicated the unidimensionality of the DASS-21 scores collected with the samples used, including the Brazilian one. Further studies with Brazilian participants are needed in order to corroborate with this finding and investigate additional evidence of validity.
Thus, the present study aimed to investigate the psychometric properties of the DASS-21 in a Brazilian sample. Through CFA, the following models were tested: one-dimensional, three correlated factors, hierarchical and bifactor, partially replicating the studies by Osman et al. (2012) and Zanon et al. (2020). Evidence indicators of convergent and discriminant validity were also generated, in addition to reliability indices. L.F.D. ROCHA et al.

Method Participants
A total of 250 Brazilian university students, aged between 18 and 60 years (M = 24.92; SD = 8.63) participated in this research, 88.8% from the Southeastern region of the country, 9.2% from the Southern region, 1.6% from the Northeastern region and 0.4% from the Midwest. Of these, 76.0% reported being female and 23.2% male. In this sample, 0.8% did not indicate their sex, but their gender (i.e., 0.4% trans women and 0.4% neutral). Regarding marital status, 84.8% declared themselves single, 13.6% married, 1.2% divorced or separated, and 0.4% widowed. Regarding the nature of the institution in which they studied, 78.8% were public and 21.2%, private. Of these participants, 77 responded to the instrument again between 25 and 39 days after the date of the first measurement (M = 30.64; SD = 2.71).

Instruments
To measure negative affectivity, it was used the DASS-21 of Lovibond and Lovibond (1995), adapted for the Brazilian population by Vignola and Tucci (2014). It is a self-report instrument composed of 21 items equally divided between the Depression, Anxiety, and Stress subscales. The respondent should indicate how much each of them has applied to their reality during the past week. Answers are given on a 4-point Likert scale ranging from "It was not applied at all" (0) to "It was applied a lot or most of the time" (3).
In the study by Vignola and Tucci (2014), item 18 ("I felt I was rather touchy") presented a higher factorial load in the depression dimension and not in stress, contrary to the results found by Lovibond and Lovibond (1995) and other validation studies. A possible explanation presented by the authors is that the terms in Portuguese, used for the translation of the word "touchy" in that study (meaning, in Portuguese, emotional or sensitive), may be culturally related to the sadness and unpleasant events characteristic of depression. Other studies that translated the instrument into Spanish and Portuguese used the word "irritable", causing the item to have a factor load in the stress dimension and not in depression (Apóstolo et al., 2011;Daza et al., 2002;Patias et al., 2016). Thus, for the present study, this item was reformulated to "I felt I was very irritable", in order to adapt the Brazilian translation to the original meaning in English.
It was also used the Neuroticism subscale of the Five-Factor Personality Inventory, adapted for Brazil by Andrade (2008). This instrument assesses the neuroticism trait of the personality, based on six items (e.g., "I see myself as someone who is temperamental, changes mood easily") that were answered using a Likert scale from 1 (I totally disagree) to 5 (I totally agree). The subscale Neuroticism obtained a Cronbach's Alpha value of 0.65 in the study by Andrade (2008).

Procedures
After approval of the project by the Research Ethics Committee of the Universidade do Estado do Rio de Janeiro (UERJ, University of the State of Rio de Janeiro), Certificado de Apresentação para Apreciação Ética (CAAE, Presentation Certificate for Ethical Appreciation) protocol nº 00237418.1.0000.5282, a virtual questionnaire was prepared using the Google Forms software containing the Informed Consent Form and the instruments of the research. The invitation to participate in the research was carried out through advertisements on social media (e.g., Facebook) and e-mails to university professors requesting the referral to their students, both containing the link to the virtual questionnaire. After agreeing with the Informed Consent Form and completing the instruments, participants were asked about their interest in participating in the second stage of the research at a future time. If so, they should register their personal e-mail for further contact.

Data analysis
The collected data were entered into the SPSS-23 Software and descriptive analysis were performed in order to verify the univariate and multivariate distribution of data. In order to compare the qualities of the adjustments to the DASS-21 factorial models, procedures similar to those of Osman et al. (2012) and Zanon et al. (2020) were adopted.
In the context of Structural Equation Modeling, they were submitted to Confirmatory Factor Analysis (CFA) in the Analysis of Moment Structures Software (AMOS 23) (Arbuckle, 2014) to verify evidence of factorial, convergent, and discriminant validity of the DASS-21. In the current study, the following indices were estimated: the Chi-square (ꭕ²), which assesses the magnitude of the discrepancy between the population covariance matrix and the sample covariance matrix; ꭕ² is a conservative estimate of the model's fit when the sample size is > 200 (in this case, the ꭕ²/gl ratio was used and the results < 2.0 are considered good by Byrne, 2016); the Standardized Root Mean Square Residual, which is the square root of the error matrix divided by degrees of freedom, values < 0.08 are considered good adjustment indicators (Hu & Bentler, 1999); the Comparative Fit Index and the Tucker-Lewis Index, which compare the adjustment of the tested model with the adjustment of the baseline model, values > 0.90 are considered good (Bentler & Bonett, 1980); the Root Mean Square Error of Approximation, which measures the discrepancy through degrees of freedom between the sample and population estimates, values < 0.05 are considered very good (Kline, 2005); the Akaike Information Criterion, an index necessary to compare various alternative models that fit the data, the lower values indicate better models (Bentler & Bonett, 1980).
To estimate the construct validity, in the context of the Structural Equation Modeling, the factorial validity was assessed by the standardized weights (λ) and the individual reliability of the items (λ 2 ). Convergent validity was assessed using Average Extracted Variance and internal consistency. The internal consistency of the DASS-21 was verified using Cronbach's Alpha and Composite Reliability Coefficients, assessed for each of the factors and for the global factor. The discriminant validity was determined by comparing the Average Extracted Variance of the factors with the square of the correlation between them (Hair, Black, Babin, & Anderson, 2018).
In the context of Classical Statistics, when searching for evidences of convergent validity, the DASS-21 scores were correlated with the scores of the Five-Factor Personality Inventory Neuroticism subscale, a construct similar to negative affectivity. Pearson's Correlation Coefficient was used for that. To test reliability, it was also used the test-retest method by calculating the Intraclass Correlation Coefficient.

Results
Examination of the DASS-21 scores revealed a Mardia's coefficient of 43.03 (normalized = 11.96), which indicated the multivariate abnormality in their distribution. However, in the univariate distribution of scores, asymmetry was < ± 1.0 and kurtosis < ± 2.0, which is not considered an extreme violation of normality. In the CFA, the Maximum Likelihood estimation method was used, which is robust even in the presence of a non-normal data distribution (Marôco, 2014).
In the CFA performed for the DASS-21 one-dimensional model (Figure 1), the quality of the adjustment to the variance-covariance matrix of the 21 items was poor (Table 1) (Figure 2), that is, Negative Affectivity (2nd order) and Depression, Anxiety and Stress (1 st order), the same indices were found for both (Table 1) and the quality of adjustment to the data was good. All items in the DASS-21 three oblique factors models and DASS-21 2nd order , had standardized factor weights (λs) ≥ 0.50 (Figures 1 and 2). This indicates that all λ 2 s are ≥ 0.25, which corresponds to the amount of total variability for each item that is explained by the factor to which it belongs, that is, appropriate individual reliability.
Convergent validity, according to Fornell and Larcker (1981), was assessed using the mean variances of the items explained per factor, called Average Variance Extracted (AVE). In both models (DASS-21 three oblique factors and DASS-21 2nd order ), the Depression and Stress factors had convergent validity (AVE ≥ 0.50), but the Anxiety factor was slightly below (Table 2).
Regarding discriminant validity, it was assessed whether items that represent one factor are sufficiently independent from other factors. The AVEs of the three factors should exceed the square of the values of the correlations between them (Hair et al., 2018). However, in both models tested, the DASS-21 three oblique factors and DASS-21 2nd order , the factors were not sufficiently discriminatory when compared (Table 2).  The reliability of the DASS-21 2nd order model was calculated using the Composite Reliability Coefficient and Cronbach's Alpha Coefficient, which revealed adequate values of internal consistency. The test-retest with 77 participants, with an interval of 25 to 39 days, found intraclass correlation coefficients that indicate good temporal stability for the measurement ( Table 2).
The correlations between the DASS-21 Depression, Anxiety and Stress factors between themselves ranged from 0.78 to 0.91. The correlations of these factors with the DASS-21 global dimension ranged from 0.88 to 0.90. Still in the perspective of Classical Statistics, the global score of the Five-Factor Personality Inventory Neuroticism subscale (Cronbach's α = 0.85) showed significant correlations with the DASS-21 global score (r = 0.57; p < 0.01) and with the scores of the Depression (r = 0.45; p < 0.01), Anxiety (r = 0.48; p < 0.01) and Stress (r = 0.58; p < 0.01) factors.
Although the adjustments of the DASS-21 three oblique factors and DASS-21 2nd order models were good, the high correlation and insufficient evidence of discriminant validity between the factors justified further analysis. A CFA of the DASS-21 bifactor model was performed (Figure 2). The bifactor measurement model assumes that the covariance between a set of items can be explained by a set of orthogonal factors including a global factor and specific factors, that is, independently (Reise, 2012). The CFA results revealed that all 21 DASS-21 items saturated with higher factor loads in the global dimension than in the specific factors (Figure 2), making evident the existence of a global dimension. In addition, some items did not have sufficient factor loads (≥ 0.32) to represent the constructs of specific dimensions, although all of them have shown to be significant (p < 0.05). The obtained indices also revealed a good fit of this model to the empirical data, surpassing the previously tested models (Table 1).
However, there is evidence that traditional general fit indices tend to favor bifactor models over other models (Gignac, 2016). Thus, it was necessary to assess the robustness of the global factor and the specific factors using additional statistical indices. The explained common variance is a useful statistic because it reveals the variance attributable to the global dimension of the total common variance of the tested model (Bentler, 2009). One-dimensional data (theoretical) is the most extreme example and reaches 1.0. In the present study, the global factor with three specific factors had the best fit to the empirical data. The calculation of the common variance explained for this DASS-21 bifactor model revealed that the global factor was responsible for 65% of the explained common variance, the Depression factor, 16%, the Anxiety factor, 9%, and the Stress factor, 8%. That is, these indices indicate that the substantial majority of the normal data variance was explained by the global factor (Negative Affectivity) and that the specific factors (Depression, Anxiety, and Stress) had weaker participation.

Discussion
The present study investigated the evidence of the latent structure of the DASS-21, as well as its convergent and discriminant validity indicators and reliability indices in a Brazilian sample. After the Confirmatory Factor Analysis, it was observed that the hierarchical model (one global second-order factor and three firstorder factors) and the three oblique factors model presented identical indicators of good adjustment.
However, despite the adequate individual reliability of the items and their convergence to the factors they belong to, by calculating the AVE, there was no evidence of sufficient discriminating validity between the DASS-21 factors. Martins et al. (2019) and Patias et al. (2016) reported similar results in other Brazilian samples. One possible explanation is the high clinical overlap of depression, anxiety, and stress symptoms (Lovibond & Lovibond, 1995), leading to a high correlation between these factors. These indicators seem to suggest the hegemonic presence of a common factor (Negative Affectivity) for the DASS-21. That is, there are indications that the instrument may be predominantly one-dimensional, a result similar to the ones found by Osman et al. (2012) and Zanon et al. (2020).
To resolve these issues, analytical resources based on bifactor models were used. In the confirmatory analysis, the DASS-21 bifactor model test showed the best fit indices to the data. The comparison test (Δχ²) with the DASS-21 three oblique factors and DASS-21 2nd order models, demonstrated the superiority of adjustment of the DASS-21 bifactor model. Additional statistical analysis, such as explained common variance and estimated hierarchical omega, indicated that the DASS-21 would be better used as a general score for Negative Affectivity rather than three separate factors of depression, anxiety and stress.
It is a fact that the DASS-21 was built to assess the multiple dimensions of depression, anxiety and stress (Lovibond & Lovibond, 1995). However, the evidence generated in the present study with scores from Brazilian participants suggests that the measure is predominantly one-dimensional, corroborating the results found by Osman et al. (2012) and Zanon et al. (2020). Thus, although it is possible to identify four constructs for the DASS-21 latent structure (i.e., depression, anxiety, stress and negative affectivity), the results of this study indicate that the specific factors are minor variations of a global factor. The current diagnostic system used in psychiatric research and practice considers these mental disorders to be categorical and independent, however, results such as those found in the present study indicate that depressive, anxious and stress states are better understood as minor variations of a broader underlying syndrome, as well as signaled by Barlow et al. (2016). Other studies that investigated the latent structure of instruments measuring different psychopathological symptoms also confirmed the presence of a global factor in children, adolescents and adults, indicating the existence of a common vulnerability to all forms of psychopathology (Caspi & Moffitt, 2018;Laceulle, Chung, Vollebergh, & Ormel, 2020;Martel et al., 2017).
Some authors suggest the adoption of the general factor 'p' nomenclature for the global factor of psychopathology, in analogy to the 'g' factor of intelligence (Caspi & Moffitt, 2018). In studies on cognitive skills, although it is found the existence of specific factors (e.g., verbal ability, visual ability and processing speed), a single factor (i.e., 'g' factor) is able to summarize the performance of the participants in the different tests used, making it possible to draw a parallel to what has been observed about the global factor of psychopathology (Caspi & Moffitt, 2018).
In clinical practice, the common factor to different symptoms may suggest transdiagnostic processes for the development of emotional disorders (i.e., mechanisms that play an important role in the etiology, maintenance and evolution of different psychopathological states), making a unified treatment approach possible (Barlow et al., 2016;Falcone & Gonçalves, 2019). Standardized protocols focusing on transdiagnostic processes have been pointed out as efficient and effective tools for the treatment of different mental disorders, some even showing greater effectiveness when compared to specific protocols (Egan, Wade, Shafran, & Antony, 2014). Although they believe that studies in this regard are still recent and that any prescription related to treatment is premature, Caspi and Moffitt (2018) encouraged clinical research on the effectiveness of transdiagnostic interventions -psychotherapeutic and pharmacological -, as the first line of treatment, in which patients who do not show significant improvement are referred to specific treatments.
The existence of a global factor for psychopathology is also of particular relevance in terms of disease prevention. As noted, the 'p' factor may suggest the presence of a vulnerability to the development of any and all psychopathological conditions. Therefore, prevention strategies that focus on these common risk factors will tend to be more comprehensive than strategies aimed at preventing specific disorders (Caspi & Moffitt, 2018).
The moderate correlations between Neuroticism and DASS-21 factors, especially the global one, reveal good convergent validity of the instrument. Similar relationships were found by other authors (Barroso, Baptista, & Zanon, 2018;Gurtman, McNicol, & McGillivray, 2014). In relation to reliability, the internal consistency indices of the DASS-21 factors, by means of Cronbach's alpha and Composite Reliability, were satisfactory and similar to those obtained by other studies (Daza et al., 2002;Osman, 2012;Patias et al., 2016). The reliability represented by the DASS-21 temporal stability showed good indices, suggesting that the constructs assessed did not vary systematically in the studied intervals, as found by other authors (Bottesi et al., 2015). All indicators of reliability of the global factor (negative affectivity) were higher than those of the subscales.
In summary, the current results indicate that the DASS-21 presented evidence of a one-dimensional structure, linked to a correlated variable, internal consistency and temporal stability, with Brazilian participants. It should be noted, however, that the present study has some limitations in relation to non-probabilistic sampling as it is composed exclusively of university students, mostly female, which can make generalizations difficult for the rest of the population. The results, therefore, will have to be considered within these limits. It is suggested that future studies investigate the validity of DASS-21 in broader samples of the Brazilian population and with different characteristics from the ones presented here, such as adults, the elderly and, mainly, with clinical samples. It is also worth noting that, even though it assesses depressive, anxious, and stress symptoms, the DASS-21 is not intended to be a diagnostic tool. Despite these limitations, it is believed that the evidence generated in this study represents a contribution to advancing the investigation of negative affectivity.