DASS-21: assessment of psychological distress through the Bifactor Model and item analysis

DASS-21: avaliação do distresse psicológico pelo Modelo Bifactor e análise dos itens

DASS-21: evaluación del distrés psicológico mediante el Modelo Bifactor y análisis de ítems

Abstract

The term distress has been used to refer to a continuous variable operationalized through symptoms of depression, anxiety, and stress. In this study, psychological distress is measured using the Depression, Anxiety, and Stress Scale (DASS-21). Confirmatory Factor Analysis compared the fit of different measurement models for the DASS-21, with the parameters of the items verified through the Andrich rating scale model. A non-clinical sample of 530 participants (mean age=24.35±6.55 years; 71.89% women) responded to the instrument. According to the theoretical hypothesis, the results indicated a better fit for the bifactor model, composed of three specific factors (depression, anxiety, and stress) and a general factor (general psychological distress). The assessment of the item properties allowed for a better understanding of the organization of the continuum represented by the construct psychological distress. It is possible to conclude that the Brazilian version of the DASS-21 is an adequate measure for psychological distress.

Keywords:
stress; anxiety; depression; internal structure; test validity

Resumo

O termo “distresse” psicológico tem sido utilizado na literatura para se referir a uma variável contínua operacionalizada por meio dos sintomas de depressão, ansiedade e estresse. Esta pesquisa propõe a utilização da Escala de Depressão, Ansiedade e Estresse (DASS-21) para avaliação do distresse psicológico. Comparou-se o ajuste de diferentes modelos de medidas propostos a DASS-21 por meio da análise fatorial confirmatória e verificou-se os parâmetros dos itens com o Andrich rating scale model. Uma amostra não clínica de 530 participantes (idade: M = 24,35, DP = 6,55, 71,89% mulheres), respondeu ao instrumento. Os resultados indicaram melhor adequação do modelo bifactor composto por três fatores específicos (depressão, ansiedade e estresse) e um fator geral (distresse psicológico geral), conforme hipótese teórica. A avaliação das propriedades dos itens possibilitou melhor compreensão da organização do contínuo representado pelo construto distresse psicológico (severidade dos sintomas). Conclui-se que a versão brasileira da DASS-21 é uma medida adequada do distresse psicológico geral.

Palavras-chave:
estresse; ansiedade; depressão; estrutura interna; validade do teste

Resumen

El término distrés psicológico se ha utilizado en la literatura para referirse una variable continua operada por medio de los síntomas de depresión, ansiedad y el estrés. En esta investigación el distrés psicológico se mide a través de la Escala de Depresión, Ansiedad y Estrés (DASS-21). El Análisis Factorial Confirmatorio comparó el ajuste de diferentes modelos de medición para la DASS-21 y los parámetros de los ítems se verificaron a través de la Andrich rating scale model. Una muestra no clínica de 530 participantes (24,35 ± 6,55, 71,89% mujeres), respondieron al instrumento. Los resultados indicaron una mejor adecuación del modelo bifactor compuesto por tres factores específicos (depresión, ansiedad y estrés) y un factor general (distrés psicológico general), lo que correspondió con la expectativa teórica. La evaluación de las propiedades del ítem permitió una mejor comprensión de la organización del continuo representado por el constructo distrés psicológico. Se concluye que la versión brasileña DASS-21 es una medida adecuada del distrés psicológico general.

Palabras clave:
estrés; ansiedad; depresión; estructura interna; validez de test

Depression, anxiety, and stress are currently considered the main impairment related to mental health (Blanco & Canto-de-Souza, 2018). The impact of these psychopathologies on individuals’ daily lives is described in the literature as being associated with physical disorders such as hypertension, diabetes, impaired immune system (Faro, 2015), cognitive issues such as poor academic and professional performance, negative anticipation of events (Blanco & Canto-de-Souza, 2018), failures in adaptive processes (Sinclair et al., 2012), and increased rates of absenteeism at work with the consequent costs of the illness process (Bachion, Peres, Belisário, & Carvalho, 1998).

According to Lee, Lee, and Moon (2019), worldwide estimates indicate the existence of approximately 615 million individuals affected by symptoms associated with these psychopathologies. According to the latest report released by the World Health Organization (2017), Brazil has the highest levels of anxiety and depression among the Latin American countries. Given the social and scientific relevance that the study of these conditions can provide, further studies are needed, notably on the concept of psychological distress. Moreover, there is a need to better understand the relationship between anxiety, stress, and depression.

The term psychological distress has been used to refer to the intricate presence of depression, anxiety, and stress. Psychological distress is operationalized as a complex and multidimensional construct, characterized by the presence of internal or external stressors, which cause an unfavorable psychological state, associated with high emotional exhaustion. Psychological distress involves symptoms related to depression and anxiety such as low self-esteem, feelings of hopelessness, anxiety, agitation, and sadness (Varela, Pereira, Pereira, & Santos, 2017). The experience of these symptoms for more than two weeks can be an indicative sign of psychological distress (Boas & Morin, 2014).

Considering this breadth, the term has been comprehended as a relevant mental health indicator that allows screening for vulnerabilities related to the future development of these psychopathologies (Faro, 2015), as well as aspects related to quality of life, treatment/intervention planning and the evolution of the condition (Varela et al., 2017). Therefore, the availability of an indicator that allows screening for this type of symptom could directly contribute to the prevention of the onset of clinical conditions, since stressful events can precipitate episodes of anxiety and depression, often leading to characteristic stress responses (Margis, Picon, Cosner, & Silveira, 2003; Pinto, Martins, Pinheiro, & Oliveira, 2015). Although the importance of this type of screening is highlighted, there is still a shortage of psychological instruments directed toward assessing psychological distress.

The literature review highlights efforts by researchers focused on the development of measures able to identify symptoms associated with distress. For the general population there are the Brief Symptom Inventory (BSI, Derogatis & Meliaratos, 1983), the Symptom Questionnaire (SQ-48, Carlier et al., 2012), the Psychological Distress Manifestation Scale (PDMS, Massé et al., 1998) and the Depression, Anxiety and Stress Scale (DASS-21, Lovibond & Lovibond, 1995), the latter being the focus of the present study.

The DASS-21 was developed by Lovibond and Lovibond (1995) and is widely used in different countries. Its original version consists of 42 items (DASS) and, its brief version, 21 items (DASS-21) that are divided into three correlated dimensions (Gomez, 2013; Osman et al., 2012). The first of these dimensions, called stress, assesses the presence of negative affects, depressed mood, insomnia, discomfort, and irritability, which can be present in depressive and anxious situations. The second, called depression, refers to the presence of depressive symptoms such as self-depreciation, devaluation of life, and hopelessness. The third, called anxiety, assesses somatic tension and hyperactivity (Patias, Machado, Bandeira, & Dell’Aglio, 2016).

Over the previous two decades, different studies have been conducted to understand the functioning of the original and reduced versions of the instrument in clinical and non-clinical populations of different age groups (Leal, Antunes, Passos, Pais-Ribeiro, & Maroco, 2009; Pais-Ribeiro, Honrado, & Leal, 2004). These studies indicate that the DASS-21 is a valid and reliable measure for assessing constructs in different cultures, having been translated into more than 44 languages (Lee et al., 2019). However, due to the fact that these studies were carried out in different cultures, variations regarding the instrument’s internal structure have been observed. Different models have been found: three factors, second order model containing three factors (stress, depression, and anxiety), tripartite model (anhedonia, physiological hyperarousal, and a general negative factor), bifactor model (in which items mainly load on one general factor of distress and one of the specific factors of stress, depression, and anxiety), two-factor model (depression and anxiety/stress), four-factor model (a general factor of distress, depression, anxiety, and stress) and a single factor of distress (Apóstolo, Tanner, & Arfken, 2012; Henry & Crawford, 2005; Le et al., 2017; Lee, 2019; Nanthakumar et al., 2017; Valencia, 2019; Yudirim, Boysan, & Kefeli, 2018).

In Brazil, Patias et al. (2016) carried out the cultural adaptation and investigation of the factorial structure of the DASS-21. The study, with a sample of 686 adults from different regions, indicated the adequacy of the structure made up of three factors, with good levels of reliability (Cronbach’s alpha), which varied between .92 and .96. Vignola and Tucci (2014) also found a structure with three factors (depression, stress, and anxiety), with reliability indices (Cronbach’s alpha) that ranged from .86 to .92, in a clinical sample composed of 242 adults from São Paulo. Silva et al. (2016) also investigated the internal structure, with a sample of 310 adolescents from Pernambuco, indicating the relevance of a two-factor structure, one factor for assessing anxiety and stress and the other with the items developed for assessing depression. These factors showed good index of internal consistency (Cronbach’s alpha) between .88 and .77, respectively.

It should be highlighted that, in both the international and national literature, there is no consensus regarding the internal structure of the DASS-21, which has shown important variations depending on the sample and the country. Part of this situation can be understood in view of the existence of a structure composed of correlated factors. In order to overcome this problem, international studies have used the bifactor structure and the results have been more satisfactory than those presented by higher order factors. The bifactor model has managed to provide responses compatible with a latent structure that is simultaneously unidimensional (loads on a common factor) and multidimensional (loads on specific factors) (Cucina & Byle, 2017). This type of analysis was used in a series of studies with the instrument (Alfonsson, Wallin, & Maathz, 2017; Gomez, 2013; Henry & Crawford, 2005; Lee & Kim, 2020; Osman et al., 2012; Ruiz, García-Martín, Suárez-Falcón, & Odriozola-González, 2017; Shaw, Campbell, Runions, & Zubrick, 2017).

Although the potential of the DASS-21 to assess psychological distress through a bifactor structure has been the target of investigation in different countries, in the Brazilian context, research based on this data analysis methodology has not yet been carried out. Few studies were found that analyzed the instrument items using Item Response Theory (IRT). Furthermore, there is a lack of studies in both the international and national literature that aim to test the invariance of the measurement model among respondents according to age and gender groups, with the samples being restricted to homogeneous groups, either composed of adolescents or adults. Studies aimed at examining invariance have most commonly been conducted, investigating this issue in samples from different countries and cultures (Bibi, Lin, Zhang, & Margarf, 2020; Scholten, Velten, Bieda, Zhang, & Margraf, 2017; Zanon et al., 2020).

In order to provide contributions to fill these gaps, this study aimed to provide new evidence of validity based on the internal structure and study of the properties of the items using IRT of the Brazilian version of the DASS-21. For this, Confirmatory Factor Analyses (CFA) were performed to compare different measurement models commonly used in the literature (unifactorial, three correlated factors, three second order factors and bifactor model), as well as the evaluation of the invariance of the measurement model according to gender and age group (adolescents and adults).

Method

Participants

The sample consisted of 530 participants, between 12 and 50 years of age (M=24.35; SD=6.55 years), 202 of whom were male, with five participants not providing this data. The sample was divided into two age groups: adolescents from 12 to 18 years of age (n=149; M=16.25; SD=2.04), of whom 89 were female, and adults aged over 19 years (n=364; M=23.81; SD=6.45), of whom 222 were female. In the adult group, two participants did not report their gender and 17 did not report their age. The participants came from non-clinical populations from the state of Pernambuco (70%) and the capital of the state of Goiás (30%).

Instruments

Depression, anxiety and stress scale - DASS-21, this instrument aims to assess and discriminate symptoms of anxiety and depression, based on the Tripartite Model (Lovibond & Lovibond, 1995), which groups the symptoms of anxiety and depression into three factors: (a) presence of negative affect (depressed mood, insomnia, irritability), (b) specific factors of depression (anhedonia, absence of positive affect) and (c) specific symptoms of anxiety (somatic tension and hyperactivity) (Watson et al., 1995).

Consisting of 21 items, each factor groups seven items answered using a four-point Likert-type scale, indicating the severity and frequency of symptoms experienced in the previous seven days. In this way, the person evaluated scores 0 - “did not apply to me at”, 1 - “applied to me to some degree, or some of the time”, 2 - “applied to me to a considerable degree or a good part of time”, and 3 - “applied to me very much or most of the time”. The duration of the application is around ten minutes. Regarding the psychometric studies of the instrument in the Brazilian population, the original model was tested and confirmed, with adequate levels of internal consistency found for each factor (between .92 and .96) (Patias et al., 2016).

Procedures

After approval of the study by the Research Ethics Committee of the Pontifical Catholic University of Goiás PUCG, under authorization number CAAE: 01259018.9.0000.0037, data collection was started. For this, the consent form was handed to those of legal age and sent to those responsible for the participants under 18 years of age. Those that chose to participate needed to return the signed consent form. Participants between 12 and 18 years of age that participated in the study also needed to hand in the consent form duly completed and signed. At a specific date and time, the instrument was collectively applied in the classroom, in a single session that lasted about thirty minutes.

Data analysis

Confirmatory Factor Analysis (CFA) was used for the comparison and verification of the internal structure of the DASS-21. Accordingly, some models were tested and compared, using the χ² difference test, DIFFTEST (Muthén & Muthén, 2012), these being: unifactorial, three factors, three second order factors and bifactor. In addition, the estimation method appropriate to the level of ordinal measurement, Robust Weighted Least Square (WLSMV) (Lara & Alexis, 2014) was used. The WLSMV is considered the most accurate for evaluating models that have variables with less than five response categories (Rhemtulla, Brosseau-Liard, & Savalei, 2012). Finally, the invariance of the measurement model (Milfont & Fisher, 2010) between female and male participants was investigated, as well as between the adolescent and adult age groups. The models were tested based on the indices recommended by Muthén and Muthén (2012): WLSMV χ² , df, CFI, TLI and RMSEA, with the following values of model fit indicators adopted as a reference: χ² /df<3; CFI and TLI>.95 and RMSEA<.06. These analyses were performed using the statistical package Mplus 7.11 (Muthén & Muthén, 2012).

For the assessment of the reliability, an internal consistency analysis using the Cronbach’s alpha and McDonald’s omega coefficients was used, with reference values equal to or greater than .70. To evaluate the reliability of the bifactor model, the hierarchical Omega ω_H procedure was used, which assesses the level of reliability of specific factors while controlling the effect of the general factor (Reise, 2012).

The second stage of the analysis consisted of evaluating the parameters of the items in the DASS-21 using IRT, more specifically, the Andrich Rating Scale Model (with calibration estimated using the maximum likelihood method). It should be highlighted that this is a model belonging to the Rasch family for polytomous items, which restricts the discrimination parameter “a” equal to 1, under the assumption that all items are equally and strongly related to the underlying dimension (Linacre, 2015). Accordingly, the level of difficulty of the items was verified, in this case, the severity of the symptom described by the item, as well as the Infit and Outfit indices, item information curve and test information curve. These analyses were conducted using the statistical software R, and the MIRT package ­(Chalmers, 2012).

Results

In agreement with the first aim of the present study, fit indices of the different models proposed for the DASS-21 were estimated as described in the literature. The results are presented in Table 1, where the fit indices for the unifactorial, three factor, three second order factor and bifactor models are verified.

Table 1
Confirmatory Fit Indices for Different Factorial Models

Model	ƛ2	df	ƛ2/df	CFI	TLI	RMSEA	RMSEA 90%CI
Unifactorial	1186.99	189	6.28	.949	.943	.088	.084 0.093
Three factors	757.966	186	4.08	.971	.967	.067	.062 0.072
Second order	757.966	186	4.08	.971	.967	.067	.062 0.072
Bifactor	586.351	168	3.49	.989	.973	.061	.055 0.066
Invariance by sex
Configural	760.085	336	2.26	.977	.971	.061	.056 0.067
Scalar	809.816	412	1.97	.978	.978	.054	.048 0.050
Age group invariance
Configural	757.502	336	2.254	.974	.968	.062	.056 0.068
Scalar	902.833	412	2.191	.970	.969	.060	.055 0.066

• Note. ƛ² = Chi-square; df = degree of freedom; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMSEA = Root Mean Square Error of Approximation; CI = confidence interval.

As observed in Table 1, the results indicated adequate fit indices for the model composed of three correlated factors and the bifactor model, and acceptable fit indices for the unidimensional model. However, the bifactor model showed significantly better fit indices when compared to the other models, confirming the initial hypothesis of a better fit for this model. The results of the factor loadings, residual variance of the items and reliability indicator (omega hierarchical) of the tested bifactor model are presented in Table 2.

Table 2
Bifactor Model for the Brazilian Version of the DASS-21

Items	General factor	Specific factors			θ2
		1	2	3
1	.568	.398			.519
6	.642	.313			.490
8	.696	.458			.305
11	.655	.496			.325
12	.740	.318			.351
14	.720	.055			.478
18	.736	.219			.410
2	.590		.256		.586
4	.630		.597		.247
7	.705		.131		.485
9	.757		.010		.427
15	.838		.096		.288
19	.685		.371		.393
20	.742		.070		.445
3	.582			.453	.455
5	.608			.118	.616
10	.744			.399	.288
13	.725			.175	.443
16	.733			.280	.384
17	.720			.404	.318
21	.712			.433	.306
ωH	.89	.02	.01	.03
α	.941	.876	.850	.866

• Note. ω_H= Omega Hierarchical; α= Cronbach’s alpha; F1= stress; F2= Anxiety; F3= Depression; θ²= residual variance of the item

The factor loadings for the specific factors varied between .010 (item 9) and .597 (item 4) indicating that 0.1 to 37.0% of the variance of each item was predicted by the specific factors. For the general factor, the loadings varied between .568 (item 1) and .838 (item 15) indicating that from 32.26 to 70.22% of the single variance of each item was predicted by the general factor. These results are complementary to those observed using the Omega Hierarchical statistic, which suggests that a very high amount of variance in the sum of the scores can be attributed to the general factor, and that the reliability of specific factors, when controlling for variance of the general factor, is very low. Finally, Table 2 shows the Cronbach’s alpha and McDonald’s Omega indicators, which presented satisfactory values, that is, greater than .8.

A second stage of the evaluation of the internal structure of the scale corresponded to the analysis of the invariance of the measurement model between female and male participants and between the adolescent and adult age groups. The results presented in the lower half of Table 1 show the equivalence of the configural model, that is, of the general internal structure proposed for the measurement model, number of latent variables and their respective observed variables, as well as the invariance of the scalar model, which evaluates, in addition to the equivalence described in the configural model, the equivalence of the intercept of the items between the referred groups. With the Mplus software, it is not possible to estimate the metric invariance for the bifactor model, which refers to the equivalence of the factor loadings between the groups. Nevertheless, the fact that the differences between the CFI and RMSEA values of the scalar and configural models (∆CFI and RMSEA) were less than .01, indicates a strong invariance of the model between the groups (Rios & Wells, 2014; Chen, 2007). With strong invariane being supported, one can assume that invariance of a less constrained model (i.e. metric invariance) would be also supported.

Through IRT, a gradual response model, the parameters of the items were verified, the results of which (level of difficulty of the transition points of the response categories (threshold), item difficulty (mean of the threshold) and fit indices: infit and outfit) are shown in Table 3.

Table 3
Item Parameters Estimated by the Gradual Response Model

Items	Thr1	Thr2	Thr3	Mean Thr	Infit	Outfit
12S	-0.54	0.214	1.54	0.405	0.91	0.87
08S	-0.736	0.553	1.48	0.432	0.93	0.91
06S	-0.503	0.476	1.71	0.561	1.05	1.04
11S	-0.662	0.633	1.72	0.564	1.00	0.99
01S	-0.497	0.83	1.57	0.634	1.21	1.27
18S	-0.24	0.579	1.72	0.686	0.93	0.87
13D	-0.55	0.875	1.76	0.695	0.97	0.95
05D	-0.461	0.591	1.98	0.703	1.19	1.19
14S	0.045	0.642	1.86	0.849	1.01	1.03
09A	0.576	0.817	1.48	0.958	0.98	0.97
19A	0.704	0.735	1.92	1.120	1.03	1.05
20A	0.454	0.902	2.05	1.135	0.96	0.94
07A	0.89	0.693	1.92	1.168	1.04	1.15
10D	0.734	1.033	1.95	1.239	0.84	0.79
17D	0.952	1.071	1.77	1.264	0.91	0.83
02A	0.566	1.225	2.08	1.290	1.19	1.35
03D	0.595	1.298	2.06	1.318	1.13	1.47
04A	1.139	0.846	2.21	1.398	1.12	1.14
21D	1.343	0.910	1.98	1.411	0.92	0.80
16D	0.892	0.840	2.60	1.444	0.90	0.84
15A	1.405	0.984	2.48	1.623	0.73	0.67

• Note. Items presented in order of difficulty; Thr1 = response threshold between categories 0 and 1; Thr2 = response threshold between category 2 and 3; Thr3 = response threshold between categories 3 and 4; S = items on the stress scale; A = items on the anxiety scale; D = items on the depression scale.

From Table 3, it can be seen that the items with lower levels of difficulty (e.g. items are more easily endorsed by the participants) correspond to those from which the content represents the symptoms of stress: items 12, 8, 6, 11, 1, and 18, involving aspects such as difficulty in relaxing, feeling nervous, feeling irritable. In the middle of the continuum are items that represent less intense symptoms of depression based on the level of difficulty, such as 13 and 5, representing behaviors such as feeling sad, as well as an item that represents greater intensity of stress (being intolerant) and, mainly, some anxiety symptoms (items 9, 19, 20 and 7), related to symptoms such as concern about feeling panic, feeling unjustified tachycardia or tremors. In other words, only the participants that present existing distress at a moderate level possibly agree with the content of these items.

A third grouping, composed of more complex items, that is, those with higher levels of difficulty (items 10, 17, 2, 3, 4, 21, 16 and 15) was found. These items are related to more intense symptoms of depression and anxiety, involving the absence of positive expectations, feelings of little value, and the absence of positive feelings or meaning of life, as well as physiological symptoms (for ex. dry mouth and difficulty breathing). These items are more strongly endorsed by individuals who are likely to present high-level of psychological distress.

Regarding the thresholds between the categories (i.e. where the scores most likely changes category within the Likert scale), it was expected that these would show a pattern of increasing values for all items. An example is item 12, in which an individual with a skill level (theta) up to -0.54 probably answers zero on the Likert scale, from that value up to a theta of 0.21 they would indicate alternative 1, between 0.214 and 1.54 they indicate alternative 2 and, from that value up, would probably indicate alternative 3. Therefore, higher levels of skill in the construct (theta), in the case of distress, leads to a greater probability of the individual indicating a higher point on the scale. Only four items (04A, 07A, 15A, 16D) showed flaws in this pattern, suggesting an overlap between the first and second response categories for these items.

Table 3 also shows the item fit indices (infit/outfit), the results of which indicate good relevance for both indices (between 0.7 and 1.3), as recommended by Bond and Fox (2001). Exceptions occur in items 02A and 03D, which exceeded the upper limit of the outfit. The item information curves and test information curve were examined, with the results shown in Figure 1.

Figure 1
Item information curve and Test information curve/standard error

The graph on the left of Figure 1 shows that the items with the highest levels of information were: “09A - worry about feeling panic”, “17D - feelings of little value” and “21D - absence of meaning in life”. The items with the lowest levels of information were: “05D - difficulty in taking initiative”, “06S - exaggerated reactions” and “11S - feeling agitated”. However, there were two sets of overlapping item information curves, the first set of overlapping curves has its highest level of information located at approximately 0.5 logit of the theta scale, while the second has its highest level of information located at approximately 1 logit on the theta scale. These characteristics of the items result in the test information curve that exceeds standardized errors in the range of -2 to 4 logits, with the best reliability range of the instrument occurring between points 0 and 2 logits on the theta scale (see image located on the right of Figure 1).

Discussion

The main objective of this study was to evaluate the potential of the DASS-21 for the assessment of global psychological distress and to investigate new evidence of validity based on the internal structure for the Brazilian version of the instrument. These efforts correspond to an international trend regarding the establishment of a new validity argument (American Educational Research Association [AERA], American Psychological Association [APA], & National Council of Measurement in Education [NCME], 2014) for instruments. In addition, this analysis makes it possible to assess the potential of the DASS-21 for the evaluation of global psychological distress and to support the inferences made in international studies (Alfonsson et al., 2017; Gomez, 2013; Henry & Crawford, 2005; Shaw et al., 2017), as well as filling an important gap in the Brazilian literature regarding the scarcity of instruments qualified to assess psychological distress.

From the CFA, different measurement models were compared for the DASS-21. The results indicated a significantly higher fit for the bifactor model, corroborating studies conducted within American (Osman et al., 2012), Australian (Gomez, 2013; Shaw et al., 2017), Swedish (Alfonsson et al., 2017), Korean (Lee & Kim, 2020), Spanish (Ruiz et al., 2020), and English (Henry & Crawford, 2005) populations. However, it should be highlighted that, with the exception of the unifactorial model that presented fit indices classified as inadequate, the other models tested also presented satisfactory results and, therefore, can also be considered adequate. This phenomenon is not uncommon in the assessment of psychological constructs and can be identified as equivalence between models (Krunis & Maris, 2016). Fit for more than one model was also reported by Kyriazos, Stalikas, Prassa, and Yotsidi (2018) in which study the three-factor model showed a better higher fit than the others.

In the present study, other results were favorable for the adoption of the bifactor model. Among them, the proposal to assess global distress using the bifactor model is methodologically and theoretically justified. According to Chen, West, and Sousa (2006), the bifactor model is applicable when hypothesizing: a) a general factor that explains shared variance among all items of the model, in this case understood as global psychological distress; b) the existence of multiple specific orthogonal factors, each hypothesized to explain the unique variance of the specific domain, in this case depression, anxiety, and stress, in addition to the general factor; and c) if researchers are interested in the specific factors of the domain, as well as in the general factor.

Also in relation to the internal structure of the DASS-21, indicators of invariance of the factorial structure were found when comparing the assessment of different groups according to the gender and age group of the participants. This result makes it possible to compare the scores of these groups in future studies (Chen, 2007; Cook, Kallen, & Amtmann, 2009; Milfont & Fisher, 2010). The same type of invariance, according to the participant’s gender, has also been reported in other studies with the instrument (Jafari, Nozari, Ahrari, & Bagheri, 2017; Kyriazos et al., 2018; Le et al., 2017; Lee & Kim, 2020; Pezirkianidis, Karakasidou, Lakioti, Stalikas, & Galanakis, 2018). Accordingly, possible differences found between groups would not be associated with measurement errors attributed to the instrument, but with possible real differences between the groups. The importance of this type of analysis is based on the fact that the lack of information related to the invariance of measurement instruments can lead researchers to make inadequate comparisons, even violating ethical issues related to psychological testing and assessment (Milfont & Fisher, 2010).

Through the item and test information curves, it was shown that better levels of reliability of the instrument are located on the positive part of the continuum. These results are consistent with the instrument’s proposal to more precisely evaluate people who have medium and high levels of psychological distress, since these are the respondents that need to be identified in a psychological screening.

The aims established for the present study were satisfactorily achieved, as evidence of validity for the DASS-21 was demonstrated, based on the internal structure and reliability, regarding the assessment of global psychological distress, using a bifactor model, this being in agreement with international studies (Henry & Crawford, 2005; Lee et al., 2019; Shaw et al., 2017; Valencia, 2019). It should be highlighted that the use of IRT enabled the investigation of characteristics of the scale not verified in previous studies (level of difficulty and fit of the items), as well as the organization of the continuum that represents general psychological distress.

The results of the study confirm the possibility of using the instrument as a general measure of psychological distress, through the use of a general factor, as well as flexibility for the use of the three subscales (depression, anxiety, and stress). The analysis of the items also showed that the general construct presents itself within a continuum of disposition of difficulty of the items that: 1) starts with symptoms such as difficulty to relax, feeling nervous, and irritation, 2) presents an intermediate stage with symptoms such as feeling of sadness, difficulty in taking initiatives and 3) ends with more complex symptoms such as lack of enthusiasm and meaning in life, and fear of panicking. In general, what is observed is that this continuum represents different intensities of psychological distress, starting from the perception of less intense symptoms (daily stress), going through excessive worry, up to the feeling of inability to deal with these symptoms and a sensation of loss of control (high level of stress).

Finally, some limits of this study can be highlighted. Firstly, the results from this study were obtained based on a convenience sample from two cities located in specific regions of Brazil, Northeast and Central-West. Secondly, it should be noted that the study was conducted within a non-clinical sample. Further studies should replicate the present results within a clinical population. Finally, caution is suggested in the generalization of the results obtained in the present study and new studies should be performed to investigate other evidence of validity of the DASS-21 for assessment of psychological distress, such as evidence based on the relationship with external variables, for example.

References

• Alfonsson, S., Wallin, E., & Maathz, P. (2017). Factor structure and validity of the Depression, Anxiety and Stress Scale-21 in Swedish translation. Journal of Psychiatric and Mental Health Nursing, 24(2-3), 154-162. doi: 10.1111/jpm.12363.
  » https://doi.org/10.1111/jpm.12363
• American Educational Research Association [AERA], American Psychological Association [APA], & National Council of Measurement in Education [NCME] (2014). Standards for educational and psychological testing. Washington, DC: APA.
• Apóstolo, J. L. A., Tanner, B. A., & Arfken, C. L. (2012). Confirmatory factor analysis of the portuguese Depression Anxiety Stress Scales-21. Revista latino-americana de enfermagem, 20(3), 590-596. doi: 10.1590/S0104-11692012000300022.
  » https://doi.org/10.1590/S0104-11692012000300022
• Bachion, M. M., Peres, A. D. S., Belisário, V. L., & Carvalho, E. C. D. (1998). Estresse, ansiedade e coping: uma revisão dos conceitos, medidas e estratégias de intervenção voltadas para a prática de enfermagem. Revista mineira de enfermagem, 2(1), 33-39. Recuperado de http://reme.org.br/artigo/detalhes/867
  » http://reme.org.br/artigo/detalhes/867
• Bibi, A., Lin, M., Zhang, X. C., & Margarf, J. (2020). Psychometric properties and measurement invariance of Depression, Anxiety and Stress Scales (DASS-21) across cultures. International Journal of Psychology. https://doi.org/10.1002/ijop.12671
  » https://doi.org/https://doi.org/10.1002/ijop.12671
• Blanco, M. B., & Canto-de-Souza, A. L. M. (2018). Ansiedade, memória e o transtorno de estresse pós-traumático. CES Psicología, 11(2), 53-65. doi: 10.21615/cesp.11.2.5
  » https://doi.org/10.21615/cesp.11.2.5
• Boas, A. A. V. & Morin, E. M. (2014). Psychological well-being and psychological distress for professors in Brazil and Canada. Revista de Administração do Mackenzie, 15(6), 201-2019. doi: 10.1590/1678-69712014/administracao.v15n6p201-219.
  » https://doi.org/10.1590/1678-69712014/administracao.v15n6p201-219
• Carlier, I., Schulte-Van Maaren, Y., Wardenaar, K., Giltay, E., Van Noorden, M., Vergeer, P., & Zitman, F. (2012). Development and validation of the 48-item Symptom Questionnaire (SQ-48) in patients with depressive, anxiety and somatoform disorders. Psychiatry Research, 200(2-3), 904-910. doi: 10.1016/j.psychres.2012.07.035.
  » https://doi.org/10.1016/j.psychres.2012.07.035
• Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29. Recuperado de: www.jstatsoft.org/v48/i06/
  » www.jstatsoft.org/v48/i06/
• Chen, F. (2007). Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Structural Equation Modeling, 14, 464-504. https://doi.org/10.1080/10705510701301834.
  » https://doi.org/https://doi.org/10.1080/10705510701301834
• Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41(2), 189-225. doi: 10.1207/s15327906mbr4102_5
  » https://doi.org/10.1207/s15327906mbr4102_5
• Cook, K. F., Kallen, M. A., & Amtmann, D. (2009). Having a fit: impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption. Quality of Life Research, 18(4), 447-460. doi: 10.1007/s11136-009-9464-4
  » https://doi.org/10.1007/s11136-009-9464-4
• Cucina, J. & Byle, K. (2017). The Bifactor Model Fits Better Than the Higher-Order Model in More Than 90% of Comparisons for Mental Abilities Test Batteries. Journal of Intelligence, 5(3), 27. doi: 10.3390/jintelligence5030027.
  » https://doi.org/10.3390/jintelligence5030027
• Derogatis, L. R. & Melisaratos, N. (1983). The Brief Symptom Inventory: An introductory report. Psychological Medicine, 13(3), 595-605. doi: 10.1017/S0033291700048017.
  » https://doi.org/10.1017/S0033291700048017
• Faro, A. (2015). Estresse e distresse: estudo com a Escala de Faces em Aracajú (SE). Temas em Psicologia, 23 (2), 341-354. doi: 10.9788/TP2015.2-08.
  » https://doi.org/10.9788/TP2015.2-08
• Gomez, R. (2013). Depression Anxiety Stress Scales: Factor structure and differential item functioning across women and men. Personality and Individual Differences, 54(6), 687-691. doi: 10.1016/j.paid.2012.11.025.
  » https://doi.org/10.1016/j.paid.2012.11.025
• Henry, J. D. & Crawford, J. R. (2005). The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. British journal of clinical psychology, 44(2), 227-239. doi: 10.1348/014466505X29657.
  » https://doi.org/10.1348/014466505X29657
• Jafari, P., Nozari, F., Ahrari, F., & Bagheri, Z. (2017). Measurement invariance of the Depression Anxiety Stress Scales-21 across medical student genders. International Journal of Medical Education, 8, 116-122. doi: 10.5116/ijme.58ba.7d8b
  » https://doi.org/10.5116/ijme.58ba.7d8b
• Kyriazos, T., Stalikas, A., Prassa, K. and Yotsidi, V. (2018). Can the Depression Anxiety Stress Scales Short Be Shorter? Factor Structure and Measurement Invariance of DASS-21 and DASS-9 in a Greek, Non-Clinical Sample. Psychology, 9, 1095-1127. doi: 10.4236/psych.2018.95069.
  » https://doi.org/10.4236/psych.2018.95069
• Lara, D. & Alexis, S. (2014). ¿Matrices Policóricas/Tetracóricas o Matrices Pearson? Un estudio metodológico. Revista Argentina de Ciencias del Comportamiento, 6 (1), 39 - 48. doi: 10.30882/1852.4206.v6.n1.6357.
  » https://doi.org/10.30882/1852.4206.v6.n1.6357
• Le, M. T. H., Tran, T. D., Holton, S., Nguyen, H. T., Wolfe, R., & Fisher, J. (2017). Reliability, convergent validity and factor structure of the DASS-21 in a sample of Vietnamese adolescents. Plos One. https://doi.org/10.1371/journal.pone.0180557
  » https://doi.org/https://doi.org/10.1371/journal.pone.0180557
• Leal, I. P., Antunes, R., Passos, T., Pais-Ribeiro, J., & Maroco, J. (2009). Estudo da escala de depressão, ansiedade e stresse para crianças (EADS-C). Psicologia, Saúde & Doenças, 10(2), 277-284. Recuperado de: http://www.scielo.mec.pt/scielo.php?script=sci_arttext&pid=S1645-00862009000200011
  » http://www.scielo.mec.pt/scielo.php?script=sci_arttext&pid=S1645-00862009000200011
• Lee, D. (2019). The convergent, discriminant, and nomological validity of the Depression Anxiety Stress Scales-21 (DASS-21). Journal of Affective Disorders, 259, 136-142. https://doi.org/10.1016/j.jad.2019.06.036.
  » https://doi.org/https://doi.org/10.1016/j.jad.2019.06.036
• Lee, B., & Kim, Y. E. (2020). Validity of the depression, anxiety, and stress scale (DASS-21) in a sample of Korean university students. Current Psychology. https://doi.org/10.1007/s12144-020-00914-x
  » https://doi.org/https://doi.org/10.1007/s12144-020-00914-x
• Lee, J., Lee, E., & Moon, S. H. (2019). Systematic review of the measurement properties of the Depression Anxiety Stress Scale-21 by applying updated COSMIN methodology. Quality of Life Research . doi: 10.1007/s11136-019-02177-x
  » https://doi.org/10.1007/s11136-019-02177-x
• Linacre, J. M. (2015). Winsteps Rasch Measurement Computer Program User’s Guide. Beaverton, OR Winsteps.com.
• Lovibond, P. F. & Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck depression and anxiety inventories. Behaviour Research and Therapy, 33(3), 335-343. doi: 10.1016/j.rbp.2012.05.003.
  » https://doi.org/10.1016/j.rbp.2012.05.003
• Margis, R., Picon, P., Cosner, A. F., & Silveira, R. D. O. (2003). Relação entre estressores, estresse e ansiedade. Revista de Psiquiatria do Rio Grande do Sul, 25(1), 65-74. Recuperado de http://www.scielo.br/pdf/rprs/v25s1/a08v25s1
  » http://www.scielo.br/pdf/rprs/v25s1/a08v25s1
• Massé, R., Poulin, C., Dassa, C., Lambert, J., Bélair, S., & Battaglini, A. (1998). The structure of mental health: higher order confirmatory factor analyses of psychological distress and well-being measures. Social Indicators Research, 45(1-3), 475-504. Recuperado de https://page-one.springer.com/pdf/preview/10.1023/A:1006992032387
  » https://page-one.springer.com/pdf/preview/10.1023/A:1006992032387
• Milfont, T. L. & Fisher, R. (2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1), 111-121. doi: 10.21500/20112084.857
  » https://doi.org/10.21500/20112084.857
• Muthén, L. K. & Muthén, B. O. (2012). Mplus User’s Guide (7th ed.). Los Angeles, CA.
• Nanthakumar, S., Bucks, R. S., Skinner, T. C., Starkstein, S., Hillman, D., James, A., & Hunter, M. (2017). Assessment of the Depression Anxiety and Stress Scale (DASS-21) in untreated obstructive sleep apnea (OSA). Psychological Assessment, 29(10), 1201-1209. doi: 10.1037/pas0000401.
  » https://doi.org/10.1037/pas0000401
• Organização Mundial de Saúde [World Health Organization] (2017). Depression and Other Common Mental Disorders - Global Health Estimates. Recuperado de http://apps.who.int/iris/bitstream/10665/254610/1/WHO-MSD-MER-2017.2-eng.pdf?ua=1
  » http://apps.who.int/iris/bitstream/10665/254610/1/WHO-MSD-MER-2017.2-eng.pdf?ua=1
• Osman, A., Wong, J. L., Bagge, C. L., Freedenthal, S., Gutierrez, P. M., & Lozano, G. (2012). The Depression Anxiety Stress Scale -21 (DASS-21): further examination of dimensions, scale reliability, and correlates. Journal of Clinical Psychology, 68(12), 1322-1338. doi: 10.1002/jclp.21908.
  » https://doi.org/10.1002/jclp.21908
• Pais-Ribeiro, J. L., Honrado, A., & Leal, I. (2004). Contribuição para o estudo da adaptação portuguesa das escalas de ansiedade, depressão e stress (EADS) de 21 itens de Lovibond e Lovibond. Psicologia, Saúde & Doenças , 5(2), 229-239. Recuperado de http://www.scielo.mec.pt/scielo.php?pid=S1645-00862004000200007&script=sci_arttext&tlng=em
  » http://www.scielo.mec.pt/scielo.php?pid=S1645-00862004000200007&script=sci_arttext&tlng=em
• Patias, N. D., Machado, W. D. L., Bandeira, D. R., & Dell’Aglio, D. D. (2016). Depression Anxiety and Stress Scale (DASS-21)-Short Form: Adaptation and Validation for Brazilian Adolescents. Psico-USF, 21(3), 459-469. doi: 10.1590/1413-82712016210302.
  » https://doi.org/10.1590/1413-82712016210302
• Pezirkianidis, C., Karakasidou, E., Lakioti, A., Stalikas, A., & Galanakis, M. (2018). Psychometric Properties of the Depression, Anxiety, Stress Scales-21 (DASS-21) in a Greek Sample. Psychology, 9(15), 2933-2950. doi: 10.4236/psych.2018.915170
  » https://doi.org/10.4236/psych.2018.915170
• Pinto, J. C., Martins, P., Pinheiro, T. B., & Oliveira, A. C. (2015). Ansiedade, depressão e stresse: um estudo com jovens adultos e adultos portugueses. Psicologia, Saúde & Doenças , 16(2), 148-163.doi: 10.15309/15psd160202.
  » https://doi.org/10.15309/15psd160202
• Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate behavioral research, 47(5), 667-696.doi: 10.1080/00273171.2012.715555.
  » https://doi.org/10.1080/00273171.2012.715555
• Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354-373. doi: 10.1037/a0029315.
  » https://doi.org/10.1037/a0029315
• Rios, J. & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26(1), 108-116. doi: 10.7334/psicothema2013.260.
  » https://doi.org/10.7334/psicothema2013.260
• Ruiz, F. J., García-Martín, M. B., Suárez-Falcón, J. C., Odriozola-González, P. (2017). The Hierarchical Factor Structure of the Spanish Version of Depression Anxiety and Stress Scale -21. International Journal of Psychology and Psychological Therapy, 17(1), 97-105. Recuperado de https://www.ijpsy.com/volumen17/num1/460.html
  » https://www.ijpsy.com/volumen17/num1/460.html
• Scholten, S., Velten, J., Bieda, A., Zhang, X. C., & Margraf, J. (2017). Testing measurement invariance of the Depression, Anxiety, and Stress Scales (DASS-21) across four countries. Psychological Assessment , 29(11), 1376-1390. https://doi.org/10.1037/pas0000440
  » https://doi.org/https://doi.org/10.1037/pas0000440
• Shaw, T., Campbell, M.A., Runions, K.C., & Zubrick, S. R. (2017) Properties of the DASS-21 in an Australian community adolescent population. Journal of Clinical Psychology , 73(7), 879-892. doi: 10.1002/jclp.22376.
  » https://doi.org/10.1002/jclp.22376
• Silva, H. A., Passos, M. H. P., Oliveira, V. M. A., Palmeira, A. C., Pitangui, A. C. R., & Araújo, R. C. (2016). Versão reduzida da Depression Anxiety Stress Scale-21: ela é válida para a população brasileira adolescente?. Einstein (São Paulo), 14(4), 486-493. doi: 10.1590/s1679-45082016ao3732.
  » https://doi.org/10.1590/s1679-45082016ao3732
• Sinclair, S. J., Siefert, C. J., Slavin-Mulford, J. M., Stein, M. B., Renna, M., & Blais, M. A. (2012). Psychometric evaluation and normative data for the depression, anxiety, and stress scales - 21 (DASS-21) in a nonclinical sample of U.S. adults. Evaluation & the Health Profesions, 35 (3), 259-279. doi: 10.1177/0163278711424282.
  » https://doi.org/10.1177/0163278711424282
• Varela, A., Pereira, A., Pereira, A., & Santos, J.C. (2017). Distress psicológico: contributos para a adaptação portuguesa do SQ-48. Psicologia, Saúde & Doenças , 18(2), 278-296. doi: 10.15309/17psd180201.
  » https://doi.org/10.15309/17psd180201
• Valencia, P. (2019). Las Escalas de Depresión, Ansiedad y Estrés (DASS-21): ¿miden algo más que un factor general?. Avances En Psicología, 27(2), 177-190. https://doi.org/10.33539/avpsicol.2019.v27n2.1796
  » https://doi.org/https://doi.org/10.33539/avpsicol.2019.v27n2.1796
• Vignola, R. C. B. & Tucci, A. M. (2014). Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. Journal of Affective Disorders , 155, 104-109. doi: 10.1016/j.jad.2013.10.031
  » https://doi.org/10.1016/j.jad.2013.10.031
• Watson, D., Clark, L. A., Weber, K., Assenheimer, J. S., Strauss, M. E., & McCormick, R. A. (1995). Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. Journal of abnormal Psychology, 104(1), 15-25. doi: 10.1037%2F0021-843X.104.1.15.
  » https://doi.org/10.1037%2F0021-843X.104.1.15
• Yildirim, A., Boysan, M., & Kefeli, M. C. (2018) Psychometric properties of the Turkish version of the Depression Anxiety Stress Scale-21 (DASS-21), British Journal of Guidance & Counselling,46(5), 582-595. doi: 10.1080/03069885.2018.1442558
  » https://doi.org/10.1080/03069885.2018.1442558
• Zanon, C., Brenner, R. E., Baptista, M. N., & Zladi, A. (2020). Examining the Dimensionality, Reliability, and Invariance of the Depression, Anxiety, and Stress Scale-21 (DASS-21) Across Eight Countries. Assessment, 27(5). doi: 10.1177/1073191119887449
  » https://doi.org/10.1177/1073191119887449

Publication Dates

• Publication in this collection
  11 Mar 2022
• Date of issue
  Oct-Dec 2021

History

• Received
  07 Dec 2019
• Reviewed
  09 Nov 2020
• Accepted
  15 Dec 2020