Confirmatory factor analysis of the portuguese Depression Anxiety Stress Scales-21

Abstracts

El objetivo de este estudio fue determinar cual de los tres modelos publicados mejor caracteriza la estructura factorial de la versión portuguesa de la Depression Anxiety Stress Scale-21 (DASS-21) y evaluar su validez y confiabilidad. Se compararon los tres modelos a través de análisis factorial confirmatoria de la DASS-21, aplicada el 1.297 pacientes adultos, del servicio de atención básica (66,7% mujeres; edad Media=48,57 años). La relación entre la DASS-21 y la Positive and Negative Affect Schedule (PANAS) también fue analizada. El modelo de tres factores correlacionados se ajusta mejor a los datos. La escala presentó buena consistencia interna con valores alfa observados en las subescalas, variando de 0,836 a 0,897. La correlación con la PANAS fue positiva y comedida con la escala de afecto negativa, y negativa y limitada con la escala de afecto positivo. Esos resultados corroboran la estructura de tres factores. La prueba presentó confiabilidad adecuada y validez de constructo, dando soporte a su uso para rastrear pacientes portugueses en el servicio de atención básica.

Análisis Factorial; Traducción; Adulto; Cuestionarios

O objetivo deste estudo foi determinar qual dos três modelos publicados melhor caracteriza a estrutura fatorial da versão portuguesa da Depression Anxiety Stress Scale-21 (DASS-21) e avaliar sua validade e confiabilidade. Compararam-se os três modelos através de análise fatorial confirmatória da DASS-21, aplicada em 1.297 pacientes adultos, do serviço de atenção básica (66,7% mulheres; idade média=48,57 anos). A relação entre a DASS-21 e a Positive and Negative Affect Schedule (PANAS) também foi analisada. O modelo de três fatores correlacionados se ajusta melhor aos dados. A escala apresentou boa consistência interna com valores alfa observados nas subescalas, variando de 0,836 a 0,897. A correlação com a PANAS foi positiva e moderada com a escala de afeto negativa, e negativa e limitada com a escala de afeto positivo. Esses resultados corroboram a estrutura de três fatores. O teste apresentou confiabilidade adequada e validade de constructo, dando suporte ao seu uso para rastrear pacientes portugueses no serviço de atenção básica.

Análise Fatorial; Tradução; Adulto; Questionários

ORIGINAL ARTICLE

Confirmatory factor analysis of the portuguese Depression Anxiety Stress Scales-21

João Luís Alves Apóstolo^I; Barry Allen Tanner^II; Cynthia Lee Arfken^III

^IPhD, Adjunct Professor, Unidade de Investigação em Ciências da Saúde - Enfermagem, Escola Superior de Enfermagem de Coimbra, Portugal

^IIPhD, Clinical Psychologist, Life Stress Center, Detroit Receiving Hospital, USA

^IIIPhD, Associate Professor, Department of Psychiatry and Behavioral Neurosciences, School of Medicine, Wayne State University, USA

^{Corresponding Author}

ABSTRACT

To determine which of three published models best characterizes the factor structure of the Portuguese version of the Depression Anxiety Stress Scales-21 and to assess its validity and reliability. Confirmatory factor analysis of Depression Anxiety and Stress Scale-21 for 1,297 adult, primary care outpatients (66.7% female, Mage = 48.57 years) comparing 3 models. The relationship between the Depression Anxiety Stress Scales-21 and the Positive and Negative Affect Schedule was analyzed. The correlated 3-factor model fit the data best. The scale demonstrated good internal consistency, with alpha scores of the subscales ranging from 0.836 to 0.897. Correlation with the Positive and Negative Affect Schedule was positive and moderate with the negative affect scale; it was negative and limited with the positive affect. These findings support the correlated 3-factor structure. The test demonstrated adequate reliability and construct validity, which supports its use for screening in primary care settings with Portuguese speakers.

Descriptors: Factor Analysis; Translation; Adult; Questionnaires.

Introduction

Depression and anxiety are serious burdens worldwide. Nurses and other professionals have a responsibility to screen patients. However, to do so they need a validated measure. The Depression Anxiety Stress Scale (DASS) is a frequently used measure⁽¹⁾ with good psychometric properties⁽²⁾. It has a demonstrated ability to discriminate between clinical and nonclinical groups⁽³⁾.

The 42-item DASS was named for its three correlated factor scales of Depression, Anxiety, and Stress⁽⁴⁾. The abbreviated DASS21 was constructed by selecting seven items from each of the three main scales to represent the full-item content⁽⁴⁾. The DASS21 has a cleaner factor structure with less cross loading of items⁽³⁾.

An important step in adapting tests to new languages is determining construct equivalence between the original and the translation, with factor analysis often used to assess the extent to which a translation of a test measures the same constructs as the original⁽⁵⁾. A similar factor structure in the original and the translation suggests that they measure the same constructs, in this case depression, anxiety, and stress.

Inconsistent results for the factor structure of the DASS have been reported in Portugal. The 3-factor structure has been reported for both the 42-item DASS⁽⁶⁾ and the DASS21^(7-8) among Portuguese college students. In contrast, other researchers⁽⁹⁾ preferred a 2-factor solution consisting of a unique depression factor and a combined anxiety-stress factor for the DASS21, as completed by psychiatric outpatients (OP).

All four of the above Portuguese studies involved exploratory factor analysis. The three studies involving college students reported 3-factor solutions while the study involving psychiatric OP did not. Furthermore, two of the studies used an oblique rotation in which the factors were allowed to correlate^(6,8), consistent with the original work on the DASS⁽⁴⁾, while the others^(7,9) used orthogonal rotation in which the factors were not allowed to correlate. This raises questions as to both the number of factors of the Portuguese DASS21 and the relationship of the factors to each other. We expected the correlated 3-factor model to fit the data best, thereby providing evidence of construct validity for the original DASS model⁽⁴⁾ with this new population.

Another aspect of establishing construct equivalence involves demonstrating convergent and divergent validity through the relationship of the translated test to other measures. Studies have reported appropriate correlations between the English language DASS and Positive and Negative Affect Schedule (PANAS)⁽¹⁰⁾ for a variety of populations^(11-12), typically large positive correlations for the Negative scale (convergent validity) and smaller negative correlations for the Positive scale (divergent validity).

In summary, we had two goals in this study. The first was to determine if the Portuguese DASS21 is best described by three correlated factors, similar to the original English language test. The second was to assess the convergent and divergent validity of the Portuguese DASS21 scales through correlations with the Negative and Positive scales of the Portuguese PANAS. We expected the DASS21 to correlate positively with the Negative scale of the PANAS and negatively with its Positive scale.

Method

Participants

The study was approved by the Human Investigation Committee of the Portuguese Regional Health Center Administration prior to collecting data at health centers in four regions. Consecutive visitors aged 18 or older were asked to participate until we had at least 325 completed protocols from each region. Unpaid volunteers, adult primary care patients in Portugal, (N=1,301) provided written, informed consent to participate. Women accounted for 66.7% of the participants for whom gender was identified. The mean age was 48.57 years (SD=19.98). A third (N=427) of the respondents who reported their level of education had 0-4 years of school, just over a quarter (348) had 5-9 years, a fifth (259) had 10-12 years, and less than a fifth (228) had more than 12 years of school. A majority (N=741) of those recording their marital status were married or in a committed relationship, with just over a fifth (263) single, 12% (161) widowed, and 8.5% (110) separated or divorced. Most of these participants (N=1,052) also completed the PANAS.

Instruments

We used a Portuguese translation of the DASS21⁽⁹⁾. The items are in the same order as the English version, with all 21 items printed on one side of the form. The test uses Likert-scales ranging from 0 ("did not apply to me at all") to 3 ("applied to me very much, or most of the time"), with instructions to circle a numeral to indicate "how much the statement applied to you over the past week". Scores can range from 0 to 21 for each of the three scales and the scale scores may be totaled for a total distress score with a maximum of 63. Four incomplete protocols were excluded from the CFA, leaving a sample size of 1,297.

We also used a Portuguese translation of the PANAS⁽¹³⁾, which consists of two columns each of 10 mixed positive and negative affective adjectives. The positive adjectives load on one factor and the negative on another, corresponding to the Positive and Negative scales⁽¹⁴⁾. Respondents refer to a Likert-scale ranging from 1 ("very slightly or not at all") to 5 ("extremely") and write the corresponding numeral in the space provided to the left of each adjective. Various instructions regarding the time period to be reported have been found to produce similar results⁽¹⁰⁾. We chose the same time period as the DASS21, instructing our participants to "Indicate to what extent you have felt this way during the past week". Scores can range from 10 to 50 for each scale.

Analysis

The competing models for the DASS21 were examined with Mplus version 6.0 using the default settings (including maximum likelihood estimation), other than for specified correlations. Model 1 specified three orthogonal factors⁽⁷⁾, Model 2 specified two orthogonal factors⁽⁹⁾, and Model 3 specified three oblique factors^(6,8), consistent with the DASS21 manual⁽⁴⁾. All other statistics were computed with SPSS version 13.

Although CFA software provides information, which allows the user to improve fit by moving or removing items, such modifications may be based on characteristics unique to a particular sample, resulting in solutions that do not generalize to other samples⁽¹⁵⁾. Furthermore, modification of the original model involves switching from confirmatory to exploratory analysis and is open to abuse⁽¹⁶⁾. For these reasons, we chose not to modify our models.

Some authorities provide specific cutoff values for fit indexes akin to the .05 level in traditional hypothesis testing⁽¹⁷⁾, while others recommend caution in applying specific cutoffs⁽¹⁸⁾ and point out that these indexes were originally intended to provide an alternative to hypothesis testing, and are better used to identify improvements in fit when comparing models⁽¹⁹⁾. Therefore, we chose to look at improvement in the fit between models rather than applying cutoff values to the fit for individual models.

Use of the comparative fit index (CFI), Tucker-Lewis index (TLI), and root mean square error of approximation (RMSEA) has been recommended for an initial evaluation. When comparing models, the Akaike information criteria (AIC) and Bayesian information criteria (BIC) are recommended⁽¹⁶⁾. In addition to those statistics, we included the standard root mean square residual (SRMR) and χ², consistent with recommendations to combine indexes with different strengths and weaknesses⁽¹⁷⁾. CFI, TLI, RMSEA, and SRMR are standardized to produce values between 0 and 1, thereby facilitating the examination of individual models. AIC and BIC, however, are not standardized and values can only be interpreted when comparing the values of related models. A significant χ² indicates departure of the data from the model being tested, but it is sensitive to sample size, with large samples attaining significance even for extremely small differences⁽²⁰⁾. Smaller values of χ², RMSEA, SRMR, AIC, and BIC indicate better fit when comparing models, while larger values of CFI and TLI indicate comparatively better fit.

One may also formally test the significance of the difference between the χ² for two related models by computing χ²_diff⁽²¹⁾. χ²_diff is used to examine the improvement in fit of successive models rather than testing the significance of the fit of any one model. While this does involve significance testing, it is consistent with comparing the relative fit of multiple models rather than requiring that individual models meet a specific level of fit.

Results

Reliability

The alpha coefficient was .88 for the Depression scale, .84 for Anxiety, .90 for Stress, and .945 for the Total.

Scale scores

Table 1 provides the scores for each scale by gender and for the overall group. Women scored significantly higher than men on all four scales: Depression (t(n=1215) = 2.69, p = .007); Anxiety (t(n=1227) = 2.633, p = .009); Stress (t(n=1219) = 3.402, p = .001); and Total (t(n=1178) = 3.247, p = .001). We examined the distribution of scores in our sample with the Kolmogorov-Smirnov test, and found a significant departure from a normal distribution for each of the scales, with all p <.001.

Factor validity

Table 2 summarizes eight fit indexes computed for this study, with the models arranged from left to right by increasing goodness of fit. The orthogonal 2-factor model performed better than the orthogonal 3-factor model, while the oblique 3-factor model fit better still. Although χ² was still significant, χ²_diff indicated a significant improvement in fit from the orthogonal 2-factor to the oblique 3-factor model (p < .001). AIC, BIC, RMSEA, and SRMR also improved (decreased in magnitude), as did CFI and TLI (increased in size) from the orthogonal 2-factor model to the oblique 3-factor model.

Convergent and divergent validity

Table 3 lists the correlations among the DASS scales and between the DASS and PANAS scales. The scales correlated between .505 and .606 with the negative affect and from -.109 to -.229 with the positive affect, as measured by the PANAS (all p < .01). Intercorrelations of the DASS21 scales are positive and strong.

Discussion

We found that the oblique 3-factor model for the DASS21 fit our Portuguese data best, consistent with the English language DASS21 from which it was translated. Our findings are consistent with those from other countries^(1-2), providing additional evidence of the applicability of the DASS21 to multiple cultures, and these findings will facilitate cross-cultural comparisons. All loadings exceeded .30, a common threshold for the minimum acceptable loading⁽²²⁾, with all but item 2 surpassing .50, a recommended threshold for "strong" loadings⁽²³⁾.

We also found that the reliability of the DASS21 scales was adequate. The values compare favorably to those in the manual for the DASS21 where Depression = .81, Anxiety = .73, and Stress = .81. The total, though, is not reported in the manual⁽⁴⁾. They are also comparable to those for the UK normative sample ⁽¹²⁾, where the alpha for Depression was .88, Anxiety was .90, Stress was .90, and the Total was .93.

In an attempt to clarify the factor structure of the Portuguese DASS21, we examined the orthogonal 2-factor⁽⁹⁾, the orthogonal 3-factor⁽⁷⁾ and the oblique 3-factor^(6,8) models in what we believe is the first published confirmatory factor analysis (CFA) of this measure. CFA allows one to test how well each hypothesized model fits the current data using multiple fit indexes, in order to informally compare most of those fit indexes, and to explicitly determine the significance of the difference between the χ² obtained for two competing models.

These findings are also in accordance with results from other studies in which the PANAS was used as a measure for the criterion validity of the DASS^(7,12), although correlation values for DASS21 and positive affect are higher in those studies. Using recommended guidelines⁽²⁴⁾, the negative correlations between the PANAS Positive scale and all of the DASS scales are small, while the positive correlations between the PANAS Negative scale and each of the DASS scales are large, as are those among the various DASS scales.

Intercorrelations between the dimensions of DASS21 showed high positive values, demonstrating a very strong association between the scores of this instrument's subscales, ranging from .74 to .77. Similar findings have been reported in other studies with both clinical and non-clinical samples^(7,9,14).

The female participants scored significantly higher on all the DASS21 scales, similar to results found in Spain⁽²⁵⁾. Scores were not normally distributed, with relatively more low than high scores. A skewed distribution makes scores based on the mean and SD difficult to interpret for any given individual, and the use of percentile norms is recommended⁽¹²⁾.

Our sample contained a disproportionate percentage of female and middle-aged participants. Future work in this area might focus on developing Portuguese percentile norms with a more representative non-patient sample in terms of age and gender.

Conclusion

The oblique 3-factor model for the DASS21 fit our Portuguese data best.

Our findings support the validity of the Portuguese translation of the DASS21. Having a validated translation allows expanded screening for depression and anxiety in places such as primary care settings. This version is shorter than the DASS42 and, therefore, should be more acceptable for users, and yet still be adequately reliable.

Received: Dec. 19^th 2011

Accepted: Marc. 12^th 2012

• 1. Allen J, Annells M. A literature review of the application of the Geriatric Depression Scale, Depression Anxiety Stress Scales and Post-traumatic Stress Disorder Checklist to community nursing cohorts. J Clin Nurs. 2009 Mar;18(7):949-59.
• 2. Page AC, Hooke GR, Morrison DL. Psychometric properties of the Depression Anxiety Stress Scales (DASS) in depressed clinical samples. Br J Clin Psychol. 2007 Sep;46(Pt 3):283-97.
• 3. Antony MM, Bieling PJ, Cox, BJ, Enns MW, Swinson, RP. Psychometric properties of the 42-item and 21-item versions of the Depression Anxiety Stress Scales in clinical groups and a community sample. Psychol Assess. 1998 Jun;10(2):176-81.
• 4. Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. Sydney: Psychology Foundation; 1995. 42 p.
• 5. Sireci SG, Patsula L, Hambleton RK. Statistical methods for identifying flaws in the test adaptation process. In: Hambleton RK, Merenda PF, Spielberger CD, editors. Adapting educational and psychological tests for cross-cultural assessment. Mahwah, New Jersey: Erlbaum; 2005. p. 93-115.
• 6. Alves G, Carvalho M, Baptista, A. Estudo das características psicométricas de uma escala de Depressão, Ansiedade e Stresse em jovens adultos. [Psychometric characteristics of a Depression Anxiety and Stress scale in young adults.] In: Soares AP, Araújo S, Caíres S, editors. Avaliação psicológica: formas e contextos. Braga: APPORT; 1999. p. 267-75.
• 7. Apóstolo JLA, Mendes AC, Rodrigues MA. Propriedades psicométricas da Escala de Depressão, Ansiedade e Stresse (DASS-21), numa amostra não clínica. [Psychometric properties of the Depression Anxiety Stress Scales (DASS-21) in a nonclinical sample.] Rev Invest Enferm. 2007;15:66-76.
• 8. Pais-Ribeiro JL, Honrado A, Leal I. Contribuição para o estudo da adaptação Portuguesa das Escalas de Ansiedade, Depressão e Stress (EADS) de 21 itens de Lovibond e Lovibond. [Study of the Portuguese adaptation of the 21-item Anxiety, Depression and Stress Scales (DASS21) of Lovibond and Lovibond]. Psicol Saúde & Doenças. 2004;5(2):229-39.
• 9. Apóstolo JLA, Mendes AC, Azeredo ZA. Adaptation to Portuguese of the Depression, Anxiety and Stress Scales (DASS). Rev. Latino-Am. Enfermagem. 2006 Nov-Dec;14(6):863-71.
• 10. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: The PANAS scales. J Pers Soc Psychol. 1988 Jun;54(6):1063-70.
• 11. Gloster AT, Rhoades HM, Novy D, Klotsche J, Senior A, Kunik M, et al. Psychometric properties of the Depression Anxiety and Stress Scale-21 in older primary care patients. J Affect Disord. 2008 Oct;110(3):248-59.
• 12. Henry JD, Crawford JR. The short-form version of the Depression Anxiety Stress Scales (DASS-21): Construct validity and normative data in a large non-clinical sample. Br J Clin Psychol. 2005 Jun;44(2):227-39.
• 13. Simões A. São os homens mais agressivos que as mulheres? [Are men more aggressive than women?] Rev Portuguesa Pedagogía. 1993;27(3):387-404.
• 14. Apóstolo J, Rodrigues M. Propriedades psicométricas da Positive and Negative Affect Schedule (PANAS) numa amostra não clínica. [Psychometric properties of the Positive and Negative Affect Schedule (PANAS) in a nonclinical sample.] Referência. 2009;5:2 série,Suppl:394.
• 15. MacCallum RC, Roznowski M, Necowitz LB. Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychol Bull. 1992 May;111(3):490-504.
• 16. Schreiber JB, Stage FK, King J, Nora A, Barlow EA. Reporting structural equation modeling and confirmatory factor analysis results: A review. J Educ Res. 2006 Jul-Aug;99(6):323-37.
• 17. Hu L-t, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1-55.
• 18. Sharma S, Mukherjee S, Kumar, A, Dillon WR. A simulation study to investigate the use of cutoff values for assessing model fit in covariance structure models. J Bus Res. 2005;58(7):935-43.
• 19. Marsh HW, Hau K-T, Wen Z. In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Struct Equ Modeling. 2004;11(3):320-41.
• 20. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2006. 475 p.
• 21. Gorsuch R. Factor analysis. In: Schinka JA, Velicer WF, Weiner IB, editors. Handbook of Psychology, v. 2. Research Methods in Psychology. New York: Wiley; 2003. p. 143-64.
• 22. Waltz CF, Strickland O, Lenz ER. Measurement in nursing and health research. 4^th ed. New York: Springer; 2010. 492 p.
• 23. Costello AB, Osborne JW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval. [periódico na Internet] 2005; [acesso 13 jan 2011]; 10(7):1-9. Disponível em: http://pareonline.net/getvn.asp?v=10&n=7
• 24. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New Jersey: Lawrence Erlbaum; 1988. 569 p.
• 25. Fonseca-Pedrero E, Paino M, Lemos-Giráldez S, Muñiz J. Propriedades psicométricas de la Depression Anxiety and Stress Scales-21 (Dass-21) en universitarios españoles. [Psychometric properties of the Depression Anxiety Stress Scales-21 (DASS-21) in Spanish university students.] Ansiedad y Estrés. 2010;16(2-3):215-26.

Publication Dates

• Publication in this collection
  12 Sept 2012
• Date of issue
  June 2012

History

• Received
  19 Dec 2011
• Accepted
  12 Mar 2012