ABSTRACT

Objective

To evaluate the interday reproducibility, agreement and validity of the construct of short version of the Depression Anxiety Stress Scale-21 applied to adolescents.

Methods

The sample consisted of adolescents of both sexes, aged between 10 and 19 years, who were recruited from schools and sports centers. The validity of the construct was performed by exploratory factor analysis, and reliability was calculated for each construct using the intraclass correlation coefficient, standard error of measurement and the minimum detectable change.

Results

The factor analysis combining the items corresponding to anxiety and stress in a single factor, and depression in a second factor, showed a better match of all 21 items, with higher factor loadings in their respective constructs. The reproducibility values for depression were intraclass correlation coefficient with 0.86, standard error of measurement with 0.80, and minimum detectable change with 2.22; and, for anxiety/stress: intraclass correlation coefficient with 0.82, standard error of measurement with 1.80, and minimum detectable change with 4.99.

Conclusion

The short version of the Depression Anxiety Stress Scale-21 showed excellent values of reliability, and strong internal consistency. The two-factor model with condensation of the constructs anxiety and stress in a single factor was the most acceptable for the adolescent population.

Affective symptoms; Surveys and questionnaires; Reproducibility of results; Adolescent; Validation studies; Psychiatric status rating scales

RESUMO

Objetivo

Avaliar a reprodutibilidade interdias, a concordância e a validade do construto da versão reduzida da Depression Anxiety Stress Scale-21 aplicada a adolescentes.

Método

A amostra foi composta por adolescentes de ambos os sexos, com idades entre 10 e 19 anos, recrutados de escolas e centros esportivos. A validade de construto foi realizada por análise fatorial exploratória, e a confiabilidade foi calculada para cada construto, por meio de coeficiente de correlação intraclasse, erro padrão de medida e mudança mínima detectável.

Resultados

A análise fatorial combinando os itens correspondentes a ansiedade e estresse em um único fator, e depressão em um segundo fator apresentou melhor adequação de todos os 21 itens, com cargas fatoriais mais altas em seus respectivos construtos. Os valores de reprodutibilidade para a depressão foram coeficiente de correlação intraclasse com 0,86, erros padrão de medida com 0,80 e mudança mínima detectável com 2,22 e, para a ansiedade/estresse, foram coeficiente de correlação intraclasse com 0,82, erro padrão de medida com 1,80 e mudança mínima detectável com 4,99.

Conclusão

A versão reduzida da Depression Anxiety Stress Scale-21 apresentou excelentes valores de confiabilidade e também uma forte consistência interna. O modelo de dois fatores com a condensação dos construtos ansiedade e estresse em um único fator foi o mais aceitável para a população adolescente.

Sintomas afetivos; Inquéritos e questionários; Reprodutibilidade dos testes; Adolescente; Estudos de validação; Escalas de graduação psiquiátrica

INTRODUCTION

The short version of the Depression Anxiety Stress Scale-21 (DASS-21) was developed to provide a self-report measure of anxiety, depression and stress signals. During the development process, it was established that the main symptoms of depression are low self-esteem, hopelessness, devaluation of life, self-deprecation and inertia. The main symptom of anxiety is physiological arousal. The stress construct of the scale emerged empirically during the development of the depression and anxiety scales, through aggregation of items relating to difficulty relaxing, tension, impatience, irritability and restlessness.(¹1. Szabó M, Lovibond PF. Anxiety, depression, and tension/stress in children. J Psychopathol Behav Assess. 2006;28(3):192-202.)

The DASS-21 has been translated and validated in many languages and used with several ethnic groups.(²2. Sica C, Ghisi M, Altoè G, Chiri LR, Franceschini S, Coradeschi D, et al. The Italian version of the obsessive compulsive inventory: its psychometric properties on community and clinical samples. J Anxiety Disord. 2009;23(2):204-11.

3. Osman ZJ, Mukhtar F, Hashim HA, Abdul Latiff L, Mohd Sidik S, Awang H, et al. Testing comparison models of DASS-12 and its reliability among adolescents in Malaysia. Compr Psychiatry. 2014;55(7):1720-5.

4. Tonsing KN. Psychometric properties and validation of Nepali version of the Depression Anxiety Stress Scales (DASS-21). Asian J Psychiatr. 2014;8:63-6.

5. Oei TP, Sawang S, Goh YW, Mukhtar F. Using the Depression Anxiety Stress Scale 21 (DASS-21) across cultures. Int J Psychol. 2013;48(6):1018-29.

6. Sinclair SJ, Siefert CJ, Slavin-Mulford JM, Stein MB, Renna M, Blais MA. Psychometric evaluation and normative data for the depression, anxiety, and stress scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Eval Health Prof. 2012;35(3):259-79.

7. Apóstolo JL, Mendes AC, Azeredo ZA. Adaptation to Portuguese of the Depression, Anxiety and Stress Scales (DASS). Rev Lat Am Enfermagem. 2006;14(6):863-71.

8. Daza P, Novy DM, Stanley MA, Averill P. The depression anxiety stress scale-21: Spanish translation and validation with a Hispanic sample. J Psychopathol Behav Assess. 2002;24(3):195-205.-⁹9. Vignola RC, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. J Affect Disord. 2014;155:104-9.) It is widely used to assess symptoms of mental suffering in clinical and non-clinical adult samples.(¹⁰10. Aboalshamat K, Hou XY, Strodl E. The impact of a self-development coaching programme on medical and dental students’ psychological health and academic performance: a randomised controlled trial. BMC Med Educ. 2015;15:134.

11. Goldhagen BE, Kingsolver K, Stinnett SS, Rosdahl JA. Stress and burnout in residents: impact of mindfulness-based resilience training. Adv Med Educ Pract. 2015;6:525-32.

12. Raymond KL, Lovell GP. Food addiction symptomology, impulsivity, mood, and body mass index in people with type two diabetes. Appetite. 2015;95:383-9.

13. de Haan L, Egberts AC, Heerdink ER. The relation between risk-taking behavior and alcohol use in young adults is different for men and women. Drug Alcohol Depend. 2015;155:222-7.

14. Del Brutto OH, Mera RM, Del Brutto VJ, Maestre GE, Gardener H, Zambrano M, et al. Influence of depression, anxiety and stress on cognitive performance in community-dwelling older adults living in rural Ecuador: results of the Atahualpa Project. Geriatr Gerontol Int. 2015;15(4):508-14.-¹⁵15. Hmwe NT, Subramanian P, Tan LP, Chong WK. The effects of acupressure on depression, anxiety and stress in patients with hemodialysis: a randomized controlled trial. Int J Nurs Stud. 2015;52(2):509-18.) A growing number of studies of adolescents apply the DASS-21 to identify signs of anxiety, depression and stress; however, most studies investigating the validity of the DASS-21 constructs were conducted with adults, making it difficult to extrapolate the scale efficiency items to recognize the physiological symptoms in adolescents.

The psychometric properties of the DASS-21 in population under 18 years of age suggest that the emotional symptoms in this age group are similar to those in adults, but this tool requires further improvement.(¹⁶16. Szabó M. The short version of the Depression Anxiety Stress Scales (DASS-21): factor structure in a young adolescent sample. J Adolesc. 2010;33(1):1-8.) There have been no construct validation studies of the Portuguese language version of the DASS-21 for adolescents. Therefore, it is necessary an exploratory factor analysis that allows data to speak for themselves, that is, would let the structure that is designed for data suggest the most appropriate factor model independently.(¹⁷17. Laros JA. O uso da análise fatorial: algumas diretrizes para pesquisadores. In: Pasquali L. Análise fatorial para pesquisadores. Brasília: LabPAM Saber e Tecnologia; 2012. p. 141-60.) Moreover, the scarcity of studies confirming the test-retest reproducibility of DASS-21 decreases the accuracy of the measurement results obtained with the instrument.(¹⁸18. Aarås A, Veierød MB, Larsen S, Ortengren R, Ro O. Reproducibility and stability of normalized EMG measurements on musculus trapezius. Ergonomics. 1996; 39(2):171-85.)

OBJECTIVE

To evaluate the interday reproducibility, the agreement and validity of the construct of the short version of the Depression Anxiety Stress Scale in adolescents.

METHODS

Sample

The sample consisted of adolescents of both sexes, aged between 10 and 19 years, who were recruited from schools and sports centers in the city of Petrolina (PE), Brazil, in 2015. The minimum sample size required for factor analysis is usually ten subjects per item and, at least, a total of one hundred subjects.(¹⁷17. Laros JA. O uso da análise fatorial: algumas diretrizes para pesquisadores. In: Pasquali L. Análise fatorial para pesquisadores. Brasília: LabPAM Saber e Tecnologia; 2012. p. 141-60.) To calculate the sample size required for reproducibility analysis, we used the G*Power 3.1.7 software,(¹⁹19. Shoukri MM, Asyali MH, Donner A. Sample size requirements for the design of reliability study: review and new results. Stat Methods Med Res. 2004;13(4):251-71.) specifying α=0.05; β=0.10 (90% power); correlation ratio for the null hypothesis (ρH₀)=0.40, proportion of correlation for alternative hypothesis (ρH₁)=0.80, and an attrition rate of 20%. The required minimum sample size was estimated at 31 subjects.

Depression Anxiety Stress Scale-21 was administered by trained evaluators to clarify possible doubts of adolescents, without interfering in their answers. The scale was applied after the participants and their legal guardians signing the Informed Consent Form. The adolescents answered the DASS-21 individually in classrooms with the presence only of the evaluator. Thirty-one adolescents were recruited to reevaluation, with a 1-week interval between assessment. Participants who failed to complete the scale were not included in the analysis. The study was approved by the Research Ethics Committee of the Universidade de Pernambuco (UPE), under protocol number 944.548, CAAE: 38321114.0.0.0000.5207.

Instrument

The Brazilian version of the DASS-21 is a validated Portuguese translation(⁹9. Vignola RC, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. J Affect Disord. 2014;155:104-9.) of the original scale. It is a self-report instrument consisting of three seven-item subscales, to assess depression, anxiety and stress over the last week. The responses are given on a 4-point Likert scale, ranging from zero if “I strongly disagree” to 3 if “I totally agree”. Overall scores for the three constructs are calculated as the sum of scores for the relevant seven items multiplied by two. Ranges of scores correspond to levels of symptoms, ranging from “normal” to “extremely serious”.(⁹9. Vignola RC, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. J Affect Disord. 2014;155:104-9.) The DASS-21 items and corresponding constructs are detailed in table 1.

Statistical analysis

Test-retest reproducibility was assessed in terms of intraclass correlation coefficient (ICC) with 95% confidence interval (95% CI); ICC values above 0.75 were interpreted as an indication of excellent reliability, values between 0.40 and 0.74 were taken as an indication of good reliability, and >0.40 as an indication of poor reliability.(²⁰20. Fleiss JL. The design and analysis of clinical experiments. Canadá: John Wiley & Sons, 1986. p. 1-31.) The standard error of measurement (SEM) was calculated to estimate the change in each score and the minimal detectable change (MDC) was also calculated to determine the minimum clinically significant change. The single-sample t-test was used to assess systematic differences between test-retest scores, with the significance level set at p=0.05. Absolute agreement was analyzed by preparing the Bland-Altman plot for the first and second assessment results, based on a scatter plot of the difference of the two evaluations and the average of the two evaluations. This provided a visual representation of biases, errors, limits of agreement, outliers and trends.

Construct validity was analyzed using exploratory factor analysis of principal components with varimax rotation. Taking into account the sample size of 310 participants, the factor loading values >0.40 were considered strong.(²¹21. Osman A, Wong JL, Bagge CL, Freedenthal S, Gutierrez PM, Lozano G. The Depression Anxiety Stress Scales-21 (DASS-21): further examination of dimensions, scale reliability, and correlates. J Clin Psychol. 2012;68(12):1322-38.) The internal consistency of the subscale for the three constructs was assessed using Cronbach’s α; values between 0.70 and 0.80 indicated a reliable scale, although values below 0.70 were considered acceptable for physiological constructs.(²²22. Field A. Descobrindo a estatística utilizando o SPSS. 2ed. Porto Alegre: Penso; 2009. p. 688.) We also calculated individual item-total correlations for each construct. This is a structural measure of the validity of a scale, indicating that a given item measures the construct with which it is associated, rather than the others. An item with good validity is more closely correlated with the total score for the construct with which it is associated than the total scores for other constructs.(⁹9. Vignola RC, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. J Affect Disord. 2014;155:104-9.)

All statistical analyses were conducted using the software Statistical Package for Social Sciences (SPSS) version 20.0, and Graph PadPrism version 5.03.

RESULTS

Sample

The sample consisted of 310 adolescents, 179 (57.7%) males, overall mean age of the sample was 14.16 (±2.12) years. Thirty-one adolescents were recruited for reproducibility analysis, but six participants dropped out; hence the reassessment sample comprised 25 adolescents (14 females), giving a statistical power of 89.64%. The interval between assessments was 1 week, and re-evaluated participants had a median age of 18 years (interquartile range=4).

Internal consistency and construct validity

The DASS-21 was found to have strong internal consistency. Anxiety Cronbach’s α was 0.80; depression was 0.80; stress was 0.77; overall was 0.88. Cronbach’s α for the combination of the anxiety and stress constructs was 0.82, also indicating strong internal consistency. The item-total correlations for each construct were anxiety with 0.77, depression with 0.78 and stress with 0.80. The correlations between constructs were 0.57 for anxiety and depression, 0.70 for anxiety and stress and 0.60 for depression and stress. The scores for the Kaiser-Meyer-Olkin measure of sampling adequacy (KMO=0.885) and the Bartlett sphericity test (approximate χ² of 2106.950, comp=0.000) confirmed that the data were suitable for factor analysis, and that the adequacy of the model was excellent.(²²22. Field A. Descobrindo a estatística utilizando o SPSS. 2ed. Porto Alegre: Penso; 2009. p. 688.)

In the construct of analysis of DASS-21 for three factors, all items corresponding to the constructs anxiety and depression obtained their highest factor loadings in their respective constructs, except for stress, as shown in table 2. The items 8, 6 and 11 of the stress construct had higher factor loading values in its source construct, since the items 14 and 18, despite having greatest factor loadings for stress, 0.48 and 0.47, respectively, they also appear with high load factor to construct anxiety, with 0.43 and 0.45, respectively. The most problematic items were 1 and 12, which greater load on anxiety, despite being nominally associated with stress. The item-total correlations indicated that all 21 items correlated more strongly with the scores for the construct with which they were associated. The item-total correlations ranged between 0.54 and 0.69 for anxiety, between 0.56 and 0.75 for depression, and between 0.57 and 0.68 for stress.

Depression Anxiety and Stress Scale-21 had a three-dimension support proposed by the original author. Nonetheless, considering that the three-factor model resulted in some items with similar or stronger loads on nominally non-core constructs, we also conducted varimax orthogonal analysis for a two-factor model (Table 3), combining the items related to anxiety and stress into a single factor, with the items related to depression contributing to the second factor. A better match for all 21 items with higher factor loadings in their respective constructs was observed. Only item 13 (depression) loaded strongly on both factors and it still loaded most strongly on the depression factor. Once again we calculated item-total correlations; based this time on two factors (anxiety-stress and depression). The item-total correlations were between 0.46 and 0.62 for anxiety-stress items, and between 0.56 and 0.75 for depression items. The model with two factors also presented items with the difference between the correlations lower than 0.20; however, to a lesser extent, it was observed in items 20, 6 and 15 of depression.

Test-retest reproducibility

The intraclass correlation coefficient values for the DASS-21 indicated excellent reliability, with 95%CI, SEM and MDC for each DASS-21 construct, and are shown on table 4. The agreement analysis for both assessments of each construct is displayed on figure 1. The Student’s t test was significant for all constructs, indicating a systematic error type. From the graphs of constructs, one can infer that this error occurred to a lesser extent, considering that the bias of the mean difference was close to zero. It is possible to observe trends in scores in the constructs anxiety and stress, and consequently, the combination scores of both constructs, with a distribution of most individuals above zero, but still within the upper and lower limits of 95%CI. The depression, stress and anxiety-stress scores included some outliers, but, in all cases, they were close to the range limit.

DISCUSSION

It is important to confirm that a scale is reliable in all populations for which it is used. However, there was only one published evaluation of the test-retest reproducibility of DASS-21, and it did not use the most appropriate statistical tests for this purpose. Bottesi et al.,(²³23. Bottesi G, Ghisi M, Altoè G, Conforti E, Melli G, Sica C. The Italian version of the Depression Anxiety Stress Scales-21: factor structure and psychometric properties on community and clinical samples. Compr Psychiatry. 2015;60:170-81.) reported values of r=0.64 for anxiety, r=0.75 for depression and r=0.64 for stress, showing a moderate and positive correlation, but the Pearson’s correlation coefficient effectively measures the relation between the data test-retest, and not the agreement between them. The ICC is considered one of the best measures of validity and, in this study, the ICC values indicated excellent test-retest validity, providing evidence that the DASS-21 is reliable to be used with adolescents.

It is also important to calculate the SEM and the MDC, since they provide further information to assess reproducibility of a test, quantifying the absolute error for scores on the test, and the minimum difference in scores that could not be attributed to measurement error. In this study, the SEM values for DASS-21 scores were small, and the probability of random and systematic error in measurement was also small. Nevertheless, this error was visible when we performed the Student’s t test and made the Bland-Altman plots for each construct. The problem most often found in the construct stress was related to the analysis of the combined anxiety-stress construct. The fact that scores were lower on retest may indicate that respondents understood the questions and response scales better having completed the entire questionnaire before. The MDC values were calculated to offer a good reference point for intervention studies. But, likewise other reproducible values, the MDC has not been calculated in the studies that validated the DASS-21.

Considering that the three DASS-21 constructs are based on multiple items, it is appropriate to calculate Cronbach’s α coefficient separately for each construct. In this study, the scale was found to be reliable and the values for internal consistency were comparable with those reported in other recent studies.(²2. Sica C, Ghisi M, Altoè G, Chiri LR, Franceschini S, Coradeschi D, et al. The Italian version of the obsessive compulsive inventory: its psychometric properties on community and clinical samples. J Anxiety Disord. 2009;23(2):204-11.,⁴4. Tonsing KN. Psychometric properties and validation of Nepali version of the Depression Anxiety Stress Scales (DASS-21). Asian J Psychiatr. 2014;8:63-6.

5. Oei TP, Sawang S, Goh YW, Mukhtar F. Using the Depression Anxiety Stress Scale 21 (DASS-21) across cultures. Int J Psychol. 2013;48(6):1018-29.-⁶6. Sinclair SJ, Siefert CJ, Slavin-Mulford JM, Stein MB, Renna M, Blais MA. Psychometric evaluation and normative data for the depression, anxiety, and stress scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Eval Health Prof. 2012;35(3):259-79.,⁹9. Vignola RC, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. J Affect Disord. 2014;155:104-9.,¹⁶16. Szabó M. The short version of the Depression Anxiety Stress Scales (DASS-21): factor structure in a young adolescent sample. J Adolesc. 2010;33(1):1-8.,²¹21. Osman A, Wong JL, Bagge CL, Freedenthal S, Gutierrez PM, Lozano G. The Depression Anxiety Stress Scales-21 (DASS-21): further examination of dimensions, scale reliability, and correlates. J Clin Psychol. 2012;68(12):1322-38.,²⁴24. Vasconcelos-Raposo J, Fernandes HM, Teixeira CM. Factor structure and reliability of the depression, anxiety and stress scales in a large Portuguese community sample. Span J Psychol. 2013;16:E10.) Cronbach’s α coefficient ranged from 0.74 to 0.86 for anxiety, 0.77 to 0.92 for depression, and 0.70 to 0.90 for stress. Three studies reported Cronbach’s α coefficient for the scale as a whole.(²2. Sica C, Ghisi M, Altoè G, Chiri LR, Franceschini S, Coradeschi D, et al. The Italian version of the obsessive compulsive inventory: its psychometric properties on community and clinical samples. J Anxiety Disord. 2009;23(2):204-11.,⁵5. Oei TP, Sawang S, Goh YW, Mukhtar F. Using the Depression Anxiety Stress Scale 21 (DASS-21) across cultures. Int J Psychol. 2013;48(6):1018-29.,²⁴24. Vasconcelos-Raposo J, Fernandes HM, Teixeira CM. Factor structure and reliability of the depression, anxiety and stress scales in a large Portuguese community sample. Span J Psychol. 2013;16:E10.) These values and the value observed in this study indicated that the DASS-21 had acceptable internal consistency, no redundant questions and was made up of independent items.(²²22. Field A. Descobrindo a estatística utilizando o SPSS. 2ed. Porto Alegre: Penso; 2009. p. 688.)The Cronbach’s α coefficient showed an increase in the analysis of joint constructs of anxiety and stress, possibly enhancing the role of items that can have more interaction.

In this study, the DASS-21 constructs were strongly and positively correlated. The correlation between anxiety and stress was highest, corroborating other studies.(²2. Sica C, Ghisi M, Altoè G, Chiri LR, Franceschini S, Coradeschi D, et al. The Italian version of the obsessive compulsive inventory: its psychometric properties on community and clinical samples. J Anxiety Disord. 2009;23(2):204-11.,⁶6. Sinclair SJ, Siefert CJ, Slavin-Mulford JM, Stein MB, Renna M, Blais MA. Psychometric evaluation and normative data for the depression, anxiety, and stress scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Eval Health Prof. 2012;35(3):259-79.,¹⁶16. Szabó M. The short version of the Depression Anxiety Stress Scales (DASS-21): factor structure in a young adolescent sample. J Adolesc. 2010;33(1):1-8.) Other investigations have reported higher correlations between anxiety and depression(²⁴24. Vasconcelos-Raposo J, Fernandes HM, Teixeira CM. Factor structure and reliability of the depression, anxiety and stress scales in a large Portuguese community sample. Span J Psychol. 2013;16:E10.) and between depression and stress(⁴4. Tonsing KN. Psychometric properties and validation of Nepali version of the Depression Anxiety Stress Scales (DASS-21). Asian J Psychiatr. 2014;8:63-6.,²¹21. Osman A, Wong JL, Bagge CL, Freedenthal S, Gutierrez PM, Lozano G. The Depression Anxiety Stress Scales-21 (DASS-21): further examination of dimensions, scale reliability, and correlates. J Clin Psychol. 2012;68(12):1322-38.,²⁵25. Patrick J, Dyck M, Bramston P. Depression Anxiety Stress Scale: is it valid for children and adolescents? J Clin Psychol. 2010;66(9):996-1007.) than we found in this study. The strong correlations between constructs is a relevant indication that there are similarities between items associated with the different constructs, which may present issues that may show signs of concomitant anxiety, depression and stress. This cannot fully explain the results for construct analysis based on three factors, as proposed in the original scale.(²⁶26. Lovibond PF, Lovibond SH. The structure of negative emotional states: comparison of the depression anxiety stress scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther. 1995;33(3):335-43.)

Three-factor model structural analysis reveals problems, with some items loading strongly on more than one construct or loading strongly on nominally unrelated constructs. The items 14 and 18 (stress) also loaded strongly on anxiety. The problem was most serious for item 12 (stress), which loaded most strongly on anxiety. Opposing the three-factor model with its floating items, the two-factor model achieved a better adaptation of the items, but there was a decline in the explained variance, which is a clear indication that the flaws in both models stem from the interpretation of the items they comprise. Our findings suggest that the representativeness of the DASS-21 dimensions remains questionable even in the two-factor model.

The results of this study do not differ from those of earlier studies(⁴4. Tonsing KN. Psychometric properties and validation of Nepali version of the Depression Anxiety Stress Scales (DASS-21). Asian J Psychiatr. 2014;8:63-6.

5. Oei TP, Sawang S, Goh YW, Mukhtar F. Using the Depression Anxiety Stress Scale 21 (DASS-21) across cultures. Int J Psychol. 2013;48(6):1018-29.-⁶6. Sinclair SJ, Siefert CJ, Slavin-Mulford JM, Stein MB, Renna M, Blais MA. Psychometric evaluation and normative data for the depression, anxiety, and stress scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Eval Health Prof. 2012;35(3):259-79.,⁹9. Vignola RC, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. J Affect Disord. 2014;155:104-9.) with DASS-21, but they still support a structure of three factors, until for presenting a greater explained variance. The study which validated a version of the DASS-21 for use in Brazil(⁹9. Vignola RC, Tucci AM. Adaptation and validation of the depression, anxiety and stress scale (DASS) to Brazilian Portuguese. J Affect Disord. 2014;155:104-9.) corroborated the original three-factor structure. Although several items loaded strongly on more than one factor, the authors only highlighted item 18, which loaded most strongly on the depression factor despite belonging to the stress construct. They offered an explanation based on cultural factors regarding the concept of sensitivity. Oei et al.,(⁵5. Oei TP, Sawang S, Goh YW, Mukhtar F. Using the Depression Anxiety Stress Scale 21 (DASS-21) across cultures. Int J Psychol. 2013;48(6):1018-29.) endorsed the three-factor model with the caveat that to achieve acceptable model fit, it was necessary to delete three items from the stress construct which loaded more strongly on the depression factor. They nevertheless found that several items loaded strongly on more than one factor. Other studies(⁴4. Tonsing KN. Psychometric properties and validation of Nepali version of the Depression Anxiety Stress Scales (DASS-21). Asian J Psychiatr. 2014;8:63-6.,⁶6. Sinclair SJ, Siefert CJ, Slavin-Mulford JM, Stein MB, Renna M, Blais MA. Psychometric evaluation and normative data for the depression, anxiety, and stress scales-21 (DASS-21) in a nonclinical sample of U.S. adults. Eval Health Prof. 2012;35(3):259-79.,²⁴24. Vasconcelos-Raposo J, Fernandes HM, Teixeira CM. Factor structure and reliability of the depression, anxiety and stress scales in a large Portuguese community sample. Span J Psychol. 2013;16:E10.) supported the three-factor model, but did not have enough data to reinforce this model or to discuss the problem encountered.

The structural problems we found in the three-factor model were also observed in other exploratory factor analyses of the DASS-21 in the adult population,(²2. Sica C, Ghisi M, Altoè G, Chiri LR, Franceschini S, Coradeschi D, et al. The Italian version of the obsessive compulsive inventory: its psychometric properties on community and clinical samples. J Anxiety Disord. 2009;23(2):204-11.,⁷7. Apóstolo JL, Mendes AC, Azeredo ZA. Adaptation to Portuguese of the Depression, Anxiety and Stress Scales (DASS). Rev Lat Am Enfermagem. 2006;14(6):863-71.,²¹21. Osman A, Wong JL, Bagge CL, Freedenthal S, Gutierrez PM, Lozano G. The Depression Anxiety Stress Scales-21 (DASS-21): further examination of dimensions, scale reliability, and correlates. J Clin Psychol. 2012;68(12):1322-38.) and in adolescents.(²⁷27. Willemsen J, Markey S, Declercq F, Vanheule S. Negative emotionality in a large community sample of adolescents: the factor structure and measurement invariance of the short version of the depression anxiety stress scales (DASS-21). Stress & Health. 2011;27(3):e120-8.) These problems are the main reason for advocating further analysis, including the testing of other models which might fit better. The exploratory factor analysis by Apóstolo et al.,(⁷7. Apóstolo JL, Mendes AC, Azeredo ZA. Adaptation to Portuguese of the Depression, Anxiety and Stress Scales (DASS). Rev Lat Am Enfermagem. 2006;14(6):863-71.) revealed flaws in the original model, and the authors proposed an alternative two-factor model, combining anxiety and stress into a single factor; when tested this model proved a better fit to the data.

Most studies that tried to validate the DASS-21 in specific languages, or for specific ethnic groups, used adult samples. There were few attempts to validate the scale for adolescent populations, since this age group is thought to have problems comprehending the scale and differentiating between some of the symptoms it assesses. The five studies(¹⁶16. Szabó M. The short version of the Depression Anxiety Stress Scales (DASS-21): factor structure in a young adolescent sample. J Adolesc. 2010;33(1):1-8.,²⁵25. Patrick J, Dyck M, Bramston P. Depression Anxiety Stress Scale: is it valid for children and adolescents? J Clin Psychol. 2010;66(9):996-1007.,²⁷27. Willemsen J, Markey S, Declercq F, Vanheule S. Negative emotionality in a large community sample of adolescents: the factor structure and measurement invariance of the short version of the depression anxiety stress scales (DASS-21). Stress & Health. 2011;27(3):e120-8.

28. Duffy CJ, Cunningham EG, Moore SM. Brief report: the factor structure of mood states in an early adolescent sample. J Adolesc. 2005;28(5):677-80.-²⁹29. Tully PJ, Zajac IT, Venning AJ. The structure of anxiety and depression in a normative sample of younger and older Australian adolescents. J Abnorm Child Psychol. 2009;37(5):717-26.) that used adolescent samples reached different conclusions about the best model for this population. Some studies(¹⁶16. Szabó M. The short version of the Depression Anxiety Stress Scales (DASS-21): factor structure in a young adolescent sample. J Adolesc. 2010;33(1):1-8.,²⁵25. Patrick J, Dyck M, Bramston P. Depression Anxiety Stress Scale: is it valid for children and adolescents? J Clin Psychol. 2010;66(9):996-1007.,²⁸28. Duffy CJ, Cunningham EG, Moore SM. Brief report: the factor structure of mood states in an early adolescent sample. J Adolesc. 2005;28(5):677-80.) emphasized the high correlations between the constructs, suggesting that they were not empirically distinguishable, especially among adolescents.

Most studies with adolescent samples concluded that a two-factor model is most appropriate in this population, but with reservations. Tully et al.,(²⁹29. Tully PJ, Zajac IT, Venning AJ. The structure of anxiety and depression in a normative sample of younger and older Australian adolescents. J Abnorm Child Psychol. 2009;37(5):717-26.) and Willemsen et al.,(²⁷27. Willemsen J, Markey S, Declercq F, Vanheule S. Negative emotionality in a large community sample of adolescents: the factor structure and measurement invariance of the short version of the depression anxiety stress scales (DASS-21). Stress & Health. 2011;27(3):e120-8.) defended a two-factor model alternative, keeping the constructs anxiety and depression, and a new construct is formed by the three constructs condensed into a single factor, called “Negative Affect”. This model, before those tested, had the best adjustment. Duffy et al.,(²⁸28. Duffy CJ, Cunningham EG, Moore SM. Brief report: the factor structure of mood states in an early adolescent sample. J Adolesc. 2005;28(5):677-80.) also argued that a two-factor model would be more appropriate, but they proposed new factors: a physiological arousal factor comprising four items from the original anxiety construct (2, 4, 7, 19), and a general negativity factor comprising all other items.

Patrick et al.,(²⁵25. Patrick J, Dyck M, Bramston P. Depression Anxiety Stress Scale: is it valid for children and adolescents? J Clin Psychol. 2010;66(9):996-1007.) argued for a one-factor model on the basis of the high inter-construct correlations, suggesting that adolescents cannot differentiate the three dimensions of the original model, and that, in adolescents, the scale only assesses a single dimension. They note that it is not possible to infer, from the available data, whether this single factor is more or less strongly associated with any of the constructs of the adult model. Szabó(¹⁶16. Szabó M. The short version of the Depression Anxiety Stress Scales (DASS-21): factor structure in a young adolescent sample. J Adolesc. 2010;33(1):1-8.) also reported strong correlations between the constructs, defended the three-factor model, but concluded that although anxiety and depression are similar in both adults and adolescents, stress is questionable in this age group.

It is important to recognize the limitations of this study. The failure to consider the educational background of the adolescents was an important limitation, since relevant data were not gathered. The wide variation in age (10 to 19 years) in our sample means that there was also wide variation in school grade and hence years of education, which may imply differences in comprehension of the questions that make up the scale. Another limitation was that we did not carry out confirmatory factor analysis; however, given the lack of consensus on the factorial structure of the scale, an exploratory factor analysis seemed more appropriate.

Considering the increasing use of the DASS-21 in studies with adolescents, it is necessary to revise some items, especially those belonging to the stress construct, to minimize problems caused by adolescents’ misinterpretation or lack of comprehension of some items. Thus, the DASS-21 can be a tool that clearly and independently identifies the signs of the three psychological states that compose it, in the adolescent population. It is also necessary to conduct further analyses to determine the reliability of the scale in other populations, so as to have an even more consistent discussion of the values found.

CONCLUSION

The short version of the Depression, Anxiety Stress Scale-21 has excellent reliability and good internal consistency. In the analysis of agreement, it was possible to infer, though small, the presence of systematic error type in all constructs. This emphasizes the importance of knowing the standard error of measurement and the minimal detectable change values if the scale is to be clinically used. Finally, the two-factor model in which the original anxiety and stress constructs are combined into a single factor appears to be more appropriate for the adolescent population. However, considering the psychometric limitations of the two models, both could be used to assess adolescents.

REFERENCES

Publication Dates

History