Introduction
Many children suffer from emotional disorders. Indeed, lifetime prevalence rates range from 7 to 15%, depending on the type of emotional disorder studied (Costello et al.
2003; Verhulst et al.
1997). Although evidence-based therapies are available to treat children with emotional disorders, around 45% of children do not respond sufficiently (Bodden et al.
2008), so there is room for improvement. To improve treatment results we need to know more about developmental and maintaining factors of disorders, but also about effective components of treatment. One factor which is presumed to play a critical role in the onset and maintenance of anxiety disorders is cognition (Beck
2005). Commonly, anxious cognition is examined by measuring negative thoughts (e.g., “I am worthless”). However, in order to investigate cognitive models of anxiety disorders it is also necessary to measure positive thoughts (e.g., “I feel good about myself”). In the current study, we describe the development of a questionnaire which incorporates negative and positive thoughts: the Children’s Automatic Thoughts Scale-Negative/Positive (CATS-N/P).
Cognitions play an important role in disorders and their treatment. For example, children with anxiety and mood disorders report more dysfunctional and negative beliefs than healthy children (Beck
2005). There are three important models which describe the contribution of thoughts to emotional disorders. According to the States-of-Mind (SOM) model (Schwartz and Garamoni
1989), the balance of positive and negative thoughts is essential for psychological well-being. A ratio of .62 between positive and negative thoughts is considered optimal or healthy. A ratio less than .31 is related to depression or anxiety (Schwartz and Garamoni
1989). A second model is Kendall’s “power of nonnegative thinking”. This model states that anxious children may benefit more from a reduction in the amount of their negative thoughts than from an increase in the amount of their positive thoughts (Kendall and Chansky
1991; Kendall and Korgeski
1979). Third, Beck’s content-specificity hypothesis focuses on dysfunctional cognitive schemata and specific cognitive content. Anxious self-talk is future-oriented, unstable, and focused on threat. Depressive self-talk is past-oriented, stable, and focused on loss and failure (Beck and Clark
1997; Ronan and Kendall
1997).
Several questionnaires have been developed to measure cognitions in children. However, several problems have been associated with the application of these questionnaires. First, the majority of cognition questionnaires for children [e.g., the Children’s Anxious Self-Statements Questionnaire (CASSQ; Ronan et al.
1988), or the Cognition Checklist for Children (CCL-C; Jolly and Dykman
1994)], are downward extensions of measures developed for adults. Children might have trouble understanding items of these questionnaires or make different self-statements than adults. Therefore, the original factor structure for adults may not hold in a younger population. A second problem is that most cognition questionnaires fail to distinguish between thoughts and symptoms. For example, the Negative Affectivity Self-Statement Questionnaire (NASSQ; Ronan et al.
1994; NASSQ-Anxiety scale; Sood and Kendall
2007), which was developed using self-statements generated by children, measures both symptoms (e.g. “I was shaking”) and thoughts (e.g. “I usually do something stupid”). This overlap in item content might artificially inflate correlations between symptom measures and cognition measures. The overlap in item content also makes it difficult to disentangle the specific contributions of symptoms and cognitions to the disorder. Third, most cognition questionnaires measure general (negative) affect and not anxiety and/or depression separately, so it is difficult to examine content specificity. Finally, there are no psychometrically sound questionnaires which incorporate negative as well as positive thoughts (and not positive affect).
A cognitions questionnaire which circumvents most of these problems is the Children’s Automatic Thoughts Scale (CATS; Schniering and Rapee
2002). This questionnaire was specifically designed for children, has been used in different international studies, and measures thoughts but not symptoms. The items of the CATS are based on self-statements made by clinically anxious, depressed, or behaviorally disturbed children (Schniering and Rapee
2002). The CATS assesses negative beliefs common to both internalizing and externalizing problems. In addition, it also contains specific items related to different disorders, which can facilitate the investigation of content-specificity (i.e. thoughts that are specific or common to these disorders). Confirmatory factor analysis of the CATS in a community sample revealed four distinct first-order factors (Physical threat, Social threat, Personal failure and Hostility) and one higher-order factor reflecting negative beliefs (Schniering and Rapee
2002). This factor structure was replicated in two other studies (Schniering and Lyneham
2007; Schniering and Rapee
2004). The CATS has consistently shown good internal reliability, with Cronbach’s alphas ranging from .82 to .96 (Bodden and Bögels
2006; Schniering and Lyneham
2007; Schniering and Rapee
2002,
2004). Test–retest reliability was good at 1 month (.66–.80) and 3 months (.68–.77; Schniering and Rapee
2002). The CATS has good discriminant validity. In fact, the CATS has been demonstrated to discriminate between children with anxiety disorders and healthy controls (Bodden and Bögels
2006; Schniering and Rapee
2002); to discriminate between anxiety, depression, and behavioral disorders (Bodden and Bögels
2006; Schniering and Rapee
2002); and to discriminate between different anxiety disorders (Bodden and Bögels
2006; Schniering and Lyneham
2007). Finally, the CATS has also been shown to be sensitive to treatment change (Mifsud and Rapee
2005; Schniering and Lyneham
2007).
While the CATS has a number of advantages over other measures of cognition, it does not assess positive thoughts. The inclusion of positive thoughts in a cognition questionnaire makes it possible to examine theoretical cognitive models like the SOM model, power of nonnegative thinking, and the content-specificity hypothesis. Therefore, to increase the applicability of this questionnaire in research on cognition in children, we decided to extend the CATS with positive self-statements. The resulting measure was named the CATS-Negative/Positive (CATS-N/P). The objective of the present study was to describe the development and psychometric properties of the CATS-N/P in a community sample of children and adolescents.
Our first research question concerned the factor structure of the CATS-N/P and consisted of two parts: (a) whether we could derive the original four-factor structure of the CATS in a Dutch population; and (b) if the factor structure for the CATS-N/P would include an extra factor for positive thoughts. Performing a factor analysis of the CATS-N/P is important for several reasons. First, adding extra items to a questionnaire might change the overall factor structure. We wanted to be confident that the original subscales were still relevant to the new questionnaire. This is important because earlier studies showed that subscales of the CATS discriminated between different disorders (Bodden and Bögels
2006; Schniering and Rapee
2002). Secondly, translating a questionnaire or using it in a different population can change the factor structure. However, the factor structure of the Dutch translation of the CATS had not been examined in previous studies. Third, a factor analysis can reveal whether the positive items will form a coherent factor. This should be determined before the balance between negative and positive thoughts can be examined in future studies using the CATS-N/P.
Our second research question concerned the internal reliability of the scale and the 8-week test–retest reliability. Our third research question focused on the convergent and discriminant validity of the CATS-N/P. We hypothesized that there would be a positive correlation between the negative beliefs factor and the measures for emotional problems and anxiety; and a negative correlation between the measures for emotional problems and anxiety and the positive thoughts scale of the CATS-N/P. Finally, age and sex differences on the CATS-N/P were examined exploratively.
Discussion
In this study we investigated an adapted version of the Children’s Automatic Thoughts Scale, the CATS-Negative/Positive (CATS-N/P), in a non-referred community sample. In order to enhance the applicability of the CATS in studying the cognitions of children with different disorders, we added an extra subscale containing positive thoughts. The main research question of the current study was whether the factor structure and the psychometric properties of the new CATS-N/P were satisfactory.
Although some modifications in the factor structure were made, the factor structure and internal reliability of the CATS-N/P in the current sample was almost equal to the results of earlier studies using the CATS (Schniering and Rapee
2002,
2004). Although the fit indices found in the current study were a little smaller than in earlier studies (Schniering and Rapee
2002,
2004), the original four-factor structure of the CATS was supported in a Dutch sample. The major difference between the findings of the current study and earlier results was the addition of an extra ‘positive thoughts’ factor. In addition, three Hostility items were found to cross-load on other factors. Closer examination of the three items revealed that these cross-loadings seemed to have face validity. The three items were all negative thoughts about hostility directed at the child itself (e.g. “Most people are against me”). In contrast, the remaining seven Hostility items described thoughts about hostility directed at other persons and focused on revenge or other people being bad or stupid (“Bad people deserve to be punished”). Another difference between the current study and earlier results is that in the current study the Hostility factor only correlated modestly with the other factors. Therefore, it was not appropriate to use a higher-order factor including items from the Hostility factor or to calculate a Total score including Hostility items. Indeed, in earlier studies the Hostility factor was also found to display the lowest intercorrelations and factor loadings relative to other factors (Schniering and Rapee
2002,
2004). However, the three aforementioned items were not seen to be problematic in earlier studies (Schniering and Rapee
2002,
2004). The results found in this study regarding the Hostility items may reflect a shift in the underlying structure caused by the addition of the positive items. Another possibility is that due to the translation, children interpreted the items slightly different than in the English version. Based on the current findings, and to facilitate comparisons between different studies using the CATS, we recommend using and reporting two different negative thoughts Total scores: one with and one without Hostility items. Indeed, this is in line with the findings from other studies using the CATS, which have found that children with internalizing disorders (anxiety and depression) report more negative thoughts concerning Physical threat, Social threat, and Failure, while children with behavior disorders score higher on Hostile intent (Schniering and Rapee
2002).
The new positive items showed good internal reliability and high factor loadings. As expected, the Positive thoughts scale correlated negatively with almost all other subscales. However, the Positive thoughts scale was positively correlated with the Hostile thoughts scale. A possible explanation for this finding may be that both types of items share a common feature: assertive/extrovert or externalizing thoughts. Indeed, the higher-order model found in the current study seems to suggest that the CATS-N/P has three different types of items: items reflecting negative, internalizing self-statements (including separate social threat, physical threat, and failure items); items reflecting negative externalizing self-statements (hostility) and items reflecting positive self-statements.
The second research question in this study was whether the test–retest reliability of the CATS-N/P was satisfactory. Although the short-term reliability of the CATS-N/P was good and comparable to earlier results with the CATS (Schniering and Rapee
2002), the reliability at a longer interval was only moderate to good. However, there were some practical constraints which may have influenced the results of the test–retest analysis. First, the retest occurred across a broad time frame, which makes it difficult to interpret the results. Second, because of the school holidays, some children did the retest at home without supervision. The administration of the measure in different settings might have influenced the answers (e.g., at home there was less group pressure). Moreover, during the school holidays, children were probably less exposed to events involving threat, potential failure, and hostility from peers than during normal school days. This may have resulted in a temporary decrease in the occurrence of their negative thoughts. Third, there was a selection bias, in that children did not automatically participate in the second part of the study and had to give separate consent for the retest. Due to the aforementioned constraints, the stability of the CATS-N/P over longer periods of time should be further investigated.
The third research question explored in this study was whether the convergent and discriminant validity of the CATS-N/P was satisfactory. As expected, the Positive thoughts subscale was negatively associated with measures assessing anxious and depressive symptoms. Furthermore, the correlations between Physical Threat, Social Threat, Failure, and Total scores and the anxiety and depression measures were all high. Hostility correlated only moderately with these measures. This result was in line with previous studies which demonstrated that Hostility distinguished between children with internalizing problems and children with behavior problems (Schniering and Rapee
2002). The low correlations between the CATS-N/P subscales and the emotional subscale of the SDQ may be explained by the fact that the SDQ measures global emotional problems. Moreover, the SDQ was filled out by the parents, while the CATS-N/P was filled out by the children.
Unexpectedly, the correlations found between the CATS-N/P and both anxiety and depression measures were equally strong. This result was in contrast to earlier research, which found that the CATS subscale Failure discriminated between depressed and anxious children (Schniering and Rapee
2002). In the current study, however, the assessment of the validity of the CATS-N/P was based on correlations between different subscales of the CATS-N/P and different symptom measures in a non-referred sample, rather than differences between clinically anxious, depressed, or behaviorally disturbed children. Therefore, the current sample may have been too homogeneous to find differences between anxiety and depression. This may especially have been the case, given that the correlation between anxiety and depression symptoms is known to be high (Costello et al.
2003). Of course, another possibility is that the CATS-N/P is not able to discriminate between anxiety and depression.
Finally, we examined age and gender differences in mean scores on the different CATS-N/P subscales. First of all, the mean scores are substantially lower overall (about 8 points for Total score and 2 points for all subscales) than in the community sample as described by Schniering and Rapee (
2002). Although the groups used in the current study and the study by Schniering and Rapee (
2002) seem comparable in terms of age, gender, and socioeconomic status, the difference in mean scores between the two studies might be explained by cultural differences. Moreover, the translation and extra positive items might have influenced responding style. However, the means found in the current study are higher (about 11 points for the Total score) than the means found in a Dutch control sample described by Bodden and Bögels (
2006). This difference in mean scores cannot be due to the translation, but interregional variations and the smaller sample size used by Bodden and Bögels may account for the difference. In clinically anxious groups, large differences in mean scores between separate studies have also been found (e.g. Bodden and Bögels
2006; Schniering and Lyneham
2007; Schniering and Rapee
2002). Further research should aim to investigate the possibility that the CATS or CATS-N/P may be very sensitive to sample characteristics.
The age differences in this study were rather unexpected. In contrast with earlier results (Bodden and Bögels
2006; Schniering and Lyneham
2007), younger children reported more negative thoughts on some scales than older children. Although one would expect older children to worry more about social threat and failure, the group setting in which these data were collected may have lead to underreporting thoughts because of social concerns. As for sex differences, the results of the current study are similar to those found in earlier studies, in that boys reported more hostile thoughts and positive thoughts than girls. These findings may reflect that boys in general display more externalizing behavior (Costello et al.
2003) and are more self-confident than girls (Birndorf et al.
2005).
Because some cross-cultural differences were found regarding the overall mean scores and differences between age groups, the generalizability of the CATS and CATS-N/P to other countries and/or cultures is uncertain. Therefore, norm tables from different samples should be interpreted cautiously. Another limitation of this study is that the factor structure and psychometric properties of the measure were not evaluated in a clinical group. Future studies should therefore focus on further establishing the discriminant validity and psychometric properties of the CATS-N/P, for example by comparing non-clinical and clinical groups with different disorders (e.g., anxiety disorders, depression, behavior disorders). Moreover, it would be interesting to examine whether the CATS-N/P can predict treatment change and whether the Positive thoughts subscale can discriminate between clinically anxious and depressed children.
The CATS-N/P is an adapted and innovative version of the CATS designed specifically to measure positive and negative thoughts in children. This was the first study to apply the CATS-N/P in a large community sample in the Netherlands. The psychometric properties of the new measure were found to be good and the added positive items formed a psychometrically sound factor. Therefore, the CATS-N/P can be a valuable tool for the facilitation of research into the role of cognitive factors in the development and maintenance of different childhood disorders. Moreover, the use of the CATS-N/P in a clinical setting might improve the insight of the clinician in the amount of dysfunctional thoughts of a child pre-treatment and whether cognitions change over the course of treatment (especially after cognitive restructuring).