Theory and treatment for childhood anxiety disorders typically implicates children’s negative cognitions, yet little is known about the characteristics of thinking styles of clinically anxious children. In particular, it is unclear whether differences in thinking styles between children with anxiety disorders and non-anxious children vary as a function of child age, whether particular cognitive distortions are associated with childhood anxiety disorders at different child ages, and whether cognitive content is disorder-specific. The current study addressed these questions among 120 7–12 year old children (53% female) who met diagnostic criteria for social anxiety disorder, other anxiety disorder, or who were not currently anxious. Contrary to expectations, threat interpretation was not inflated amongst anxious compared to non-anxious children at any age, although older (10–12 year old) anxious children did differ from non-anxious children on measures of perceived coping. The notion of cognitive-content specificity was not supported across the age-range. The findings challenge current treatment models of childhood anxiety, and suggest that a focus on changing anxious children’s cognitions is not warranted in mid-childhood, and in late childhood cognitive approaches may be better focussed on promoting children’s perceptions of control rather than challenging threat interpretations.
Opmerkingen
Author Note
Cathy Creswell, School of Psychology and Clinical Language Sciences, University of Reading; Lynne Murray, School of Psychology and Clinical Language Sciences, University of Reading, Stellenbosch University; Peter Cooper, School of Psychology and Clinical Language Sciences, University of Reading, Stellenbosch University.
Anxiety disorders in childhood are common, frequently chronic, associated with impaired psychosocial functioning and are known to present a risk for the development of other serious psychological disturbance (e.g., major depressive disorder and substance abuse; Kim-Cohen et al. 2003). Cognitive behaviour therapy has been established as an effective treatment for childhood anxiety disorders; however, typically, 40–50% of children retain their primary anxiety disorder diagnosis following treatment (e.g., James et al. 2005). A more developed understanding of the factors maintaining childhood anxiety disorders is crucial for improved treatment outcomes.
A core component of cognitive-behavioural treatments for childhood anxiety disorders is a focus on modifying maladaptive thinking styles (e.g., Kendall 2011), based on the premise that dysfunctional cognitions play a central role in the maintenance of anxiety (e.g., Beck et al. 1985). Consistent with this premise, a number of studies have demonstrated that children with anxiety disorders interpret ambiguous situations in a more negative manner than non-anxious children (e.g., Barrett et al. 1996; Creswell et al. 2005). Furthermore, changes in anxious self-statements have been found to mediate treatment gains amongst children with anxiety disorders (Kendall and Treadwell 2007). Studies to date have, however, typically grouped together children from broad age ranges, and, as such, have failed to take into account potentially important age effects when considering the association between cognition and anxiety. Furthermore, there has been a lack of clarity regarding the specific cognitive distortions that are associated with anxiety disorders among children of different ages (e.g., attribution of threat, anticipation of low coping). Finally, the issue of whether cognitive distortions apply to all kinds of content, or are disorder-specific (e.g., social content for socially but not non-socially anxious children) needs to be addressed (the ‘cognitive content hypothesis’).
A tendency to make negative interpretations in response to ambiguous stimuli has been commonly reported among anxious adults (e.g., MacLeod et al. 1986). More recently, the same tendency has been demonstrated among anxious children (e.g., Barrett et al. 1996; Creswell et al. 2005). The most commonly used method for assessing such responses in children involves interviewing children about hypothetical ambiguous stories. Using this paradigm, clinically anxious youth have been found to give more negative responses than non-anxious youth in three cognitive domains: expectations relating to threat, negative emotion, and perceived control (e.g., Bogels and Zigterman, 2000; Waters et al. 2008b). Studies showing these effects have typically involved children from a broad age range (e.g., 7–14 years; Barrett et al. 1996), but they have not addressed the questions of whether the identified associations are consistent across the age-range, and whether specific cognitive themes may be particularly closely linked to anxiety at different ages. It is likely that the nature of the association between such negative responses and anxiety varies at different stages in development, because, with increasing age, children have a greater body of experiences to inform their thinking styles, and children’s thinking styles become more stable and global (e.g., Nolen-Hoeksema et al. 1992). Furthermore, the cognitive dimensions that are associated with anxiety may differ at different ages. It has been suggested, for example, that, in childhood, an underestimation of one’s ability to cope with potentially difficult situations may play a more central role in the maintenance of anxiety than thoughts focussed on threat or danger (e.g., Alfano et al. 2002; Creswell and O’Connor 2011; Waters et al. 2008a). Clarity on these issues will have important implications for therapy because the extent and focus of therapeutic endeavours which aim to alter cognitions, in the context of anxiety, may need to vary through childhood.
A clearer understanding of the association between thinking styles and childhood anxiety may be of particular clinical value in the case of social anxiety disorder. Within anxiety clinic populations, about 60% of children (7–12 years) meet criteria for social anxiety disorder, and this is the principal complaint in about 20% of cases (Hudson et al. 2010). As in adults, social anxiety disorder in childhood is characterised by an intense fear of negative evaluation by others, leading to avoidance of social encounters and restricted social functioning. For example, children with social anxiety disorder report loneliness, few friends, and restricted social relationships, and may display heightened oppositional and school refusal behaviours (e.g., Beidel et al. 1999).
The most common approach to the treatment of childhood social anxiety disorder has been to use trans-diagnostic CBT protocols, such as ‘Coping Cat’ (Kendall and Hedtke 2006), which target processes considered to maintain anxiety across disorders. These typically include a core combination of challenging negative thoughts and behavioural exposure (e.g., Kendall 2011). Typically, these studies have failed to find significant disorder-specific variation in outcomes (e.g., Barrett et al. 1996; Kendall et al. 1997), but have lacked the power to address this issue properly. More recently, Hudson et al. (2010) reported, from a large archive sample of clinically anxious children (7–12 years), that those children who had a primary diagnosis of social anxiety disorder had markedly poorer outcomes following a trans-diagnostic treatment than those children where social anxiety disorder was not primary. Similarly in a large multisite study of treatment for anxiety disorders in young people (7–17 years) the presence of a diagnosis of social phobia at baseline was associated with reduced remission rates compared to those for other disorders (Ginsburg et al. 2011). In contrast to these generic approaches, a small number of trials have evaluated specific treatments for social anxiety disorder in pre-adolescents, and the results of these appear to be favourable (e.g. Beidel et al. 2000; Spence et al. 2000). The extent to which cognitive aspects of such treatment contribute to success remains unclear. This is highlighted by the fact that treatment outcomes following cognitive-behavioural treatment (58% free of social anxiety disorder; Spence et al. 2000) do not appear to be superior to behavioural treatments which have no cognitive component at all (67% free of social anxiety disorder; Beidel et al. 2000).
As is the case for anxiety disorders in general, age is likely to be an important factor to consider in relation to the association between cognitions and social anxiety. Concerns about social evaluation typically become more common from mid to late childhood (e.g. Vasey et al. 1994). This escalation in social concerns has often been assumed to be a key factor in the increase in prevalence of social anxiety disorder with age between childhood and adolescence (with the mean age of onset being 13 years; Kessler et al. 2005). However, Spence et al. (1999) proposed that negative expectancies about social relationships are not a key factor in the development of social anxiety disorder, but may initially be a response to a lack of social success which later maintains anxiety by leading to avoidance of social situations. Further clarification of the association between cognition and social anxiety at different ages in childhood is clearly required.
Both models of the maintenance of childhood anxiety generally (e.g., Daleiden and Vasey 1997; Kendall and Chansky 1991) and specific models of adult social anxiety disorder (e.g., Clark and Wells 1995; Rapee and Heimberg 1997) incorporate the ‘cognitive content specificity’ hypothesis that individuals with different anxiety disorders experience disorder-specific thinking (e.g., Beck et al. 1985). However, only a few studies have examined this hypothesis among children, and these have produced mixed findings. Among community populations, some evidence has emerged for differences between anxiety subtypes in interpretation of subtype-specific ambiguous scenarios (e.g., Bögels et al. 2003); but other studies have not supported this conclusion (Muris et al. 2000a). Community based studies, however, have been limited by the high degree of association between scores on the various anxiety subscales (e.g., Muris et al. 2000a), and as such, a failure to find clear evidence for specificity could reflect a problem of measurement. Notably, however, findings from studies of anxious clinic populations have failed to find differences in the number of social situations interpreted as threatening by children with a principal diagnosis of social anxiety disorder compared to other anxiety groups (Barrett et al. 1996). Thus, on the basis of children’s responses to ambiguous scenarios, there is no clear evidence of social anxiety disorder-specific interpretation.
While examining responses to ambiguous scenarios is a widely used approach to assess anxiety-related thinking styles, this approach relies on participants imagining how they would respond to a hypothetical situation, and little is known about the ecological validity of this method of assessment. Notably, research on child cognitive vulnerability to depression shows that, at an age when children do not typically manifest vulnerability using hypothetical scenario or questionnaire methods (5 years), vulnerability does manifest itself in the context of real life, ecologically valid assessments (Murray et al. 2001), and, moreover, is predictive of depression in adolescence (Murray et al. 2011). Thus, an alternative approach is to elicit children’s thoughts ‘in vivo’. Using such an approach, studies to date have suggested that children with social anxiety disorder do differ from non-anxious children in their pre-task performance predictions (Alfano et al. 2006; Beidel et al. 1999; Spence et al. 1999). Crucially, it has not been addressed whether this effect would also emerge in children with other forms of anxiety, and, as such, the specificity of this finding remains unclear. Furthermore, whether the negative expectations of socially anxious children occur specifically in response to social (as opposed to non-social) situations has also not been addressed.
In summary, cognitive-behavioural treatments which include the modification of negative interpretation as a core component are commonly delivered to clinically anxious children as young as 7 years of age (e.g., James et al. 2005), but little is known about the characteristics of the thinking styles of clinically anxious children in mid to late childhood. The first aim of this study, therefore, was to consider whether differences in thinking styles between children with anxiety disorders and non-anxious children vary as a function of child age. Accordingly, this study includes groups of anxious and non-anxious children aged 7 to 12 years, who were balanced on child age. Little is known about age-effects in relation to the association between cognition and anxiety in children, so this particular age group was selected in response to studies of low mood that suggest a substantial shift in the nature of the association between thinking styles and affect from mid to late childhood (e.g., Nolen-Hoeksema et al. 1992).
The second aim of this study was to clarify whether negative interpretations of social situations are specifically associated with social anxiety disorder. Specificity is considered here in the following ways: (a) specificity to social anxiety disorder versus other anxiety disorders; (b) specificity to social, versus non-social situations; and (c) specificity in relation to particular cognitive themes (i.e., threat, anticipated negative emotion, and perceived control). In addition, in the case of evidence of specificity being found, care was taken to ensure that these effects were not due to overlapping symptoms of low mood and behavioural disturbance, given the high frequency of comorbid psychiatric conditions among children with social anxiety disorder (60%; Beidel et al. 1999) and conflicting reports regarding the specificity of interpretation biases in relation to anxiety symptoms and disorders (e.g., Creswell and O’Connor 2011; Eley et al. 2008; Reid et al. 2006).
A further consideration of this study was a methodological one, namely, to examine whether ambiguous hypothetical and real-life challenge events elicit similar responses, and have similar patterns of association with child anxiety. In order to effect this examination, children’s expectations were assessed in relation to both hypothetical ambiguous social and non-social scenarios measures, and ‘in vivo’ social and non-social challenge tasks. Specifically it was hypothesized that:
1.
Clinically anxious children will interpret potentially challenging situations as presenting higher levels of threat than non-anxious children, and this effect will be more marked in older children.
2.
Clinically anxious children, across the age range, will anticipate lower levels of coping (based on negative emotions and perceived control), in relation to potentially challenging situations, compared to non-anxious children.
3.
Children meeting diagnostic criteria for social anxiety disorder will differ from other anxious and non-anxious children in their predictions regarding social, but not non-social potentially challenging situations. Specifically:
3.1.
Socially anxious children will demonstrate higher levels of threat interpretation than other anxious and non-anxious children in relation to social situations, and this effect will be more marked in older children.
3.2.
Socially anxious children, across the age range, will demonstrate lower levels of anticipated coping than other anxious and non-anxious children in relation to social situations.
Method
Participants
One-hundred and twenty children aged 7–12 years took part in the study. The participants were 40 children with a principal diagnosis of social anxiety disorder (SA), 40 children with a diagnosed anxiety disorder but no social anxiety disorder (ANX), and 40 non-referred children (NONANX). Forty consecutive eligible participants were recruited to the SA group; participants in the other two groups were selected from a larger group of eligible participants (ANX, n = 49; NONANX, n = 51) in order to balance the groups for age and gender. Children were balanced for age by ensuring that there was an equal distribution of children in the 7–9 and 10–12 year age ranges. Accordingly, children did not differ across groups on mean age (months) (F(2, 117) = 0.63, p = 0.53); however, they did differ on the basis of family socio-economic status (χ2(1) = 8.83, p = 0.01), with fewer children among the SA group coming from families who were classified as higher/professionals than others (National Statistics Socio-Economic Classification (NS-SEC; HMSO 2005), see Table 1). The great majority of children in all conditions came from families of White, British origin and groups did not differ significantly on this basis (χ2(2) = 4.35, p = 0.11).
Table 1
Sample characteristics
Social anxiety disorder
Other anxious
Nonanxious
N = 40a
N = 40a
N = 40a
Age (months) (mean, SD)
123.33 (21.22)
119.60 (19.55)
118.80 (16.64)
F(2, 117) = 0.63
Gender (% female)
53%
53%
53%
χ2(1) = 1.00
Family SES % ‘Higher/professional’
47%
78%
73%
χ2(1) = 8.83**
Ethnicity % White British
91%
90%
76%
χ2(2) = 4.35
SCAS-c Total (mean, SD)
b43.95 (23.09)
36.00 (13.77)c
25.56 (11.76)b c
F(2, 112) = 11.45***
SCAS-c Social Phobia (mean, SD)
8.19 (4.15)b c
5.08 (3.29)b
4.54 (2.85)c
F(2, 112) = 12.25***
SCAS-p Total (mean, SD)
41.03 (18.60)
30.93 (13.80)
13.98 (6.58)
F(2, 94) = 36.79***
SCAS-p Social Phobia (mean, SD)
10.24 (4.82)b c
4.93 (3.78)b
3.63 (2.72)c
F(2, 94) = 29.58***
SDQ –p Behavioral problems (mean, SD)
7.05 (3.89)b c
5.35 (3.48)b
4.28 (3.85)c
F(2, 116) = 5.52**
SMFQ-c (mean, SD)
8.55 (6.13)b c
6.10 (3.15)b
4.02 (3.40)c
F(2, 115) = 10.32***
SCAS-c: Spence Children’s Anxiety Scale- child report; SCAS-p: Spence Children’s Anxiety Scale-parent report; SDQ-p Behavioral problems: Strengths and Difficulties Questionnaire- parent report (conduct problems and hyperactivity scales); SMFQ-c: Short Mood and Feelings Questionnaire- child report
amax available n (some SES data not provided; some questionnaires not sufficiently complete to calculate total scores)
b and c denote groups that significantly differ on basis of posthoc analyses
** = p < 0.01, *** = p < 0.001
Children in both anxiety disorder groups were recruited through referrals to the Berkshire Child Anxiety Clinic at the University of Reading by local health and education service personnel. In addition to the specified age, the other inclusion criterion was that they meet criteria for a current primary anxiety disorder diagnosis. Exclusion criteria were (a) significant physical or intellectual impairment (where this would impede reliable completion of measures), (b) current prescription of psychotropic medication (or if on medication this should have been stable for a month and (c) previous receipt of six or more sessions of cognitive behaviour therapy (i.e., treatment specifically targeting the processes under investigation).
Children were only included in the SA condition if they had a primary diagnosis of social anxiety disorder. Primacy of disorder was allocated to the disorder given the highest clinical severity rating (CSR) based on a structured diagnostic interview with children and parents (Anxiety Disorders Interview Schedule for DSM-IV- Child/Parent versions; ADIS-C/P; see below). Children were not excluded on the basis of comorbid diagnoses, and so represent a typical clinic population. 70% (n = 28) of children had a comorbid diagnosis, as follows: Separation Anxiety Disorder (58%), Specific Phobia (35%) Generalized Anxiety Disorder (53%), Obsessive Compulsive Disorder (5%), Post Traumatic Stress Disorder (2.5%), Dysthymia (10%), Major Depressive Disorder (15%), Oppositional Defiant Disorder (20%), Attention Deficit and Hyperactivity Disorder (23%).
Children in the ANX condition were included on the basis of having an anxiety disorder other than social anxiety disorder as their principal diagnosis, and they were excluded if they met criteria for a comorbid diagnosis of social anxiety disorder. Principal anxiety disorders of this group were as follows: Separation Anxiety Disorder (23%), Specific Phobia (28%), Agoraphobia without panic disorder (10%), Generalised Anxiety Disorder (33%), Anxiety Disorder Not Otherwise Specified (8%). The overall frequency of disorders within this group was as follows: Separation Anxiety Disorder (40%), Specific Phobia (47.5%) Generalized Anxiety Disorder (53%), Agoraphobia without Panic Disorder (13%), Anxiety Disorder Not Otherwise Specified (8%), Oppositional Defiant Disorder (13%), Attention Deficit and Hyperactivity Disorder (8%). The number of children who had a comorbid diagnosis in addition to their principal anxiety disorder was 62.5% (n = 25).
The SA and ANX groups differed significantly on their mean number of anxiety disorders (SA mean = 2.82 (SD = 1.68); ANX mean = 1.75 (SD = 0.84); t(57.34) = 3.62, p = 0.001) and all diagnoses (SA mean = 3.5 (SD = 2.18); ANX mean = 1.95 (SD = 1.04); t(55.72) = 4.06, p < 0.001). There was also a significant difference in the frequency of comorbid mood disorder (i.e. Major Depressive Disorder or Dysthymia) which were only present amongst the SA, and not the ANX group (χ2(1) = 11.43, p = 0.001). While the frequency of comorbid behavioral disturbance (ADHD and oppositional defiant disorder) appeared substantially higher in the SA compared to the ANX group (35% vs 20%) this was not statistically significant (χ2(1) = 2.26, p = 0.13).
Control participants were volunteers, recruited through invitation letters, sent predominantly through schools and local after-school clubs, specifically asking for children to form a non-anxious comparison group. The inclusion criteria were that children must be between 7 and 12 years of age, and have anxiety levels within the normal range based on parent and child report. Exclusion criteria were (a) significant physical or intellectual impairment (where this would impede reliable completion of measures), and (b) current prescription of psychotropic medication. Children were screened on the basis of child and parent (primary caregiver) report on the Spence Children’s Anxiety Scale-child and parent versions (SCAS-c/p; see below), as both measures have been found to discriminate between clinically anxious and non-anxious groups, and normative data are available (e.g., Spence 1998; Nauta et al. 2004). Where children scored above the normal range (i.e., in the ‘borderline’ or ‘abnormal’ categories) they were not invited for further inclusion in the study. Families in the control condition were given gift tokens in exchange for taking part.
Measures
Structured Diagnostic Interview with Children and Parents
Children were assigned diagnoses on the basis of the Anxiety Disorders Interview Schedule for DSM IV for Children- Child and Parent Versions (ADIS-C/P; Silverman and Albano 1996), a structured diagnostic interview with well established psychometric properties (Silverman et al. 2001). Where children met symptom criteria for a diagnosis (based on either child or parent report) they were assigned a clinical severity rating (CSR) ranging from 0 (complete absence of psychopathology) to 8 (severe psychopathology). As is conventional, only those children who met symptom criteria with a CSR of 4 or more (moderate psychopathology) were considered to meet diagnostic criteria. Assessors (psychology graduates) were trained on the standard administration and scoring of the ADIS-C/P through verbal instruction, listening to assessment audio-recordings and participating in diagnostic consensus discussions. The first 20 ADIS-child and ADIS-parent interviews conducted were then discussed with a consensus team, led by an experienced diagnostician (Consultant Clinical Psychologist). The assessor and the consensus team independently allocated diagnoses and CSRs. Following the administration of 20 child and 20 parent interviews, inter-rater reliability for each assessor was checked, and if assessors achieved reliability of at least 0.85 they were then required to discuss just one in six interviews with the consensus team to prevent inter-rater drift. Overall reliability for the team was excellent. Reliability for presence or absence of diagnosis was kappa = 0.98; and for the CSR intra-class correlation = 0.99.
Symptoms of Anxiety
The Spence Children’s Anxiety Scale (SCAS-c/p; Spence 1998; Nauta et al. 2004) was administered to assess child and parent reported symptoms of anxiety. The child version is a self report questionnaire that requires children to rate how often they experience each of the 38 anxiety symptoms, presented alongside six positive filler items, on a 4 point scale from 0 (never) to 3 always). The SCAS-C has demonstrated high internal reliability and concurrent validity (Spence 1998), with children from 7 years of age (Muris et al. 2000c). In the present study the social phobia scale was used as an indicator of self and parent reported social anxiety symptoms. The social phobia scale of the SCAS has been found to correlate highly with the Social Anxiety Scale for Children (SASC-R) (Muris et al. 2000b). Internal consistency was acceptable for both the total and the social phobia scales (Child: SCAS total, α = 0.89; SCAS social phobia, α = 0.70; Parent: SCAS total, α = 0.94; SCAS social phobia, α = 0.90).
Symptoms of Low Mood
The Short Mood and Feelings Questionnaire (SMFQ-c; Angold et al. 1995) was administered to assess child reported symptoms of low mood. The SMFQ-c is a brief, 13 item measure which requires children to report how often in the past 2 weeks they have experienced a number of symptoms on a 3 point scale from 0 (not true) to 2 (certainly true). The SMFQ-c has demonstrated high concurrent validity (e.g., Angold et al. 1995), and good internal consistency and predictive validity with children from 7 years of age (Sharp et al. 2006). Internal consistency was acceptable based on the current dataset (α = 0.76).
Symptoms of Behavioral Disturbance
Two scales from the parent report Strengths and Difficulties Questionnaire (SDQ-p; Goodman 1997) were used to assess behavioural disturbance: the conduct problems scale (5 items) and the attention and hyperactivity scale (5 items). Parents respond to each item on a 3 point scale from 0 (not true) to (certainly true). The SDQ-p also has good psychometric properties (Goodman 1997). The parent report version of the SDQ was used, as parents are often considered to be most reliable in terms of providing information on children’s externalising symptoms (Grills and Ollendick 2002). Internal consistency for the combined conduct problems and attention and hyperactivity scales was acceptable (α = 0.76).
Children’s Expectations
Children’s expectations regarding social and non-social events were assessed using (a) an adapted version of the Ambiguous Scenarios Questionnaire (ASQ; Barrett et al. 1996; Creswell et al. 2006), and (b) a rating scale asking children about their expectations before completing two (social and nonsocial) in-vivo challenge tasks.
Hypothetical Ambiguous Scenarios
The adapted Ambiguous Scenarios Questionnaire (ASQ) comprised 12 hypothetical situations (six social; six nonsocial). After being presented with a scenario (e.g., ‘You arrange to have a party at 4 o’clock and by half past 4 no one has arrived’- social; ‘You are lying in bed at night when you hear a big crash in the night’ - nonsocial), children were asked to (a) rate how they would feel in this situation (0 = not at all upset; 10 = very upset; negative emotion), (b) give a free response to the question ‘Why do you think this is happening?’ (threat free response), (c) rate how much they would be able to do about this situation (0 = nothing, 10 = a lot) (perceived control), (d) choose which of two alternatives (threat/nonthreat) they would be more likely to think in this situation (threat forced choice). The order of threat and nonthreat choices was counterbalanced across the 12 scenarios.
Responses to the open ended questions were coded by a psychology postgraduate blind to child group and scores on all other measures. Free responses regarding the cause of the event were coded as ‘Threat’ (e.g., ‘Nobody wants to come to my party’) or ‘Nonthreat’ (e.g., ‘They must be in a traffic jam’). A second independent coder (undergraduate psychologist) coded a sample of the responses (n = 30) in order to assess inter-rater reliability. Intraclass correlations were good, ICC = 0.91. Scores for each domain (distress, threat (free report), control, threat (forced choice)) were totalled across the 12 scenarios. As in previous reports (e.g., Creswell et al. 2006), the free and forced choice threat responses were combined to reduce the number of variables, which also reduced the skew of this particular measure so that its distribution did not differ significantly from normality (see below). Although lower for scales which comprised totals of dichotomous variables, internal consistency for each scale was acceptable (negative emotions: social α = 0.79, nonsocial α = 0.70; threat: social α = 0.56; nonsocial α = 0.59; control: social α = 0.64, nonsocial α = 0.76).
In Vivo Challenge Tasks: Expectations
A second set of cognitive measures involved expected performance ratings for two real-life challenge tasks, which differed according to whether they involved social or nonsocial content. In the social task children were asked to give a presentation to a video camera on a tripod manned by a research assistant. Children were told that they would be asked to give a 3 to 5 min presentation to the researcher which would be recorded on the video camera. Children were given a choice of topics to talk about (‘My hobbies’, ‘My ideal day’, ‘My family’, ‘My favourite holiday’) and were told that they would be left for five minutes to prepare (with their parent’s support) and then would be asked to give the speech to the camera. Before starting to prepare for the task, children were asked to provide ratings to indicate (a) how they felt about doing the task (0 = not scared at all, 10 = very scared; negative emotion), (b) how well they thought they would do the task (0 = not well at all, 10 = very, very well; threat), and (c) how much they could do about how the task goes (0 = nothing at all, 10 = a lot; perceived control).
The nonsocial task was selected in order to present a potential physical threat. In the nonsocial task, children were presented with a black box with a hole in each of its four sides, obscured by a black material curtain. Children were told that there were four scary items in the black box and that we would like the child to find out what is inside. Prior to starting the task children were asked to provide ratings on the same scales as for the social task. The box contained four fluffy or squidgy toys or substances (e.g., slime).
Ethical Considerations
This study was reviewed by the Local Research Ethics Committee on behalf of the National Health Service and the University of Reading Research Ethics Committee. Parents and children were both provided with written and verbal information about the study. In order to participate in the study written parental consent and child assent were both required.
Procedure
Following referral by a health or education service professional to the children’s anxiety clinic, potential participants were invited for an initial assessment appointment in which the ADIS-C/P, SCAS-c/p, SMFQ-c and SDQ-p were administered to parents and children in separate rooms. Children were assisted in completing questionnaires by a research assistant as appropriate (e.g., reading out of items). If children met criteria for either the SA or ANX groups they were invited to take part in a laboratory based research assessment before initiating treatment. Eighty-six percent of eligible children who were invited to participate agreed to take part in this second assessment. For the control children, parents were mailed the SCAS-P in advance of the meeting. If parents reported that children scored within normal limits on the SCAS-P, an appointment was scheduled for the laboratory based research assessment. Control children completed the SCAS-C during the lab assessment with the help of a research assistant. The lab-based tasks were conducted as part of a broader assessment of factors associated with the development of childhood anxiety disorders. On arrival for the lab assessment, children and their parents were first introduced to the overall plan for the session, and were given some time to acclimatise while playing a familiar game (‘Connect Four’). Children were then taken to a neighbouring room and were administered the adapted ambiguous scenarios questionnaire by the research assistant in an interview format (audio recorded). The child was then taken back to the lab where the presentation task was described to the child and their parent. The child was then asked to give ratings of his/her expectations, prior to doing the task. The same procedure was then followed for the Black Box task. Although we were principally interested in children’s expectations prior to doing the task, it was important to administer the tasks in full in order to be able to accurately elicit expectations when the second task was presented. The nonsocial challenge task was always presented last to minimise social evaluative concerns inherent in laboratory based observation tasks, by giving participants time to get used to the lab setting.
Data Analysis
Continuous data were screened in relation to the assumptions of parametric tests (Tabachnik and Fidell, 2000). In most cases, variables met the necessary assumptions, however, where assumptions were violated, confirmatory analyses were conducted by running analyses with 1,000 bootstrap samples. In all cases, the results were consistent, suggesting that the original analyses were robust to the violations of assumptions, so here results based on the original (nonbootstrapped) analyses are presented. Preliminary analyses were conducted to explore associations between ASQ and challenge expectation ratings and to identify potential confounding variables to include when testing hypotheses, these focussed on demographic characteristics that distinguished groups (i.e., SES) and common comorbid problems (i.e., low mood and behavioral disturbance).
The three hypotheses were tested by two multivariate analyses of covariance (MANCOVA). The dependent variables in the MANCOVAs were threat, negative emotions and perceived control assessments from, first, the Ambiguous Scenarios Questionnaire (ASQ) and, second, the challenge tasks. In both cases group (SA vs. ANX vs. NONANX) was the between subjects factor, and the identified potential confounds and the group by age interaction were entered as covariates. For all multivariate analyses of variance, Roy’s largest root test statistic was used as the primary focus was on comparisons with the social anxiety disorder group. In all cases the homogeneity of covariance matrix assumption was tenable.
Results
Preliminary Analyses
To address the methodological considerations, first, bivariate correlations were conducted to establish the extent of the association between ASQ responses and their equivalent ratings on the challenge tasks. The control dimensions correlated significantly (social, r(116) = 0.30, p = 0.001; nonsocial, r(118) = 0.20, p = 0.03). Correlations between task and ASQ variables were not significant, however, for threat (social, r(116) = 0.08, p = 0.38; nonsocial, r(116) = 0.01, p = 0.92) or negative emotion ratings (social, r(117) = 0.03, p = 0.71; nonsocial, r(119) = 0.12, p = 0.19). Second, repeated measures analyses were conducted with standardised ASQ and task ratings as the within subjects variables to compare ratings obtained using the two methods, and to examine task x child anxiety group interactions. For all dimensions there were significant effects of task, but there were no significant task x group interactions. Specifically, the social ASQ scenarios were rated higher than the social challenge task on negative emotion (F(1,114) = 8.09, p = 0.005, partial η2 = 0.07), threat (F(1,113) = 75.87, p < 0.001, partial η2 = 0.40) and lower on control (F(1,113) = 83.90, p < 0.001, partial η2 = 0.43). The non-social ASQ was rated lower than the non-social challenge task on negative emotion (F(1,116) = 4.99, p = 0.03, partial η2 = 0.04), but also lower on control F(1,115) = 25.70, p < 0.001, partial η2 = 0.18) and higher on threat ratings (F(1,113) = 44.35, p < 0.001, partial η2 = 0.28). Although there was no evidence of participant groups differing significantly on their responses to the different methods, as correlations were small or modest, the (unstandardised) ASQ and challenge task expectation ratings were analysed separately for all subsequent analyses.
Analyses were then conducted to confirm group differences on total and social anxiety and to identify potential confounds to be controlled for when testing hypotheses. On the basis of continuous measures (see Measures section), and as would be expected, significant group differences were found on scores of total anxiety (SCAS-c, SCAS-p) and social phobia. The groups also differed, however, on low mood (SMFQ-c) and behavioral disturbance (SDQ conduct and hyperactivity- p,). As shown in Table 1, posthoc tests identified significant differences between the anxious (SA and ANX) and nonanxious groups on total anxiety scores, but the SA and ANX groups did not differ significantly. On all other scales (social phobia, low mood and behavioral disturbance) there were significant differences between the SA group and the other two groups (ANX and NONANX), but there were not significant differences between the ANX and NONANX groups. Where groups differed on potential confounding variables, exploratory analyses were conducted to identify whether these factors were associated with any of the dependent variables (i.e., ASQ and challenge task expectations).
Socio-economic status was significantly associated with children’s expectations of control in the nonsocial challenge task (t(106) = 2.15, p = 0.03), with children from higher SES families giving higher control ratings (Higher SES mean (SD) = 6.39 (2.84); Lower SES mean (SD) = 5.08 (3.23)). SES was not significantly associated with any other ASQ or task expectation ratings. Behavioral disturbance, as assessed by the conduct problems and hyperactivity scales of the SDQ-p, was significantly positively associated with ASQ social negative emotions (r(119) = 0.33, p < 0.001) and ASQ social threat (r(116) = 0.36, p < 0.001), but no other ASQ or task expectation ratings. Low mood, assessed by the SMFQ-c, was significantly positively associated with ASQ social negative emotions (t(118) = 0.21, p = 0.03), and negatively associated with expectations of control in the non-social threat task (r(117) = −0.18, p = 0.05). Further analyses, therefore, initially controlled for SES, behavioral disturbance and low mood, as appropriate, in relation to the relevant dependent variables and the main effect of child age was considered throughout. Since symptoms of low mood (SMFQ-c), socio-economic status and child age did not have a significant effect when included in any of the models, however, the results presented below are adjusted for behavioural disturbance (SDQ) only. In order to ensure that controlling for comorbid symptoms was not obscuring findings due to potentially overlapping constructs, all analyses were conducted without controlling for SDQ behavioural disturbance and effect sizes were consistent in all cases. Similarly, analyses were repeated excluding participants who met diagnostic criteria for a mood disorder or ADHD (as these were particularly elevated in the SA group) and again the findings remained consistent.
Hypothesis Testing
Ambiguous Scenarios (ASQ)
In order to examine the three hypotheses in relation to children’s interpretation of ambiguous scenarios, a multivariate analysis of covariance was conducted with group (social anxiety disorder vs. other anxiety disorder vs. nonanxious) as the independent variable, ASQ nonsocial and social threat, control and negative feelings as the dependent variables, and SDQ-p and the group x age interaction entered as covariates. There were significant effects of behavioral disturbance (SDQ-p, F(6, 102) = 4.64, p < 0.001; partial η2 = 0.21), and group (F(6, 103) =3.15, p = 0.007; partial η2 = 0.16), but there was also a significant age by group interaction (F(6, 104) = 3.32, p = 0.005; partial η2 = 0.16). Follow up between subjects tests indicated that behavioral disturbance was positively associated with social threat (F(1, 107) = 7.78, p = 0.006; partial η2 = 0.07) and social negative feelings (F(1, 107) = 12.39, p = 0.001; partial η2 = 0.10). Anxiety disorder status was significantly associated with nonsocial threat (F(2, 107) = 4.56, p = 0.01; partial η2 = 0.08) and nonsocial negative feelings (F(2, 105) = 6.43, p = 0.002; partial η2 = 0.11). The age x group interaction was also significantly associated with nonsocial threat (F(3, 107) = 3.55, p = 0.02; partial η2 = 0.09) and nonsocial negative feelings (F(3, 107) = 4.39, p = 0.006; partial η2 = 0.11). As shown in Fig. 1, younger children in the SA group appear to report lower negative emotions and threat than ANX and NONANX children in nonsocial situations, whereas from about 10 years of age SA children report higher negative emotions in response to nonsocial ambiguous scenarios compared to the ANX and NONANX groups. In order to examine these interactions, further follow up analyses were conducted with the younger (7–9 years) and older (10–12 years) children separately. Examination of the plots of age x group interactions indicated that inclusion of the whole age range may be masking potentially important group differences present at different child ages, therefore exploratory analyses, within smaller age groups, were conducted for all dependent variables. In order to detect a medium effect (f2 = 0.15; α = 0.05, Power = 0.90) in a MANOVA with three groups, four predictors and six dependent variables, a total sample of 52 participants is required. Thus the current sample provides sufficient power to conduct these follow-up analyses within more restricted age groups.
Fig. 1
Age (months) by group interactions
×
When only younger (7–9 years) children were considered, there was a significant group effect (F(6, 42) = 2.77, p = 0.02; partial η2 = 0.28). Tests of between subjects effects identified that there was a significant group effect for nonsocial negative emotions (F(2, 46) = 4.77, p = 0.01, partial η2 = 0.17), with planned contrasts showing that the SA group gave significantly lower negative emotion ratings than the other anxious group (k = 11.57, p = 0.003; means (sds): SA, 16.06 (8.89); ANX, 26.82 (11.09); NONANX, 21.72 (10.71)). Apart from this, however, there were no group effects on ASQ indices. Thus, among younger children, there was no evidence of increased threat interpretation or reduced coping expectations in either of the anxious groups in comparison to nonanxious children, and little evidence for specificity of responses in the context of social anxiety disorder.
Among the older children (10–12 years), the group effect was again significant (F(6, 56) = 3.23, p = 0.008), however, in this case tests of between subjects effects identified significant differences across groups for nonsocial threat (F(2, 60) = 4.01, p = 0.02, partial η2 = 0.12), social control (F(2, 60) = 7.03, p = 0.002, partial η2 = 0.19), and nonsocial control (F(2, 60) = 6.24, p = 0.003, partial η2 = 0.17). Planned contrasts, accounting for the covariates, indicated that the SA group produced significantly higher nonsocial threat scores than the ANX group (k = 1.88, p = 0.009; means (sds): SA, 5.14 (2.17); ANX, 4.19 (2.20); NONANX, 4.45 (2.11)). Both anxious groups, however, gave significantly lower ratings than the nonanxious group on social control (SA vs. NONANX, k = 12.10, p = 0.001; ANX vs. NONANX, k = 7.76, p = 0.01; means (sds): SA, 18.36 (11.89), ANX, 21.43 (10.16); NONANX, 28.77 (7.81)) and nonsocial control (SA vs. NONANX, k = 10.45, p = 0.01; ANX vs. NONANX, k = 13.20, p = 0.001; means (sds): SA, 24.55 (13.13); ANX, 21.14 (11.06), NONANX 34.18 (13.04)). Thus, among older children, compared to nonanxious children, those in the two anxious groups produced increased threat interpretation and reduced control expectations, however, there was again little evidence of disorder specific effects.
In Vivo Challenge Tasks
As above, in order to examine the three hypotheses in relation to children’s expectations regarding the challenge tasks, a multivariate analysis of covariance was conducted with group (social anxiety disorder vs. other anxiety disorder vs. nonanxious) as the independent variable, nonsocial and social challenge task expectations (threat, control and negative feelings) as the dependent variables, and the age x group interaction entered as covariates. The effect of group was not significant (F(6, 107) = 0.1.10, p = 0.37; partial η2 = 0.06), but the group x age interaction approached significance (F(6, 108) = 2.14, p = 0.055; partial η2 = 0.11). Follow up between subjects tests indicated that the age x group interaction was significantly associated with nonsocial threat (F(3, 111) = 25.15, p = 0.02; partial η2 = 0.08). As shown in Fig. 1, this appeared to reflect higher levels of threat rated by younger other anxious compared to socially anxious and nonanxious children. As before, follow up analyses were conducted with the younger (7–9 years) and older (10–12 years) children separately in order to deconstruct this interaction effect, and explore group differences that may be limited to children within a smaller age range. Among the 7–9 year olds, the group effect was not significant (F(6, 44) = 1.58, p = 0.18, partial η2 = 0.18), however, it was significant for the 10–12 year olds (although notably the effect size was only marginally higher) (F(6, 59) = 2.32, p = 0.04, partial η2 = 0.19). For 10–12 year olds, tests of between subjects effects identified a group difference for control in the social challenge task only (F(2, 63) = 4.30, p = 0.02, partial η2 = 0.12). Planned contrasts indicated a significant difference between the SA and NONANX groups (k = 2.14, p = 0.005; mean(sd): SA, 5.77 (2.67); ANX, 6.55 (2.65), NONANX 7.91 (1.95)), the difference between the ANX and NONANX groups also approached significance (k = 1.36, p = 0.07). Thus, as was found with the ambiguous scenarios paradigm, among older children, those with SA anticipated lower levels of control in a social challenge task in comparison to nonanxious children. Evidence for disorder specificity, however, was lacking in that there were no significant differences between the two anxious groups.
Discussion
The primary aim of this study was to examine the thinking styles of clinically anxious children across middle childhood. The results of the current study failed to support the first hypothesis that anxious children would show increased threat interpretation in comparison to a nonanxious comparison group, as the clinically anxious groups did not differ from the nonanxious group on threat interpretation at any age. The second hypothesis was partially supported, in that the two anxious groups did report lower perceived control than the nonanxious group in both social and nonsocial ambiguous scenarios and the social challenge task. Notably, however, this group difference was only statistically significant when analyses were limited to the older (10–12 year old) children (although developmental changes in the association between cognitions and anxiety would not be expected to occur in such a discrete manner). It is particularly interesting that the extent of similarity and difference between groups on cognitive measures varied with child age, suggesting that the nature of the association between cognition and anxiety may alter through development. The current study, therefore, provides partial support for a central premise of cognitive models of anxiety disorders that negative thinking styles maintain anxiety, but only in later childhood. Future studies should attempt to identify the developmental factors that are likely to influence the changing association between cognition and affect in middle and late childhood.
A second central premise of cognitive models of anxiety disorders is that of cognitive-content specificity (e.g., Beck et al. 1985). In line with predictions, differences in threat interpretation were only found between the two anxious groups among the older children. However, contrary to predictions, older socially anxious children in the current study produced higher rates of threat interpretation than other anxious children in nonsocial ambiguous situations. It is important to point out that, while we included only children for whom social anxiety disorder was the primary diagnosis, we did not exclude on the basis of comorbid anxiety disorders, as we were keen to recruit a representative clinical sample of socially anxious children, and this may have influenced our results. However, in the current study, the difference between the anxious groups appears to be as much a reflection of the somewhat low scores of the other (nonsocially) anxious group as of the somewhat high scores of the socially anxious group.
With regard to coping-related cognitions (negative emotions and control), younger children in the social anxiety disorder group produced lower ratings on anticipated negative emotions in response to nonsocial ambiguous situations compared to the other anxiety disorders group. While this might appear to be consistent with the cognitive-content specificity hypothesis, it is important to note that the anxiety disorder groups did not differ significantly from the nonanxious group on this measure, so the findings suggest that the social anxiety disorder group gave particularly low ratings on anticipated negative emotions, rather than the other anxiety disorder group giving particularly high ratings. Overall, the findings, therefore, provide little evidence for the cognitive content specificity hypothesis across the age range studied.
The findings with older children are consistent with studies that have suggested that children with social anxiety disorder differ from nonanxious children in their interpretations and expectations relating to social situations (e.g., Alfano et al. 2006; Spence et al. 1999), but they are also consistent with those that have suggested that there is a lack of specificity between different anxiety subtypes (e.g., Barrett et al. 1996; Bögels et al. 2003; Muris et al. 2000a). Both sets of findings are, nevertheless, in line with the suggestion that ‘negative thoughts’ in late childhood (i.e., preadolescents) may be better characterised by thoughts about a lack of coping, rather than thoughts about threat (e.g., Alfano et al. 2002). Specifically, differences were most consistently found between anxious and nonanxious older children in relation to children’s predicted control. The general lack of significant findings in relation to threat interpretation is at variance with studies that have included adolescents, which have found differences between clinically anxious and nonanxious young people on this cognitive dimension (e.g., Barrett et al. 1996; Creswell et al. 2005) (though notably more negative interpretation biases have been found in anxious compared to non-anxious preschoolers; Dodd et al. 2011). By contrast our findings, at least with older children, are consistent with a study that considered a broader range of cognitive themes among preadolescent children (Waters et al. 2008a) and found that children with anxiety disorders (aged 7–12 years) did not differ significantly from nonanxious children on danger judgements, while having significantly lower scores on their perceived ability to influence situations.
In addition to perceived control, the other aspect of interpretation in which significant differences were found between anxious and nonanxious children, was anticipated negative emotions. Contrary to expectations, however, socially anxious children produced lower ratings of anticipated negative emotions in response to ambiguous hypothetical social scenarios than nonanxious children. This finding presents something of a puzzle, but may reflect a tendency for what is referred to as ‘self-enhancement’ or ‘unrealistic optimism’ amongst the clinically anxious participants, a process which is considered to be a mechanism that acts to protect self-esteem (e.g., Sedikides and Strube 1997). Notably, the same pattern was not seen in the in-vivo tasks,this situation only occurred in relation to the rating of anticipated emotional response, where it is likely that the low negative emotion response may have been more clearly socially desirable.
These findings relate to our methodological interest in the extent to which children’s responses to ambiguous hypothetical scenarios correspond to their responses to novel in vivo challenge situations. Ambiguous situations have been extremely widely used in the child and adult anxiety literature (see e.g., Hadwin et al. 2006), however they have been subjected to little evaluation in relation to their validity In the current study we found that, although participant groups did not significantly differ from each other in how they responded to the two different methods of assessment, only ratings of perceived control were significantly associated with their counterpart ratings from the real-life challenge tasks. This is notable, given that control ratings were also most consistently associated with child anxiety group, and, hence may be a particularly useful domain to focus on, amongst 10–12 year olds. The real-life challenge tasks used here involved a social challenge (having to give a speech to a research assistant with a video camera), and a nonsocial challenge (having to explore the contents of a mysterious box containing ‘scary’ items). It is of course possible that different sorts of tasks may have elicited different ratings and, potentially, more disorder specific ratings. For example, children with social anxiety disorder have been found to differ from nonanxious controls in their self-evaluations in relation to tasks which involved explicit evaluation (Spence et al. 1999) or interaction with peers (Alfano et al. 2006). Future studies would benefit from inclusion of a broader range of disorder-relevant tasks, as well as consideration of other cognitive themes (e.g., self-focused attention; Clark and Wells 1995). Nonetheless, it is notable that none of the hypothesised differences between anxious and nonanxious groups were found among the younger sample, whether assessed by either ambiguous hypothetical situations or challenge tasks, suggesting that the null findings were not simply a reflection of limitations of using hypothetical scenarios to assess thinking styles in this age group.
One of the significant strengths of the current study was the inclusion of an age and gender balanced anxious comparison group, and consideration of the potential confounding effects of, for example, behavioural disturbance, low mood and socioeconomic status. In this regard it was notable that rates of both mood and behavioural disorders (particularly ADHD) were higher in the social anxiety group compared to both the other anxious and nonanxious groups, and family socioeconomic status was also lower (although these differences did not alter the pattern of results in relation to cognitions and anxiety). Previous studies have either excluded participants on the basis of comorbid disorders (e.g., Spence et al. 1999) or have not included an other anxious comparison group (e.g., Beidel et al. 1999) so we are unable to consider these relative rates of disturbance in the socially anxious group in relation to other samples. Our findings, however, suggest that children with social anxiety disorder may represent a particularly complex group in relation to comorbidity and social circumstances and this warrants further research attention. From our findings we cannot however conclude that there are differences between children with social anxiety disorder and those with any other specific anxiety disorders, as here we have grouped together children on the basis of meeting or not meeting diagnostic criteria for social anxiety disorder. Due to the high levels of comorbidity typical of anxious youth more specific comparisons present a challenge for future research.
It is important to acknowledge that task order in the current study was not counterbalanced. The order was fixed to allow participants the opportunity to get used to the social challenge inherent in participating in an observed task prior to doing the ‘nonsocial’ task. As a result, however, order effects cannot be ruled out. A final, but essential, consideration is the fact that the study involved a cross-sectional design. Where significant associations between child anxiety group and cognitions were found, therefore, conclusions cannot be drawn regarding the direction of effects. The use of prospective and experimental designs (such as cognitive bias modification, e.g., Vassilopoulos et al. 2009) will be useful to identify whether cognitions act to maintain anxiety. The findings here, however, lead to the suggestion that ongoing longitudinal and experimental investigations may do well to consider the varying nature of the association between cognition and anxiety in middle and late childhood, and to shift focus from threat interpretation to perceived coping abilities when working with preadolescents. In cognitive therapy with adults the importance of identifying the ‘hot thought’ is emphasised (e.g., Williams and Garland 2002), which in anxiety treatment commonly represents the underlying threat. With children, however, it may not be necessary (or may even be unhelpful) to persist in questioning until a child has generated a threatening explanation for their fear. Instead, it may be more helpful to focus on helping children challenge the perspective that they would not be able to take positive action should a challenging situation arise. This possibility warrants further research attention. Consideration of similar research questions to those addressed here with adolescent samples will also be valuable in identifying at what point in development threat interpretation is more clearly associated with anxiety, as will consideration of what factors influence its development for example, a lack of social success experiences (e.g., Spence et al. 1999), and parental discourse (Murray et al. 2013).
In summary, the findings from the current study suggest that differences between anxious and nonanxious children become clear in late childhood, particularly in relation to perceived control and anticipated negative emotions. Across the whole age range, however, there was a lack of evidence for cognitive-content specificity in relation to childhood social anxiety disorder. The findings suggest that treatments for childhood anxiety disorders would benefit from being developmentally sensitive to differences in the nature of the association between cognitions and anxiety throughout childhood. While patterns of results were consistent when considering children’s perceived control, future research would do well to include real life challenge tasks to assess other aspects of children’s cognition.
Acknowledgments
The authors would like that thank the participating families, as well as staff at the Berkshire Child Anxiety Clinic; Lucy Willetts, Rachel Gitau, Sue Cruddace, Sarah Shildrick, Adela Apetroaia, Jenny Crosby, Amy Corcoran, Sarah Cook, Ray Percy and Gemma Denton for their help collecting and coding data. We also thank Kiri Clarke for her help preparing the manuscript. Cathy Creswell was funded by an MRC Clinical Scientist Fellowship (G0601874).
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
Author Note
Cathy Creswell, School of Psychology and Clinical Language Sciences, University of Reading; Lynne Murray, School of Psychology and Clinical Language Sciences, University of Reading, Stellenbosch University; Peter Cooper, School of Psychology and Clinical Language Sciences, University of Reading, Stellenbosch University.
Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.
