Social anxiety disorder (SAD) is a mental health condition characterized by an excessive fear about being judged negatively in social situations (American Psychological Association, 2013). Feared social situations include circumstances where the individual may experience scrutiny, such as when performing in front of others (e.g., a speech), being observed (e.g., eating in public), or in social interactions (e.g., meeting new people). Individuals with SAD experience a range of cognitive, emotional, and behavioural symptoms that cause significant distress and/or impair daily functioning, often leading to decreased quality of life and occupational functioning (Barrera & Norton, 2009; Bruch et al., 2003).
In the general population, ruminating or ‘thinking about’ an approaching or past social event can sometimes be an adaptive way of planning and preparing, as well as creating meaning and understanding, depending on the content and ability to process information (Penney & Abbott, 2015; Vassilopoulos, 2008). Cognitive models of SAD (Clark, 2001; Heimberg et al., 2010; Hofmann, 2007) provide a framework for understanding the key symptoms and maintaining processes, with all three models suggesting that individuals with SAD engage in maladaptive levels of negative rumination. In SAD, negative rumination is both repetitive and intrusive focusing on negative evaluations of the individuals social engagement leading up to (i.e., pre-event rumination) and following (i.e., post-event rumination) an event (Clark & Wells, 1995). Literature exploring negative rumination for individuals with SAD often uses the terms ‘pre-event rumination’ and ‘anticipatory processing’ interchangeably. Similarly, the terms ‘post-event rumination’ and ‘post-event processing’ are also comparable.
Within one model of social anxiety, Hofmann (2007) suggests that individuals with SAD experience increased social apprehension and heightened negative self-focused attention leading up to a social event, due to high perceived social standards. This experience triggers a number of cognitive processes whereby the individual overestimates the probability and social costs of negative social outcomes, as well as increased negative perception of themselves, and decreased perception of their social efficacy and ability to control their anxiety (Hofmann, 2007). As a result, individuals with SAD anticipate social mishaps (i.e., pre-event rumination), engage in safety behaviours and/or avoidance, and engage in post-event rumination. Both pre- and post-event rumination are understood as interwoven processes, with post-event rumination hypothesised to confirm negative perceptions about the self and social situations, as well as consolidate biased memories of perceived social incompetence (Brozovich & Heimberg, 2013). Correspondingly, pre-event rumination likely reactivates negative imagery of perceived social failures, strengthening memory biases, as well as directly increasing anticipatory anxiety and potential avoidance of social situations (Heimberg et al., 2010). Given the nature of negative rumination, it is also theorised as one of the key variables that hinders individuals with SAD from becoming desensitized to social events, despite regular exposure to events that have seemingly neutral or positive outcomes (Penney & Abbott, 2014). Negative rumination impacts treatment efficacy, in that increased pre-event rumination (Wong et al., 2017) and post-event rumination (Price & Anderson, 2011) at baseline impedes social anxiety symptom reduction during psychological treatment. Thus, pre- and post-event rumination are seen to play fundamental roles in maintaining SAD and impacting treatment effectiveness. Therefore, the development and evaluation of valid and reliable measures of pre- and post-event rumination in social anxiety is crucial.
A range of current measures exist that assess negative rumination relevant to social anxiety. The Anticipatory Social Behaviours Questionnaire (ASBQ: Hinrichsen and Clark, 2003) and the Anticipatory Processing Questionnaire (APQ: Vassilopoulos, 2004) both pertain to pre-event rumination, whereas the Rumination Questionnaire (RQ: Mellings and Alden, 2000), the Post Event Processing Questionnaire (PEPQ: Rachman et al., 2000) and the Post-Event Processing Inventory (PEPI: Blackie and Kocovski, 2017) center on post-event rumination. Though pre-event and post-event rumination are largely assessed using separate measures for social anxiety they are theorised to have similar features including intrusiveness, content, and imagery related to negative evaluation (Rachman et al., 2000; Vassilopoulos, 2004).
The ASBQ is 12-item adaption from the Social Behaviour Questionnaire (Clark and Wells, 1995). The ASBQ was created to further investigate the various cognitive strategies people with social anxiety engage in prior to social situations (Hinrichsen & Clark, 2003). Exploratory factor analysis (N = 861) and subsequent confirmatory factor analysis (N = 485) found that a 10-item two-factor solution (i.e., avoidance and preparation subscales) best fit the data in a sample of undergraduate students as well as in a sample of undergraduates with high social anxiety (N = 59) (Mills et al., 2013). Mills et al. (2013) found evidence that the avoidance subscale significantly predicted anxiety scores at baseline and immediately preceding a social interaction for individuals with high social anxiety, whereas the preparation subscale did not predict anxiety scores at these same time points. This suggests that pre-event rumination associated with avoidance and escape could be more maladaptive than anticipatory rumination related to preparation and planning in social anxiety. Future research is yet to fully investigate the differences between the 10-item and 12-item ASBQ, as well as determine the replicability of the two-factor structure in a clinical SAD sample. Another measure of pre-event rumination is the APQ. The APQ assesses content, frequency, and intrusiveness of thoughts leading up to a unspecified social situation in which participants also experienced anxiety (Vassilopoulos, 2004). The APQ asks participants to rate questions using a visual analogue scale from 1 to 100. The APQ was found to have a one-factor structure in undergraduate participants (N = 147). The 15-item APQ was also found to have a significant and positive correlation between measures of social anxiety. Though the APQ appears to have good measurement properties, it is yet to be psychometrically assessed in a high social anxiety sample or clinical SAD sample (Vassilopoulos, 2004).
The PEPQ is a 13-item measure of post-event rumination (Rachman et al., 2000). The PEPQ, similar to the APQ, asks participants questions about their thoughts (i.e., intrusiveness, frequency) in relation to an unspecified anxiety-inducing social situation that they experienced recently over the past few months. The initial study found that the 13-item measure had significant and positive correlations with measures of social anxiety in an undergraduate sample (N = 130). Principal component analysis suggested a one-factor solution, however, three items did not significantly load onto this factor, (i.e., Items 5, 9 and 10 in Table 1) suggesting that these items may not reflect post-event rumination. McEvoy and Kingsep (2006) conducted a follow-up exploratory factor analysis in a clinical sample of individuals with SAD (N = 117), and found the same items did not significantly load, as well as one additional item, therefore the study recommended the use of a 9-item scale, namely the PEPQ-Revised (PEPQ-R). Though the PEPQ-R was significantly related to measures of depression, anxiety, and stress, the PEPQ-R was not significantly related to measures of social anxiety, which may indicate that the PEPQ-R reflects depressive and general anxiety (i.e., worry) rumination, rather than content specific to social anxiety. These findings cast doubt on the psychometric properties of the PEPQ and PEPQ-R, as a measure of rumination in SAD. An alternative self-report measure is the PEPI (Blackie & Kocovski, 2017). The PEPI has two 12-item parallel versions that assess state (i.e., PEPI-S)and trait (i.e., PEPI-T) aspects of post-event processing. Principal components analysis suggested the PEPI-S and PEPI-T are both represented by three factors in an undergraduate sample (N = 220). The three factors reflected (1) intensity of post-event rumination, (2) frequency of post-event processing, and (3) self-judgement. Subsequent confirmatory factor analysis replicated this factor structure and indicated that the higher-order general post-event processing factor could be inferred from the three factors, in a sample of undergraduate students (N = 199) as well as for individuals seeking help for social anxiety (N = 155) (Blackie & Kocovski, 2017; Blackie and Kocovski, 2019). The PEPI also demonstrated good construct and convergent validity, as both measures were significantly and positively correlated with measures of social anxiety and another measure of state based post-event rumination (Blackie & Kocovski, 2017). Overall, the PEPI appears to have a stable factor structure and good psychometric properties, and therefore research should aim to validate this measure in a clinical SAD population. The RQ is another measure that has 5-items used to measure post-event rumination, specifically in relation to a social interaction task (Mellings & Alden, 2000). The RQ was found to have good construct validity, as it was significantly and positively related to negative bias in self-judgements and self-related information following the interaction (Mellings & Alden, 2000). However, the measurement and psychometric properties are yet to be assessed in a community, high social anxiety, or SAD sample.
The Socially Anxious Rumination Questionnaire (SARQ: Abbott and Rapee, 2004; Edwards et al., 2003), originally developed as the negative rumination subscale of the Thoughts Questionnaire (Abbott & Rapee, 2004; Edwards et al., 2003), has the option to measure both pre-event and post-event rumination. The SARQ is a measure of negative rumination relative to a pre-determined social threat task such as a speech or social interaction. The SARQ has two parallel versions that contain the same 12 items, with the difference between the two versions being a change in tense to reflect pre-event rumination (i.e., SARQ-pre) or post-event rumination (i.e., SARQ-post). The SARQ-pre utilises future tense and the SARQ-post utilises past tense, and this allows for comparison of pre- and post-event rumination in relation to the same task, as well as the ability to contrast different types of social threat situations. Though the SARQ is a widely used measure of pre- and post-event rumination (Modini et al., 2018a; Wong et al., 2017), psychometric analysis has only been recently completed. Donohue et al. (2021) explored the psychometric properties of the SARQ in relation to a speech task, with a large sample of individuals with SAD (N = 505). The study found that in relation to an impromptu speech, the SARQ exhibited a one-factor structure, with good construct, excellent internal consistency, good test-retest reliability, and treatment sensitivity to group cognitive behavioural therapy (CBT). In addition, the SARQ demonstrated excellent clinical utility in discriminating individuals with SAD from non-clinical participants. Importantly, results were comparable for pre-event and post-event rumination. Thus, the SARQ was endorsed as a robust measure of pre- and post-event rumination to be utilised with individuals with SAD in relation to a speech. However, the SARQ is yet to be psychometrically validated in relation to a social interaction. This is important as increased negative rumination for individuals with SAD is not limited to performance tasks, such as speeches, but rather to a broad range of social situations, including social interactions (Brozovich & Heimberg, 2008). For example, Coles et al. (2001) found that individuals with SAD engage in increased negative rumination preceding and following a speech as well as a social interaction, when compared to those without SAD, with studies yet to explore whether negative rumination is significantly worse before or after different social tasks.
Interestingly studies have tended to predominantly measure post-event rumination, but not pre-event rumination, in relation to a social interaction, using the SARQ (Gaydukevych & Kocovski, 2012; Zou & Abbott, 2012) and RQ (Mellings & Alden, 2000). This is likely due to the hypothesised ‘post-mortem’ (i.e., Clark and Wells, 1995, p. 74) that takes place for individuals with SAD following social situations, in the context of the inherent ambiguity of social interactions in allowing people with SAD to establish whether they have met the perceived standards to avoid negative evaluation (Zou & Abbott, 2012). Though performance situations, such as speeches, can be ambiguous, they are inherently more structured (i.e., about one topic; time-limited), able to be planned and rehearsed, and have regulated roles (e.g., speaker and audience). Thus, it seems plausible that this leads to a ‘higher stakes’ appraisal preceding such performance tasks, since the individual has a defined role with inferred standards that resembles similar situations (e.g., at school or university), with direct feedback, grades and associated evaluation. This could potentially lead to greater pre-event rumination preceding a speech, in comparison to a conversation. By comparison, social interactions are less structured, and the role of speaker is dynamic. This means that the individual needs to draw upon a complex set of interpersonal skills to successfully engage and interact. It is this combination of interpersonal complexity and ambiguity during a social interaction that is hypothesised to increase the potential bias that one behaved in a socially ‘unacceptable’ way (Voncken & Bogels, 2008). From this perspective, an increase in post-event rumination following a social interaction, relative to a speech, may occur as a result of negative self-perceptions being exaggerated following the conversation. Such biases may be activated and maintained by the inherent ambiguity in knowing whether one has achieved the inferred standard during a social interaction, thereby inducing a repetitive review of one’s performance (including thoughts and feelings experienced during the task) in an effort to find confirmatory evidence of a social mishap warranting negative evaluation in the post-event period (Zou & Abbott, 2012). Though individuals with SAD are hypothesised to experience increased pre-event and post-event rumination following different social tasks, no study has explored whether pre-event rumination is more pronounced following a speech nor whether post-event rumination is more exaggerated following a social interaction with clinically diagnosed individuals with SAD. This may be clinically relevant for developing targeted interventions for individuals with SAD who experience anxiety in either performative or interactional situations. Additionally, it could be the case that the processes involved in pre-event and post-event rumination are different depending on the type of social threat (performance versus interaction), and so being able to reliably measure these processes across different social contexts is important in developing our theoretical understanding of these constructs.
Thus, the current study aimed to assess the psychometric properties of the SARQ in relation to a social interaction task (i.e., extension of Donohue et al., 2021). The psychometric properties will be assessed using components of the quality criteria for measurement properties outlined by Terwee et al. (2007). We hypothesise that a one-factor solution will emerge for both pre-event and post-event versions of the SARQ (i.e., content validity), and that the SARQ will demonstrate adequate internal consistency, construct validity, treatment sensitivity (i.e., responsiveness), and no floor or ceiling effects. In addition to evaluating the psychometric properties of the SARQ we also aim to assess the potential differences in the frequency of pre-event and post-event rumination following a speech versus a social interaction task.
Method
Participants
Clinical participants (n = 134) were aged between 18 and 77 years (M = 32.14; SD = 10.69; 50.7% female) and met criteria for SAD as their principal diagnosis. Regarding the highest level of education attained, 30.3% had completed high school, 21.1% had completed a certificate or diploma, and 44.4% had completed a university degree.
Diagnostic status of SAD was established using the Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV: DiNardo et al., 1994). Doctorate-level clinical psychology students under the supervision of experienced clinical psychologists administered the ADIS-IV. Individuals with a principal Axis I diagnosis other than SAD were excluded from the study, as were individuals who were actively suicidal or displaying symptoms of psychosis. Clinician severity ratings (CSR) were determined according to symptom severity and associated impairment in functioning and were rated on a 9-point scale ranging from 0 to 8. The mean CSR for participants with a principal diagnosis of SAD was 6.01 (SD = 0.85), corresponding to a severe level of impairment or distress caused by social anxiety. Secondary diagnoses for those in the clinical group included: generalized anxiety disorder (26.1%), unipolar depression (23.7%), specific phobia (7%), dysthymia (6.3%), panic disorder with or without agoraphobia (1.4%), obsessive-compulsive disorder (1.4%), and other diagnoses (7.7%), with (34.5%) having no secondary diagnosis. Diagnostic interviews were recorded, and 20% were selected at random (n = 29) to be coded by a blind independent rater for reliability. Interrater reliability for the agreement of SAD as the principal diagnosis was good (κ = 0.86), with good agreement on clinician severity ratings (ICC = 0.85).
Measures
The Socially Anxious Rumination Questionnaire (SARQ)
The SARQ (Abbott & Rapee, 2004; Donohue et al., 2021; Edwards et al., 2003) is a state measure of the frequency of pre- and post-event rumination. The SARQ has two parallel versions that differ slightly in terms of tense used for instructions, as the SARQ-pre utilizes future tense (e.g., ‘I will look stupid’), whilst the SARQ-post utilizes past tense (e.g., ‘I looked stupid’). The SARQ-pre reflects pre-event rumination and is administered prior to the social task, and the SARQ-post reflects post-event rumination and is administered 1 week following the task. Both SARQ-pre and SARQ-post have 12-items rated on a 5-point Likert scale, ranging from never (0) to very often (5), with a maximum score of 60, with higher scores equating to increased negative rumination. The SARQ demonstrated excellent psychometric properties in relation to a speech in a SAD sample, with excellent construct validity, excellent internal consistency, good test-retest reliability, as well as demonstrating treatment sensitivity and clinical utility (Donohue et al., 2021).
Social Phobia Scale and the Social Interaction Anxiety Scale (SPS and SIAS)
The SPS and SIAS (Mattick & Clarke, 1998) are two commonly used questionnaires with 20 items in each. The SPS assesses fears of social observation or scrutiny, and the SIAS evaluates anxiety regarding social interactions. Both scales possess excellent psychometric properties, a stable factor structure, good construct validity and high levels of internal consistency and test-retest reliability for individuals with SAD (Mattick & Clarke, 1998; de Beurs et al., 2014; Wong et al., 2019). Cronbach’s alpha for the current sample was excellent for the SPS (α = 0.91) and good for the SIAS (α = 0.84).
Speech Performance Questionnaire (SPQ)
The SPQ is a state-based measure used to identify the individual’s self-evaluation of their public speaking ability (Rapee & Lim, 1992). The SPQ-pre measures self-appraisals about ones predicted performance immediately before a task, whereas, the SPQ-post measures actual self- appraisals after the task. Higher scores on the SPQ elucidate more positive self-appraisals (e.g., kept audience interested; spoke well) in relation to the task. Cronbach’s alpha was good (α = 0.85).
State Anxiety Rating Scale (SAR)
The SAR (Rapee & Abbott, 2007) is a 10-item questionnaire developed to measure current feelings of anxiousness in relation to the task that the individual has just completed (e.g., ‘I felt nervous’). The SAR was measured immediately following the social interaction.
Cronbach’s alpha was excellent for the current sample (α = 0.91).
The Brief Fear of Negative Evaluation Scale (BFNE)
The BFNE (Leary, 1983), a 12-item self-report scale that captures fear of negative evaluation by others. The total score of the BFNE was used due to its excellent psychometric properties demonstrated in individuals with SAD (Modini et al., 2015), with excellent internal consistency, excellent construct validity, as well as treatment sensitivity in a SAD sample (Weeks et al., 2005). Cronbach’s alpha for the BFNE was adequate, α = 0.76, in the current sample.
The Depression Anxiety Stress Scales (DASS-21)
The DASS-21 (Lovibond & Lovibond, 1995) is a 21-item self-administered measure of symptoms related to depression, anxiety, and stress. The DASS-21 has been found to have good construct validity and good internal consistency in a sample of individuals with SAD (Antony et al., 1998). Cronbach’s alpha for each subscale ranged from adeqaute to excellent for depression (α = 0.91), anxiety (α = 0.79) and stress (α = 0.89).
Procedure
The study was approved by the Human Research Ethics Committee at Macquarie University (HREC approval number: RO5197), as part of a larger study previously detailed in Rapee et al. (2013). Potential participants contacted the Centre for Emotional Health Clinic at Macquarie University. These participants were screened via telephone and those who appeared to have anxiety-related difficulties were invited to participate in the structured clinical interview, and subsequent social anxiety tasks and CBT treatment. At the assessment session all participants provided informed consent and this was followed by the completion of the structured clinical interview (i.e., ADIS-IV) as well as trait and state questionnaire measures (i.e., SIAS, SPS, DASS-21, and BFNE). Participants were then informed that they would be giving a speech as well as having a conversation in one week at a scheduled research session. At the research session, participants completed separate SARQ-pre questionnaires for the speech and conversation, assessing levels of rumination during the preceding week (n = 134). Participants then completed the speech and conversation in counterbalanced order. Regarding the conversation, participants were asked to engage in a 5-minute conversation with a member of the research team (i.e., a confederate). Confederates were trained to maintain a neutral manner and encourage the participant to talk without directing the conversation, this included refraining from asking questions or suggesting topics to discuss. If after 30 s the participant remained silent, the confederate was allowed to prompt the conversation and ask one of five open ended statements (e.g., ‘It’s a nice day today’). If the participant responded with closed statements following two of these prompts, the confederate was provided with three questions that they could ask aimed to encourage conversation (e.g., ‘What kind of movies do you like?’). For the speech, participants were asked to give a 3-minute presentation on a topic of their choice. One week after the research session, participants were asked via email to complete separate SARQ-post for the conversation and speech, with a large percentage of participants completed these questionnaires (92.5%). SARQ-pre scores were compared for participants who did and did not complete the SARQ-post, showing no significant difference. Participants in this trial were not financially compensated for their time. Participants in this study were a subsection of the SAD group in Donohue et al. (2021) that completed the SARQ-pre and SARQ-post for both the social interaction task as well as the speech task.
All participants then went on to complete a 12-week group CBT intervention, consisting of 12, 2-hour sessions. The CBT intervention involved thought challenging, examining of core beliefs, training of attentional resources away from threat, as well as realistic feedback of social performances and in vivo exposure of social experiences with the elimination of subtle avoidance and safety behaviours. The treatment did not directly target negative rumination. More information regarding the CBT intervention can be sourced from Rapee et al. (2009), and a copy of the manual can be obtained from Jonathan E. Gaston or Ronald M. Rapee. Treatment was delivered by doctoral level clinical psychology students under the supervision of senior clinical psychologists experienced in the assessment and treatment of anxiety disorders. After treatment, participants were administered the ADIS-IV again for diagnoses met at pre-treatment to ascertain current diagnostic status. A large percentage of participants also completed a post-treatment research session (76%). This post-treatment research session replicated the procedure of the pre-treatment research session. This included administration of the SARQ-pre (n = 102) and that SARQ-post (n = 92) for a second (post-treatment) conversation and speech.
Data Analysis
AMOS, v12 (Arbuckle, 2013) was used to conduct confirmatory factor analysis. Exploratory factor analysis and confirmatory factor analysis had established that the unidimensional model demonstrated best fit for the SARQ in relation to a speech (Donohue et al., 2021). Thus, a unidimensional model was fitted to the data for the SARQ-pre and SARQ-post. The current study followed the guidelines provided by Hu and Bentler (1999) and Browne and Cudeck (1993) to evaluate the model fit, this included RMSEA values in the range of 0.05 to 0.08 to indicate fair fit, and fit index (i.e., IFI, TLI, CFI) above 0.90 as acceptable. In terms of sample size, the the 5:1 ratio of sample size to number of items was met (Bentler, 1989). Within-factor correlated errors with covariances of greater than 4.0 were permitted to covary within each model provided that the items were sufficiently similar (Brown, 2015). For example, SARQ-pre item 1 (i.e., ‘I will feel very anxious’) was permitted to covary with item 4 (i.e., ‘I will look nervous/anxious’), to improve model fit. IBM SPSS Statistics, v24.0 for Windows (IBM, New York, USA) was utilized for the remaining analyses. Construct validity was assessed using Pearson correlations with the following measures: SPS, SIAS, BFNE, DASS-21 subscales, and CSR. Internal consistency was examined with Cronbach’s alpha. Treatment sensitivity was assessed via paired samples t-tests. Change scores from pre-treatment to post-treatment were also calculated for the SARQ-pre, SARQ-post, CSR and SIAS, and correlated using Pearson correlations to further assess sensitivity to treatment. In addition, paired samples t-tests were used to determine if levels of pre-event rumination, using the SARQ-pre, (n = 134) and post-event rumination, using the SARQ-post (n = 124) were significantly higher for a conversation or speech at pre-treatment for individuals with SAD.
Results
Confirmatory Factor Analysis
The SARQ-pre demonstrated good fit for the unidimensional factor model. The SARQ-post also had good fit for the one-factor model. See Tables 1 and 2 for fit indices and factor loadings, respectively.
Table 1
Fit indices from confirmatory factor analyses for the SARQ-pre and SARQ-post in relation to a social interaction
Scale | χ2 /df | IFI | TLI | CFI | RMSEA | PCLOSE |
|---|---|---|---|---|---|---|
SARQ-pre | 1.77 | 0.97 | 0.96 | 0.97 | 0.08 | 0.06 |
SARQ-post | 1.35 | 0.99 | 0.98 | 0.99 | 0.05 | 0.41 |
Table 2
Standardised factor loadings for SARQ-pre (n = 134) and SARQ-post (n = 124) in relation to a social interaction
SARQ-pre | Factor Loading | SARQ-post | Factor Loading |
|---|---|---|---|
1. I will feel very anxious | 0.80 | 1. I felt very anxious | 0.82 |
2. I will make a lot of mistakes | 0.78 | 2. I made a lot of mistakes | 0.87 |
3. The investigator won’t like me | 0.73 | 3. The investigator didn’t like me | 0.76 |
4. I will look nervous/anxious | 0.81 | 4. I looked nervous/anxious | 0.86 |
5. I will make a fool of myself | 0.89 | 5. I made a fool of myself | 0.87 |
6. I always do badly at this sort of thing | 0.75 | 6. I always do badly at this sort of thing | 0.81 |
7. I will look stupid | 0.88 | 7. I looked stupid | 0.86 |
8. I will feel very self-conscious | 0.83 | 8. I felt very self-conscious | 0.88 |
9. I will feel like a failure | 0.76 | 9. I felt like a failure | 0.85 |
10. I will feel awkward | 0.74 | 10. I felt awkward | 0.87 |
11. My heart will pound very fast | 0.52 | 11. My heart was pounding very fast | 0.57 |
12. I will make a bad impression | 0.75 | 12. I made a bad impression | 0.88 |
Floor and Ceiling Effects
Across the SARQ-pre, the total score ranged from 0 to 46. Only 3.7% of participants achieved the lowest possible score and 0% achieved the highest possible score. Across the SARQ-post, the total score ranged from 0 to 48, with 4.7% of participants attaining the lowest possible score and 0% reaching the highest possible score.
Normality Measures
Inspection of the Q-Q plots for both the SARQ-pre and SARQ-post revealed no deviating outliers. On inspection of the histogram there is a slight negative skew, this was expected as the sample of individuals with SAD was rated as having a severe level of clinical impairment according to the CSR. In line with central limit theorem, we proceeded with parametric testing, as it states that large sample sizes (N > 40) are normally distributed (Ghasemi & Zahediasl, 2012).
Internal Consistency
Excellent internal consistency was found for both the SARQ-pre (α = 0.95) and SARQ-post (α = 0.96).
Construct Validity
Both SARQ-pre and SARQ-post were significantly and positively related to measures of social anxiety, including the BFNE, SIAS and SPS. The subscales on the DASS-21 as well as CSR, also demonstrated a significant positive relationship with the SARQ-pre and SARQ-post. Please see Table 3 for correlations.
Treatment Sensitivity
Evidence of treatment sensitivity was assessed by comparing pre-treatment and post-treatment scores for both the SARQ-pre and SARQ-post. Paired samples t-test found a significant reduction in SARQ-pre scores, t(101), 8.90, p < .001, 95% CI [7.85–12.35]), as well as SARQ-post scores, t(91), 10.64, p < .001, 95% CI [10.449–15.245], following a 12-week group CBT intervention. Please see Table 4 for SARQ-pre and SARQ-post mean scores and standard deviations at pre-treatment and post-treatment, including pre-post Cohen’s d effect sizes. A change score was calculated for the difference between pre-treatment to post-treatment scores for the SARQ-pre, SARQ-post, CSR and SIAS. There were significant correlations between the change in SARQ-pre and SARQ-post, r (91) = 0.25, p = .02, as well as change in SARQ-pre and SIAS, r (98) = 0.37, p < .001. Change in SARQ-pre was not significantly correlated with change in CSR. Similarly, the change in SARQ-post was not significantly correlated with changes in CSR or SIAS (all p’s > 0.06).
Table 3
Correlations between SARQ-pre and SARQ-post with anxiety and related measures
Scale | SARQ-pre | SARQ-post |
|---|---|---|
SARQ-pre | 1.00 | 0.57** |
SARQ-post | 0.57** | 1.00 |
SPS | 0.46** | 0.49** |
SIAS | 0.57** | 0.44** |
BFNE | 0.47** | 0.30** |
SPQ-pre | − 0.50** | NS |
SPQ-post | NS | − 0.67** |
SAR | 0.62** | 0.61** |
DASS 21 Depression | 0.49** | 0.37** |
DASS 21 Anxiety | 0.35** | 0.33** |
DASS 21 Stress | 0.46** | 0.33** |
CSR | 0.28** | 0.35** |
Table 4
Descriptive statistics and pre-post effect sizes for cognitive behavioural therapy
Pre Treatment Mean (SD) | Post Treatment Mean (SD) |
d
| |
|---|---|---|---|
SARQ-pre (n = 102) | 25.64 (12.03) | 15.08 (10.75) | 0.84 |
SARQ-post (n = 92) | 21.78 (13.79) | 8.94 (10.45) | 1.05 |
Social Conversation vs. Speech
At pre-treatment SARQ-pre scores were significantly higher preceding the speech (M = 29.16; SD = 10.63), in comparison to the social interaction (M = 25.67; SD = 12.03), t(133), 4.30, p < .001, 95% CI [-5.10 - -1.90]. In contrast, SARQ-post scores were significantly higher following the social conversation (M = 21.34, SD = 13.40), in comparison to the speech (M = 18.41, SD = 12.14), t(123), 3.82, p < .001, 95% CI [1.41–4.45].
Discussion
Pre- and post-event rumination are two cognitive processes implicated in the maintenance of SAD, which are experienced across a variety of social situations. The current study aimed to extend previous research by assessing the psychometric properties of a widely used measure of negative rumination – the SARQ – in relation to a social conversation. Confirmatory factor analyses indicated that the SARQ was a stable unidimensional construct, and the total score for both SARQ-pre and SARQ-post was recommended. Both SARQ-pre and SARQ-post demonstrated good construct validity, with significant positive correlations with three measures of social anxiety (i.e., SPS, SIAS, and BFNE) and state anxiety immediately following the interaction (i.e., SAR). In addition, there was a significant negative correlation between the SARQ and self-appraisals on the speech task (i.e., SPQ). This indicates that the SARQ captures negative rumination pertinent to individuals with SAD, specifically in relation to a social interaction. The SARQ also demonstrated excellent internal consistency, with no floor or ceiling effects present. There was evidence of treatment sensitivity as the SARQ-pre and SARQ-post scores significantly decreased following a 12-week group CBT treatment. The reduction in pre-event and post-event rumination demonstrated a large effect size. Thus, the findings indicate that the SARQ is a sound psychometric instrument to measure pre- and post-event rumination. Given that the present findings are consistent with those of Donohue et al. (2021) who utilized a speech, we conclude that the SARQ is readily generalisable to a range of social threats pertinent to the experience of social anxiety. Thus, the SARQ offers the unique benefit of being both a pre-event and post-event rumination measure that has been validated in a clinical SAD sample for two different types of social situations.
Interestingly, the observed patterns of rumination preceding and following the two social threats were significantly different. In anticipation of an upcoming speech, participants with SAD demonstrated significantly higher levels of pre-event rumination on the SARQ in comparison to their ratings anticipating an upcoming social interaction. On the other hand, participants with SAD showed significantly higher post-event rumination following the social interaction, in comparison to the speech. That is, people with SAD ruminate more in anticipation of a performance-based speech but experience greater levels of post-event rumination in relation to a social interaction with a stranger. One hypothesis to explain these findings is that the two types of social experiences may differ in the levels of ambiguity that they elicit (Clark & Wells, 1995). That is, social interactions are intrinsically more ambiguous and complex, thus appraisals of perceived failure and negative images of the self are more readily available as there is increased bias that one may have behaved in a potentially unacceptable way (Modini et al., 2018a). Despite being more structured, the speech may be perceived as a ‘higher stakes’ task leading to increased anxiety and threat and associated rumination. Additionally, people with SAD may perceive the threat of negative evaluation from a poor speech as less than that associated with potential rejection from a perceived poor social interaction. The negative outcomes thought to be associated with social rejection appear to be greater, and possibly less recoverable, for people with SAD. This likely relates to the socio-evolutionary perspective of importance of group cohesion and the ability to develop and maintain relationships, so that the individual is not ostracised from the group (Weeks et al., 2011). In the context of a speech, multiple processes including change in state anxiety, threat appraisals (i.e., probability, cost and coping), and performance appraisals have been shown to mediate the relationship between social anxiety and pre-event rumination, whereas only changes in threat appraisals mediate the relationship between social anxiety and post-event rumination (Modini et al., 2018b). This indicates that the factors at play before and after a speech may not identically mirror each other. It is therefore possible that different factors, or a different combination of factors, could potentially mediate the relationship between social anxiety and pre- and post-event rumination in relation to social interactions. Future research should explore the processes mediating social anxiety and negative rumination in the context of a social interaction.
In terms of limitations, though the current study had a large clinical sample of individuals with SAD, there was no control nor non-clinical group included as a comparison to assess clinical utility. This meant that we could not compute group differences to determine if the SARQ exhibited significantly higher scores of pre-event and post-event rumination in comparison to a non-clinical or other-clinical control group in the context of a social interaction. Additionally, we could not provide a clinical cut-off score for the SARQ in relation to a social interaction using Receiver Operating Characteristic (ROC) curve analysis. Future studies should aim to collect community or non-clinical data for the SARQ in relation to a social conversation to compare if there are significant group differences. Furthermore, although we tested for treatment sensitivity and found a large effect size for both pre-event and post-event rumination following CBT, we did not have an active treatment nor waitlist group to compare to the CBT group. Test re-test reliability analyses are also required to determine if SARQ scores in relation to social interaction are stable across time.
Therefore, the SARQ pre-event and SARQ post-event can be utilized as a robust measure of negative rumination in individuals with SAD. It is currently the only measure that has been validated in SAD in two different social tasks. In relation to a social interaction, the SARQ demonstrated a stable unidimensional factor structure, along with good construct validity, excellent internal consistency, and treatment sensitivity. These findings lead us to endorse the use of the SARQ as a measure of rumination preceding and following an anxiety-provoking social conversation task to further understand the process of rumination in SAD.
Complicance with Ethical Standards
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical Approval
All studies were approved by the Macquarie University Human Research Ethics Committee.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.