Figures
Abstract
Objective
The Metacognitions Questionnaire 30 assesses metacognitive beliefs and processes which are central to the metacognitive model of emotional disorder. As recent studies have begun to explore the utility of this model for understanding emotional distress after cancer diagnosis, it is important also to assess the validity of the Metacognitions Questionnaire 30 for use in cancer populations.
Methods
229 patients with primary breast or prostate cancer completed the Metacognitions Questionnaire 30 and the Hospital Anxiety and Depression Scale pre-treatment and again 12 months later. The structure and validity of the Metacognitions Questionnaire 30 were assessed using factor analyses and structural equation modelling.
Results
Confirmatory and exploratory factor analyses provided evidence supporting the validity of the previously published 5-factor structure of the Metacognitions Questionnaire 30. Specifically, both pre-treatment and 12 months later, this solution provided the best fit to the data and all items loaded on their expected factors. Structural equation modelling indicated that two dimensions of metacognition (positive and negative beliefs about worry) were significantly associated with anxiety and depression as predicted, providing further evidence of validity.
Citation: Cook SA, Salmon P, Dunn G, Fisher P (2014) Measuring Metacognition in Cancer: Validation of the Metacognitions Questionnaire 30 (MCQ-30). PLoS ONE 9(9): e107302. https://doi.org/10.1371/journal.pone.0107302
Editor: Una Macleod, Supportive care, Early DIagnosis and Advanced disease (SEDA) research group, United Kingdom
Received: February 20, 2014; Accepted: August 11, 2014; Published: September 12, 2014
Copyright: © 2014 Cook et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This research was conducted as part of a Population Health Scientist Fellowship (Ref G0802425) funded by the Medical Research Council. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Metacognition refers to the knowledge, beliefs and cognitive processes involved in the monitoring, control and appraisal of cognition [1], [2]. The metacognitive model of psychological disorder [2], [3] states that emotional distress is maintained by maladaptive and prolonged patterns of thinking (such as persistent worry or rumination) which are activated and driven by underlying metacognitive beliefs. Two types of metacognitive belief are thought particularly important: positive beliefs about the benefits of specific strategies for coping with distressing thoughts and feelings (e.g. worrying will help me cope); and negative beliefs about the danger and uncontrollability of perseverative thinking (e.g. my worrying is uncontrollable). Positive beliefs about the value of worrying and rumination are thought to activate use of these strategies as a means of regulating emotion and cognition. These strategies become pathological when negative metacognitive beliefs are also activated so that worry or rumination itself becomes the focus of negative appraisal – causing additional worry about worry (meta-worry). In addition, negative beliefs about the need to control thinking may lead to attempts to suppress unwanted thoughts or worries, which typically has a paradoxical effect, increasing their salience and intensifying emotional distress. A second important component of metacognition for understanding emotional distress is the cognitive processes that control and monitor cognition. In particular, the metacognitive model suggests that increased use of selective attention to, and monitoring of, cognition leads to unwanted thoughts and feelings becoming more salient [4]. A recent meta-analysis [5] concluded that Metacognitive Therapy, which challenges metacognitive beliefs, is an effective intervention for both anxiety and depressive disorders. Such findings provide clear support for the value and importance of the metacognitive model for understanding the maintenance of emotional distress. The Metacognitions Questionnaire (MCQ) was developed by Cartwright-Hatton and Wells [6] to explore the metacognitive dimensions that are central in the metacognitive model of emotional disorder. The initial 65-item, questionnaire (MCQ-65) consisted of five subscales based on factor analyses, three of which assess beliefs, including: ‘Positive beliefs about worry’; ‘Negative beliefs about the danger and uncontrollability of worry’; and ‘negative beliefs about thoughts in general’. The remaining two subscales assess the tendency to focus on cognitive events, ‘Cognitive self-consciousness’; and confidence in cognitive abilities, particularly memory and attention, ‘Cognitive confidence’. The MCQ-65 uses a four-point Likert response scale: 1 (do not agree); 2 (agree slightly); 3 (agree moderately); 4 (agree very much).
However, despite excellent psychometric properties (see Wells [1] for a review), the usefulness of the MCQ-65 was compromised by its length; consequently a shorter 30-item version was developed [7]. This MCQ-30 retained the factor structure and the response scale of the longer measure, with six items selected to represent each metacognitive dimension on the basis of highest factor loading and item clarity in previous studies.
Initial psychometric properties of the MCQ-30 were found, in a sample of 182 student and community participants, to be broadly similar to those of the longer measure [7]. Internal consistency of the subscales ranged from an adequate 0.72 to an excellent 0.93 with adequate test-retest reliability for four out of five subscales (ranging from r = 0.59 ‘Negative beliefs about worry’ to r = 0.87 ‘Cognitive self-consciousness’). Confirmatory and exploratory factor analysis confirmed an acceptable fit of the original five factor model with most items loading on their predicted factors except in the case of ‘Need to control thoughts’ where only three out of six items loaded significantly. In addition, all five subscales were significantly and positively correlated with measures of worry (Penn State Worry Questionnaire, PSWQ [8]) and Trait anxiety (Stait - Trait Anxiety Inventory, STAI [9]) with the subscale ‘Negative beliefs about worry’ showing the strongest associations. Further studies have since assessed the psychometric properties of the MCQ-30 in mixed student and community samples in the UK [10] and Turkey [11]. In both cases the original five factor structure was replicated and positive correlations demonstrated with theoretically appropriate measures of worry (PSWQ), anxiety and depression.
Recently, interest has grown in applying the metacognitive model to understanding emotional distress in cancer [12], [13]. Thewes et al [13] used the MCQ-30 to explore for the first time the association of metacognitive beliefs with Fear of Cancer Recurrence (FCR) among young women with early stage breast cancer. They found that the subscale ‘Negative beliefs about worry’ was the most highly correlated with FCR and that the MCQ-30 total score accounted for 36% of the variance in this outcome, leading them to conclude that maladaptive metacognitions play an important role in FCR. However, caution is warranted in the interpretation of such findings because without formal psychometric testing we do not yet know how the MCQ-30 operates in a cancer population.
Consequently, the current study aims to explore for the first time the validity of the MCQ-30 in cancer. The primary aim is to explore whether the established 5-factor structure of the MCQ-30 is valid in this population and to investigate the internal consistency of its subscales. A second aim is to explore whether the theoretically expected associations between specific subscales of the MCQ-30 and anxiety and depression demonstrated in previous research (Wells & Cartwright-Hatton, 2004; Spada et al, 2008; Yilmaz et al, 2008) are replicated, thus providing evidence of concurrent validity in this population. As the association of metacognitive beliefs with emotional distress in cancer had not been investigated before this study, this analysis was exploratory with only one a priori hypothesis: that the subscale ‘Negative beliefs about worry’ would be the main predictor of variance in both anxiety and depression, as this relationship has been consistently documented in mental health [1] physical health [14] and student and community populations [10], [11].
Methods
Ethics statement
This research was approved according to UK guidelines, by the NHS North West 5 Research Ethics Committee (reference: 09/H1010/70). There are no conflicts of interest to be declared.
Participants
Participants were recruited from patients at least 18 years old attending routine pre-treatment clinics at a National Health Service (NHS) teaching hospital, after receiving a diagnosis of primary non-metastatic breast or prostate cancer. Patients were excluded if they had recurrent or metastatic disease, or were considered by the clinical team or researcher to be too distressed or confused to give informed consent.
Measures
The Metacognitions Questionnaire 30- (MCQ-30) [7] assesses metacognitive beliefs and processes. It comprises five subscales: ‘Positive beliefs about worry’; ‘Negative beliefs about worry’; ‘Cognitive confidence’; ‘Need to control thoughts’; and ‘Cognitive self-consciousness. For each subscale, six items are scored 1–4, yielding total scores of 6 to 24. High scores indicate, respectively, more positive and negative beliefs about worry, reduced confidence in memory, greater belief in the need to control thoughts and an increased tendency towards self-focussed attention. The MCQ-30 has excellent internal consistency and good convergent and predictive validity in normal populations [7], [10], [11].
The Hospital Anxiety and Depression scale (HADS) [15] was used to assess anxiety and depression. The HADS is a well-established measure of emotional distress specifically developed for use in physically ill populations. Fourteen items are scored on a 4-point scale yielding two subscale scores of 0-21 with high scores indicating great anxiety or depression. The HADS has been extensively validated for use in cancer [16], [17] and is one of the most widely employed measures of anxiety and depression symptoms in this population.
Procedure
Data for this study was collected as part of a larger prospective study exploring the association of metacognitive beliefs with emotional distress after cancer [18]. Suitable participants were identified by clinic staff, who gave them recruitment letters and information sheets for the study along with their appointment letters for routine pre-treatment consultations and explained that participation in the research was entirely voluntary. When patients attended the clinic, those willing to see the researcher were given further information and asked for written consent. Participants were asked to complete the study questionnaires in clinic and were given the choice of electronic (hand-held PC) or paper formats. Those unable to complete the questionnaires in clinic took a copies (paper version) home and returned them by post. Twelve months later participants were mailed a second questionnaire pack which they completed and returned by post.
Data analysis
To explore the validity of the MCQ-30 over time and under different circumstances, the data were analysed separately for both time points (pre-treatment & 12 months later).
Construct validity of the MCQ-30 was first assessed using Confirmatory Factor Analysis (CFA) to test the published five-factor measurement model. As the primary aim of this study was to assess validity rather than achieve the best possible model fit, the decision was taken not to make minor modifications to the model based on the data (unless strongly supported by theory) as such modifications often just reflect idiosyncratic characteristics of the sample [19]. Instead, Exploratory Factor Analysis (EFA) was used to explore whether an alternative model would be more appropriate for this sample. Both sets of analyses (CFA and EFA), were performed in Mplus version 6.12 [20], using the robust weighted least squares estimator (WLSMV [21], [22]) recommended for ordinal categorical data [23]. The EFA tested models up to and including a five-factor structure without dictating where items should load. As previous studies identified MCQ-30 subscales as inter-correlated, an oblique rotation (Geomin) was used to establish the optimum pattern of item loadings. For both analyses (CFA & EFA), adequacy of model fit was assessed based on two incremental fit indices: the Comparative Fit Index (CFI); and the Tucker-Lewis Fit Index (TLI), with values close to 0.95 indicating a well-fitting model [24], and two absolute misfit indices: the Root mean Square Error of Approximation (RMSEA) with values <.05 indicating good fit and 0.5– .08 adequate fit [25]; and the Weighted Root Mean Square Residual (WRMR) with values less than .95 indicating good fit [26]. For the EFA the Standardised Root mean Square (SRMR) was used, instead of the WRMR, with values <.05 indicating good fit. Inter-correlations amongst the five latent factors of the published model were examined and the internal consistency of each subscale assessed using Cronbach's alpha.
Concurrent validity of the MCQ-30 was then assessed (at each time point) by fitting the data to a structural model in which latent variables for anxiety and depression (each indicated by their seven constituent HADS items), were regressed onto the MCQ-30 factors. Adequacy of model fit was again assessed using the fit indices described above. As the MCQ-30 and HADS subscales were not normally distributed and the study sample relatively small, bootstrapping techniques were used to test the robustness of the findings.
Results
Sample characteristics for the participants at each time point are shown in Table 1.
Factorial Structure
Confirmatory factor analysis of the MCQ-30 five factor model showed overall a marginally adequate fit of the model to the data at the pre-treatment assessment: χ2 (395) = 787.448. p<. 01, RMSEA = .066 (90% CI = .059–.073), CFI = .91, TLI = .90, WRMR = 1.218.
Exploratory Factor analysis which, unlike CFA, does not dictate where items should load, confirmed that a five-factor solution nevertheless provided the best model. Moreover, the fit indices (χ2 (295) = 439.692. P<.001, RMSEA = .046 (90% CI = .037–.055), CFI = .97, TLI = .95, SRMR = 0.046) together indicate a good fit of the model to this data. As shown in Table 2, all items loaded >0.4 on their expected factors [7]. However, as the items were allowed to load freely across any factors, minor discrepancies were observed between the EFA–derived solution and the published five factor model. Specifically, two items, MCQ3 and MCQ13, had their highest loadings on factors other than the expected ones. Item MCQ3 loaded higher on ‘Negative beliefs about worry’ (F1) than on its expected factor - ‘Cognitive self-consciousness’ (F4). Item MCQ13 had equivalent loadings on both its expected factors - ‘Need for control over thoughts’ (F5) - and ‘Cognitive self-consciousness’ (F4). Two further items (MCQ5 & MCQ29) also demonstrated significant (>.4) cross-loadings although for both the highest loading remained consistent with the published factor structure.
At the 12 month follow-up, CFA indicated an adequate fit of the data to the published five-factor model: χ2 (395) = 684.184. p<. 01, RMSEA = .060 (90% CI = .053–.068, (p RMSEA<.05<.015), CFI = .95, TLI = .95, WRMR = 1.048. Therefore no Exploratory Factor Analysis was performed.
The Mean and SDs of the five MCQ-30 subscales and the correlations amongst the five latent variables (CFA standardised solution) at both time points are presented in Table 3. The internal consistency of the subscales was assessed using Cronbach's alpha (Table 3) and ranged from 0.73 to 0.89 pre-treatment and from .79 to .91 at 12 month follow-up, indicating adequate to excellent internal consistency. At both time points the subscale with the lowest alpha coefficient was ‘Need for Control’.
Convergent validity
The hypothesised model of the relationship between metacognitive beliefs (using the MCQ-30's published factor structure) and concurrent anxiety and depression is shown in Figure 1. Overall, the fit indices for this latent variable SEM (see Table 4) indicated an acceptable fit of the model. At both time points, ‘Negative beliefs about worry’ explained significant variance in both anxiety and depression and, as hypothesised, was the strongest of all the predictors. ‘Positive beliefs about worry’ also explained variance in anxiety at both time points but not depression. At the pre-treatment time-point ‘Need for control over thoughts’ was associated with fewer symptoms of anxiety, although this association fell just short of significant (p = .057) at the twelve-month follow-up. There was no significant relationship between ‘Cognitive confidence’ or ‘Cognitive self-consciousness’ and anxiety or depression at either time-point.
N.B. Rectangles indicate observed variables on MCQ-30 (MCQ) or HADS (H); ellipses indicate latent factors. Latent factors: Positive beliefs about worry (POS); Negative beliefs about worry (NEG); Cognitive Confidence (CC); Need to control thoughts (NC); Cognitive Self-consciousness (CSC); HADS Anxiety (HADS-A); HADS Depression (HADS-D). Figures show standardised path coefficients and their significance at pre-treatment and (in brackets) 12-month follow-up. Errors not shown; *** p<.001 ** p<.01 * p<.05
Discussion
The present study provides the first evidence to support the published five-factor structure of the MCQ-30 [7] as valid and replicable in a cancer population. Although at the pre-treatment time point CFA showed only a marginal fit, subsequent EFA confirmed that a five-factor solution still provided the best solution. The improved fit observed for the EFA over the CFA was the result of items being allowed to load freely across any of the factors. However, all items still loaded on their expected factors with only minor discrepancies between the two models. At 12-month follow-up, fit was acceptable and comparable to that reported by the measure's developers [7]. It is not clear why the fit should be slightly better at the 12 month follow-up. The mode of administration differed between the two time points with pre-treatment assessments largely being carried out on hand-held PCs while 12 month follow-up were completed on paper. Improved fit at follow-up might therefore arise because the procedure for this assessment is closer to how the questionnaires have been administered during previous validation studies. Equally, the observed improvement in fit could be partly due to the timing of assessments in that the pre-treatment assessment was conducted relatively soon after diagnosis, during a period that is clinically busy and often emotionally turbulent. In contrast, the twelve month follow-up for most is likely to be a more settled time, at least clinically. However, taken together, these CFA and EFA results suggest that the established five factor structure of the MCQ-30 is valid for use in a cancer population and that it remains valid across one year post-diagnosis and changing illness/treatment circumstances. In addition, the results indicate that the subscales possess good internal consistency comparable to those found in previous studies [7], [10], [11].
Two items (MCQ3 & MCQ13) had their highest loadings on a different factor to that expected. However, only one of these loaded higher on that factor; Item MCQ3 (‘I think a lot about my thoughts’) had its highest loading on ‘Positive beliefs about worry’ rather than the expected factor ‘Cognitive self-consciousness’. Both of these items have also been found to cross-load on different factors previously [11] although, in that study, item MCQ3 loaded >0.4 on ‘Negative beliefs about worry’ not on ‘Positive beliefs about worry’ as in the present study.
Preliminary evidence of the measure's convergent validity is provided by the structural equation model of the relationship of the MCQ-30 latent factors with anxiety and depression. As hypothesised, and as shown previously in mental health, physical health, student and community populations, ‘Negative beliefs about worry’ was the strongest predictor of both anxiety ([7], [10], [11], [14] and depression [10], [11]. In addition, ‘Positive beliefs about worry’ predicted anxiety at both time points. However, in contrast, ‘Need for control over thoughts’ was negatively related to anxiety at pre-treatment although this relationship was marginally non-significant at twelve month follow-up. This suggest that participants with lower conviction about the need to control their thinking experience greater anxiety. Such findings are unexpected as previous studies in mental health, student and community samples have indicated that greater belief in the need to control thoughts, predict higher rather than lower levels of anxiety. This result may indicate a difference between this and previously studied mental health, student and community populations. However, further work would be required to establish whether this is a true population difference or just an artefact of the present data.
It is important to note that, by structural equation modelling standards, the study employed a small sample size, which may reduce the stability of the findings. Consequently, further work is required to establish whether the apparent differential item functioning and observed patterns of associations represent real population differences or are idiosyncratic to this data set. In addition, as only breast and prostate cancer patients were included in the study, it remains important to explore whether study findings can be replicated across different cancer diagnoses.
In summary, the current study provides initial evidence that the established five-factor structure of the MCQ-30 is valid for use in a cancer population and that the subscales possess good internal consistency. Positive and negative beliefs about worry were associated with concurrent anxiety and depression as expected, although the negative relationship of anxiety with ‘Need for control over thoughts’ is unexpected and therefore intriguing. Despite the limitations discussed above we conclude from this study that, the MCQ-30 is a sufficiently valid measure for assessing metacognitive beliefs and processes in breast or prostate cancer populations in the first year after diagnosis.
Acknowledgments
We gratefully thank the staff and patients of the Royal Liverpool & Broadgreen University NHS Hospital Trust for supporting this study.
Author Contributions
Conceived and designed the experiments: SAC PS GD PF. Performed the experiments: SAC. Analyzed the data: SAC GD. Wrote the paper: SAC PS PF.
References
- 1.
Wells A (2009) Metacogntive Therapy for Anxiety and Depression. New York Guilford Press. 316 p.
- 2.
Wells A (2000) Emotional disorders and metacognition: Innovative cognitive therapy. Chichester, UK: Wiley.
- 3. Wells A, Mathews G (1996) Modelling cognition in emotional disorder: the S-REF model. Behaviour Research and Therapy 32: 867–870.
- 4.
Wells A, Mathews G (1994) Attention and emotion: a clinical perspective. Hove, UK: Erlbaum.
- 5. Normann N, van Emmerik AA, Morina N (2014) The efficacy of metacognitive therapy for anxiety and depression: a meta-analytic review. Depress Anxiety 31: 402–411.
- 6. Cartwright-Hatton S, Wells A (1997) Beliefs about worry and intrusions: the metacognitions questionnaire and its correlate. Journal of Anxiety Disorders 11: 279–296.
- 7. Wells A, Cartwright-Hatton S (2004) A short form of the metacognitions questionnaire: properties of the MCQ-30. Behaviour Research and Therapy 42: 385–396.
- 8. Meyer TJ, Miller ML, Metzger RL, Borkovec TD (1990) Development and validation of the Penn State Worry Questionnaire. Behav Res Ther 28: 487–495.
- 9.
Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs G (1983) Manual for the stait-trait anxiety inventory. Palo Alto, CA: Consulting Psychology Press.
- 10. Spada MM, Mohiyeddini C, Wells A (2008) Measuring metacognitions associated with emotional distress: Factor structure and predictive validity of the metacognitions questionnaire 30. Personality and Individual Differences 45: 238–242.
- 11. Yilmaz AE, Gencoz T, Wells A (2008) Psychometric characteristics of the Penn State Worry Questionnaire and Metacognitions Questionnaire-30 and metacognitive predictors of worry and obsessive-compulsive symptoms in a Turkish sample. Clin Psychol Psychother 15: 424–439.
- 12. McNicol K, Salmon P, Young B, Fisher P (2013) Alleviating emotional distress in a young adult survivor of adolescent cancer: a case study illustrating a new application of metacognitive therapy. Clinical Case Studies 12: 22–38.
- 13. Thewes B, Bell ML, Butow P (2013) Fear of cancer recurrence in young early-stage breast cancer survivors: the role of metacognitive style and disease-related factors. Psychooncology 22: 2059–2063.
- 14. Allott R, Wells A, Morrison AP, Walker R (2005) Distress in Parkinson's disease: contributions of disease factors and metacognitive style. Br J Psychiatry 187: 182–183.
- 15. Zigmond AS, Snaith RP (1983) The hospital anxiety and depression scale. Acta Psychiatr Scand 67: 361–370.
- 16. Moorey S, Greer S, Watson M, Gorman C, Rowden L, et al. (1991) The factor structure and factor stability of the hospital anxiety and depression scale in patients with cancer. Br J Psychiatry 158: 255–259.
- 17. Vodermaier A, Millman RD (2011) Accuracy of the Hospital Anxiety and Depression Scale as a screening tool in cancer patients: a systematic review and meta-analysis. Support Care Cancer 19: 1899–1908.
- 18.
Cook SA, Salmon P, Dunn G, Fisher P in press.
- 19. MacCallum RC, Roznowski M, Necowitz LB (1992) Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin 111: 490–504.
- 20.
Muthen LK, Muthen BO (1998–2010) Mplus User's Guide. Sixth Edition. Los Angeles, CA: Muthen & Muthen.
- 21. Muthen B (1984) A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika 49: 115–132.
- 22.
Muthen B, du Toit SHC, Spisic D (1997) Robust inference using weighted least squares and quadratic estimating equations in latent variable modelling with categorical and continuous outcomes (Technical Report). Los Angeles: Los Angeles: University of California.
- 23.
Brown TA (2006) Confirmatory factor analysis for applied research. New York: Guildford Press.
- 24. Hu L-T, Bentler BM (1999) Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6: 1–55.
- 25.
Browne MW, Cudeck R (1993) Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park, CA: Sage. pp.136–162.
- 26.
Yu C-Y (2002) Evaluating cutoff criteria of model fit indices for latent variable models with binary and continuous outcomes [Doctoral dissertation]. Los Angeles: University of California.