Hostname: page-component-7c8c6479df-5xszh Total loading time: 0 Render date: 2024-03-26T13:28:38.361Z Has data issue: false hasContentIssue false

Processing confusing procedures in the recent re-analysis of a cognitive bias modification meta-analysis

Published online by Cambridge University Press:  20 March 2018

Anne-Will Kruijt*
Affiliation:
Postdoctoral fellow, Department of Psychology, Stockholm University, Stockholm, Sweden
Per Carlbring
Affiliation:
Professor, Department of Psychology, Stockholm University, Stockholm, Sweden.
Rights & Permissions [Opens in a new window]

Abstract

Type
Correspondence
Copyright
Copyright © The Royal College of Psychiatrists 2018 

Those worried about the cognitive bias modification (CBM) field being affected by ever-moving goal posts may have thought their concerns confirmed by Grafton and colleagues’ re-analysis of the meta-analysis by Cristea and colleagues.Reference Cristea, Kok and Cuijpers1, Reference Grafton, MacLeod, Rudaizky, Holmes, Salemink and Fox2 The paper concludes with the suggestion that we should only call CBM CBM if it is successful. To provide a treatment-inspired analogue: ‘This? No, this is just water, it's only homoeopathy if it works’.

It seems that we witness an almost prototypical disagreement between experimentalists and treatment evaluationists about which question to ask and which data to include. Importantly, the two author groups appear quite agreed that the answer to the question ‘whether assigning an anxious individual to engage in a CBM procedure will result in direct symptom reduction’ would be ‘not likely’. Perhaps Grafton and colleagues had better direct their critical attention towards the work by ‘field insiders’ in which CBM is quite consistently touted as a treatment, not to mention the apparent push for clinical dissemination and premature commercial exploitation. Thus, the question meta-analysed by Cristea and colleagues, authors specialising in meta-analytical evaluation of (proposed) treatments, appears perfectly legitimate.

Grafton and colleagues’ exposé on the correct question to meta-analyse, reads uncomfortably like a perceived-damage-containing mission. The discomfort is aggravated by the presented re-analysis, applying dichotomising and partly mystifying criteria to distil a subset of eligible studies from those selected for the original meta-analysis. Specifically, the requirements for a study to pass ‘Criterion 3: effect size computed by Cristea et al reflects legitimate emotional vulnerability assessment’ (Table 1, p. 268) remain unknown, as do the rules governing the final dividing criterion ‘intended CBM procedure successfully induced the process of bias modification’ (p. 268).

One could attempt to reconstruct the criterion rules from the tables provided, but it matters little. The analysis by Grafton and colleagues is flawed in a manner that must have escaped the attention of authors, reviewers and editors alike, even after Cristea and colleagues pointed it out in their commentary.Reference Cristea, Kok and Cuijpers3 To be very explicit: Grafton et al meta-analysed the study effect-size estimates calculated by Cristea et al.

In their original paper,Reference Cristea, Kok and Cuijpers1 Cristea and colleagues state clearly that (a) for studies reporting multiple symptom outcomes, these were averaged into a single effect-size estimate (p. 8), and (b) effect-size estimates reflect symptoms assessed post-training, excluding assessments following a stressor procedure (p. 9).

Based on the narrative, it appears that criterion 3 has to do with each study either (a) assessing symptoms on trait (rather than state) measures, yet effect-size estimates averaging across state and trait measures were analysed, or (b) employing a post-training stressor procedure, yet symptoms assessed preceding such stressor procedures were analysed. Surely, we are not to assume reliable retro-active impact of unannounced stressors, nor that excluding studies with state measures only, results in adjustment of state measures retained for other studies. Therefore, we must conclude that this small yet crucial detail has gone unnoticed.

A meta-analysis by Grafton and colleagues, assessing evidence for their hypotheses, could perfectly exist alongside the meta-analysis by Cristea and colleagues. The currently presented re-analysis, however, does not convince.

References

1Cristea, IA, Kok, RN, Cuijpers, P. Efficacy of cognitive bias modification interventions in anxiety and depression: meta-analysis. Br J Psychiatry 2015; 206: 716.Google Scholar
2Grafton, B, MacLeod, C, Rudaizky, D, Holmes, EA, Salemink, E, Fox, E, et al. Confusing procedures with process when appraising the impact of cognitive bias modification on emotional vulnerability. Br J Psychiatry 2017; 211: 266–71.Google Scholar
3Cristea, IA, Kok, RN, Cuijpers, P. Invited commentary on … Confusing procedures with process in cognitive bias modification research. Br J Psychiatry 2017; 211: 272–3.Google Scholar
Submit a response

eLetters

No eLetters have been published for this article.