Choose (and use) your tools wisely: “Validated” measures and advanced analyses can provide invalid evidence for/against a theory

Excerpt

Doyle and Mullan’s letter ( 2016) provides an opportunity to clarify some important issues in methods and theory not explicit in Leventhal, Phillips, and Burns ( 2016). Doyle and Mullan ( 2016) claim that (1) Leventhal et al. ( 2016) were biased in their overview of the Commonsense Model (CSM) by not citing the meta-analyses showing weak relationships between certain measures of CSM constructs and measures of treatment adherence (thereby harming the science by not including null results of the meta-analyses); and (2) the lack of meta-analytic support for the CSM means that the theory should be abandoned. Both claims are false for two main, related reasons: first, the meta-analyses were not relevant to the purpose of Leventhal et al. ( 2016) and so were not left out due to bias; second, and more importantly, even if the purpose of Leventhal et al. ( 2016) had been to provide a systematic review of research evidence for the CSM, the meta-analyses cited by Doyle and Mullan do not provide adequate or even valid evidence against the CSM. We will provide three specific arguments in support of these issues after re-stating the goals of the article targeted for criticism. …