The development and evaluation of mindfulness-based interventions for a variety of psychological and medical disorders have grown exponentially over the past 20 years. Yet, calls for increasing the rigor of mindfulness research and recognition of the difficulties of conducting research on the topic of mindfulness have also increased. One of the major difficulties is the measurement of mindfulness, with varying definitions across studies and ambiguity with respect to the meaning of mindfulness. There is also concern about the reproducibility of findings given few attempts at replication. The current secondary analysis addressed the issue of reproducibility and robustness of the construct of self-reported mindfulness across two separate randomized clinical trials of mindfulness-based relapse prevention (MBRP), as an aftercare treatment for substance use disorder. Specifically, we tested the robustness of our previously published findings, which identified a latent construct of mindfulness as a significant mediator of the effect of MBRP on reducing craving following treatment. First, we attempted to replicate the findings in a separate randomized clinical trial of MBRP. Second, we conducted sensitivity analyses to test the assumption of the no-omitted confounder bias in a mediation model. The effect of MBRP on self-reported mindfulness and overall mediation effect failed to replicate in a new sample. The effect of self-reported mindfulness in predicting craving following treatment did replicate and was robust to the no-omitted confounder bias. The results of this work shine a light on the difficulties in the measurement of mindfulness and the importance of examining the robustness of findings.