Implementation Reporting Recommendations for School-Based Mindfulness Programs

Our set of recommendations are derived from a review of implementation standards used in research with youth and in complex social settings, as well as an examination of standards developed for research in healthcare, clinical, and educational settings. In the past two decades, scientists have endorsed the use of rigorous reporting standards for disseminating the results of clinical trials. One of the earliest efforts to create a set of these reporting standards came from the Consolidated Standards of Reporting Trials (CONSORT;Altman et al., 2001; Moher et al., 2001) which provided guidance on reporting methods and outcomes for randomized controlled trials. The CONSORT guidelines have been extremely influential and are commonly required by top-tier journals. Other more recent efforts have sought to create reporting standards for a broader range of study designs. For instance, Hoffman et al. (2014) developed the Template for Intervention Description and Replication (TIDieR)—a checklist that provides more generic and comprehensive reporting guidelines for an array of study designs. Similarly, other groups in the field of education have developed broadly applicable reporting standards (e.g., Standards for Reporting on Empirical Social Science Research in AERA Publications; American Educational Research Association, 2006).

Taken as a whole, these reporting standards are intended to create accurate, transparent, consistent, and complete records of study procedures, which allows for readers to make informed decisions about a study’s scientific merits, supports the building of a rigorous evidence base, facilitates comparisons across studies, and aids practitioners in their implementation of new programs (Montrosse-Moorhead & Griffith, 2017). Requiring reporting standards also nudges researchers toward considering and reporting on study attributes that they may otherwise ignore (e.g., participant characteristics or contextual factors)—opening new avenues of scientific investigation through increased attention to and awareness of previously underrepresented participants or communities. Reporting standards also facilitate replication trials, as full descriptions of procedures are made transparent to independent investigators. Finally, reporting standards can help reviewers and journal editors to establish consistent standards and expectations of what needs to be reported on in studies.

We drew upon four reporting tools that were most aligned and suited for our aims of designing and developing a set of reporting recommendations for SBMPs: (1) the JARS (Appelbaum et al., 2018; Levitt et al., 2018); (2) the Standards for Reporting Implementation Studies (StaRI) (Pinnock et al., 2017); (3) the Oxford Implementation Index (Montgomery et al., 2013), and (4) the TIDieR (Hoffmann et al., 2014). Two derivatives of the TIDieR were also examined. One iteration emphasizes reporting on specific contextual factors for applied health contexts and documenting heterogeneity of interventions in non-therapeutic settings (Cotterill et al., 2018). Another iteration offers specific reporting recommendations for studies of mindfulness-based programs (MBPs) with adults in therapeutic settings (Crane & Hecht, 2018). Lastly, we utilized a CC approach for the design of our reporting recommendations derived from an implementation reporting framework for studies of programming conducted in youth contexts (Dymnicki et al., 2020).

The reporting formats of the aforementioned tools inform the present recommendations. For instance, the StaRI recommends that researchers report on and provide more detailed descriptions of contextual factors and adaptations made to the intervention based on specific local needs—an approach that lends itself to research being conducted in complex and dynamic school settings. The StaRI also provides guidance for reporting the “logic pathway”—also referred to as the logic model, theory of change, or cause-and-effect diagram—which involves detailing how the implementation strategy will guide the intervention delivery and the mechanisms through which the intervention is expected to produce outcomes (Pinnock et al., 2017).

Additionally, TIDieR and its iterations informed several recommendations reported herein (Cotterill et al., 2018; Crane & Hecht, 2018; Hoffmann et al., 2014). Crane and Hecht (2018) adapted the TIDieR guidelines specifically for researchers studying MBPs with adult populations. This pioneering work focused mostly on implementation elements central to teaching integrity. For instance, they recommend that researchers report on teacher competence (using the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC); Crane et al., 2013; Crane & Kuyken, 2019) and teachers’ training in mindfulness-based practices, as well as their adherence to the norms of good practice for teaching mindfulness.

Given the context of SBMPs, Crane and Hecht’s (2018) suggestions are of great use, but are not fully applicable to educational settings for several reasons. MBPs with adults (1) have quite different intervention and therapeutic goals than SBMPs, (2) rely on voluntary participation or those seeking help, and (3) depend solely on trained teachers or facilitators to implement the program in a relatively controlled setting. Conversely, SBMPs are often universally administered (by external facilitators, classroom teachers, or both) to all students as part of the standard curriculum. A recent review of studies of SBMPs found only a small percentage of studies (11%) were of programs administered to a targeted subgroup of students (Roeser et al., 2022a). SBMPs are also implemented with participants at various stages of development and their developmental needs inform the type of programming offered, the program goals, and approaches to program implementation. External facilitators and teachers also deliver SBMPs within complex and uncontrolled school settings that are hierarchically nested contexts (e.g., students, classrooms, schools, districts, communities). Thus, the conditions and participants within each level of these complex social settings influence how a program is implemented and the types of supports that are offered to support implementation. Finally, implementing and studying SBMPs requires buy-in from teachers, staff, administrators, parents, and community members—a factor that is often not a consideration with MBPs carried out with adults (Wilde et al., 2019).

Despite the many upsides of the various reporting standards described above, they fail to account for unique considerations vital to SBMP research. There are important elements in the educational context that are critical to understanding the relationship between methods employed and inferences concluded, both of which are highly dependent on the nature of the phenomena under investigation and the context in which such study occurs. Research investigating SBMPs includes such phenomenological (mindfulness-based) and contextual (school systems) factors that were not captured in any of the existing reporting standard guidelines.

To guide our set of reporting recommendations, we examined reviews and articles of SBMPs that focused on implementation (Dariotis et al., 2017; Emerson et al., 2020; Espil et al., 2020; Gould et al., 2016; Meixner et al., 2019; Montero-Marin et al., 2022; Rempel, 2012; Tudor et al., 2022; Wilde et al., 2019), as well other reviews of SBMPs that offer suggestions about SBMP implementation in their discussion sections (e.g., Felver et al., 2016; Roeser et al., 2022a). In particular, we prioritized two reviews that focused on SBMP implementation, as opposed to SBMP outcomes (Emerson et al., 2020; Gould et al., 2016). These two systematic reviews highlight several areas that are inconsistent and under-reported in SBMP studies. Based on our examination of the extant literature, six main areas emerged warranting greater attention and more consistent and complete reporting: (1) reporting and providing detail on multiple implementation elements; (2) using consistent implementation terminology; (3) providing a study theory of change, especially as it relates to Hypothesized and/or Validated CCs and operationalizes mindfulness in the theory; (4) reporting on teacher training and competence; (5) reporting on contextual factors; and (6) reporting on participant characteristics and experiences—both those delivering and those receiving the SBMP.

Generally, studies of SBMPs offer little detail and provide inconsistent information about implementation, and also use different terminology and approaches to assess implementation, making it difficult to draw comparisons across studies (Emerson et al., 2020; Gould et al., 2016; Roeser et al., 2022a; Tudor et al., 2022). For example, in one review of SBMP studies, fewer than 20% of studies assessed aspects of implementation beyond dosage—components such as participant responsiveness, uptake, and integrity were significantly underreported (Gould et al., 2016). There also exists terminological confusion regarding implementation. For example, in some studies, dosage is used to refer to the number of sessions delivered, while in others it refers to the number of sessions that participants attended (see Tudor et al., 2022).

Most studies of SBMPs fail to outline Hypothesized and/or Validated CCs in their theory of change for the program of study, as well as fail to operationalize mindfulness in this theory. SBMPs often involve multi-component programming and varying approaches to program structure and delivery, which necessitates the need to identify Hypothesized CCs to ultimately understand and empirically validate which implementation elements are driving the effects observed (see Felver et al., 2022). One review of SBMP studies that focused on implementation found only 10% of studies articulated Hypothesized CCs and only 6% referenced a logic model or theory of change (Gould et al., 2016). Another review of SBMP studies found that only 13% of studies outlined the relationship between elements of the mindfulness intervention and the outcomes assessed (Felver et al., 2016). Furthermore, studies of SBMPs offer disparate definitions of mindfulness—no agreed-upon definition currently exists in the field—and often researchers do not operationalize or include mindfulness in their theory of change, making it difficult to discern how mindfulness might generate outcomes or which core mindfulness competencies under the broad umbrella of mindfulness (e.g., self-awareness, non-judging) are of focus (see Felver et al., 2022).

Studies of SBMPs also involve a range of facilitation approaches (e.g., external facilitator, trained teacher, combination) and lack consistency, transparency, and detail regarding the reporting of teacher and facilitators’ training and competence to deliver SBMPs. Roeser et al. (2022a) found a range of facilitation approaches used with SBMPs, whereby 50% were administered by external facilitators, 41% were administered by trained classroom teachers, and 7% were administered by a combination of classroom teachers and external facilitators. Very few studies or reviews of SBMPs have examined the ways in which these different administration approaches impact outcomes. A recent review of SBMPs found no studies that specified in sufficient-enough detail about the teacher training (Emerson et al., 2020), as was determined by criteria for teacher training and competence used to assess MBPs with adults (Crane et al., 2013, 2017). This same review also revealed that only 6% of studies used and/or reported an assessment of teacher competence. However, some of the most recent studies of SBMPs have started to provide more detailed reports of teacher training and competence using these criteria, as well as begun exploring the relationship between teacher/facilitator factors and program outcomes (Crane et al., 2020; Montero-Marin et al., 2022). Since facilitator competency has been linked to program outcomes (Atkinson & Wade, 2015), this is an area in need of further and more consistent documentation.

Finally, our current examination of the literature found incomplete reporting of contextual elements, participant characteristics, and participant responsiveness, as well as vague descriptions (if any) of the relationships between these elements and the program’s implementation and its outcomes (Emerson et al., 2020; Felver et al., 2016). For example, in one review, 71% of studies failed to provide information about participant or community socio-economic status (SES) or students with identified disabilities (Felver et al., 2016). A more recent review found that 43% of studies did not report on the SES of the participating students or the local community (Roeser et al., 2022a). In addition, SBMP studies have largely failed to collect in-depth assessments of participant responsiveness and experiences, critical aspects of implementation that have been shown to contribute to intended outcomes (e.g., Monteiro, 2020; Roeser et al., 2022a). In their review of SMBP studies, Tudor et al. (2022) found only one study that explored the relationship between participant responsiveness and outcomes (Metz et al., 2013). As the field moves toward larger-scale studies, school-related factors will be important to consider (e.g., teacher buy-in, psychological supports, existing programming) as it may have significant implications for program design, delivery, and outcomes. Without consistent and complete reporting of these contextual and participant factors, it will remain unclear for whom SBMPs work best and under which conditions.

The extensive effort to create implementation reporting standards in other fields has generated substantial benefit in terms of improved reporting and study quality (Montrosse-Moorhead & Griffith, 2017). Similarly, we hope the creation of implementation reporting recommendations for SBMPs will increase the rigor and interpretability of evidence in the field. In this next section, we offer our conceptual framework of SBMP implementation elements and the corresponding key terms that guide and inform a set of implementation reporting recommendations specifically for studies of SBMPs. This present work encourages researchers to identify Hypothesized CCs of SBMPs and to incorporate them into their theories of change—linking implementation elements with outcomes—and measurement strategies. In addition, the recommendations shed light on implementation elements requiring more attention and complete reporting in studies of SBMPs, such as school context, participant characteristics, teacher training and competence, and developmental considerations for implementing SBMPs with students across the PreK-12 spectrum.

The goal of this work is to propose an inclusive conceptual framework to foster a common approach for conceptualizing implementation and reporting on SBMP implementation elements. We refer to the framework as the SBMP Implementation Framework (SBMP-IF; see Fig. 1). The SBMP-IF is broad and designed to provide an overarching set of categories and constructs to inform our reporting recommendations. It is not meant to be exhaustive; rather, it focuses on those implementation elements that are most relevant to the field of SBMP research. To inform the development of the framework, we drew from the school-based prevention literature (Domitrovich & Greenberg, 2000; Domitrovich et al., 2008), literature on the implementation of mindfulness programs (Broderick et al., 2019; Crane & Hecht, 2018; Emerson et al., 2020; Dariotis et al., 2017; Espil et al., 2021; Gould et al., 2016; Meixner et al., 2019; Monteiro, 2020; Montero-Marin et al., 2022; Rempel, 2012; Tudor et al., 2022; Wilde et al., 2019), and the implementation science literature (Berkel et al., 2011; Blase & Fixsen, 2013; Durlak & DuPre, 2008; Dymnicki et al., 2020). These categories and constructs are not necessarily distinct nor are they completely hierarchical in nature; however, they provide guidance for reporting on the implementation elements of SBMPs that might help to eventually identify CCs (see Dymnicki et al., 2020). Indeed, the SBMP-IF is intended to be of heuristic value to clarify and support more consistent and complete reporting, measurement, and testing of SBMP elements, thereby better identifying factors that lead to, mediate, and/or moderate outcomes.

Additionally, we offer a set of corresponding key terms and definitions (see Table 1) found both in the SBMP-IF and in the reporting recommendations. By explicitly establishing this implementation terminology, it is our hope to create a common language related to SBMP implementation that provides greater consistency in reporting and enhances comparability across studies. We encourage researchers and practitioners to adopt this language and/or provide explanation for deviating from it.

The SBMP-IF framework (Fig. 1) focuses on SBMP Implementation Elements, which we define as the key elements of SBMPs that, when specified, allow researchers to identify the CCs of the program and discern for whom and under what conditions the program works. The SBMP implementation elements encompass all elements that are important to consider when developing, implementing, and studying SBMPs. Next, we have conceived of four broad implementation categories of SBMP implementation elements: the program (as designed to be implemented), participants, context, and implementation (the program as it is actually delivered and received). These four categories adapted from Dymnicki et al. (2020) encompass the myriad of elements that influence and are involved in SBMP implementation. Within these four broad categories, we describe sub-dimensions to provide further guidance for conceptualizing and reporting on implementation. For definitions of the categories and sub-dimensions outlined below, refer to Table 1. For further detail on the reporting recommendations related to these categories and sub-dimensions, refer to the full reporting recommendations provided in Table 2.

The full set of reporting recommendations for SBMP Implementation Elements can be found in Table 2. The items listed as “recommendations” in the table (see column, “Recommended Reporting Items”) are central and should be included somewhere in the write-up and reporting of study findings, while those that are “suggestions” (see column, “Supplementary Reporting Suggestions”) are illustrative suggestions to enhance further rigor and/or specificity. The supplementary items included depend on the kind of study being conducted, context within which the research occurs, and the stage of the research. The recommendations are meant to be flexible enough that they can be applied to quantitative, qualitative, and mixed-methods studies.

The SBMP-IF (Fig. 1) guided our decisions for organizing and including various recommendations and suggestions. The reporting recommendations are categorized into sections that correspond to the general organization and formatting of peer-reviewed journal publications, like that of the JARS and StaRI reporting standards. This organization provides consistent and complete reporting at the publication stage. Also, beginning the research design process with the publication in mind encourages researchers to consider implementation elements and implementation reporting at the very start of the study design process. These recommendations can inform and guide the study design, implementation, analytic plan, and reporting processes.

On the surface, it might appear that the recommendations are at times redundant; however, the intention is to encourage researchers to report on implementation elements thoroughly and consistently throughout their publications. Take for example, the case of reporting on CPCs and the ISS. In the introduction section of a paper, we recommend that researchers clearly articulate the Hypothesized CPCs (eventually Validated CPCs) and the ISS elements. Then, later in the measures section, researchers are encouraged to measure and report program integrity in relation to CPCs. And finally, in the results section, we suggest that researchers also analyze outcomes in relation to Hypothesized and/or Validated CPCs (see suggested reporting in Table 2). In essence, researchers should consider CPCs throughout the design, implementation, and, subsequently, the reporting phases of a study. This grounds the documentation of what was intended to happen, and what actually occurred in relation to Hypothesized and/or Validated CPCs and the theoretical rational or theory of change of the study. The recommendations facilitate a focus on the outlined theoretical or empirical rationale and nudge researchers to address the ways in which implementation meets or fails to meet integrity to Hypothesized and/or Validated CCs.

Many of the recommendations offered are important for any study publication; however, there are several recommendations specific to SBMPs. For one, studies of SBMPs have been inconsistent in their definitions of mindfulness (or the contemplative area of focus), as well as not situated or considered mindfulness or mindfulness competencies in their theories of change (see Roeser et al., 2022b). To develop a better understanding about the role of mindfulness in influencing outcomes, researchers should explicitly define and operationalize this construct in their theory of change for students of a specific age. This recommendation encourages researchers to document how mindfulness, or the specified contemplative area of the study, addresses the articulated problem of focus. More consistent and complete reporting will promote greater understanding of how mindfulness is defined and measured, as well as support measurement and analytic efforts across studies. It may even help to move the field toward a more unifying definition of mindfulness that takes into account developmental considerations (see Roeser et al., 2022b).

The provided recommendations highlight the need to report on developmental considerations and how they inform the program itself and the theory of change. For instance, many SBMPs are derived from adult programming (e.g., MBSR and MBCT), whereby adult programming is adapted and therefore may not be developmentally appropriate or engaging for children or youth. In fact, Roeser et al. (2022a) found that 52% of SBMPs included in their review were adapted from adult programming. If SBMPs are to be conducted in the unique context of school settings with youth at different developmental stages, more reporting is needed to identify the programmatic aspects that are developmentally unique/appropriate for students of different ages insofar as they target specific elements of mindfulness or mindfulness competencies (Roeser et al., 2022b).

The recommendations are also intended to support the navigation of the multifaceted nature of implementation in complex social settings like schools. These recommendations nudge researchers toward reporting on contextual elements at both the school and broader levels, facilitating a more ecological approach to reporting (Roeser et al., 2021). Researchers are encouraged to collect as much school-level data as might be relevant to their study, detailing data on representativeness and documenting school characteristics that might pertain to implementation (e.g., principal support, teacher attitudes and readiness). Additionally, gathering school, classroom, and community-level data can support greater understanding of implementation and the factors driving program outcomes. For instance, researchers can collect quantitative and qualitative data to report on factors like relational trust, which have been shown to affect program implementation (Mischenko et al., 2022).

We developed a set of implementation reporting recommendations for SBMPs to provide more consistent and complete documentation of both the implementation design (what was intended) and the implementation delivery (what actually happened), as well as the factors that impact delivery of SBMPs. The development of reporting recommendations can support greater consensus about the Core Components (CCs) of SBMPs, improve assessment of whether a program was implemented well enough to anticipate relevant participant outcomes (Gould et al., 2016), and inform for whom and under what conditions SBMPs work. In the recent My Resilience in Adolescence (MYRIAD) trial—the largest randomized controlled trial of an SBMP—researchers explored a range of implementation factors and analyzed their impact on program outcomes (e.g., set a priori benchmarks related to student dosage, explored the effects of fidelity, quality, and student practice on program outcomes) (Montero-Marin et al., 2022). Collecting this type of data can help to draw conclusions about the factors impacting and/or driving program outcomes. Studies like this one help to increase understanding about implementation in the field of SBMPs, however, without reporting recommendations and consensus on a priori benchmarks for elements like QOI or practice, reporting will happen in a non-systematic and inconsistent manner. Thus, in developing these reporting recommendations, the intention is that researchers in the field of SBMPs will be more likely to attend to a wide range of implementation elements in the design, implementation, and reporting stages of research and will do so in a consistent manner—an effort that we hope will be reinforced through heightened journal expectations related to reporting. Practitioners and policymakers can also gain further insight about these programs, as they will be equipped with more information about implementation and the program itself, which can then inform decisions about selecting and implementing SBMPs for specific contexts and sets of participants.

Many of these reporting recommendations are aspects of implementation reporting that have been unaddressed in the early years of the study of SBMPs. As such, much more attention to and reporting on teacher training and competence, participant characteristics, responsiveness and uptake of mindfulness practices, and the school and broader level contexts is necessary for understanding findings and informing future research—reporting that involves refinement, consensus, and standardization of the measures used to assess these implementation elements. Furthermore, significantly more developmental consideration is needed when it comes to program adoption and implementation, as well as measurement of mindfulness and related constructs (see Roeser et al., 2022b).

With regard to measurement, following these recommendations and engaging in more detailed implementation reporting will likely require the development of new measures to capture a broader array of implementation elements. As such, we recommend starting with the development of a clear theory of change that incorporates implementation, as this focus will influence what is measured and tested. To guide this process, researchers of SBMPs might find it helpful to utilize the CORE process model (Gould et al., 2016) as a resource for supporting rigorous measure development from feasibility studies to full program scale-up efforts. Additionally, interdisciplinary collaborations with researchers from clinical, educational, and developmental science backgrounds may afford strengths for developing Common Implementation Measures that attend to the contextual features of life in schools and can be used across studies of different SBMPs (Gould et al., 2016). One such example of a Common Implementation Measure is the TMEOS (Broderick et al., 2019)—a measure that has been used to assess quality of mindfulness instruction and teacher competency in K-12 classroom settings.

It may seem daunting to collect the contextual data outlined here, especially for those unfamiliar with school contexts. So, we recommend collecting and reporting on these data as an ideal to be attained, recognizing that schools offer varying ranges of data and data access. For example, demographic and attendance data are typically collected by schools and easily obtainable. Other characteristics and data may require additional partnership and collaboration with participating school systems. Building trusting relationships with these school systems can facilitate data access and collection, bolstering both the scientific validity of the outcomes reported and the contextual relevance (Bryk et al., 2011).

These reporting recommendations can support the study of SBMPs in underserved and marginalized communities—an often under-studied context in SBMP research—as they encourage more detailed reporting on participant characteristics and school contexts. Recent reviews of studies of MBPs with adults and SBMPs with youth have shown a lack of focus and representation of participants from underserved and marginalized populations (Eichel et al., 2021; Roeser et al., 2022a). To remedy this, we see more complete reporting and attention to the collection of participant voices and data (e.g., participant engagement and receptivity to programs, data on potential harms, attention to identity, agency, and belonging) as important avenues for understanding the reception of programs and for identifying ways to adapt them to be more sensitive and responsive to the diverse needs of marginalized communities. Careful articulation of specific implementation elements can shed light on the effects of SBMPs for different groups of students and various school contexts (e.g., Luthar et al., 2020). For instance, it could be that certain implementation elements drive effects in more affluent and privileged communities, while other elements may influence implementation and outcomes for underserved and marginalized communities. Taking these steps, the field of SBMP research has the potential to become more “transformative” in its approach—helping to interrupt the “reproduction of inequitable educational environments” (Rivas-Drake et al., 2021, p. 1).

There are limitations to our approach. For one, the review of reporting tools in other behavioral science fields and the review of the SBMP literature were not exhaustive. As such, there are potential resources and approaches that we did not explore. Secondly, the set of recommendations offered may not be fully comprehensive (e.g., certain methodological considerations might not have been included; see Roeser et al., 2022b). Thirdly, this work centered around SBMPs, which we defined (based on conventions in the broader field of mindfulness research) as implementation of any program implemented within a school context (PreK-12) with a central feature of mindfulness and/or contemplative principles and practices. However, this is a poorly operationalized definition of what actually constitutes an SBMP as it includes any programming that self-identifies as teaching mindfulness and/or contemplative principles or practices despite significant heterogeneity in the actual programming (Felver et al., 2016). Similar to past efforts that brought operational specificity to the field of SEL programming research (Weissberg et al., 2015), future research should carefully consider identified constituent domains and core mindfulness competencies of MBPs for youth (Felver et al., 2022) to support understanding of the effects and processes of SBMPs. Finally, the recommendations have yet to be pilot tested, which limits the claims that can be made regarding their simplicity, practicality, and usability. One future research effort might involve a consensus-building effort with a larger audience of peers who are experts in the fields of SBMPs and implementation (i.e., a Delphi study) to adapt and solidify these recommendations into a set of standards, building off the work of Felver et al. (2022). We hope studies will adopt these reporting recommendations to generate new learnings about implementation of SBMPs and identify advantages and disadvantages of using the recommendations.

With the proliferation of SBMPs, more consistent and complete reporting of implementation is needed to strengthen the evidence base and determine the Core Components (i.e., of the program and the surrounding support system), as well as discern for whom and under what conditions these programs work (Roeser et al., 2022a). The set of recommendations offered here are the first proposed in the field of SBMPs. As such, they are novel and bold. They are designed to be usable and flexible with the intent of supporting researchers from a wide range of disciplines and their myriad study designs and aims. Finally, they are meant to be practical, equipping practitioners and policymakers with more information about SBMPs to inform both their program selection decisions and overall approaches to implementation based on specific contexts and sets of participants.