Introduction

Children and adolescents need a balanced set of cognitive, social and emotional skills in order to achieve positive outcomes in school, in work, and in life more generally (OECD 2015). Social and emotional skills such as understanding and managing emotions, navigating social conflicts effectively, and making responsible decisions have been shown to influence numerous measures of social outcomes, including improved health, life satisfaction, subjective wellbeing, and reduced odds of engagement in anti-social behaviours (Goodman et al. 2015). Social and emotional skills do not play a role in isolation, they interact with cognitive skills which further enhance children’s likelihood of achieving positive outcomes in life (OECD 2015).

A common approach to supporting the development of children’s social and emotional skills has been school-based interventions (Jones and Bouffard 2012; Barry et al. 2017). Schools have been identified as a key setting for building social, emotional, and behavioural outcomes because students spend a substantial amount of time there. The school also provides a socialising context in which students are able to learn a range of life skills, many of which are associated with academic success (Taylor et al. 2017; Durlak et al. 2011; Sklad et al. 2012). Many school-based programmes have targeted an interrelated set of skills that fall under the headings of mental health promotion, character education, social and emotional learning (SEL), bullying prevention, life skills, strengths-based approaches, and youth development. Over the last three decades, the concept of social and emotional learning has served as an umbrella framework for a range of approaches and appears to have the largest and most rigorously evaluated evidence base. Social and emotional learning is defined as the process through which students acquire and effectively apply the knowledge, attitudes, and skills necessary to recognise and mange emotions, solve problems effectively, and establish positive relationships with others (CASEL 2005).

A growing body of research suggests that social and emotional skills are malleable and can be effectively taught using a variety of approaches and formats including classroom-based programming and whole school approaches (Jones and Bouffard 2012). Research indicates that interventions yield most successful outcomes when they are integrated into daily practice and school culture, seek to engage all staff, reinforce skills outside of the classroom such as hallways and playgrounds, support parental engagement, and coordinate work with outside agencies (Barry et al. 2017; Jones and Bouffard 2012; Weare and Nind 2011; Ttofi and Farrington 2011; Adi et al. 2007a, b; Wilson et al. 2003). Together, these characteristics point to the importance of adopting a whole school approach to enhancing children and young people’s social and emotional skill development. Jones and Bouffard (2012) highlighted key principles of social and emotional skill development that supports a move toward that adoption of a whole school approach: (i) continuity and consistency are essential for skill development, thus efforts need to be school wide, span age ranges, and consistent across multiple contexts within the school; (ii) social, emotional, and academic skills are interdependent and, therefore, efforts should be made to promote these skills simultaneously, reducing time pressures for teachers; (iii) social and emotional skills develop in social contexts, hence relationships between students and staff and among students are an important focus in their own right; (iv) classroom and schools operate as systems and both classroom- and school-wide efforts can set positive standards and expectations that promote and reinforce social and emotional competencies.

A whole school approach aims to integrate skill development into daily interactions and practices using collaborative efforts that include all staff, teachers, families, and children (Jones and Bouffard 2012; Meyers et al. 2015). Based on the World Health Organization’s definition of a Health Promoting School (WHO 1998), a whole school approach defines the entire school community as the unit of change and involves coordinated action between three interrelated components: (i) curriculum, teaching, and learning; (ii) school ethos and environment; (iii) family and community partnerships. Effective curriculum teaching and learning involve teaching skills through the implementation of evidence-based programmes, as well as modelling social emotional competencies, and providing continuous and consistent opportunities to practice these skills during everyday classroom situations (Oberle et al. 2016). At the school level, skills are reinforced in non-curriculum-based ways through policies, social relations, whole staff training, organisational structure, and daily activities in the school that are designed to promote a positive school climate which, in turn, helps children to develop positively across academic, social, emotional, and behavioural domains (Jones and Bouffard 2012; Meyers et al. 2015). Family and community partnerships involve extending learning to the home and community contexts. Embedding families within a whole school approach reinforces the complementary roles of families and educators and extends opportunities for learning across the two contexts in which children spend most of their time. Community partners provide links with external support and mental health services in the community, thereby ensuring there is access to services for students needing additional social and emotional support.

Several countries have launched national initiatives that adopt a school-wide approach to social and emotional learning. In Australia, for example, KidsMatter Primary is a mental health and wellbeing whole school framework that supports primary schools in implementing social and emotional learning school-wide (Dix et al. 2012). Through KidsMatter Primary, schools undertake a two-to-three-year cyclical process where they plan and take action to (i) promote social and emotional learning; (ii) work authentically with parents, carers, and families; and (iii) provide support for students who may be experiencing mental health difficulties. At second level, MindMatters provides professional development, curriculum, and whole school resources aimed at improving the mental health and wellbeing of young people (Wyn et al. 2000). In the UK, the Social and Emotional Aspects of Learning (SEAL) programme was developed as a whole school framework to support the social and emotional skill development of children and young people. This resource includes a curriculum element which is designed to support both universal and targeted work and whole school materials including resources relating to staff development, school organisation, management and leadership, and school ethos (Hallam 2009; Banerjee et al. 2014).

Despite extensive investment in whole school interventions, their effectiveness remains unclear. Reviewers of the evidence to date conclude that taking a whole school approach is more likely than individual classroom-based interventions to result in enduring positive change, because of its multi-component focus (Weare and Nind 2011; Adi et al. 2007a, b; Tennant et al. 2007; Jane-llopis et al. 2005; Wells et al. 2003). However, some recent reviews suggest that whole school interventions adopting a whole school approach are failing to show impact (e.g. Durlak et al. 2011; Langford et al. 2015). To date, however, no meta-analysis has been carried out specifically on interventions adopting a whole school approach to social and emotional learning. The aim of this review was, therefore, to examine the impact of these interventions on children and young people’s outcomes including social and emotional adjustment, behavioural adjustment, academic achievement, and internalising symptoms. A secondary aim was to assess the impact of moderating variables on programme outcomes.

Methods

The PRISMA guidelines for conducting a meta-analysis (Moher et al. 2009) were followed for the planning, conducting, and reporting of results.

Selection of studies

To be eligible for inclusion, studies were required to meet four methodological criteria: (i) utilised an experimental or quasi-experimental design with a control/comparison group; (ii) reported outcomes that could be transformed to Cohen’s d effect sizes; (iii) was published after 1998, in line with the Word Health Organization’s recommendation for schools to focus on the adoption of a whole school approach (WHO 1998); (iv) was published in English. In addition, the intervention (i) adopted a whole school approach as defined by the WHO (1998), (ii) was aimed at children and young people aged 4–18 years attending primary or secondary school, (iii) adopted a competency enhancement focus or was aimed at reducing problem behaviours through the application of social and emotional skills (e.g. bullying prevention interventions). Whole school interventions and frameworks which focused on behaviour management were not included in the current review, as a separate meta-analysis of this has already been carried out (Solomon et al. 2012).

Search strategy

Academic databases including Embase, PsycInfo, Scopus, and ERIC were searched. Eleven education databases were searched: NREPP, Child Trends US, Blueprints for Healthy Youth Development, Office of Justice Programs US, RAND Promising Practice Network on Children Families and Communities, California Evidence-based Clearing House for Child Welfare, Office of Adolescent Health, Crime Solutions US, Washington State Institute Public Policy, CASEL, and Education Endowment Foundation Database, UK. Eight health promotion and public health databases were searched: Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Centre); University of York National Health Service Centre for reviews and dissemination; National Institute of Clinical Excellence (NICE); British Education Index, Databases of Abstracts of Reviews of Effectiveness (DARE); Health Technology Assessment (HTA); Cochrane Database of Systematic Reviews; the Campbell Collaboration; WHO programmes and projects. Additional sources included Google Scholar and reference lists of relevant articles, book chapters, and reviews. Key individuals and organisations identified through the search process were contacted to identify further details on publications. The electronic search strategy used across all databases is provided in Table 1. The search for studies was conducted between 15 August and 8 October 2015. A repeated search was conducted between 30 July and 7 August 2017 to include articles published up to July 2017.

Table 1 Original search strategy for electronic databases

Results literature search

The results of the search and study selection are shown in Fig. 1. The original search, in 2015, identified 6626 citations from the academic databases and a further 32 citations from the other sources. After the removal of duplicates, 4402 abstracts were screened. A total of 392 full text articles were screened for eligibility. Of these, 348 did not fulfil the inclusion criteria and were excluded. Forty-four studies were selected for inclusion in the meta-analysis. The updated search which was carried out in August 2017 identified a further six studies. The combined searches resulted in a total of 50 articles which fulfilled the inclusion criteria. Six of the included articles reported findings from one evaluation study of the whole school intervention Positive Action. These six articles were therefore perceived as one study, resulting in 45 studies that underwent meta-analysis.

Fig. 1
figure 1

Flowchart study selection, first search (Aug 2015) and updated search (Aug 2017) combined

Coding

A coding sheet was developed by three authors (J.G., A.C., and M.S.). The purpose of the coding sheet was to extract all relevant information from the studies including methodological, intervention, and recipient characteristics. To check for inter-rater reliability, 10% of all the included articles were independently coded by the first and last authors (J.G. and A.C.). For all the checked articles, the percentage of agreement was above 85% and was perceived as sufficient. All differences were discussed and resolved.

Assessing quality of evidence

All studies that fulfilled the criteria for this meta-analysis underwent an assessment of their methodological quality. Following the guidelines of the Cochrane Public Health Group (Jackson and Waters 2005), the methodological quality of the intervention evaluations was assessed using the Quality Assessment Tool for Quantitative Studies (EPHPP 1998). Studies were assessed for selection bias, study design, confounders, blinding, data collection method, and dropouts. Each study was rated independently by two reviewers (J.G. and A.C.). The quality assessment results were compared and disagreements were resolved through discussion. Based on the ratings of the six components, each study received an overall rating of strong, moderate, or weak.

Outcomes and moderators

In line with other meta-analyses in this field (Payton et al. 2008; Sklad et al. 2012, four outcome categories were defined:

  1. i.

    Social and emotional adjustment: this category included measurements of social or emotional skills, and attitudes toward self and others.

  2. ii.

    Behavioural adjustment: this category included positive social behaviour, conduct problems, victimisation, and risky behaviour (e.g. substance abuse, unprotected sexual intercourse)

  3. iii.

    School performance: this category included all measurements of academic achievement, for example reading achievement scores or academic competence rated by the teacher

  4. iv.

    Internalising symptoms: this category included outcomes related to reducing psychopathology, such as depression and anxiety, and also feelings of wellbeing.

To assess the impact of moderators on programme outcomes, a number of variables were selected. Methodological variables included the following: research design (randomised controlled trial/quasi-experimental), number of months between pre- and post-intervention (12 or less/13 or more), implementation fidelity (no problems reported/problems reported/not reported), quality assessment (strong/moderate/weak), self-report measure (yes/no), and year of publication (2010 or earlier/2011 or later). Intervention characteristics selected as moderators included the following: intervention focus on behavioural problems (yes/no), school level (primary/secondary), community component included (yes/no), and targeted component included for students at risk of problem behaviour (yes/no). Lastly, a potential moderator regarding the recipient characteristics was the country where the study was carried out (United States/outside United States).

Analysis plan

Outcomes reported in the included studies were categorised into one of the four main outcome categories. The intervention and control group’s means, standard deviations, and sample size were extracted for each measurement. Standardised difference between two means “Cohen’s d” (or Cohen’s g in the case of small sample sizes) was then calculated per measurement based on means and standard deviations or other appropriate data (e.g. odds ratios). When studies failed to report means, standard deviations, or proportions, effect sizes were calculated using a t test, F-statistic, or p value and sample size.

For each effect size, the direction was determined in such a way that positive values indicated a more beneficial outcome favouring the intervention group over the control group. The variance of each effect size was calculated based on the sample sizes. It was also necessary to determine whether the design effect of clustering was taken into account in each study. If a design effect was not in place, the variances of the effect sizes were adjusted based on intercluster correlations (ICCs) of the same (or a comparable) article, resulting in true variances.

For each study, mean effect sizes for each outcome category were calculated. If different articles presented data about the same cohort, the data was combined into one mean effect size per outcome. For the mean effect sizes, it was determined whether the direction of the effect size was positive or negative. Effects sizes of 0.20 were considered small, 0.50 moderate, and 0.80 high (Lipsey and Wilson 1993). For each mean effect size, a variance was calculated based on an estimated correlation (r) and the (true) variance of the individual effect sizes (Morris 2008).

To analyse the general effectiveness of the interventions on the four main outcome categories, the Comprehensive Meta-Analysis (CMA) programme was used. A random effects model was used with a maximum likelihood estimation procedure to arrive at effect sizes (ES) and 95% confidence intervals. Outliers were removed from the main analyses. Studies were considered as outliers when their standardised residual exceeded the norm of 3.00. CMA was also used to calculate statistical heterogeneity, publication bias, and the influence of possible moderators. Heterogeneity was determined by calculating I2 values, indicating the degree of inconsistency across studies in a meta-analysis (Higgins et al. 2003). I2 values are derived from H2 values through the following formula: (H2–1) ∕ H2. Publication bias was estimated by funnel plots, Egger et al.’s (1997) regression tests, and Duval and Tweedie’s (2000) trim and fill analyses. The influence of possible moderators was determined by calculating Q values (heterogeneity between groups) and their corresponding p values, based on the Z-values of the different moderator categories (Borenstein et al. 2010). Few studies reported outcomes for subgroups or follow-up data, so it was decided not to do separate analysis for subgroups or follow-up studies.

Results

Characteristics of included interventions

Table 2 provides the summary characteristics for each study included in the meta-analysis. The 45 studies represented 30 different interventions. The total sample across the 45 studies was 496,299 students aged between 4 and 16 years. The number of students per study varied between 150 (Hennessey 2007) and 300,000 (Kärnä et al. 2011a, b), with a median of 2242 students. All studies included mixed-sex samples. Twenty-five studies (56%) were carried out outside of the United States. Twenty-eight studies (62%) employed randomised controlled designs. Only two studies (4%) presented a clear distinction between post-intervention and follow-up measurement. In total, 28 studies (62%) reported on social and emotional adjustment outcomes, 41 studies (91%) reported on behavioural adjustment outcomes, 10 studies (22%) reported on internalising symptoms outcomes, and 8 studies (18%) reported on school performance outcomes. Eighteen studies (40%) reported a high level of programme fidelity, 6 studies (13%) reported problems with fidelity, and 21 studies (47%) did not report fidelity. Of the 45 studies, 44 were published in peer-reviewed journals (98%). Regarding the quality of the evidence, 22 studies (49%) received a strong quality assessment rating, 15 studies (33%) received a moderate rating, and 8 studies (18%) received a weak quality assessment rating.

Table 2 Characteristics of included interventions

Seventeen of the whole school interventions were implemented in primary school (57%), seven interventions (23%) were implemented in secondary school, and six interventions were implemented across primary and secondary school (20%). All interventions provided teacher training and a programme manual. Ten interventions (33%) were aimed at enhancing social and emotional skills and 20 interventions (67%) were aimed at reducing bullying behaviour through the application of social and emotional skills. All interventions included a classroom curriculum, strategies addressing the whole school ethos and environment, and a parent component. Seventeen interventions (57%) included a community component. Fourteen interventions (47%) contained a targeted component aimed at addressing the needs of children at risk of developing emotional or behavioural problems.

Intervention effects

At post-intervention, there were a sufficient number of studies for each outcome category. As shown in Table 3, a small, but statistically significant increased mean effect size in the desired direction, was detected for social and emotional adjustment (d = 0.220), behavioural adjustment (d = 0.134), and internalising symptoms (d = 0.109). Across all outcomes the level of heterogeneity was high with I2 ranging from 87.99 to 99.40. The mean effect size for school performance was not found to be increased significantly (d = 0.193, 95% CI − 0.105 to 0.490, p = 0.204). When analysing the results on school performance more extensively, it is found that one highly positive effect size (Snyder et al. 2010) widened the confidence interval of the mean effect size, thus making it non-significant. However, based on the standardised residual of this study (1.44), this study was not perceived as an outlier.

Table 3 Effect size by outcome category (excluding outliers)

Publication bias

Visual inspection of the funnel plots (supplementary materials) showed no evidence of publication bias for the outcomes “social and emotional adjustment” and “behavioural adjustment”. The distribution of both these outcomes appears close to symmetrical and Egger’s regression tests showed no signs of publication bias (respectively p values of 0.28 and 0.33). Also, Duval and Tweedie’s trim and fill indicated that for both these outcomes, no studies needed to be filled or trimmed. For “school performance”, the funnel plot showed more studies to the left of the mean. Egger’s regression test was not significant (p = 0.71), but Duval and Tweedie’s trim and fill analysis indicated that five studies needed to be trimmed, adjusting the effect size to d = 0.605. For “internalising symptoms”, the funnel plot showed more studies to the right of the mean. Egger’s regression test was again not significant (p = 0.40), but Duval and Tweedie’s trim and fill analysis indicated that three studies needed to be trimmed, adjusting the effect size to d = 0.060.

Moderators of effect sizes

Moderator analyses were carried out to determine the effect of methodological, intervention, and recipient characteristics on the two most reported outcomes: social and emotional adjustment and behavioural adjustment (Table 4). Regarding methodological characteristics, the moderator “quality assessment” was of significant influence on students’ behavioural adjustment (Q = 9.141, df = 1, p = 0.001), with the highest effect sizes reported for studies that received a moderate or weak quality assessment rating (d = 0.410 vs d = 0.090). Regarding the intervention characteristics, the inclusion of a community component as part of a whole school intervention was shown to have a significant impact on participants’ social and emotional adjustment (Q = 5.092, df = 1, p = 0.024). Results indicated that whole school interventions which contained a community component showed significantly higher effect sizes than interventions without a community component (d = 0.447 vs d = 0.152). Lastly, regarding recipient characteristics, moderator analysis revealed that interventions evaluated in the United States were found to have significantly higher social and emotional effect sizes than whole school interventions evaluated outside of the United States (d = 0.450 vs d = 0.120; Q = 4.255, df = 1, p = 0.039).

Table 4 Moderators of effect sizes (including outliers)

Discussion

International research highlights the importance of implementing social and emotional skill development within the context of a whole school approach which embraces the wider school, family, and community contexts (Barry et al. 2017; Oberle et al. 2016; Jones and Bouffard 2012; Weare and Nind 2011). Whole school interventions have received significant investment in the past decade; however, their impact remains unclear. The aim of this meta-analysis was to examine the effectiveness of whole school interventions on a range of social, emotional, behavioural, and academic outcomes. Whilst previous meta-analyses have examined the impact of universal social and emotional learning programmes (e.g. Durlak et al. 2011; Sklad et al. 2012; Taylor et al. 2017), this current meta-analysis is, to our knowledge, the first to examine the effectiveness of interventions that adopt a whole school approach to social and emotional skill development. The main findings indicate that whole school interventions yield small, but significant positive effects on social and emotional adjustment, behavioural adjustment, and internalising symptoms. The highest estimate obtained was for social and emotional adjustment (d = 0.220), with estimates for behavioural adjustment and internalising symptoms ranging from 0.109–0.134. Previous meta-analyses of universal social and emotional learning interventions have reported higher effect sizes across social, emotional, behavioural, and academic domains (e.g. Durlak et al. 2011; Sklad et al. 2012).

It is likely that implementation had a significant role to play in the lower effect sizes reported in this current meta-analysis. Findings from implementation science provide evidence on the importance of quality of implementation in producing programme outcomes (Durlak and DuPre 2008; Fixsen et al. 2005). Durlak and colleagues’ (Durlak et al. 2011) meta-analysis of universal social and emotional skill-based interventions reported that interventions implemented with high quality produced larger effect sizes across all six social, emotional, behavioural, and academic outcomes, compared with interventions implemented with lower quality which only achieved significant effects in two outcome categories (attitudes and conduct problems). Researchers have found that comprehensive school-wide interventions frequently encounter problems with implementation (Durlak and Dupre 2008; Wilson and Lipsey 2007; Wilson et al. 2003). These interventions require substantial planning and support as skill development extends beyond the classroom, connecting and extending learning through the school ethos and environment and in partnership with families and communities. There is a need for significant infrastructure and capacity to support system-wide implementation of whole school interventions; however, this is often missing (Spoth et al. 2013). One of the interventions included in the current meta-analysis (SEAL) was shown to have no impact on students’ social and emotional skills (Wigelsworth et al. 2012). A process evaluation revealed that a lack of buy in from staff, perceived need for the programme, insufficient training, and teachers’ self-efficacy were all related to variability in the level of implementation of the programme (Lendrum et al. 2013). Implementation research is critical to understanding the range of factors operating at the level of the intervention, provider, community, delivery, and support system which impact the quality with which a programme is implemented.

Results from this meta-analysis revealed whole school interventions did not have a significant impact on academic performance. This finding is in contrast to results from previous meta-analyses, one of which reported an 11 percentile gain in academic achievement among students who received a social and emotional learning intervention (Durlak et al. 2011). A possible explanation for the non-significant finding in this meta-analysis is the small number of studies (N = 8) in this outcome category. In addition, one study with a large effect size widened the confidence interval of the mean effect size, thus making the mean effect size non-significant. Furthermore, only one of the eight studies utilised standardised achievement test scores as a measure of academic performance. The remaining studies utilised teacher self-report data of academic competence. The low number of studies combined with the use of teacher self-report data weakens the confidence that can be placed in this finding, and as a result, this finding should be viewed with caution.

Moderator analysis indicated that studies from the United States (US) had a significantly higher effect size that non-US studies. Two possible reasons could explain this finding. Weare and Nind (2011) in their review of social and emotional skill-based interventions suggested that, compared with European and Australian interventions, US interventions are more prescriptive in their training, programme manual, and requirements for programme fidelity. They contend that some non-US interventions fail to show impact as a result of their more flexible, bottom-up approach to skill development which can make it difficult for teacher to know what to implement and how it should be implemented. Another possible reason relates to the level of district and national supports for social and emotional learning across countries. It is argued that whole school interventions are most likely to be successful, effective, and sustainable when the necessary support from educational stakeholders at national level is in place (Mart et al. 2015; Barry et al. 2017). Support from the education system at national level has the power to catalyse systemic change at school level by communicating a culture of “what matters” in school learning (Oberle et al. 2016). This support can include advocating for policies that support whole school integration of social and emotional learning, defining age-specific standards for student outcomes across social and emotional learning domains and allocating the required resources for the adoption of evidence-based interventions. All of which can assist in creating the necessary conditions for school-wide social and emotional learning. Over the past number of years, the US has made significant advancements in embedding support for social and emotional learning at district and state levels. Examples of such advancements include the development of learning standards for social and emotional skill development (Dusenbury et al. 2015) and the establishment of CASEL’s Collaborative Districts Initiative (CDI) and Collaborative States Initiative (CSI) which are designed to build systemic support for social and emotional learning. Further research is required to advance our understanding of the type of support required by schools from the education system to implement evidence-based programmes with high quality and embed them within the school system.

Additional moderator analysis revealed the impact of implementing a community component as part of a whole school approach. Examples of community components included the following: additional support from community specialists for children considered “at risk” of developing problems; the involvement of community members in school components; the implementation of intervention activities through subgroups of the community including community leaders, the media, and social workers. Results from this meta-analysis indicated that the implementation of a community component was associated with a significant higher effect size on children’s social and emotional adjustment. This finding highlights the importance of schools working collaboratively with the wider community in reaching out to and receiving support from parents and local agencies. The development of links with community agencies and services has been argued to be the most essential component of the health-promoting school approach (Goltz et al. 1997; Lister-Sharp et al. 1999; WHO 1996). Community partners provide links with external support and mental health services in the community, thereby ensuring there is access to services for students needing additional support.

Whilst the interventions varied in their inclusion of a community component and what this consisted of, all of the interventions contained a classroom curriculum, a family component, and strategies aimed at enhancing the school ethos and environment. Furthermore, a number of key practices within these components were common across the majority of interventions. At the school level, practices included the following: the provision of a guide to support schools in developing whole school policies based on the intervention’s principles; the establishment of a school committee whose goal is to plan and implement the intervention; staff meetings dedicated to planning the whole school approach, monitoring its progress and arranging professional development. The majority of interventions identified in this meta-analysis established school-wide expectations, defined school-wide rules, displayed posters in school corridors that reflected intervention concepts, and implemented a school-wide system of encouraging the use of skills. Whole staff training was a feature of all interventions and included the following: training on the application of teaching strategies throughout the school day; instructional methods in interactive teaching, positive communication, problem solving, and cooperative learning; and strategies to support collaboration with parents. Key strategies used to engage parents included letters sent to parents providing information on the intervention’s key principles, teacher-parent meetings, the provision of a parent education programme or workshop targeting risk, and protective factors in the home environment.

Limitations

The findings from this meta-analysis need to be interpreted in the context of their limitations. Firstly, just over half of the studies (54%) received a strong quality assessment rating, the remaining studies received a moderate (29%) or weak (17%) rating. Moderator analysis revealed studies with a moderate or weak rating showed significantly larger effect sizes for behavioural adjustment than high-quality studies. Lower quality studies were mostly attributed to a high rate of dropout or failure to report dropout at post-intervention. This is an issue which has been identified as one of the most common pitfalls of efficacy trials (Clarke and Barry 2015). Accurate reporting of dropout and analysis on the extent to which dropout may have introduced bias is recommended for future studies in order to strengthen the quality of the evidence base. Secondly, although a total of 48 studies were identified for inclusion in the meta-analysis, the number of studies which examined the impact of whole school interventions on academic achievement and internalising outcomes was low. As a result, the power to detect significant effect sizes was reduced. Thirdly, although significant efforts were made to identify every relevant study for this meta-analysis, we cannot be sure that we identified all possible studies, in particular, unpublished studies which have a tendency to report null effects. A final limitation was the search for English-only studies which limits the representativeness and generalisability of the analysis.

Implications for research

Eight studies (16%) provided data on academic achievement. These results highlight the need for future studies to examine the impact of whole school interventions on academic achievement. Furthermore, the use of standardised achievement test scores in determining programme impact should be prioritised. Additional research on the long-term impact of whole school interventions is required in order to determine the durability of programme outcomes. Furthermore, whilst there is emerging evidence on the economic case for investing in school-based SEL (e.g. Belfield et al. 2015), cost-benefit analysis of whole school interventions would provide an important insight into their economic returns as educational investments.

There is a need for future research to identify the essential ingredients/components of whole school interventions that have been found through research to positively affect proximal outcomes. There is a clear overlap in the core components used across whole school interventions. Identifying the components that are essential for building social and emotional skills at the level of the individual, the classroom, the whole school, the family, and the community could facilitate the use of the most effective strategies with the greatest potential for impact. Research on the identification of “essential ingredients” across evidence-based youth mental health treatment and prevention interventions has shown some promising results in this area (e.g. Chorpita and Daleiden 2009; Boustani et al. 2015). Knowing the essential components of whole school interventions could assist schools in implementing whole school programme with high quality and at the same time making the necessary adaptations to suit the local context and specific population needs.

The results from this meta-analysis highlight the current lack of emphasis on programme implementation (46% of studies did not monitor programme implementation). Programme outcomes cannot be interpreted appropriately without information on the quality of programme information or the extent of intervention delivery. A greater focus on implementation research is required in order to understand a programme’s true value and to offer guidance in terms of continual improvement to programme delivery (Durlak 2015). Furthermore, there is a need for specific implementation guidelines and tools to support the effective adoption of whole school interventions in natural contexts. CASEL’s School Theory of Action (School ToA) resource is an example of a framework which provides specific tools and resources for setting up and sustaining a whole school approach to social and emotional learning (Oberle et al. 2016). Factors known to promote sustainability and impact are addressed, including the following: ongoing professional development, ongoing assessment and evaluation, infrastructure and school-wide integration, family-school-community partnerships, and ongoing communication. The further development and testing of implementation tools and resources are essential in addressing barriers to effective integration of social and emotional learning within the wider school system.

Implications for policy and practice

The findings from this meta-analysis have a number of implications for policy makers and practitioners, responsible for implementing social and emotional skill-based interventions. The successful implementation of evidence-based whole school practices requires individual motivation, individual and organisational capacity, supportive policies, and attention to the challenges involved in changing practices as well as addressing the challenges posed by the need to adapt programmes once implemented (Oberle et al. 2016). In order for whole school interventions to be used more widely, further effort is needed to assist schools in selecting an evidence-based intervention that fits their needs, implementing it with quality and adopting methods to sustain the intervention over the long term (Wandersman and Florin 2003).

There is also a need for investment in professional development and learning for building organisational capacity in adopting a whole school approach to social and emotional learning. As part of this, there is a need to promote partnerships between teacher preparation programmes, programme developers, and implementation experts to equip trainee teachers with the competencies necessary to embed social and emotional learning within the curriculum and wider school, family, and community contexts. The provision of leadership support and ongoing professional development is also required to support the integration and sustainability of whole school interventions.

Conclusions

A growing body of research suggests that for optimal impact, social and emotional skill development needs to be embedded within a whole school, multi-modal approach. This approach typically involves coordinated action between curriculum, teaching, and learning; the school ethos and environment; and family and community partnerships. The results from this meta-analysis indicate a small, but significant positive impact of these interventions on students’ social and emotional adjustment, behavioural adjustment, and internalising symptoms. Whole school interventions were not shown to impact on academic achievement. Moderator analysis provides evidence that interventions implemented and evaluated in the US were more effective in enhancing participants’ social and emotional adjustment than non-US interventions. In order to advance our understanding of the conditions necessary to achieve successful outcomes, a greater focus on implementation research and the identification of essential components of whole school interventions is required.