Skip to main content

Advertisement

SpringerLink
Log in

The Effectiveness and Micro-costing Analysis of a Universal, School-Based, Social–Emotional Learning Programme in the UK: A Cluster-Randomised Controlled Trial

  • Original Paper
  • Published:
School Mental Health Aims and scope Submit manuscript

Abstract

There are a growing number of school-based interventions designed to promote children’s social and emotional learning. One such intervention, PATHS (Promoting Alternative Thinking Strategies), was evaluated in a randomised controlled trial involving 5074 pupils aged 4–6 years at baseline in 56 primary schools across a large city in the UK. The programme was implemented for two academic years. The primary outcome measure was the teacher-rated Strengths and Difficulties Questionnaire (SDQ). A secondary measure was the PATHS Teacher Rating Scale (PTRS). Observations of child and teacher behaviours were undertaken in a third of intervention and control schools using the Teacher–Pupil Observation Tool (T-POT). Regarding fidelity, dose and adherence were measured via weekly logs completed by teachers, and a semi-structured questionnaire completed by PATHS coaches was used as a global measure of fidelity (capturing adherence, dose and quality). A cost-consequence analysis examined programme costs from a multi-agency public sector perspective. At 1 year post-baseline, there were no statistically significant differences between the programme and control groups on the SDQ subscales or the SDQ total difficulties and impact scores. There were statistically significant differences favouring the programme group for six out of 11 subscales on the secondary outcome measure (PTRS). At 2 years post-baseline, there were no statistically significant differences between the groups on either measure. Fidelity, according to the global measure, was relatively strong, and there was no relationship between fidelity and treatment effects. The average cost of PATHS was £12,666 per school or £139 per child. The study, which was fully powered and independent of the programme developer, shows no statistically significant effect of the programme on child behaviour or emotional well-being.

Trial registration site and number: www.controlled-trials.com: ISRCTN 32534848.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. www.casel.org

  2. For the trial protocol, see http://www.nets.nihr.ac.uk/__data/assets/pdf_file/0019/81712/PRO-10-3006-01.pdf.

  3. This concerns children aged 4-7, equivalent to pre-kindergarten, kindergarten and first grade in the USA.

  4. Discounting helps to “make fair” comparisons of programmes whose costs and outcomes occur at different times. In order to compare, and add up, costs that accrue at different times, it is necessary to calculate their present value, which expresses them as an equivalent amount of today’s pounds. Discounting converts the £ value of costs in different time periods to present value. This is necessary because a £ in the future is worth less than a £ now.

  5. Micro-costing is bottom-up approach used to estimate the cost of setting up and delivering an intervention. It involves collecting detailed information about the resources required to deliver an intervention, and subsequently assigning economic unit costs to each component of resource use. The alternative approach would be gross costing, a top-down approach where the total cost invoiced is divided by the total resource use to obtain an average cost of resource use. The micro-costing approach is accepted as being more accurate than gross costing and is widely used in costing studies (e.g. Charles, Edwards, Bywater, & Hutchings, 2013; Tarricone, 2006; Xu, Nardini, & Ruger, 2014).

  6. There is no evidence-based threshold for fidelity advised by the programme developer. The 80 % threshold used here is purely indicative, and other intervention studies have set lower thresholds for acceptable and high fidelity.

References

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, fifth edition (DSM-5). Amercian Psychiatric Association.

  • Bateman, A., Boyes, S., Hines, J., Lacey, T., Kerrigan, B., Hopkins, C., et al. (2012). Implementing the PATHS program in Birmingham, England. In P. M. Brown, M. W. Corrigan, & A. Higgins-DAlessandro (Eds.), Handbook of prosocial education. Rowman: Blueridge.

    Google Scholar 

  • Bierman, K. L., Domitrovich, C. E., Nix, R. L., Gest, S. D., Welsh, J. A., Greenberg, M. T., et al. (2008). Promoting academic and social-emotional school readiness: The head start REDI program. Child Development, 79(6), 1802–1817.

    Article  PubMed  PubMed Central  Google Scholar 

  • Blueprints. (2012). Blueprints for Violence Prevention website. http://www.colorado.edu/cspv/blueprints/modelprograms/PATHS.html. Accessed 26th September 2012.

  • Blueprints. (2014). Promoting Alternative Thinking Strategies (PATHS) http://www.blueprintsprograms.com/factSheet.php?pid=b6692ea5df920cad691c20319a6fffd7a4a766b8. Accessed 3rd January 2014.

  • Brown, B., & Russell, K. (1997). Methods of correcting for multiple testing: operating characteristics. Statistics in Medicine, 16, 2511–2528.

    Article  PubMed  Google Scholar 

  • Bywater, T., & Sharples, J. (2012). Effective evidence-based interventions for emotional well-being: lessons for policy and practice. Research Papers in Education, 27(4), 389–408.

    Article  Google Scholar 

  • Campbell, M. K., Elbourne, D. R., Altman, D. G., & for the CONSORT Group. (2004). The CONSORT statement: extension to cluster randomised trials. British Medical Journal, 328, 702–708.

    Article  PubMed  PubMed Central  Google Scholar 

  • Collaborative for Academic, Social and Emotional Learning (CASEL). (2014). http://www.casel.org/social-and-emotional-learning. Accessed 2nd January 2014

  • Charles, J. M., Edwards, R. T., Bywater, T., & Hutchings, J. (2013). Micro-costing in public health economics: Steps towards a standardized framework, using the Incredible Years Toddler Parenting Program as a worked example. Prevention Science, 14(4), 377–389.

    Article  PubMed  Google Scholar 

  • Conduct Problems Prevention Research Group (CPPRG). (1999). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting and Clinical Psychology, 67(5), 648–657.

    Article  Google Scholar 

  • Conduct Problems Prevention Research Group (CPPRG). (2010). The effects of a multiyear universal social-emotional learning program: the role of student and school characteristics. Journal of Consulting and Clinical Psychology, 78(2), 156–168.

    Article  Google Scholar 

  • Curtis, L., & Netten, A. (2004). Unit costs of health and social care 2004. Canterbury: PSRRU.

    Google Scholar 

  • Curtis, C., & Norgate, R. (2007). An evaluation of the PATHS curriculum at Key Stage 1. Educational Psychology in Practice, 23, 33–44.

    Article  Google Scholar 

  • Denham, S. A., & Brown, C. (2010). “Plays nice with others”: social-emotional learning and academic success. Early Education and Development, 21(5), 652–680.

    Article  Google Scholar 

  • Department for Education and Skills (DfES). (2005). Primary Social and Emotional Aspects of Learning (SEAL): Guidance for Schools. Nottingham: DfES.

    Google Scholar 

  • Department for Education and Skills (DfES). (2007). Social and Emotional Aspects of Learning (SEAL): Guidance for secondary schools. Nottingham: DfES Publications.

    Google Scholar 

  • Domitrovich, C., Cortes, R., & Greenberg, M. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” Curriculum. Journal of Primary Prevention, 28(2), 67–91.

    Article  PubMed  Google Scholar 

  • Domitrovich, C. E., Greenberg, M. T., Cortes, R., & Kusché, C. A. (2005). The preschool PATHS curriculum. Deerfield, MA: Channing-Bete Publishers.

    Google Scholar 

  • Drummond, M. F., Sculpher, M. J., Torrance, G. W., O’Brien, B. J., & Stoddart, G. L. (2005). Methods for the economic evaluation of health care programmes (3rd ed.). Oxford: Oxford University Press.

    Google Scholar 

  • Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432.

    Article  PubMed  Google Scholar 

  • Ecclestone, K., & Hayes, D. (2008). The dangerous rise of therapeutic education. London: Routledge.

    Google Scholar 

  • Edwards, R. T., Hounsome, B., Linck, P., & Russell, I. T. (2008). Economic evaluation alongside pragmatic randomised trials: Developing a Standard Operating Procedure for clinical trials units. Trials, 14, 64–65.

    Article  Google Scholar 

  • Edwards, R. T., ÓCéilleachair, A., Bywater, T., Hughes, D. A., & Hutchings, J. (2007). Parenting programme for parents of children at risk of developing conduct disorder: cost effectiveness analysis. British Medical Journal, 10, 334–682.

    Google Scholar 

  • Formby, E., Coldwell, M., Stiell, B., Demack, S., Stevens, A., Shipton, L., Wolstenholme, C. & Willis, B. (2011). Personal, Social, Health and Economic (PSHE) Education: A mapping study of the prevalent models of delivery and their effectiveness. Research Brief DFE-RB080, Department for Education, London.

  • Furedi, F. (2009). Wasted: why education isn’t educating. London: Continuum.

    Google Scholar 

  • Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38, 581–586.

    Article  PubMed  Google Scholar 

  • Goodman, R. (2001). Psychometric properties of the Strengths and Difficulties Questionnaire (SDQ). Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345.

    Article  PubMed  Google Scholar 

  • Goodman, R., Ford, T., Simmons, H., Gatward, R., & Meltzer, H. (2000a). Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. British Journal of Psychiatry, 177, 534–539.

    Article  PubMed  Google Scholar 

  • Goodman, R., Renfrew, D., & Mullick, M. (2000b). Predicting type of psychiatric disorder from Strengths and Difficulties Questionnaire (SDQ) scores in child mental health clinics in London and Dhaka. European Child and Adolescent Psychiatry, 9, 129–134.

    Article  PubMed  Google Scholar 

  • Goodman, R., & Scott, S. (1999). Comparing the Strengths and Difficulties Questionnaire and the Child Behavior Checklist: Is small beautiful? Journal of Abnormal Child Psychology, 27(1), 17–24.

    Article  PubMed  Google Scholar 

  • Greenberg, M. T., & Kusché, C. A. (1998). Preventive intervention for school-age deaf children: The PATHS curriculum. Journal of Deaf Studies and Deaf Education, 3(1), 49–63.

    Article  PubMed  Google Scholar 

  • Greenberg, M. T., & Kusché, C. A. (2002). Promoting Alternative Thinking Strategies (PATHS): Blueprints for Violence Prevention (Book 10). Boulder, CO: Center for the Study and Prevention of Violence, University of Colorado.

    Google Scholar 

  • Hobbs, T., Axford, N., & Jodrell, D. (2011). Getting the measure of child health and development outcomes (2): The picture for a local authority in England. Child Indicators Research, 4(1), 81–100.

    Article  Google Scholar 

  • Hoffman, D. M. (2009). Reflecting on social emotional learning: A critical perspective on trends in the United States. Review of Educational Research, 79(2), 533–556.

    Article  Google Scholar 

  • Humphrey, N. (2013). Social and emotional learning: A critical appraisal. London: Sage.

    Book  Google Scholar 

  • Humphrey, N., Kalambouka, A., Bolton, J., Lendrum, A., Wigelsworth, M., Lennie, C., & Farrell, P. (2008). Primary Social and Emotional Aspects of Learning (SEAL): Evaluation of small group work. London: Department for Children, Schools and Families.

    Google Scholar 

  • Humphrey, N., Lendrum, A., & Wigelsworth, M. (2010). Social and Emotional Aspects of Learning (SEAL) programme in secondary schools: National evaluation. London: Department for Education.

    Google Scholar 

  • Kam, C.-M., Greenberg, M. T., & Kusché, C. A. (2004). Sustained effects of the PATHS curriculum on the social and psychological adjustment of children in special education. Journal of Emotional and Behavioral Disorders, 12, 66–78.

    Article  Google Scholar 

  • Little, M., Berry, V., Morpeth, L., Blower, S., Axford, N., Taylor, R., et al. (2012). The impact of three evidence-based programmes delivered in public systems in Birmingham. UK, International Journal of Conflict and Violence, 6(2), 260–272.

    Google Scholar 

  • Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). Hoboken, NJ: Wiley.

    Google Scholar 

  • Malti, T., Ribeaud, D., & Eisner, M. P. (2011). The effectiveness of two universal preventive interventions in reducing children’s externalizing behavior: A cluster randomized controlled trial. Journal of Clinical Child and Adolescent Psychology, 40(5), 677–692.

    Article  PubMed  Google Scholar 

  • Martin, P. A., Daley, D., Hutchings, J., Jones, K., Eames, C., & Whitaker, C. J. (2010). The Teacher-Pupil Observation Tool (T-POT). School Psychology International, 31(3), 229–249.

    Article  Google Scholar 

  • Netten, A., & Knight, J. (1999). Annuitizing the human capital investment costs of health service professionals. Health Economics, 8, 245–255.

    Article  PubMed  Google Scholar 

  • Ofsted. (2013). Not yet good enough: Personal, social, health and economic education in schools. London: Ofsted.

    Google Scholar 

  • Riggs, N. R., Greenberg, M. T., Kusché, C. A., & Pentz, M. A. (2006). The mediational role of neurocognition in the behavioral outcomes of a social-emotional prevention program in elementary school students: Effects of the PATHS Curriculum. Prevention Science, 7, 91–102.

    Article  PubMed  Google Scholar 

  • Ross, S. M., Cheung, A., Slavin, R., Sheard, M. K., & Elliott, L. (2011). Promoting primary pupils’ social-emotional learning and pro-social behaviour: Longitudinal evaluation of the Together 4 All Programme in Northern Ireland. Effective Education, 3, 61–81.

    Article  Google Scholar 

  • Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group. (2010). CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. British Medical Journal, 23(340), d40.

    Google Scholar 

  • Seifer, R., Gouley, K., Miller, A. L., & Zakriski, A. (2004). Implementation of the PATHS curriculum in an urban Elementary School. Early Education and Development, 15(4), 471–486.

    Article  Google Scholar 

  • Spence, S. H., & Shortt, A. L. (2007). Research review: Can we justify the widespread dissemination of universal, school-based interventions for the prevention of depression among children and adolescents? Journal of Child Psychology and Psychiatry, 48(6), 526–542.

    Article  PubMed  Google Scholar 

  • StataCorp. (2009). Stata statistical software: Release 11. College Station, TX: StataCorp LP.

    Google Scholar 

  • Sundell, K., Hansson, K., Löfholm, C. A., Olsson, T., Gustle, L. H., & Kedesjö, C. (2008). The transportability of multisystemic therapy to Sweden: Short-term results from a randomized trial of conduct-disordered youths. Journal of Family Psychology, 22(4), 550–560.

    Article  PubMed  Google Scholar 

  • Tarricone, R. (2006). Cost-of-illness analysis: What room in health economics? Health Policy, 77(1), 51–63.

    Article  PubMed  Google Scholar 

  • The Social Research Unit (SRU). (2009). A service design manual for implementing PATHS in Birmingham, UK. Dartington: The Social Research Unit.

    Google Scholar 

  • Weare, K. (2004). Developing the emotionally literate school. London: Sage.

    Google Scholar 

  • Weare, K. (2010). Mental health and social and emotional learning: Evidence, principles, tensions, balances. Advances in School Mental Health Promotion, 3(1), 5–17.

    Article  Google Scholar 

  • Weare, K., & Gray, G. (2003). What works in promoting children’s emotional and social competence and wellbeing?. Nottingham: DfES Publications.

    Google Scholar 

  • White, I. R., Horton, N. J., Carpenter, J., & Pocock, S. J. (2011). Strategy for intention to treat analysis in randomised trials with missing outcome data. British Medical Journal, 342, d40.

    Article  PubMed  PubMed Central  Google Scholar 

  • World Health Organization. (2004). ICD-10: International statistical classification of diseases and related health problems (Tenth Revision). Geneva: World Health Organization.

    Google Scholar 

  • WSIPP (Washington State Institute for Public Policy). (2015). Promoting Alternative Thinking Strategies (PATHS). http://www.wsipp.wa.gov/BenefitCost/Program/94.

  • Xu, X., Nardini, H. K. G., & Ruger, J. P. (2014). Micro-costing studies in the health and medical literature: Protocol for a systematic review. Systematic Reviews, 3(1), 47.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

The authors would like to acknowledge the hard work and dedication of Birmingham City Council’s (BCC) “Brighter Futures” team, including the PATHS coaches who collected fidelity data for the study. In addition, we thank the teachers and schools that took part in the trial, contributing valuable data on children’s well-being. Professor Daniel F. Perkins was involved in the design of the study. Drs Pam Martin and Catrin Eames undertook the independent observations of classroom behaviour. Pat Linck undertook initial analysis of the cost data. The North Wales Trials Unit (NWORTH) provided the randomisation service for this study.

Funding

This study was funded by Birmingham City Council (BCC) as part of its “Brighter Futures” strategy for children’s services.

Author information

Authors and Affiliations

  1. NIHR CLAHRC South West Peninsula (PenCLAHRC), Plymouth University Peninsula Schools of Medicine and Dentistry, Room N10, ITTC Building, Plymouth Science Park, Plymouth, PL6 8BX, UK

    Vashti Berry

  2. Dartington Social Research Unit, Dartington, UK

    Nick Axford & Kate Tobin

  3. Department of Health Sciences, University of York, York, UK

    Sarah Blower & Tracey Bywater

  4. Institute of Health Research, University of Exeter Medical School, Exeter, UK

    Rod S. Taylor

  5. Centre for Health Economics and Medicines Evaluation, Bangor University, Bangor, UK

    Rhiannon Tudor Edwards & Carys Jones

Authors
  1. Vashti Berry
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nick Axford
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sarah Blower
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rod S. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rhiannon Tudor Edwards
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kate Tobin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Carys Jones
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tracey Bywater
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vashti Berry.

Ethics declarations

Conflict of interest

None.

Ethical standard

The Warren House Group Ethics Committee.

Appendix: CONSORT Diagram: Flow of Schools Through Trial

Appendix: CONSORT Diagram: Flow of Schools Through Trial

figure a

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Berry, V., Axford, N., Blower, S. et al. The Effectiveness and Micro-costing Analysis of a Universal, School-Based, Social–Emotional Learning Programme in the UK: A Cluster-Randomised Controlled Trial. School Mental Health 8, 238–256 (2016). https://doi.org/10.1007/s12310-015-9160-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12310-015-9160-1

Keywords

Search

Navigation