Skip to main content
SpringerLink
Log in

Antibullying Programs in Schools: How Effective are Evaluation Practices?

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Bullying is a problem for schools around the world, and is an important topic for research because it has been associated with negative outcomes on numerous social, psychological, and academic measures. Antibullying prevention and intervention programs have varied greatly in their outcomes, with some studies reporting positive results while others have reported little or no positive impacts. Prompted by accountability demands, many agencies have developed standards with which to assess whether social programs are effective. Antibullying program evaluations have not been systematically reviewed to determine whether these types of standards are being applied. The purpose of this study was to assess the rigor of recent peer-reviewed antibullying program evaluations. Thirty-one peer-reviewed evaluations of antibullying programs, published within the last 10 years, were identified and coded for study characteristics. Shortcomings were identified in many of these program evaluations. In order to improve evaluation practices, researchers should consider using more rigorous designs to identify cause-effect relationships, including control conditions and random assignment, using more appropriate pre-post intervals, using more advanced methods of analyses such as hierarchical linear modeling, and systematically verifying program integrity to obtain dosage data that can be used in the outcome analyses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alsaker, F. D. (2004). Bernese programme against victimization in kindergarten and elementary school. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 289–306). New York: Cambridge University Press.

    Google Scholar 

  • Alsaker, F. D., & Valkanover, S. (2001). Early diagnosis and prevention of victimization in kindergarten. In J. Juvonen & S. Graham (Eds.), Peer harassment in school: The plight of the vulnerable and victimized (pp. 175–195). New York: Guilford Press.

    Google Scholar 

  • American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.

    Google Scholar 

  • Baldry, A. C., & Farrington, D. P. (2004). Evaluation of an intervention program for the reduction of bullying and victimization in schools. Aggressive Behavior, 30, 1–15. doi:10.1002/ab.20000.

    Article  Google Scholar 

  • Baldry, A. C., & Farrington, D. P. (2007). Effectiveness of programs to prevent school bullying. Victims and Offenders, 2, 183–204. doi:10.1080/15564880701263155.

    Article  Google Scholar 

  • Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.

    Google Scholar 

  • Chatterji, M. (2007). Grades of evidence: Variability in quality of findings in effectiveness studies of complex field interventions. The American Journal of Evaluation, 28, 239–255. doi:10.1177/1098214007304884.

    Article  Google Scholar 

  • Coie, J. D. (2004). The impact of negative social experiences on the development of antisocial behavior. In J. B. Kupersmidt & K. A. Dodge (Eds.), Children’s peer relations: From development to intervention (pp. 243–267). Washington, DC: American Psychological Association.

    Google Scholar 

  • Connolly, J., Pepler, D., Craig, W., & Taradash, A. (2000). Dating experiences of bullies in early adolescence. Child Maltreatment, 5, 299–310. doi:10.1177/1077559500005004002.

    Article  PubMed  CAS  Google Scholar 

  • Cowie, H., & Olafsson, R. (2000). The role of peer support in helping the victims of bullying in a school with high levels of aggression. School Psychology International, 21, 79–95. doi:10.1177/0143034300211006.

    Article  Google Scholar 

  • Crick, N. R., & Bigbee, M. A. (1998). Relational and overt forms of peer victimization: A multiinformant approach. Journal of Consulting and Clinical Psychology, 66, 337–347. doi:10.1037/0022-006X.66.2.337.

    Article  PubMed  CAS  Google Scholar 

  • Cross, D., Hall, M., Hamilton, G., Pintabona, Y., & Erceg, E. (2004). Australia: The Friendly Schools Project. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 187–210). New York: Cambridge University Press.

    Google Scholar 

  • Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23–45. doi:10.1016/S0272-7358(97)00043-3.

    Article  PubMed  CAS  Google Scholar 

  • DeRosier, M. E. (2004). Building relationships and combating bullying: Effectiveness of a school-based social skills group intervention. Journal of Clinical Child and Adolescent Psychology, 33, 196–201. doi:10.1207/S15374424JCCP3301_18.

    Article  PubMed  Google Scholar 

  • Edwards, D., Hunt, M. H., Meyers, J., Grogg, K. R., & Jarrett, O. (2005). Acceptability and student outcomes of a violence prevention curriculum. The Journal of Primary Prevention, 26, 401–418. doi:10.1007/s10935-005-0002-z.

    Article  PubMed  Google Scholar 

  • Fekkes, M., Pijpers, F. I. M., & Verloove-Vanhorick, P. (2006). Effects of antibullying school program on bullying and health complaints. Archives of Pediatrics & Adolescent Medicine, 160, 638–644. doi:10.1001/archpedi.160.6.638.

    Article  Google Scholar 

  • Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., et al. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151–175. doi:10.1007/s11121-005-5553-y.

    Article  PubMed  Google Scholar 

  • Frey, K. S., Hirschstein, M. K., Snell, J. L., Edstrom, L. V. S., MacKenzie, E. P., & Broderick, C. J. (2005). Reducing playground bullying and supporting beliefs: An experimental trial of the Steps to Respect program. Developmental Psychology, 41, 479–491. doi:10.1037/0012-1649.41.3.479.

    Article  PubMed  Google Scholar 

  • Gollwitzer, M., Eisenbach, K., Atria, M., Strohmeier, D., & Banse, R. (2006). Evaluation of aggression-reducing effects of the “Viennese Social Competence Training.” Swiss Journal of Psychology, 65, 125–135. doi:10.1024/1421-0185.65.2.125.

    Article  Google Scholar 

  • Greenberg, M. T., Domitrovich, C. E., Graczyk, P. A., & Zins, J. E. (2005). The study of implementation in school-based preventive interventions: Theory, research, and practice (Vol. 3). Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration.

    Google Scholar 

  • Hall, G. E., & Hord, S. M. (2006). Implementing change: Patterns, principles, and potholes (2nd ed.). New York: Pearson.

    Google Scholar 

  • Heydenberk, R. A., Heydenberk, W. R., & Tzenova, V. (2006). Conflict resolution and bully prevention: Skills for school success. Conflict Resolution Quarterly, 24, 55–69. doi:10.1002/crq.157.

    Article  Google Scholar 

  • Hunt, C. (2007). The effect of an education program on attitudes and beliefs about bullying and bullying behaviour in junior secondary school students. Child and Adolescent Mental Health, 12, 21–26. doi:10.1111/j.1475-3588.2006.00417.x.

    Article  Google Scholar 

  • Jennifer, D., & Shaughnessy, J. (2005). Promoting non-violence in schools: The role of cultural, organisational and managerial factors. Educational and Child Psychology, 22, 58–66.

    Google Scholar 

  • Kim, J. (2006). The effect of a bullying prevention program on responsibility and victimization of bullied children in Korea. International Journal of Reality Therapy, 26, 4–8.

    Google Scholar 

  • Kyler, S. J., Bumbarger, B. K., & Greenberg, M. T. (2005). Technical assistance fact sheet on evidence-based programs. University Park, PA: Prevention Research Center for the Promotion of Human Development. Retrieved Oct. 1, 2007 from http://www.prevention.psu.edu/pubs/documents/EBP_factsheet.pdf.

  • Leadbeater, B., & Hoglund, W. (2006). Changing the social contexts of peer victimization. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 15, 21–26.

    PubMed  Google Scholar 

  • Leadbeater, B., Hoglund, W., & Wood, T. (2003). Changing contexts? The effects of a primary prevention program on classroom levels of peer relational and physical victimization. Journal of Community Psychology, 31, 397–418. doi:10.1002/jcop.10057.

    Article  Google Scholar 

  • Leff, S. S., Power, T. J., & Goldstein, A. B. (2004). Outcome measures to assess the effectiveness of bullying prevention programs in schools. In D. L. Espelage & S. M. Swearer (Eds.), Bullying in American schools: A social-ecological perspective on prevention and intervention (pp. 269–293). Mahwah, NJ: Lawrence Erlbaum.

    Google Scholar 

  • Limber, S. P., & Small, M. A. (2003). State laws and policies to address bullying in schools. School Psychology Review, 32, 445–455.

    Google Scholar 

  • Limber, S. P., Nation, M., Tracy, A. J., Melton, G. G., & Flerx, V. (2004). Implementation of the Olweus Bullying Prevention programme in the Southeastern United States. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 55–79). New York: Cambridge University Press.

    Google Scholar 

  • McLaughlin, L., Laux, J. M., & Pescara-Kovach, L. (2006). Using multimedia to reduce bullying and victimization in third-grade urban schools. Professional School Counseling, 10, 153–160.

    Google Scholar 

  • Menesini, E., Codecasa, E., Benelli, B., & Cowie, H. (2003). Enhancing children’s responsibility to take action against bullying: Evaluation of a befriending intervention in Italian middle schools. Aggressive Behavior, 29, 10–14.

    Google Scholar 

  • Merrell, K. W., Gueldner, B. A., Ross, S. W., & Isava, D. M. (2008). How effective are school bullying intervention programs? A meta-analysis of intervention research. School Psychology Quarterly, 23, 26–42. doi:10.1037/1045-3830.23.1.26.

    Article  Google Scholar 

  • Meyer, N., & Lesch, E. (2000). An analysis of the limitations of a behavioural programme for bullying boys from a subeconomic environment. South African Journal of Child and Adolescent Mental Health, 12, 59–69.

    Google Scholar 

  • Mihalic, S. (2007). Matrix of programs. Boulder, CO: Center for the Study of Prevention of Violence, Blueprints for Violence Prevention. Retrieved on September 24, 2007, from www.colorado.edu/cspv/blueprints.

  • Mooij, T. (2005). National campaign effects on secondary pupils’ bullying and violence. The British Journal of Educational Psychology, 75, 489–511. doi:10.1348/000709904X23727.

    Article  PubMed  Google Scholar 

  • Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Marton, B., & Scheidt, P. (2001). Bullying behaviors among US youth: Prevalence and association with psychosocial adjustment. Journal of the American Medical Association, 285, 2094–2100. doi:10.1001/jama.285.16.2094.

    Article  PubMed  CAS  Google Scholar 

  • Naylor, P., & Cowie, H. (1999). The effectiveness of peer support systems in challenging school bullying: The perspectives and experiences of teachers and pupils. Journal of Adolescence, 22, 467–479. doi:10.1006/jado.1999.0241.

    Article  PubMed  CAS  Google Scholar 

  • Newman-Carlson, D., & Horne, A. M. (2004). Bully Busters: A psychoeducational intervention for reducing bullying behavior in middle school students. Journal of Counseling and Development, 82, 259–267.

    Google Scholar 

  • No Child Left Behind Act of 2001, P.L. 107-110, 115 Stat.1425 (2002).

  • Okumura, T. (2007). Sample size determination for hierarchical linear models considering uncertainty in parameter estimates. Behaviormetrica, 34, 79–93. doi:10.2333/bhmk.34.79.

    Article  Google Scholar 

  • Olweus, D. (1991). Bully/victim problems among school children: Basic facts and effects of a school-based intervention program. In D. Pepler & K. H. Rubin (Eds.), The development and treatment of childhood aggression (pp. 411–448). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Olweus, D. (1993). Bullying at school: What we know and what we can do. Malden, MA: Blackwell.

    Google Scholar 

  • Olweus, D. (2005). A useful evaluation design, and effects of the Olweus Bullying Prevention Program. Psychology, Crime & Law, 11, 389–402. doi:10.1080/10683160500255471.

    Article  Google Scholar 

  • O’Moore, A. M., & Minton, S. J. (2004). Ireland: The Donegal Primary Schools’ anti-bullying project. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 275–287). New York: Cambridge University Press.

    Google Scholar 

  • O’Moore, A. M., & Minton, S. J. (2005). Evaluation of the effectiveness of an anti-bullying programme in primary schools. Aggressive Behavior, 31, 609–622. doi:10.1002/ab.20098.

    Article  Google Scholar 

  • Orpinas, P., & Horne, A. M. (2006). Bullies: The problem and its impact. In P. Orpinas & A. M. Horne (Eds.), Bullying prevention: Creating a positive school climate and developing social competence (pp. 11–31). Washington, DC: American Psychological Association.

    Chapter  Google Scholar 

  • Orpinas, P., Home, A. M., & Staniszewski, D. (2003). School bullying: Changing the problem by changing the school. School Psychology Review, 32, 431–444.

    Google Scholar 

  • Ortega, R., Del Rey, R., & Mora-Merchan, J. A. (2004). SAVE model: An anti-bullying intervention in Spain. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 167–185). New York: Cambridge University Press.

    Google Scholar 

  • Patchin, J. W., & Hinduja, S. (2006). Bullies move beyond the schoolyard: A preliminary look at cyberbullying. Youth Violence and Juvenile Justice, 4, 148–169. doi:10.1177/1541204006286288.

    Article  Google Scholar 

  • Pawson, R., & Tilley, N. (1998). Cook-book methods and disastrous recipes: A rejoinder to Farrington. Evaluation, 4, 211–213. doi:10.1177/13563899822208545.

    Article  Google Scholar 

  • Pellegrini, A., & Bartini, M. (2000). An empirical comparison of methods of sampling aggression and victimization in school settings. Journal of Educational Psychology, 92, 360–366. doi:10.1037/0022-0663.92.2.360.

    Article  Google Scholar 

  • Peterson, L., & Rigby, K. (1999). Countering bullying at an Australian secondary school with students as helpers. Journal of Adolescence, 22, 481–492. doi:10.1006/jado.1999.0242.

    Article  PubMed  CAS  Google Scholar 

  • Rahey, L., & Craig, W. M. (2002). Evaluation of an ecological program to reduce bullying in schools. Canadian Journal of Counselling, 36, 281–296.

    Google Scholar 

  • Roland, E. (2002). Bullying, depressive symptoms and suicidal thoughts. Educational Research, 4, 55–67. doi:10.1080/00131880110107351.

    Article  Google Scholar 

  • Rosenbluth, B., Whitaker, D. J., Sanchez, E., & Valle, L. A. (2004). The Expect Respect project: Preventing bullying and sexual harassment in US elementary schools. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 211–233). New York: Cambridge University Press.

    Google Scholar 

  • Salmivalli, C. (2001). Peer-led intervention campaign against school bullying: Who considered it useful, who benefited? Educational Research, 43, 263–278. doi:10.1080/00131880110081035.

    Article  Google Scholar 

  • Salmivalli, C., Kaukiainen, A., Voeten, M., & Sinisammal, M. (2004). Targeting the group as a whole: The Finnish anti-bullying intervention. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 251–273). New York: Cambridge University Press.

    Google Scholar 

  • Salmivalli, C., Kaukiainen, A., & Voeten, M. (2005). Anti-bullying intervention: Implementation and outcome. The British Journal of Educational Psychology, 75, 465–487. doi:10.1348/000709905X26011.

    Article  PubMed  Google Scholar 

  • Smith, P. K., Pepler, D., & Rigby, K. (Eds.) (2004). Bullying in schools: How successful can interventions be? New York: Cambridge University Press.

    Google Scholar 

  • Smith, J. D., Schneider, B. H., Smith, P. K., & Ananiadou, K. (2004). The effectiveness of whole-school anti-bullying programs: A synthesis of evaluation research. School Psychology Review, 33, 548–561.

    Google Scholar 

  • Smith, J. D., Ryan, W., & Cousins, J. B. (2007). Antibullying programs: A survey of evaluation activities in public schools. Studies in Educational Evaluation, 33, 120–134. doi:10.1016/j.stueduc.2007.04.002.

    Article  Google Scholar 

  • Standards of Evidence Committee. (2004). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Retrieved September 5, 2008 from http://www.jstor.org.proxy.bib.uottawa.ca/stable/pdfplus/1049953.pdf.

  • Stevens, V., Van Oost, P., & De Bourdeaudhuij, I. (2000). The effects of an anti-bullying intervention programme on peers’ attitudes and behaviour. Journal of Adolescence, 23, 21–34. doi:10.1006/jado.1999.0296.

    Article  PubMed  CAS  Google Scholar 

  • Stevens, V., Van Oost, P., & De Bourdeaudhuij, I. (2001). Implementation process of the Flemish antibullying intervention and relation with program effectiveness. Journal of School Psychology, 39, 303–317. doi:10.1016/S0022-4405(01)00073-5.

    Article  Google Scholar 

  • Stevens, V., Van Oost, P., & De Bourdeaudhuij, I. (2004). Interventions against bullying in Flemish schools: Programme development and evaluation. In P. K. Smith, D. Pepler, & K. Rigby (Eds.), Bullying in schools: How successful can interventions be? (pp. 141–165). New York: Cambridge University Press.

    Google Scholar 

  • Thompson, B. (2002). What future quantitative social science research could look like: Confidence intervals for effect sizes. Educational Researcher, 31, 25–32. doi:10.3102/0013189X031003025.

    Article  Google Scholar 

  • Torgerson, C. J., & Torgerson, D. J. (2001). The need for randomized controlled trials in educational research. British Journal of Educational Studies, 49, 316–328. doi:10.1111/1467-8527.t01-1-00178.

    Article  Google Scholar 

  • Vreeman, R. C., & Carroll, A. E. (2007). A systematic review of school-based interventions to prevent bullying. Archives of Pediatrics & Adolescent Medicine, 161, 78–88. doi:10.1001/archpedi.161.1.78.

    Article  Google Scholar 

  • Whitaker, D. J., Rosenbluth, B., Valle, L. A., & Sanchez, E. (2004). Expect Respect: A school-based intervention to promote awareness and effective responses to bullying and sexual harassment. In D. L. Espelage & S. M. Swearer (Eds.), Bullying in American schools: A social-ecological perspective on prevention and intervention (pp. 327–350). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Wolff, N. (2000). Using randomized controlled trials to evaluate socially complex services: Problems, challenges and recommendations. The Journal of Mental Health Policy and Economics, 3, 97–109. doi:10.1002/1099-176X(200006)3:2<97::AID-MHP77>3.0.CO;2-S.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Ottawa, Ottawa, ON, Canada

    Wendy Ryan & J. David Smith

Authors
  1. Wendy Ryan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. David Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wendy Ryan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ryan, W., Smith, J.D. Antibullying Programs in Schools: How Effective are Evaluation Practices?. Prev Sci 10, 248–259 (2009). https://doi.org/10.1007/s11121-009-0128-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-009-0128-y

Keywords

Search

Navigation