Bullying in schools, defined as repeated aggressive behavior including intentions to be hurtful or harmful and some sort of power imbalance between those involved, is a widespread public health problem (Eriksen et al. 2014; Olweus 1993; Ttofi and Farrington 2011).

Bullying can be physical, relational, or verbal and occur “face-to-face” and online (cyber-bullying). Based on a study conducted in 33 European and North American countries, the lowest bullying prevalence rates were found in Sweden and England (3–5%), while the highest rates were found in Lithuania and Belgium (17–29%) (Chester et al. 2015).

Being a victim of bullying has been shown to increase the risk of both direct, and long-term mental health problems, such as psychosomatic problems, school satisfaction, social support, future optimism, and self-esteem, as well as worse educational and labor market outcomes (Gini and Pozzoli 2009; Smokowski et al. 2014; Takizawa et al. 2014). For example, a recent review on the long-term consequences of bullying showed a strong association between being a victim of bullying and later onset in life of depression, poor academic achievement, school absenteeism, and suicidal behavior (Wolke and Lereya 2015).

Most previous research studies on bullying prevention programs have focused on the effectiveness in reducing the prevalence of bullying but have neglected the cost-effectiveness. Hence, this study aims to address the knowledge gap regarding the cost-effectiveness of bullying prevention programs and specifically to evaluate the cost-effectiveness of a wide spread bullying prevention program: the KiVa program.

As part of bullying prevention work, many schools have implemented manual-based bullying prevention programs. Results from a meta-analysis on manual-based bullying prevention programs showed that some (but far from all) programs show evidence of being effective, with an average reduction in bullying prevalence of about 20% (Ttofi and Farrington 2011). However, studies also show that a substantial proportion of the programs lack any beneficial effect and many schools also implement programs where there are no research studies or data on effectiveness at all (Evans et al. 2014; Ferguson et al. 2007; Flygare et al. 2013; Merrell et al. 2008).

A relatively new program that has gained increasing attention is “KiVa” (Salmivalli et al. 1998), which has been evaluated in a number of randomized experiments with promising results in terms of reducing the prevalence of bullying (Kärnä et al. 2013; Kärnä et al. 2011a, b).

As with any public health program (or investment/action in general), adopting a bullying prevention program comes with costs, both direct program costs and indirect costs. If a school implements a costly bullying prevention program, it may imply that funds for, e.g., teachers, facilities, and material, will be displaced. It is therefore essential to consider if a specific investment provides reasonable value for money. To the best of our knowledge, there are only two published economic evaluations of bullying prevention programs, both focusing on the historically popular Olweus Bullying Prevention Program (Beckman and Svensson 2015; Persson and Svensson 2013). Beckman and Svensson (2015) utilized a decision-tree model to evaluate the cost-effectiveness of the Olweus bullying prevention program implemented in grades 7 to 9 in a Swedish school context. The results indicated that the Olweus Bullying Prevention Program is moderately cost-effective. And in Persson and Svensson (2013), the Olweus Bullying Prevention program is used as an example to showcase how to conduct an economic evaluation of bullying prevention programs. The results indicated that it is motivated to implement the Olweus Bullying Prevention program compared to no specific program at all (“status quo”).

The aim of this study was to address the knowledge gap regarding the cost-effectiveness of bullying prevention programs and specifically to evaluate the cost-effectiveness of the KiVa program. We developed a decision-analytic model in the form of a Markov cohort model to assess the cost-effectiveness of the KiVa program in comparison to status quo (i.e., “treatment as usual”). The results can thus be interpreted as the expected value of the cost-effectiveness for an elementary school, grades 1 to 9 in a Swedish context, moving from the typical/average bullying prevention work to implementing the manual-based KiVa program in a whole-school approach with all its components.

The KiVa Program

KiVa (acronym for the Finnish term of “against bullying”) was developed in 2006 at the University of Turku in Finland for students between the ages 7 to 15 years (Salmivalli et al. 2011). The development of the program was initiated by the Finnish government and conducted in collaboration between the Department of Psychology and the Center for Learning Research at the University of Turku (Kärnä et al. 2011a). The goals of KiVa are to reduce the prevalence of bullying, counteract new bully-victim relationships, and reduce the negative consequences of bullying. The program takes a whole-school approach, which implies that it requires commitment and engagement from staff, students, and their parents. Consequently, KiVa is meant to be a part of everyday antibullying efforts and not implemented as a temporary project (Salmivalli et al. 2013). The KiVa program has three different versions adjusted for ages/grades (ages 7–9, 10–12, and 13–15 years) and is composed of two types of activities: universal actions and indicated actions (see Fig. 1).

The universal actions mainly consist of student lessons (primary school), themes (secondary school), and virtual learning environments. The manual-based student lessons are held by a classroom teacher and contain ten double lessons that cover a variety of topics, such as mechanisms of bullying, forms of bullying, group pressure, strategies to end ongoing bullying, and ways to support the victim. Students learn by discussions, group work, role-play exercises, and videos (Salmivalli et al. 2013). The themes are similar and are introduced to students in lessons, theme days, or at similar occasions (Kärnä et al. 2011a). The universal actions also include computer games (younger ages) and an online environment called “KiVa Street” (older ages). Furthermore, the school is equipped with KiVa symbols (e.g., posters and vests), with the intention of reminding students and staff of their work against bullying. Parents also receive a guide about bullying and ways to identify and prevent bullying (Haataja et al. 2014). The indicated actions refer to the management of acute cases of bullying and are carried out by a KiVa team of three teachers or other school staff in the form of individual and/or group discussions with the victim and the bully

Fig. 1
figure 1

Universal and Indicated Action Components of the KiVa Program

Full size image

.

Method and Material

Cost-Effectiveness Analysis

Cost-effectiveness analysis (CEA) is a method of economic evaluation that compares differences in costs to differences in benefits between two (or more) programs/treatments. CEA is increasingly used as a tool to inform decision makers on the “value for money” in prioritization between alternative programs and investments (ISPOR 2013). CEA with an “utility-based” outcome measure is also referred to as cost-utility analysis (CUA), but we use the generic term of CEA throughout this paper.

The result in a CEA is typically presented as an incremental cost-effectiveness ratio (ICER). In our setting, the ICER will show the difference in cost divided by the difference in effectiveness of adopting KiVa compared to status quo (SQ). The ICER is calculated as follows: (CostKiVa − CostSQ)/(EffectivenessKiVa − EffectivenessSQ). The ICER can be interpreted as the price/cost to gain one additional unit of the effectiveness measure. Thus, the lower the ICER the more cost-effective is the evaluated program (in relation to the comparator). In our analysis, we calculated the ICER using two different measures of effectiveness: the number of victim-free years and the number of quality adjusted life years (QALYs).

A CEA can be conducted alongside (randomized) controlled experiments and thus be based on primary data collection of individual-level costs and benefits. But, it is also common to conduct a CEA in the form of decision analytic models, where data on costs and benefits are based on secondary data (e.g., from reviews or administrative sources). Decision makers must decide on implementation (or no implementation) on a large range of programs and treatments where we lack primary data from randomized studies, thus creating a great need for decision-analytic modeling studies in the absence of large randomized studies.

The Modeling Approach: a Markov Cohort Model

The simplest modeling approach to conduct a CEA is to set up a decision-tree that represents the possible decision(s) and the associated consequences with each decision. However, in contexts where a program has consequences over several years, it quickly becomes cumbersome to represent the model using a decision-tree. An alternative modeling approach is to construct a Markov cohort model (state-transition model), which easily deals with recurring events over a longer time period (Caro et al. 2012). A Markov model consists of mutually exclusive states, where a certain share of the cohort transitions between states over time, where costs and effectiveness outcomes are assigned to each state (Briggs et al. 2006).

In our context, the two exclusive states were “bullied” and “not bullied.” For each cohort entering the school, there were different probabilities assigned for transitioning into a bullied or non-bullied state. Furthermore, between each grade, it was possible to stay in the current state or transition to the other state. Hence, a student could transition from bullied to not bullied one cycle (grade year) and move back to bullied at a later cycle. Figure S1 in the Online Supplement shows a simple graphical representation of the model. The model is run for each intervention arm, i.e., the KiVa-arm and the status quo-arm, and results from both runs were used to calculate the ICER of KiVa vs. status quo.

The Policy and Model Context

The target population for the model was an average cohort of students in a Swedish elementary school where children enter the first grade at age 6/7 years and graduate in grade nine at age 15/16 years (Swedish National Agency for Education 2006). We assumed a cohort size of 75 students divided in 3 classes, which is in line with average data for a Swedish school. Sensitivity analyses were carried out varying these assumptions.

The decision-model takes a “payer” perspective, which in the Swedish policy context implies a municipality context. Elementary schools in Sweden are funded by each municipality (in total there are 290 municipalities in Sweden), and decisions about implementing specific programs, such as the KiVa program, are taken at the municipality and/or school level. This implies that the costs included in the evaluation are costs borne by the school and municipality, i.e., direct costs to implement the program and the teacher and school staff resources that will be directed into the program.

The model followed a cohort of school entry students (grade 1) until graduation (grade 9). When performing economic evaluations, it is generally advised to take into account the timing of costs and effects using a social discount rate, which entails that future consequences are valued less than present consequences (Briggs et al. 2006). Costs and effectiveness outcomes were thus discounted at the point in time they occurred, with the rate of 3%, which is the recommended social discount rate in Sweden for health economic evaluations and in line with international recommendations (Siegel et al. 1996; TLV 2003). Costs were measured in the price level of 2017. Cost data in Euros were adjusted to Swedish kronor (€1 = 9.5 SEK).

To measure the effectiveness of KiVa we considered two different metrics. The first metric was the number of victim-free years of bullying. The primary objective of a bullying prevention program is to reduce the prevalence of bullying, and we thus evaluated the KiVa program in terms of if and how many victim-free school years it provides. A student that is never a victim of bullying will thus have nine victim-free years. The second effectiveness metric we considered was the Quality Adjusted Life Years (QALYs). QALY is a measure of health status in a single combined function, including length of life and quality of life (Pliskin et al. 1980; Zeckhauser and Shepard 1976). QALY weights can be between 0 and 1 (they may be negative as well, but not of relevance in this study), where 0 is interpreted as “equal to being dead” and 1 is interpreted as the best possible health state. Considering that bullying reduces health-related quality of life, a program that can reduce the prevalence of bullying will thus imply higher health-related quality of life and therefore more QALYs.

Input Parameters

Due to lack of relevant trial data on the full set of necessary costs and outcomes, which is often the case in health economic evaluations (Briggs et al. 2006), the input parameters for the Markov model consisted of secondary data gathered from the best available sources. The parameter estimates used in the model are summarized in Table 1 and further described below.

Table 1 Markov model: input parameters for costs, program treatment effects, and quality of life
Full size table

Transition Probabilities

The assumption was that for a cohort entering a school in the status quo situation, the initial prevalence of bullying was 4%, as acquired from an extensive national Swedish survey (Currie et al. 2012). In order to assess the transitions between being bullied and not being bullied, we could identify one Swedish study with longitudinal data on bullying experience (at the individual level) from grade 1 to grade 9 (Beckman et al. 2016). We were able to use the primary data from that study to construct the annual transitions between bullied and not bullied states (see Table S2 in the Online Supplement for the transition probabilities used).

Program Effect

The data on the treatment effect of the KiVa program was estimated based on a systematic review (see Table S3 in the Online Supplement for the search strategy and results). We included studies that evaluated the KiVa program compared to status quo using randomized experiments. The odds ratios (ORs) of being bullied without KiVa compared to intervention schools with KiVa were found to range from 1.13 to 1.83 (Garandeau et al. 2014; Kärnä et al. 2011a, b; Williford et al. 2013; Yang and Salmivalli 2013). The weighted mean of the treatment effect of KiVa from these studies correspond to a relative risk of being bullied in a school with KiVa (compared to a status quo school) of 0.58.

Cost Data

Cost data was retrieved from the suppliers and KiVa copyright-holder together with wage data from Swedish registers. The alternative to KiVa in the cost-effectiveness analysis was the status quo, which implies no change in current cost levels.

The program has a number of direct implementation costs including training and licensing fees, and per unit costs for teacher manuals, parental guides, and poster and vests. The indirect program costs consist of the time used to implement the program that will necessarily “crowd out” other school activities. These are monetized based on the time use (hours) multiplied by the teacher wage per hour (including social fees), which is what the municipalities actually pay for each hour of additional teacher time. The indirect program costs (time use) are larger than the direct implementation costs.

Quality of Life Data

As described in the “The Policy and Model Context” section, the effectiveness was measured in the number of victim-free years as well as in QALYs. We use the latter since this facilitates comparisons with other health and public health programs. In order to assess the total number of QALYs in each cohort, we had to assign health-related quality of life data for the two states. Students that are not bullied were assigned a health-related quality of life score of 0.89 (the “QALY weight”), based on data from a Swedish population survey (Burström et al. 2001). We were able to identify one study with estimates on the reduction in quality of life score due to being a victim of bullying (Beckman et al. 2016). The study used responses from 758 adolescents on the SF-6D survey to estimate health-related quality of life scores on the 0–1 index. The average reduction was estimated at 0.06 (Brazier et al. 2002), and the health-related quality of life (QALY weight) of a bully victim was set to 0.83.

Uncertainty

Uncertainty refers to presumptions in the model itself or to uncertainty regarding the input parameters. Both a deterministic (DSA) and a probabilistic sensitivity analysis (PSA) were carried out assessing parameter uncertainty (Briggs et al. 2006). In the DSA, one input parameter from the model was changed individually to investigate the impact on the ICER. This type of analysis clarifies what happens if one particular parameter alters, which can be used to identify critical parameters (Briggs et al. 2012). The PSA asses the uncertainty by jointly varying parameter inputs and in this case jointly changing the following parameters: transition probabilities, KiVa treatment effect, QALY weights, and costs of implementing KiVa (see Table 1 for the ranges and assumed distributions).

Results

Base-Case Results

Table 2 shows the base-case results as well as results from the deterministic sensitivity analysis (all results are expressed per student). The total discounted cost for the KiVa program over a time horizon of 9 years is 3686 SEK per student (€388). This should be interpreted as the additional cost per student (from the status quo) if implementing the KiVa program. The benefits show an increase of 0.47 victim-free years (from 8.04 to 8.59) and a gain in QALYs of 0.03 (from 6.88 to 6.91). The implied associated cost-effectiveness is as follows: cost per victim-free year 7879 SEK (€829) and cost per QALY 131,321 SEK (€13,823).

Table 2 The cost-effectiveness of KiVa: base-case results and deterministic sensitivity analysis
Full size table

Deterministic Sensitivity Analysis

Table 2 also includes the results of the deterministic/univariate sensitivity analysis, in which we deterministically change the values of the parameters one at a time, both increasing and decreasing the value from the base-case.

Overall, the changes of the different parameters behaved as we expected in terms of the effect of the cost per victim free-year and cost per QALY. One important parameter, with a relatively large impact, was the number of years/grades over which KiVa is implemented. The base-case result assumed that the program is implemented and run over 9 years (i.e., starting in grade 1 and continuing up to grade 9). However, many Swedish schools are divided in such a way that they only have grades 1–6 or grades 6/7–9, which makes it impossible to implement the program for 9 years in the same school. Assuming that KiVa is implemented over 3 years (grades 7–9) implied a cost per QALY of 604,988 SEK (€63,683) and a cost per victim-free year of 36,229 SEK (€3814).

Another important variable is the relative risk reduction of the KiVa program, i.e., the effectiveness of the program in terms of reducing the prevalence of bullying. For instance, if the relative risk changes to 0.3 (from 0.58), the cost per QALY and cost per victim-free year decrease to 86,699 SEK (€9126) and 5202 SEK (€548). On the other hand, if KiVa is less effective than our base-case assumption and the relative risk would be 0.7, the cost per gained QALY and cost per victim-free year increase to 179,664 SEK (€18,912) and 10,780 SEK (€1135).

The discount rate, total cost of the program, cohort size, and initial bullying prevalence rates do not impact the cost-effectiveness by a large magnitude. Thus, the critical parameters seem to regard the effectiveness of the program, but primarily, the number of years over which KiVa is implemented.

Probabilistic Sensitivity Analysis

Figure 2 shows the cost-effectiveness plane based on 5000 Monte-Carlo simulations were all parameter estimates were changed jointly, i.e., each dot represents one specific ICER from each of the 5000 iterations. Almost all ICERs are in the north-east quadrant of the cost-effectiveness plane, which implies that the program is very likely to increase costs but also increase the number of QALYs.

Fig. 2
figure 2

Cost-effectiveness plane based on probabilistic sensitivity analysis (5000 iterations)

Full size image

Figure 3 shows the same underlying data represented in a cost-effectiveness acceptability curve, which shows the probability that KiVa is cost-effective given different willingness to pay per QALY thresholds (Briggs et al. 2006). The higher threshold value, the higher probability the intervention is cost-effective.  For the assumed Swedish informal threshold of 500,000 SEK per QALYs (€52,632/QALYs), the probability that KiVa is cost-effective is close to 100%. At a threshold value of 100,000 SEK per QALY (€10,526) and 200,000 SEK per QALY (€21,053), the probability that KiVa is cost-effective is 68 and at 96%, respectively.

Fig. 3
figure 3

Cost-effectiveness acceptability curve: the probability that KiVa is cost-effective at different values per QALY

Full size image

Discussion

Being a victim of bullying causes significant suffering for children and adolescents (Takizawa et al. 2014). The importance of various bullying prevention programs has been acknowledged during the past decades, and numerous of them are run and implemented in schools. However, very few bullying prevention programs have strong evidence regarding effectiveness (Flygare et al. 2013), and there are more or less no evidence regarding the cost-effectiveness of bullying prevention programs.

In this study, we evaluated the cost-effectiveness of the KiVa bullying prevention program in the context of a Swedish elementary school. A Markov cohort model with a time frame of 9 years, or in other words over the full compulsory education in Sweden (grades 1 to 9), estimated the ICER at 131,321 SEK (€13,823) per QALY. This estimate can be compared to the Swedish national threshold value for care programs, which states that a ratio below 500,000 SEK (€52,632) per QALY can be considered moderately cost-effective (The National Board of Health and Welfare 2011). The results from the deterministic sensitivity analysis showed that the cost-effectiveness was substantially related to the number of grades/years for which the program was implemented. For example, in a school with only grades 7–9, which is relatively common in the Swedish school context, the cost per QALY was estimated at 605,000 SEK (€63,684), which is higher than the threshold value of 500,000 SEK (€52,632). This highlights that the investment gives a substantially better cost-effectiveness if implemented over more grades, the primary reason that the implementation comes with some costs that are almost “fixed,” which have to be paid irrespective of the number of grades for which the program is implemented. The program therefore shows a better “return on investment” if implemented over as many grades as possible.

Compared to a recent study on the cost-effectiveness of the Olweus Bullying Program implemented in grades 7–9 (Beckman and Svensson 2015), the results here indicate that KiVa is more cost-effective. The cost-effectiveness of the Olweus Program showed a cost per victim-free year of 131,250 SEK (€13,816), which can be compared to 12,222 SEK (€1287) with the KiVa program (or 36,299 SEK for a 3-year KiVa program). The cost per student of the two programs was fairly similar, which implies that the main reason for the better cost-effectiveness of the KiVa program is due to the evidence in favor of a higher effectiveness in terms of reducing the prevalence of bullying (Table 2). However, the two programs have not been compared “head-to-head,” so all comparisons here rely on indirect comparisons, which implies that the different results should be interpreted with caution.

Transferability of the results as the basis for decision making to other settings is reliant on the suitability of the assumed parameter values (Table 1). For instance, in a country with a higher bullying prevalence than Sweden, the ICER would be slightly lower (i.e., KiVa would be more cost-effective). In a country with lower total costs of implementing KiVa, e.g., due to lower labor costs or larger classes (cohort size), also would imply a lower ICER.

Some important limitations need to be mentioned. To begin with, the evaluation was based on a modeling approach and the results are therefore dependent on the assumed parameter values as informed by secondary literature. Since KiVa is a relatively new program, limited research and data on the effectiveness of the program is currently available, and there is no evidence on the long-term effects of KiVa. Further, the effectiveness of KiVa may vary depending on degree of implementation fidelity, e.g., engagement and accuracy among those involved (Salmivalli et al. 2014), which implies that there is a need for further high-quality RCTs where KiVa is evaluated in different school contexts.

It is also a possibility that the estimated costs of the KiVa program was overestimated since the evaluation context compares KiVa to status quo (treatment as usual). Even if the typical school is not utilizing a formal manual-based program, there are most likely some activities and costs associated with the status quo activities. If implementing the KiVa program, these activities could be replaced by the KiVa program, and therefore, the incremental (additional cost) of the KiVa program would be lower than the explicit costs with the KiVa program as used in our model.

This study only considers the short-term costs and effects, as more long-term consequences of bullying are currently insufficiently explored to conduct a fair and realistic analysis (Takizawa et al. 2014). There are studies indicating that bullying causes continuing negative effects into adulthood (Sigurdson et al. 2015), but more knowledge is necessary to confirm such allegations. With more credible knowledge about potential long-term beneficial effects of preventing bullying, it may be that the cost-effectiveness of KiVa is underestimated.

Conclusions

The cost-effectiveness of the KiVa bullying prevention program was estimated to 7879 SEK (€829) for each additional victim-free year gained and to 131,321 SEK (€13,823) for each additional QALY gained. There is a lack of evidence on the effectiveness of bullying prevention programs, and assessing effectiveness and cost-effectiveness is therefore important to inform decision makers with fixed budget constraints. The results indicate that KiVa is cost-effective compared to the Swedish cost-effectiveness threshold value if implemented during 6 or 9 years (grades 1–6 or grades 1–9), but not cost-effective if only implemented in lower secondary school for 3 years (grades 7–9).