A synthesis of meta-analytic evidence of behavioral interventions to reduce HIV/STIs

Since the 1980s and 1990s, numerous trials have been conducted to test the efficacy of behavioral interventions that aim to prevent sexually transmitted infections (STIs) and human immunodeficiency virus (HIV) by encouraging people to use condoms or reduce their number of sexual partners. In turn, in the last 10–15 years a large number of meta-analyses have been published. Some of these have focused on different target groups such as African Americans (Darbes et al., 2008), adolescents (Johnson et al., 2003), or men who have sex with men (MSM) (Johnson et al., 2005), or different types of interventions such as the use of computer-technology (Noar et al., 2009) or social media (Swanton et al., 2015). Despite their different foci, these meta-analyses often show positive pooled effect sizes for changes in condom use and other sexual risk behaviors. However, the effect sizes have been found to be significantly heterogeneous, which has led some researchers to explore which factors moderate intervention efficacy through stratified analysis and meta-regression techniques.

Given the growing numbers of meta-analyses that have conducted moderator analyses, researchers are now turning to systematically reviewing the meta-analytic studies themselves. Five such meta-reviews, or meta-syntheses, have been published in recent years. Each provide different insights into the moderators of intervention efficacy effect size (Johnson et al., 2014; Lorimer et al., 2013; Noar, 2008; Protogerou & Johnson, 2014; Vergidis & Falagas, 2009).

Four out of the five meta-reviews have focused their attention on meta-analyses of interventions targeted at specific groups such as MSM, adolescents, or specific ethnicities (Lorimer et al., 2013; Noar, 2008; Protogerou & Johnson, 2014; Vergidis & Falagas, 2009). A range of factors have been shown to be associated with larger intervention effects. Sessions delivered to single-ethnicity or single-gender groups were more efficacious than mixed ethnicity/gender sessions (Noar, 2008). For African Americans, greater efficacy was found for interventions that involved peer education, whereas for Latinos the effect was larger in interventions targeted at same sex groups (Vergidis & Falagas, 2009). Group and community-level interventions increased condom use and reduced unprotected anal intercourse in interventions delivered to MSM (Lorimer et al., 2013). The use of motivation enhancement skills training and use of theory was linked to efficacy in interventions targeted to adolescents (Protogerou & Johnson, 2014).

Unlike these four meta-reviews, Johnson et al. (2014) did not restrict their synthesis to prior meta-analyses focused on particular target groups. They focused instead on the 56 behavioral HIV prevention meta-analyses that had been included in a meta-synthesis of behavior change interventions conducted by Johnson et al., (2010). Two intervention content dimensions, skills training and motivational enhancement, were identified as being significantly associated with greater risk reduction behaviors in multiple meta-analyses. However, the synthesis lacked detail about the results found for all intervention content dimensions. In particular, their focus was on identifying only the significant moderators; the non-significant dimensions were not identified. This limits our ability to explore not only the reasons for lack of consensus in results between meta-analyses (i.e., why is a dimension a significant moderator in one meta-analysis but not another?), but also to identify dimensions that never, or rarely, produce significant effects (i.e., which dimensions do not make a difference to intervention effectiveness?)

This limitation is addressed in the meta-review reported in this paper, in which we present a comprehensive and detailed synthesis of previous meta-analyses that have tested the significance of intervention dimensions. The intervention dimensions selected for analysis are listed and defined in Table 1 and include mode of delivery dimensions (e.g., number of sessions, group delivery) and communicator dimensions (e.g., matched ethnicity, expert delivery), as well as the content dimensions (e.g., individual tailoring, condom skills training) analyzed by Johnson et al. (2014). Also, unlike the meta-reviews conducted by Lorimer et al. (2013), Noar (2008), Protogerou and Johnson (2014), and Vergidis and Falagas (2009) we did not restrict our analysis to meta-analyses that had focused on particular target groups like MSM, adolescents, or specific ethnicities.

As shown in Fig. 1, 37 meta-analyses were included in this meta-review. Table 2 shows the data extracted from each study. The meta-analyses varied in terms of how inclusive they were with some focusing on specific types of populations such as adolescents (Chin et al., 2012; Johnson et al., 2003; Johnson et al., 2011; Mullen et al., 2002), STI clinic patients (Crepaz et al., 2007; Scott-Sheldon et al., 2010), African Americans (Crepaz et al., 2007, 2009; Darbes et al., 2008; Henny et al., 2012; Johnson et al., 2009; Reid et al., 2014), Hispanics (Crepaz et al., 2007; Herbst et al., 2007), MSM (Herbst et al., 2005; Higa et al., 2013; Johnson et al., 2005), heterosexuals (Henny et al., 2012; LaCroix et al., 2013; Neumann et al., 2002; Tyson et al., 2014), women only (Crepaz et al., 2009; Lennon et al., 2012), men only (Henny et al., 2012), or drug users (Meader et al., 2013; Prendergast et al., 2001). Beyond the interventions tested on North American populations, which were included in most of the meta-analyses, others were restricted to particular countries like South Africa (Scott-Sheldon et al., 2013) and China (Liu et al., 2014; Xiao et al., 2012; Zheng & Zheng, 2012), or Asian countries (Tan et al., 2012).

Some reviews also placed restrictions on the types of interventions that were included. Restrictions included excluding interventions where recipients engaged in behaviors like role playing or condom-use skills (Albarracin et al., 2003), pamphlet studies (Johnson et al., 2003; Johnson et al., 2011), or mass-media interventions (Scott-Sheldon et al., 2011). Others restricted themselves to interventions that were group-based (Chin et al., 2012), multi-session (Meader et al., 2013), single session (Eaton et al., 2012), face-to-face (Huedo-Medina et al., 2010; Lennon et al., 2012), used computer-technology (Noar et al., 2009), used new media (Swanton et al., 2015), or were informed by the Theory of Planned Behavior (Tyson et al., 2014).

The final types of restrictions were concerned with the study design or information provided in the intervention reports. Some meta-analyses only included studies that comprised both a pre-test and post-test (Albarracin et al., 2005; Albarracin et al., 2003; Albarracin et al., 2008; Durantini et al., 2006; Earl & Albarracin, 2007), or where information was provided about the interventionist (Durantini et al., 2006) or percentage of Latinos in the sample (Albarracin et al., 2008), or where depression measures were obtained and separate results were provided for women (Lennon et al., 2012).

Although these restrictions reduce the overlap between the meta-analyses included in this meta-review, several of the meta-analyses share the same intervention studies. For example, the analyses reported by Durantini et al. (2006) and Earl and Albarracin (2007) were both based on a sub-set of papers reviewed by Albarracin et al. (2005). All of the studies included in Johnson et al. (2003) were included in the later meta-analysis reported in Johnson et al. (2011), and Reid et al. (2014) report a secondary analysis of studies included in Johnson et al. (2009). The overlap is particularly important to consider when synthesizing and interpreting the results of the moderator analyses.

A growing number of meta-analyses of STI/HIV prevention interventions have explored the sources of heterogeneity of effect sizes by testing the extent that various study characteristics moderate effect sizes. This meta-review synthesizes the results from 37 meta-analyses identified through a systematic search of the published literature. A range of mode of delivery, communicator and content dimensions were examined and consistent positive effects were found for a small number of characteristics including matching the gender or ethnicity of the communicator to the intervention recipients, group targeting or tailoring of the intervention, use of a theory to underpin intervention design, providing factual information, presenting arguments designed to change attitudes, and providing condom skills and intrapersonal skills training.

Although the use of theory moderator was not specific to a particular theory, our findings do lend support to the Information-Motivation and Behavioral Skills (IMB) model of HIV preventive behavior (Fisher & Fisher, 1992). This model proposes that information and behavioral skills are necessary, but not sufficient, for HIV prevention. People’s attitudes towards HIV prevention are also an important determinant of their motivation to initiate and maintain preventive behavior. The role of motivational enhancement and skills training was also highlighted in the meta-review conducted by Johnson et al. (2014), but the broader scope of our analysis has identified the potentially important roles of features such as matching the person delivering the intervention and targeting the content to the characteristics of the recipient. This highlights the value of designing and delivering interventions which are aimed at modifying IMB components in a group-appropriate fashion.

Also, by reporting the non-significant and negative effects alongside the positive effects, our meta-review highlights dimensions that either make no difference or could potentially compromise intervention efficacy. This includes dimensions that we might have expected to make a positive difference, such as the overall duration, number of sessions, peer delivery, tailoring to the individual, use of threat/fear induction methods, and normative arguments.

However, non-significant effects were quite prevalent and we need to be cautious about ruling out the potential value of some dimensions when in some meta-analyses the lack of significance might be attributable to lack of statistical power. We highlighted k < 20 as a small sample where lack of power might be an issue, although it should be noted that even with 20 studies the moderator effect size would need to be quite large to produce a significant effect. Meta-analyses probably need at least 50 or 60 studies to have sufficient power to detect even medium moderator effect sizes. Notably only 10 of the 37 meta-analyses included in this meta-review were based on 50 or more studies and only three of those included literature published within the last 5 years: LaCroix et al. (2014) k = 58; Scott-Sheldon et al. (2011) k = 67; and Tan et al. (2012) k = 52. Notably the largest reviews that include over 100 studies do not include any literature published within the last 10 years: Albarracin et al. (2005) k = 200; Albarracin et al. (2008) k = 350; Durantini et al. (2006) k = 166; Earl and Albarracin (2007) k = 180. This is probably because the most recent meta-analyses have tended to adopt increasingly restrictive inclusion criteria (i.e., focussing on particular types of interventions or population groups) which limit the potential to statistically examine moderators of intervention efficacy.

There are some limitations to this meta-review that need to be considered when interpreting the findings. Firstly, although we conducted a systematic and thorough search of the literature, we cannot rule out the possibility that relevant meta-analyses were not included. Secondly, we are reliant on the original authors’ literature search, data extraction, and analysis. Our synthesis relies not only on the thoroughness of the literature search and reliability of the coding of dimensions, but also the adequacy and accuracy of the statistical methods used to compute effect sizes and test moderator effects. Bearing in mind that all of the meta-analyses are published in peer reviewed journals we have placed some faith in the fact that the meta-analyses were conducted appropriately. However, there were some differences in the methods used to compute effect sizes (e.g., whether they were adjusted for baseline differences), and to test moderator effects (e.g., whether analyses were based on fixed, random, or mixed effects assumptions and use of Bonferroni corrected significance values), that may contribute towards the different patterns of results found between meta-analyses. There is also the possibility that we may have miscategorized the dimensions. Although the coding was checked between two researchers, the definitions used by some meta-analysts for their tested moderators were not always provided in detail. Also, some dimensions had quite broad definitions that may have picked up on subtly different issues. Group targeting/tailoring for example included both whether an intervention was targeted at a particular group and also whether the information was designed to be specific to the target audience. We grouped these two features together, but this could have masked different effects on intervention efficacy. Finally, the insights gained from this meta-review are somewhat restricted to identifying the moderators of intervention effect sizes for behavioral outcomes like condom use, rather than biomarker-confirmed outcomes such as STI/HIV infection rates. Our insights were restricted because only 7 of the 37 meta-analyses tested moderator effects on STI/HIV incidence. If we want to demonstrate the clinical relevance of behavioral interventions, there clearly needs to be more research which evaluates the effects on STI/HIV infection rates and considers their role relative to innovations in pharmacological prevention such as pre-exposure prophylaxis (Centers for Disease Control and Prevention, 2014).

Despite its limitations, this meta-review has advanced our understanding of factors linked to improved efficacy of behavioral interventions. It has also highlighted deficiencies in the existing meta-analytic literature including the tendency to narrow the focus and inclusion criteria. The narrow focus of many of the meta-analyses conducted in recent years has undermined the reliability of the moderator analyses that have been conducted. To further our understanding an up-to-date and less restricted meta-analysis of the HIV prevention literature is needed. A less restricted meta-analysis might also enable not only more rigorous multivariate tests of moderating factors but also an exploration of how the intervention delivery, communicator, and content factors interact with each other and other characteristics, like the study date, type of recipients, or country the study was conducted in. This could include testing some of the interactions tentatively highlighted in this meta-review, for example whether skills-based techniques work better with adolescents or threat/fear induction messages backfire when delivered to certain cultural groups. Exploring the role of factors like the study date would also provide an indication of whether the efficacy of behavioral interventions has changed over time. This type of analysis could provide insights into whether intervention efficacy has been influenced by innovations in the design of interventions or by changing external circumstances such as improved treatment or the broader social context.

The findings of this meta-review suggest that HIV/STI prevention interventions should involve a number of features. Researchers should consider who delivers the intervention, as interventions that match the gender or ethnicity of the communicator to the recipients tend to be more successful. In terms of content, there seems to be value in designing interventions that are group targeted or tailored, use theory to underpin intervention design, provide factual information, present arguments designed to change attitudes, and provide condom skills/intrapersonal skills training. In designing interventions, it is worth noting that the duration and number of sessions did not affect intervention success. Also, expert delivery was not more successful than peer delivery. These findings have important implications for the field and highlight how less labor-intensive (and thus cheaper) interventions may be as successful as those that are more labor-intensive. The specific method of delivery might however be important and a priority for future research is to compare traditional face-to-face approaches against novel methods which use social media and mHealth applications.