Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Meta-meta-analysis on the effectiveness of parent-based interventions for the treatment of child externalizing behavior problems

  • Tanja Mingebach ,

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing

    tanja.mingebach@med.uni-marburg.de

    Affiliation Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Marburg, Germany

  • Inge Kamp-Becker,

    Roles Supervision, Validation, Writing – review & editing

    Affiliation Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Marburg, Germany

  • Hanna Christiansen,

    Roles Supervision, Validation, Writing – review & editing

    Affiliation Department of Clinical Child and Adolescent Psychology, Philipps University Marburg, Marburg Germany

  • Linda Weber

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Visualization, Writing – review & editing

    Affiliation Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Marburg, Germany

Abstract

Objective

The aim of this study is to perform the first meta-meta-analysis on the effectiveness of parent-based interventions for children with externalizing behavior problems. Even though parent-based interventions are considered as effective treatments the effects reported in meta-analyses are heterogeneous and the implementation in clinical practice is suboptimal. Recapitulative valid effect predictions are required to close the still existing gap between research findings and clinical practice. The meta-meta-analytic results on changes in child behavior shall result in a clear signal for clinical practice.

Methods

This meta-meta-analysis encompasses 26 meta-analyses identified via search in electronic databases (PsycINFO, Medline, PubMed). Meta-analyses had to report effects of parent-based interventions on child behavior and focus on children under the age of 13 years with externalizing behavior problems in a clinical setting. Analyses were based on random-effects models. To combine results, the effect estimates of the meta-analyses were transformed to SMD and weighted to correct for primary study overlap. The meta-meta-analysis is registered on PROSPERO, registration number CRD42016036486 and was conducted in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement (PRISMA).

Results

The results indicate a significant moderate overall effect for child behavior (SMD = 0.46) as well as for parent reports (SMD = 0.51) and observational data (SMD = 0.62). Further analyses focusing on child externalizing behavior yielded significant and moderate effects (SMD = 0.45). All effects remained stable to follow-up. Considerable heterogeneity was observed within results.

Conclusion

Parent-based interventions are shown to be effective in improving behavior in children with externalizing behavior problems, as assessed using parent reports and observational measures. The present results should encourage health care providers to apply evidence-based parent-based interventions.

Introduction

Externalizing behavior disorders constitute one of the major reasons for the referral of children to mental health agencies [1]. Rates of externalizing disorders in preschool children are similar or even slightly higher than in older age [2,3]. Due to the high prevalence rates and long-lasting impairment associated with an early beginning of disruptive behaviors [4,5], effective interventions are needed that target children at risk of disturbed developmental processes.

The role of parents is essential in the complex interaction concerning child development. Parents need to exhibit effective parenting behaviors as well as responsiveness and sensitivity to child signals and behaviors to facilitate socialization processes in children [6]. Moreover, parenting practices such as inconsistent discipline or negative emotional expressiveness negatively affect child emotion regulation, which in turn can lead to disruptive behavior problems [7]. In particular, cycles of escalating coercive child-parent interactions as well as harsh and inconsistent parenting practices contribute to the development and maintenance of behavior problems [8,9]. Effective parenting behaviors that enhance children’s self-regulatory skills, facilitate prosocial behaviors, and that are related to preventive effects on behavior problems are therefore essential.

Parent-based interventions specifically target these relevant parental behaviors. For the treatment of externalizing behavior problems there is evidence that parent-based interventions show positive effects on parental characteristics such as parenting behavior, parental perceptions and parental mental health [10] and improve child behavior [1113]. However, despite the broad data base, the implementation in clinical practice and access to parent-based interventions is limited [14]. Furthermore, effects of parent-based interventions are heterogeneous, and range from small to large effect sizes making estimation of effects difficult [10]. This variation in meta-analytic effects may be caused by methodological and content-related heterogeneity [15]. Results can vary depending on the source of the rating [9] and the definition of the outcome category. Furthermore, the inclusion criteria differ between the studies, and the studies include different populations (e.g. clinical vs. non-clinical trials), interventions, or study designs.

Additionally, effects of parent-based interventions on child behavior are mostly assessed by parent report, which raises doubts about the reliability of these results [16]. Positive attributions of parent-based interventions may emerge, as otherwise, the invested costs are perceived as being too great [17]. That is why more recent studies take into account observational measures as a more objective source of information on child behavior. Those indicate a stable small to moderate effect of parent-based interventions on child behavior, inflicting further heterogeneity.

Aims of this study

To clarify the magnitude of effects and to facilitate the implementation of parent-based interventions in clinical practice, the aim of the present study is to conduct a comprehensive review and meta-meta-analysis summarizing all existing meta-analytic estimates concerning the impact of parent-based interventions on child behavior in a population of children with externalizing behavior problems. The effects on parental characteristics will be presented elsewhere [18].

The term externalizing behavior problems includes children with externalizing behavior problems and diagnosed disorders as conduct disorder (CD), oppositional defiant disorder (ODD), and attention-deficit hyperactivity disorder (ADHD). This broad understanding reflects a dimensional conceptualization of externalizing behavior problems as well as the relevant diagnostic classifications and there is evidence for this conceptualization [1921]. Surely this dimensional understanding of externalizing behavior problems raises the risk of heterogenous populations and the inclusion of results from a preventive setting while this review focuses on effects in a clinical setting. Nevertheless, the opportunity of including a larger database and the closeness to clinical practice are a strong argument in favor of including both children diagnosed with disorders as ODD, CD and ADHD as well as children described as presenting with externalizing behavior problems.

Meta-meta-analyses (also called second-order meta-analyses) are meta-analyses of meta-analyses. They apply similar techniques to meta-analyses of primary studies in order to combine results of meta-analyses [2225]. In a first step, the effects of parent-based interventions on child behavior are summarized irrespective of the source of information. In a second step, effects on child behavior are analyzed separately for parent report and observational data. Furthermore, a third analysis is restricted to meta-analyses explicitly reporting results on externalizing behavior problems alone. All displayed analyses are conducted for post and follow-up measures. To reach an unbiased evaluation of the effectiveness of parent-based interventions, overlap of primary studies in different meta-analyses is taken into account.

Methods

The meta-meta-analysis is registered on PROSPERO, registration number CRD42016036486 (S2 File) and was conducted in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement (PRISMA, S1 Table).

Inclusion criteria

Meta-analyses were eligible if they measured the efficacy of parent-based interventions for preschool- and school-aged children with externalizing behavior problems (see above). They had to focus on parent-based interventions for children with mental health disorders (ODD, CD, ADHD) or externalizing behavior problems and could not be conducted solely in a preventive setting. At least one child behavior outcome had to be reported. Meta-analyses published in English and German were considered.

The following two outcome categories were stated: (1) child behavior overall defined as a broad child behavior outcome including positive behaviors (e.g. prosocial), externalizing behavior problems (e.g. noncompliance, aggressive behavior, disruptive behavior) and internalizing symptoms (e.g. anxiety, depressive symptoms); consequently measures of any kind of non-performance-related child behavior were considered, on the one hand global measures as e.g. Child Behavior Checklist (CBCL), Dyadic Parent-Child Interaction Coding System (DPICS), Teacher Assessment of Social Behavior (TASB) and on the other hand specific measures as e.g. Children’s Depression Inventory (CDI), Social Behaviour Questionnaire (SBQ) (2) externalizing child behavior defined as solely disruptive behaviors that violate social norms (e.g. noncompliance, aggressive behavior); only measures restricted to externalizing behavior problems were considered, e.g. Eyberg Child Behavior Inventory (ECBI), Child Behavior Checklist-Externalizing (CBCL-E), Strengths and Difficulties Questionnaire Conduct Scale (SDQ-Conduct).

Search strategy

A systematic search in different electronic databases (PsycINFO (EBSCO), Medline (OVID), PubMed) was undertaken in March 2016 and updated in April 2018 to determine relevant meta-analyses. The following search terms were employed to look for relevant meta-analysis: meta-analysis AND parent* training OR parent* intervention OR parent* program AND children OR preschool OR toddler OR childhood OR infant. In the databases PubMed and Medline all fields were searched. In the database PsycINFO only the abstracts were searched, because searching all field was not selective enough. No further limits were set. The full electronic search strategy is illustrated in Appendix A in S1 File.

Screening of records and data extraction

Eligibility assessment was independently realized by two reviewers (LW, TM). Disagreements were subsequently resolved by consensus. First, all abstracts were screened (n = 297) and the following aspects led to exclusion of studies (n = 220): no meta-analysis; no reference to topic of interest; children with disorders/problems other than disruptive behavior problems (e.g. autism spectrum disorders, physical impairment); merely preventive interventions; publication in a language other than English or German; full text not available; update of article available. Furthermore, unpublished studies were excluded.

In a next step, screening of the full texts of the remaining articles was undertaken (n = 77) and the following aspects led to exclusion: a main focus other than the topic of interest (n = 22); sample too specific (e.g. teenage parents) or topic too specific (e.g. home visitation) (n = 7); not enough statistical information (n = 9); no outcomes of child behavior (n = 3); merely preventive interventions (n = 8). We placed our focus on face-to-face parent training interventions to obtain maximum homogeneity. Studies on children with (developmental) disabilities were included if change in externalizing behavior was the primary outcome.

Relevant information was gathered from articles using a data extraction sheet (Table A in S3 File). Two reviewers checked the extracted data, and disagreements were resolved by consulting the articles and by discussion between the two authors (LW, TM). When information regarding sample size was not directly stated in the text, sample sizes were estimated from tables as exactly as possible. If only an overall sample size was stated, the sample size was divided into two to gain approximated values for the experimental and control groups. In exceptional cases, primary studies were consulted due to ambiguous information in meta-analyses. Additionally, if relevant information was lacking, further information was requested from authors of meta-analyses.

Some meta-analyses reported overall results as well as results for subsets of primary studies. In the case of meta-analyses on Triple P (delivering different models of parent interventions differing by intensity of intervention), only data for Levels 4 (broad focus parent training) and 5 (intensive family intervention) were included (if possible), because these levels target parents of children with more severe behavior problems and are more intense [26]. Some meta-analyses also reported effect sizes for statistically defined subsets of studies (e.g. only high-quality studies, without outliers, etc.). In such cases, overall effect sizes were included to obtain a larger data base. Furthermore, only non-weighted effect sizes were included, as synthesis of effects that are weighted for moderators or mediators can be very challenging [23].

Risk of bias in included studies and quality ratings

The availability of risk of bias assessment and its quantification of influence on effect estimates in the included meta-analyses was registered.

Two reviewers independently rated the quality of every meta-analysis according to the PRISMA statement employing the PRISMA 2009 Checklist [27]. Each item was assessed on a 3-point Likert scale coding 0 (“item not fulfilled”), 1 (“item partially fulfilled”) or 2 (“item completely fulfilled”). Thus, quality ratings could range between 0 and 54. To ascertain inter-rater reliability, the intraclass correlation coefficient (ICC) was estimated for total scores using the SPSS version 22 [28]. The intraclass correlation was .886 (p < .001) and can thus be deemed excellent [29]. To depict a quality index, the means of the total scores of both raters were taken.

Correction of primary study overlap

Given that overlap of primary studies might lead to distortion of results [24], overlap of primary studies was corrected for (see Appendix B in S1 File for formulae). The correction of primary study overlap had to be executed individually for each meta-meta-analysis.

To establish that each primary study contributed only once to the meta-meta-analysis, an adjusted value was estimated for each primary study that was included in multiple meta-analyses. For this purpose, the number of meta-analyses in which each primary study was included was determined. If primary studies could not be positively related to specific effect sizes in a meta-analysis, a conservative approach was adopted by correcting for all primary studies included in the meta-analysis. The inverse of this number was considered as the uniqueness of this study. For each meta-analysis, the uniqueness values of the included primary studies were summed up to determine the adjusted number of primary studies (kadj). Due to statistical reasons, meta-analyses with kadj less than or equal to three were subsequently excluded and kadj of the remaining studies were recomputed. Based on the adjusted number of primary studies (kadj), the standard errors for the effects from meta-analyses and finally the overlap-corrected weights for each meta-analysis were estimated.

Meta-analytic procedure

We conducted random effects models using Comprehensive Meta-Analysis 2.0 (CMA) by Biostat [30] and R [31] package metafor [32].

For each meta-analysis, effect estimates were transformed to standardized mean differences (SMD). Only effect estimates that were based on at least two primary studies were used. If a meta-analysis stated multiple effect sizes for one outcome, these estimates were aggregated into a single effect size. Thus, each meta-analysis contributed only one effect size to the meta-meta-analysis. An overall SMD was computed from meta-analyses for each outcome. At this juncture, effect sizes of meta-analyses were weighted according to our colleagues [24] by the adjusted number of included primary studies kadj to obtain an overall effect estimate accounting for primary study overlap (Appendix B in S1 File). In accordance with Cohen [33], 0.2 was interpreted as a small effect size, 0.5 as moderate, and 0.8 as large.

Heterogeneity was estimated via the Q test and I2 statistic. With regard to I2, heterogeneity can be interpreted as low (25%), moderate (50%) or high (75%) [34].

Publication bias

Visual inspections of funnel plots displaying at least ten studies were carried out [15]. Furthermore, fail-safe Ns (Nfs) [35] were calculated for each outcome category to detect the number of studies with null results needed to reduce the observed result to a small effect size of 0.1.

Results

Search results

The qualitative synthesis encompasses the data of 28 meta-analyses (Fig 1). Due to the requirement of kadj > 3, two additional meta-analyses [36,37] were ruled out, so that 26 meta-analyses were finally included in the quantitative synthesis.

thumbnail
Fig 1. PRISMA flow diagram for studies included in and excluded from the meta-meta-analysis.

https://doi.org/10.1371/journal.pone.0202855.g001

Study characteristics

The study characteristics of the included meta-analyses are illustrated in Table 1. On average, individual meta-analyses included 33.68 primary studies (SD = 34.84, range 4–156). While 20 meta-analyses exclusively considered controlled trials, 8 meta-analyses included controlled as well as uncontrolled trials. Therefore, effects were estimated on the one hand as comparisons of experimental and control groups and on the other hand on pre- to post-/follow-up-measures. While at post-measurement most meta-analyses solely included controlled trials (n = 23), data at follow-up measurement are based on controlled trials as well as pre- to follow-up-measures.

23 studies contributed data for post measurement. Ten meta-analyses provided data on long-term effects, two of which could not be included in the meta-meta-analysis due to incomplete statistical information [46,52]. The eight included meta-analyses on average assessed follow-up between 2.88 (SD = 1.96) and 31.25 (SD = 20.51) months after completion of intervention (range 1–72 months). Overall, at post-measurement, 411 primary studies (reported in 436 articles) were included in the meta-meta-analyses and 135 at follow-up (reported in 148 articles).

The majority of the studies related to behavioral parent interventions, which are characterized by teaching parents effective behavioral strategies and skills to manage child behavior. Some studies [11,12,43] also included non-behavioral interventions. These interventions focus, for example, on parent-child communication or problem-solving strategies. Furthermore, some meta-analyses focused on specific parent-based interventions (Triple P, Stepping Stones Triple P, Incredible Years program, PCIT; e.g., [13,42,48,53,56]).

Basic descriptive statistics were often stated only partially. Although parental and child gender were not declared in all meta-analyses, one can act on the assumption that the majority of data was provided by mothers reporting on their sons (e.g., [40,50,56]). In most instances studies solely included parents of children under the age of 13 (range 0–18 years), only eight studies included youth older than 13 to some extent (e.g., [39,56]). The mean age could not be calculated due to missing information in the primary meta-analyses.

Child behavior overall as well as externalizing child behavior was assessed through parent reports (e.g. Child Behavior Checklist (CBCL)) or through observation of child behavior (e.g. Dyadic Parent–Child Interactive Coding System), and in several cases via teacher reports (e.g. Sutter-Eyberg Student Behavior Inventory (SESBI); Table B in S3 File).

Risk of bias in included studies and quality ratings

The assessments of the potential risk of bias were conducted in 25 out of 28 meta-analyses (Table 1). Of those meta-analyses, 18 reported results concerning the influence of a potential risk of bias on study results (mainly publication bias), of which 12 found an influence.

Quality indices based on the quality ratings of every meta-analysis employing the PRISMA 2009 Checklist are illustrated in Table 1. The mean quality index was 38.88 (SD = 6.85, range 24–49.5). Overall, the quality of the included meta-analyses is satisfactory as the mean quality index exceeds the benchmark of two-thirds of the total score.

Syntheses of results

Results are presented separately for child behavior overall and externalizing child behavior and are split into post-measurements and follow-up outcomes. Additional information is available in Tables C and D in S3 File.

Child behavior overall

Data for post-measurements of child behavior based on parent report and observational data (overall effect) emerge from 23 meta-analyses. Meta-meta-analysis revealed a statistically significant and moderate effect for parent-based interventions (SMD 0.46, 95% CI 0.38 to 0.54, p = < .0001). Strong evidence of heterogeneity was observed (Q = 150.30, p = < .0001, I2 = 85.36%).

In a further step, separate effect sizes for parent report and observational data were examined. Meta-meta-analysis on parent report data revealed a statistically significant moderate effect size (SMD 0.51; 95% CI 0.39 to 0.64, p = < .0001), but with evidence of significant heterogeneity (Q = 124.66, p = < .0001, I2 = 84.76%).

The analysis of observational data revealed a significant moderate effect size (SMD 0.62, 95% CI 0.18 to 1.06, p = 0.0062) with no significant heterogeneity (Q = 2.72, p = 0.61, I2 = 0%).

Meta-meta-analysis of follow-up effects revealed a statistically significant moderate effect size for overall child behavior outcome (SMD 0.49, 95% CI 0.35 to 0.63, p = < .0001) with evidence of significant heterogeneity (Q = 65.10, p = < .0001, I2 = 89.25%).

Follow-up data on parent report of child behavior revealed a statistically significant moderate effect size (SMD 0.51, 95% CI 0.31 to 0.72, p = < .0001) with evidence of significant heterogeneity (Q = 40.22, p = < .0001, I2 = 87.57%).

Follow-up data on observational child behavior revealed a statistically significant moderate effect size (SMD 0.59, 95% CI 0.24 to 0.94, p = 0.0010) with no significant heterogeneity (Q = 0.02, p = 0.8922, I2 = 0%).

Externalizing child behavior

Meta-meta-analysis on externalizing child behavior at post-measurement revealed a statistically significant moderate effect size (SMD 0.45, 95% CI 0.35 to 0.55, p = < .0001) with evidence of significant heterogeneity (Q = 71.26, p = < .0001, I2 = 84.56%).

Follow-up data on externalizing child behavior revealed a statistically significant moderate effect size (SMD 0.49, 95% CI 0.32 to 0.66, p = < .0001) with evidence of significant heterogeneity (Q = 26.76, p = < .0001, I2 = 88.79%.

Risk of bias across studies

Due to the exclusion of unpublished studies, the risk of publication bias is certainly raised. On the other hand, including unpublished studies could also be a threat to validity, e.g. due to poor methodological quality [15].

The visual inspections of three funnel plots (S1, S2 and S3 Figs) display some asymmetry, thus suggesting a publication bias.

The fail-safe Ns ranged between 10 and 96 (Figs 29) and indicate that at least quadruple to quintuple studies with null results are necessary to reduce effect sizes to a small level of 0.1. This relates to all conducted meta-meta-analyses and emphasizes the robustness of our findings.

thumbnail
Fig 2. Meta-meta-analysis of child behavior overall (post-intervention).

https://doi.org/10.1371/journal.pone.0202855.g002

thumbnail
Fig 3. Meta-meta-analysis of parent report of child behavior overall (post-intervention).

https://doi.org/10.1371/journal.pone.0202855.g003

thumbnail
Fig 4. Meta-meta-analysis of observational data of child behavior overall (post-intervention).

https://doi.org/10.1371/journal.pone.0202855.g004

thumbnail
Fig 5. Meta-meta-analysis of child behavior overall (follow-up).

https://doi.org/10.1371/journal.pone.0202855.g005

thumbnail
Fig 6. Meta-meta-analysis of parent report of child behavior overall (follow-up).

https://doi.org/10.1371/journal.pone.0202855.g006

thumbnail
Fig 7. Meta-meta-analysis of observational data of child behavior overall (follow-up).

https://doi.org/10.1371/journal.pone.0202855.g007

thumbnail
Fig 8. Meta-meta-analysis of externalizing child behavior (post-intervention).

https://doi.org/10.1371/journal.pone.0202855.g008

thumbnail
Fig 9. Meta-meta-analysis of externalizing child behavior (follow-up).

https://doi.org/10.1371/journal.pone.0202855.g009

Discussion

Summary of evidence

To our knowledge, this is the first meta-meta-analysis on the effectiveness of parent-based interventions. Overall, the results suggest significant and moderate effects of parent-based interventions on child behavior that are stable over time. There is a great variability in the current literature regarding the effectiveness of parent-based interventions, ranging from small to large effects. Meta-analyses reporting large or small effects on child behavior are often based on a small number of studies [13,44], which reduces the probability of detecting the true effect [58,59]. The results shown in the current paper are based on 411 primary studies at post-measurement and on 135 studies at follow-up, and thus provide strong evidence for the true effect of parent-based interventions. The smallest effect (SMD = 0.25) was reported by Menting and colleagues [48] analyzing the effectiveness of the Incredible Years program in about 59 studies. Beyond that overall effect, severity of child behavior is reported to be a relevant moderator of effect size; subset analyses indicate a moderate effect size for the program in a treatment study context. These results are in line with the moderate overall effect detected in the present meta-meta-analysis. The meta-analyses by Serketich and Dumas [54] as well as de Graaf et al. [42] report large effect sizes. The meta-analysis by Serketich and Dumas [54] reaches the lowest score in the quality rating and therefore needs to be interpreted carefully. The meta-analysis by de Graaf et al. [42] analyzing the Triple P Program Level 4 shows good results in the quality rating and sets strict limits concerning outcome measure (only ECBI) and intervention (Triple P Level 4). Nevertheless, the effects of the included studies are very heterogenous (range 0.50–1.27) and subset analyses without outliers indicate a moderate effect size comparable to the results of the meta-meta-analysis.

Our results reach beyond the existing evidence by pointing to similar positive moderate and stable effects on child behavior for both parental and observational reports. Thus, they counteract common doubts concerning the evaluation of parent-based interventions. Often, the reliability of parental reports on child behavior in conjunction with parent-based interventions is called into question as due to a justification of efforts, child behavior changes might be reported more positively by parents [16,17]. The present meta-meta-analysis cumulates the existing results of five meta-analyses relating to 59 primary studies supporting parent training effectiveness via observational measures. These results refer to child behavior in general, future studies should also take observational measures of externalizing child behavior into account.

For the child outcome ‘child behavior overall’, different aspects of child behavior are assessed, including measures of disruptive behavior, conduct problems and ADHD symptoms [16,41] as well as measures of internalizing symptoms or prosocial behavior [9,48]. Although these results do represent the effect of parent-based interventions on child behavior overall, it remains unclear which aspects of child behavior are altered to which amount. To counteract this confusion, a separate effect for parent-based interventions based solely on externalizing symptoms was calculated. This more homogenous analysis revealed moderate effects on externalizing child behavior, representing the core symptoms of externalizing disorders in children. Additional analyses with respect to prosocial behavior or internalizing symptoms were not possible due to a lack of data.

Future prospects

Future research should distinguish between different child outcome categories to allow for conclusions regarding the differential effectiveness of parent-based interventions. Since externalizing symptoms are the main target of parent-based interventions, a larger effect for this category can be expected compared to more distal child outcome categories such as internalizing symptoms or prosocial behavior. Combining these outcome categories leads to a somewhat biased appraisal of the effectiveness of parent-based interventions.

The impact of parent-based interventions on internalizing symptoms needs to be evaluated critically, since these often accompany childhood externalizing disorders [2]. Furthermore, research on the effectiveness of parent-based interventions for the treatment of children with internalizing disorders is scarce [10] and needs to be extended. Although some meta-analyses have already presented results on the effect of parent-based interventions on children’s prosocial behavior and reported small effects [48], further research is needed.

Another aim of future research should be to evaluate the comparative effectiveness of different interventions for children with externalizing behavior problems. In the present analyses, it was not possible to calculate separate effect sizes of parent-based interventions compared to different comparison groups (e.g. waitlist, no treatment, treatment as usual, active control group), as most studies aggregated these groups. Although effect sizes can be expected to decrease when comparing parent-based interventions to active control groups [16,43], studies with active control groups were included in our analysis in order to gain a more comprehensive impression of the effectiveness of parent-based interventions [41,44,48]. Confining the analyses to meta-analyses with only one type of comparison group would have restricted the database considerably, and would have excluded many relevant meta-analyses. Furthermore, some meta-analyses included studies with and without control groups (pre- to post-measures) for their effect estimation (e.g., [46,50,56]). Indeed, there is evidence that the quality of the included studies is more relevant than the inclusion of control groups [50].

Beyond that, moderator analyses considering possible variables influencing the effectiveness of parent-based interventions (e.g. symptom severity, child age, delivery format), or mediator analyses on changes in parenting behavior which mediate child behavior changes, are currently not possible and should be an aim for future research.

Limitations

As combining various independent studies inevitably increases the risk of heterogeneity due to the heterogeneous design of the studies [15], the present results should be interpreted with caution [34]. In the present meta-meta-analysis, strict inclusion criteria were set to obtain homogeneity. However, since analyses are based on different meta-analyses, which for their part also include a wide array of primary studies, the achievable homogeneity is limited, as there is still considerable heterogeneity among primary studies (e.g. participants, interventions). A comparison with other meta-meta-analyses is restricted due to the limited number of second-order meta-analyses existing so far. Two previous meta-meta-analyses in the area of interventions for mental health problems did not find significant heterogeneity [24,60]. Nevertheless, meta-meta-analyses in the field of medicine or learning research did find heterogeneity [61,62].

Some results need further consideration. On the one hand the CI for observed behavior data at post-treatment and follow-up was quite wide, although no significant heterogeneity was found. This can on the one hand be due to different observational measures applied in different studies and on the other hand may be due to the small number of meta-analyses included in these analyses. Moreover, the limitation of small numbers of meta-analyses imbedded in analyses concerns all follow-up analyses. Therefore, the observational and follow-up results need to be interpreted with special caution.

Since the main focus of the current meta-meta-analysis was to evaluate the effectiveness of parent-based interventions as an intervention for treating children with externalizing behavior problems respectively externalizing disorders, we excluded studies that focused on the efficacy of parent-based interventions exclusively in preventive settings. However, parent-based interventions can include the parents of children as young as one to two years, and in this age range, the majority of interventions are preventive in nature. Thus, there were many meta-analyses which included prevention studies and clinical studies, and children with symptoms in the non-clinical and in the clinical range (e.g., [39,42,45,51,57]). As research indicates that effects of parent-based interventions in preventive settings are smaller, this could have negatively affected the magnitude of the derived effect sizes. Overall, the research suggests that the magnitude of initial problem severity affects the magnitude of the treatment effect, and for this reason, it is an important moderator with regard to the effectiveness of parent-based interventions [63]. Future studies should distinguish more carefully between prevention and clinical studies, or at least take special care to report more details of the included populations, especially concerning initial problem severity. In this way, the impact of initial problem severity on the effectiveness of parent-based interventions, and the possibility of a differential use of different forms of parent-based interventions could be examined.

Furthermore, it is to be taken into account that behavioral as well as non-behavioral parent-based interventions were included in the analyses. Research points to larger effects for behavioral parent-based interventions [64] thus the current analyses may underestimate the effects of behavioral parent-based interventions. Separate analyses for behavioral and non-behavioral parental interventions were not possible as some meta-analyses delivered combined results for both interventions. Nevertheless, most results rely on behavioral parent-based interventions, only three meta-analyses include behavioral as well as non-behavioral parent-based interventions. A related limitation concerns the combined analyses of interventions for children with ADHD, ODD and CD. On the one hand children with the various disorders could respond differently to parent-based interventions, on the other hand the diagnostic differentiation between these externalizing disorders especially in early childhood is difficult and considerable comorbidity exists. Moreover, most meta-analyses and primary studies include children with behavior problems or externalizing symptoms in general which precludes more differentiated analyses concerning ADHD, ODD and CD.

Another issue of detailed reporting in primary studies and meta-analyses relates to the description of child outcome categories. Unfortunately, meta-analyses often present somewhat heterogeneous child outcome measures, reporting, for example, the effect of parent-based interventions on child behavior, which represents a global measure of the achieved improvement, but also makes further and more detailed analyses impossible.

As meta-meta-analyses are still in their infancy, there are currently no guidelines or quality standards for such studies. When undertaking the present meta-meta-analyses, we drew on Cochrane recommendations and PRISMA guidelines formulated for meta-analyses and applied them as well as possible. Considering the risk of publication bias, one must take into account that the incomplete retrieval of unpublished studies certainly increases bias. On the other hand, the quality of results is weakened by the inclusion of unpublished studies [15]. Although we endeavored to extract data thoroughly, the lack of or insufficiently reported data in the primary meta-analyses limited our efforts, and extracted data might be imprecise in some cases. In case of doubt, data were derived conservatively (e.g. study overlap) to avoid overestimation of effects. To enable more precise analyses and correct interpretation of effects, we appeal to authors of meta-analyses to illustrate all relevant information precisely and to adhere to high quality standards (e. g. PRISMA guidelines [27]).

Conclusions

Parent-based interventions are effective in treating children with externalizing behavior problems by means of reducing child problem behavior overall and externalizing child behavior in particular. Additionally, there are hints of cost-effectiveness [8] and positive long-term effects [51]. Based on these results, this meta-meta-analysis supports the classification of parent-based interventions as an evidence-based intervention for the treatment of children with externalizing behavior problems and disorders. Furthermore, there are some particular interventions that are already based on a broad empirical data base (e.g. Incredible Years, PCIT, Triple P). Goals for future research are to examine the differential effectiveness of parent-based interventions, and to close the still existing gap between research findings and clinical practice [65,66] by implementing this evidence-based intervention in practice.

Based on the solid data base indicating effectiveness of parent-based interventions, there should be a shift towards a broader offer of these interventions. Parents are crucial for the development of their children and are a relevant factor in the genesis of externalizing disorders. Therefore, the inclusion of caregivers in the treatment of child disorders is essential [66]. The long-term consequences of externalizing behavior problems and disorders are impairing for the affected children, their family and their environment, and are also expensive. It is thus crucial to enhance the mental health care for children with externalizing behavior problems and disorders, and we appeal to health care providers to make use of evidence-based parent-based interventions.

Supporting information

S1 File.

Appendix A. Full electronic search strategy. Appendix B. Correction of primary study overlap according to Munder et al. (2013) [24].

https://doi.org/10.1371/journal.pone.0202855.s001

(DOCX)

S2 File. Protocol: PROSPERO, registration number CRD42016036486.

https://doi.org/10.1371/journal.pone.0202855.s002

(PDF)

S1 Fig. Funnel plot displaying all studies on child behavior overall (post-intervention).

https://doi.org/10.1371/journal.pone.0202855.s005

(TIF)

S2 Fig. Funnel plot displaying all studies on parent report of child behavior overall (post-intervention).

https://doi.org/10.1371/journal.pone.0202855.s006

(TIF)

S3 Fig. Funnel plot displaying all studies on child externalizing behavior (post-intervention).

https://doi.org/10.1371/journal.pone.0202855.s007

(TIF)

References

  1. 1. Steiner H., Remsing L., & Work Group on Quality Issues. (2007). Practice parameter for the assessment and treatment of children and adolescents with oppositional defiant disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 46(1), 126–141. pmid:17195736
  2. 2. Egger H.L., & Angold A. (2006). Common emotional and behavioral disorders in preschool children: presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry, 47(3–4), 313–337. pmid:16492262
  3. 3. Lavigne J.V., LeBailly S.A., Hopkins J., Gouze K.R., & Binns H.J. (2009). The prevalence of ADHD, ODD, depression, and anxiety in a community sample of 4-year-olds. Journal of Clinical Child & Adolescent Psychology, 38(3), 315–328. pmid:19437293
  4. 4. Burke J.D., Rowe R., & Boylan K. (2014). Functional outcomes of child and adolescent oppositional defiant disorder symptoms in young adult men. Journal of Child Psychology and Psychiatry, 55, 264–272. pmid:24117754
  5. 5. Nock M.K., Kazdin A.E., Hiripi E., & Kessler R.C. (2007). Lifetime prevalence, correlates, and persistence of oppositional defiant disorder: results from the National Comorbidity Survey Replication. Journal of Child Psychology and Psychiatry, 48(7), 703–713. pmid:17593151
  6. 6. Grusec J.E. (2011). Socialization Processes in the Family: Social and Emotional Development. Annual Review of Psychology, 62, 243–269. pmid:20731599
  7. 7. Duncombe M.E., Havighurst S.S., Holland K.A., & Frankling E.J. (2012). The contribution of parenting practices and parent emotion factors in children at risk for disruptive behavior disorders. Child Psychiatry & Human Development, 43(5), 715–733. pmid:22392414
  8. 8. Furlong M., McGilloway S., Bywater T., Hutchings J., Smith S.M., & Donnelly M. (2013). Cochrane review: behavioural and cognitive-behavioural group-based parenting programmes for early-onset conduct problems in children aged 3 to 12 years (Review). Evidence-Based Child Health a Cochrane Review Journal, 8(2), 318–692. pmid:23877886
  9. 9. Sanders M.R., Kirby J.N., Tellegen C.L., & Day J.J. (2014). The Triple P-Positive Parenting Program: A systematic review and meta-analysis of a multi-level system of parenting support. Clinical Psychology Review, 34(4), 337–357. pmid:24842549
  10. 10. Herr L., Mingebach T., Becker K., Christiansen H., & Kamp-Becker I. (2015). A systematic review of the effectiveness of parent-based interventions for children aged two to twelve years. Kindheit und Entwicklung, 24(1), 6–19.
  11. 11. Dretzke J., Davenport C., Frew E., Barlow J., Stewart-Brown S., Bayliss S., et al. (2009). The clinical effectiveness of different parenting programmes for children with conduct problems: a systematic review of randomised controlled trials. Child and Adolescent Psychiatry and Mental Health, 3:7. pmid:19261188
  12. 12. Lundahl B., Risser H.J., & Lovejoy M.C. (2006). A meta-analysis of parent training: moderators and follow-up effects. Clinical Psychology Review, 26(1), 86–104. pmid:16280191
  13. 13. Thomas R., & Zimmer-Gembeck M.J. (2007). Behavioral outcomes of Parent-Child Interaction Therapy and Triple P-Positive Parenting Program: a review and meta-analysis. Journal of Abnormal Child Psychology, 35(3), 475–495. pmid:17333363
  14. 14. Prinz R. J., & Sanders M. R. (2007). Adopting a population-level approach to parenting and family support interventions. Clinical Psychology Review, 27(6), 739–749. pmid:17336435
  15. 15. Higgins J.P.T., & Green S. (Eds.). (2011). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated march 2011]. The Cochrane Collaboration.
  16. 16. Wilson P., Rush R., Hussey S., Puckering C., Sim F., Allely C.S., et al. (2012). How evidence-based is an ‘evidence-based parenting program’? A PRISMA systematic review and meta-analysis of Triple P. BMC Medicine, 10:130. pmid:23121760
  17. 17. Klein E.D., Bhatt R.S., & Zentall T.R. (2005). Contrast and the justification of effort. Psychonomic Bulletin & Review, 12(2), 335–339.
  18. 18. Weber L., Kamp-Becker I., Christiansen H., & Mingebach T. (2018). Treatment of child externalizing behavior problems: a comprehensive review and meta-meta-analysis on effects of parent-based interventions on parental characteristics. European Child and Adolescent Psychiatry. pmid:29948228
  19. 19. Coghill D., & Sonuga-Barke E.J.S. (2012). Annual research review: categories versus dimensions in the classification and conceptualisation of child and adolescent mental disorders-implications of recent empirical study. Journal of Child Psychology & Psychiatry & Allied Disciplines, 53(5), 469–489. pmid:22288576
  20. 20. Herpers P.C.M., Klip H., Rommelse N.N.J., Taylor M.J., Greven C.U., & Buitelaar J.K. (2017). Taxometric analyses and predictive accuracy of callous-unemotional traits regarding quality of life and behavior problems in non-conduct disorder diagnoses. Psychiatry Research, 253, 351–359. pmid:28427034
  21. 21. Wakschlag L.S., Estabrook R., Petitclerc A., Henry D., Burns J.L., Perlman S.B., et al. (2015). Clinical Implications of a Dimensional Approach: The Normal:Abnormal Spectrum of Early Irritability. Journal of the American Academy of Child & Adolescent Psychiatry, 54(8), 626–634. pmid:26210331
  22. 22. Becker L.A., & Oxman A.D. (2008). Chapter 22: Overviews of reviews. In Higgins J.P.T. & Green S. (Eds.), Cochrane Handbook for Systematic Reviews of Interventions (pp. 607–631). Chichester (UK): John Wiley & Sons.
  23. 23. Cooper H., & Koenka A.C. (2012). The overview of reviews: Unique challenges and opportunities when research syntheses are the principal elements of new integrative scholarship. American Psychologist, 67(6), 446–462. pmid:22352742
  24. 24. Munder T., Brutsch O., Leonhart R., Gerger H., & Barth J. (2013). Researcher allegiance in psychotherapy outcome research: an overview of reviews. Clinical Psychology Review, 33(4), 501–511. pmid:23500154
  25. 25. Schmidt F.L., & Oh I.-S. (2013). Methods for second order meta-analysis and illustrative applications. Organizational Behavior and Human Decision Processes, 121(2), 204–218.
  26. 26. Sanders M.R., & Prinz R.J. (2005). The Triple P system: A multi-level, evidence-based, population approach to the prevention and treatment of behavioral and emotional problems in children. The Register Report, 31, 42–46.
  27. 27. Liberati A., Altman D.G., Tetzlaff J., Mulrow C., Gøtzsche P.C., Ioannidis J.P.A., et al. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. PLOS Medicine 6(7), e1000100. pmid:19621070
  28. 28. IBM Corp. (2013). IBM SPSS Statistics for Windows (Version 22.0). Armonk, NY: IBM Corp.
  29. 29. Hallgren K.A. (2012). Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutorials in quantitative methods for psychology, 8(1), 23–34. pmid:22833776
  30. 30. Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2010). Comprehensive Meta-Analysis. Englewood: NJ: Biostat.
  31. 31. R Core Team (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
  32. 32. Viechtbauer W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.
  33. 33. Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd edn.). Hillsdale, NJ: Laurence Erlbaum Associates.
  34. 34. Higgins J.P.T., Thompson S.G., Deeks J.J., & Altman D.G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327(7414), 557–560. pmid:12958120
  35. 35. Orwin R.G. (1983). A Fail-Safe N for Effect Size in Meta-Analysis. Journal of Educational and Behavioral Statistics, 8(2), 157–159.
  36. 36. Kok L., van der Waa A., Klip H., & Staal W. (2016). The effectiveness of psychosocial interventions for children with a psychiatric disorder and mild intellectual disability to borderline intellectual functioning: A systematic literature review and meta-analysis. Clinical Child Psychology and Psychiatry, 21(1), 156–171. pmid:25633367
  37. 37. Zwi M., Jones H., Thorgaard C., York A., & Dennis J.A. (2011). Parent training interventions for Attention Deficit Hyperactivity Disorder (ADHD) in children aged 5 to 18 years. Cochrane Database of Systematic Reviews, Issue 12. Art. No.: CD003018. pmid:22161373
  38. 38. Buchanan-Pascall S., Gray K.M., Gordon M., & Melvin G.A. (2018). Systematic Review and Meta-analysis of Parent Group Interventions for Primary School Children Aged 4–12 Years with Externalizing and/or Internalizing Problems. Child Psychiatry & Human Development, 49, 244–267. pmid:28699101
  39. 39. Burkey M.D., Hosein M., Morton I., Purgato M., Adi A., Kurzrok M., et al. (2018). Psychosocial interventions for disruptive behavior problems in children in low- and middle-income countries: A systematic review and meta-analysis. The Journal of Child Psychology and Psychiatry. pmid:29633271
  40. 40. Carr A., Hartnett D., Brosnan E., & Sharry J. (2017). Parents Plus Systemic, Solution-Focused Parent Training Programs: Description, Review of the Evidence Base, and Meta-Analysis. Family Process, 56, 652–668. pmid:27232287
  41. 41. Charach A., Carson P., Fox S., Ali M.U., Beckett J., & Lim C.G. (2013). Interventions for Preschool Children at High Risk for ADHD: a comparative effectiveness review. Pediatrics, 131(5), e1584–1604. pmid:23545375
  42. 42. de Graaf I., Speetjens P., Smit F., de Wolff M., & Tavecchio L. (2008). Effectiveness of The Triple P Positive Parenting Program on Behavioral Problems in Children: A Meta-Analysis. Behavior Modification, 32(5), 714–735. pmid:18475003
  43. 43. Dretzke J., Frew E., Davenport C., Barlow J., Stewart-Brown S., Sandercock J., et al. (2005). The effectiveness and cost-effectiveness of parent training/education programmes for the treatment of conduct disorder, including oppositional defiant disorder, in children. Health Technology Assessment, 9(50). pmid:16336845
  44. 44. Gardner F., Montgomery P., & Knerr W. (2015). Transporting Evidence-Based Parenting Programs for Child Problem Behavior (Age 3–10) Between Countries: Systematic Review and Meta-Analysis. Journal of Clinical Child & Adolescent Psychology, 1–14. pmid:25785902
  45. 45. Leijten P., Melendez-Torres G., Gardner F., van Aar J., Schulz S., & Overbeek G. (2018). Are Relationship Enhancement and Behavior Management “The Golden Couple” for Disruptive Child Behavior? Two Meta-analyses. Child Development. pmid:29557553
  46. 46. Maughan D.R., Christiansen E., Jenson W.R., Olympia D., & Clark E. (2005). Behavioral Parent Training as a Treatment for Externalizing Behaviors and Disruptive Behavior Disorders: A Meta-Analysis. School Psychology Review, 34(3), 267–286.
  47. 47. McCart M.R., Priester P.E., Davies W.H., & Azen R. (2006). Differential effectiveness of behavioral parent-training and cognitive-behavioral therapy for antisocial youth: a meta-analysis. Journal of Abnormal Child Psychology, 34(4), 525–541. pmid:16838122
  48. 48. Menting A.T.A., Orobio de Castro B., & Matthys W. (2013). Effectiveness of the Incredible Years parent training to modify disruptive and prosocial child behavior: A meta-analytic review. Clinical Psychology Review, 33(8), 901–913. pmid:23994367
  49. 49. Mulqueen J.M., Bartley C.A., & Bloch M.H. (2015). Meta-Analysis: Parental Interventions for Preschool ADHD. Journal of Attention Disorders, 19(2), 118–124. pmid:24071773
  50. 50. Nowak C., & Heinrichs N. (2008). A comprehensive Meta-Analysis of Triple P-Positive Parenting Program Using Hierarchical Linear Modeling: Effectiveness and Moderating Variables. Clinical Child & Family Psychology Review, 11(3), 114–144. pmid:18509758
  51. 51. Piquero A.R., Jennings W.G., Diamond B., Farrington D.P., Tremblay R.E., Welsh B.C., et al. (2016). A meta-analysis update on the effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology, 12, 229–248.
  52. 52. Rimestad M.L., Lambek R., Zacher Christiansen H., & Hougaard E. (2016). Short- and Long-Term Effects of Parent Training for Preschool Children With or at Risk of ADHD: A Systematic Review and Meta-Analysis. Journal of Attention Disorders. pmid:27179355
  53. 53. Ruane A., & Carr A. (2018). Systematic review and meta-analysis of Stepping Stones Triple P for Parents of Children with disabilities. Family Process. pmid:29520764
  54. 54. Serketich W.J., & Dumas J.E. (1996). The effectiveness of behavioral parent training to modify antisocial behavior in children: A meta-analysis. Behavior Therapy. 27(2), 171–186.
  55. 55. Skotarczak L., & Lee G.K. (2015). Effects of parent management training programs on disruptive behavior for children with a developmental disability: A meta-analysis. Research in Developmental Disabilities, 38, 272–287. pmid:25577178
  56. 56. Tellegen C.L., & Sanders M.R. (2013). Stepping Stones Triple P-Positive Parenting Program for children with disability: A systematic review and meta-analysis. Research in Developmental Disabilities, 34(5), 1556–1571. pmid:23475006
  57. 57. Van Aar J., Leijten P., Orobio de Castro B., & Overbeek G. (2017). Sustained, fade-out or sleeper effects? A systematic review and meta-analysis of parenting interventions for disruptive child behavior. Clinical Psychology Review, 51, 153–163. pmid:27930935
  58. 58. Button K.S., Ioannidis J.P.A., Mokrysz C., Nosek B.A., Flint J., Robinson E.S.J., et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376. pmid:23571845
  59. 59. Slavin R., & Smith D. (2009). The Relationship Between Sample Sizes and Effect Sizes in Systematic Reviews in Education. Educational Evaluation and Policy Analysis, 31(4), 500–506.
  60. 60. Rebar A.L., Stanton R., Geard D., Short C., Duncan M.J., & Vandelanotte C. (2015). A meta-meta-analysis of the effect of physical activity on depression and anxiety in non-clinical adult populations. Health Psychology Review, 9(3), 366–378. pmid:25739893
  61. 61. Katerndahl D.A., & Lawler W.R. (1999). Variability in meta-analytic results concerning the value of cholesterol reduction in coronary heart disease: a meta-meta-analysis. American Journal of Epidemiology, 149(5), 429–441. pmid:10067902
  62. 62. Tamim R.M., Bernard R.M., Borokhovski E., Abrami P.C., & Schmid R.F. (2011). What Forty Years of Research Says About the Impact of Technology on Learning: A Second-Order Meta-Analysis and Validation Study. Review of Educational Research, 81(1), 4–28.
  63. 63. Reyno S.M., & McGrath P.J. (2006). Predictors of parent training efficacy for child externalizing behavior problems-a meta-analytic review. Journal of Child Psychology & Psychiatry & Allied Disciplines, 47(1), 99–111. pmid:16405646
  64. 64. Evans S.W., Owens J.S., Wymbs B.T., & Ray A.R. (2018) Evidence-Based Psychosocial Treatments for Children and Adolescents With Attention Deficit/Hyperactivity Disorder. Journal of Clinical Child & Adolescent Psychology, 47(2), 157–198. pmid:29257898
  65. 65. Comer J.S., Chow C., Chan P.T., Cooper-Vince C., & Wilson L.A.S. (2013). Psychosocial treatment efficacy for disruptive behavior problems in very young children: a meta-analytic examination. Journal of the American Academy of Child & Adolescent Psychiatry, 52(1), 26–36. pmid:23265631
  66. 66. Kazak A.E., Hoagwood K., Weisz J.R., Hood K., Kratochwill T.R., Vargas L.A., et al. (2010). A meta-systems approach to evidence-based practice for children and adolescents. American Psychologist, 65(2), 85–97. pmid:20141264