This study evaluated the effectiveness of the Triple P (Primary Care and/or Group) parenting support program on various aspects of the parenting experience through a quasi-experimental pretest – post-test protocol with an active comparison group (Care as usual). A sample of 384 parents assigned to two groups (n Triple P = 291; n Care as usual = 93) completed three subscales of the Parenting Stress Index – 4 – Short Form (Parental Distress, Parent-Child Dysfunctional Interaction, and Difficult Child) and the Parental Self-Agency Measure. The amount of intervention received by each parent was considered in the analyses. Path analysis showed that the type (Triple P vs Care as usual) and the amount of intervention (number of sessions attended) contributed independently to predicting changes in parenting experience between pretest and post-test. Receiving Triple P was systematically associated with more positive outcomes than receiving care as usual, whereas more intervention was positively associated with improved self-efficacy and parent-child relationship quality. Overall, the model explained between 4.0% and 12.6% of the variance in individual change, depending on the outcome, suggesting that unmeasured drivers of change came into play. A follow-up of parents in the Triple P group two to four years after the end of the program (n follow-up = 164) showed that the observed changes were maintained over time, with moderate to large effect sizes. These results help to broaden the evidence base on the effectiveness of this program in a sociocultural and linguistic context different from that in which it was developed.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Parenting refers to taking care of children, supervising their behavior, and ensuring their development (Hoghughi & Long, 2004). For Lacharité et al. (2016), the parenting experience refers to attitudes, beliefs, values about parenting, satisfaction with parenting, parental self-efficacy, and parental stress. Although most parents find the parenting experience fulfilling (Lavoie & Fontaine, 2016), some conditions can make parenting more difficult (Nelson et al., 2014), sometimes leading to parental stress and reduced self-efficacy. In the United States, 13% of children live with at least one parent with high levels of parental stress (Raphael et al., 2010). In a Quebec populational survey, 15% of parents of young children (0–5 years) reported being often or always stressed in relation with their child’s difficulties or behaviors, and 21% displayed low self-efficacy (Lavoie & Fontaine, 2016).
One of the most common strategies to prevent or counteract these parenting difficulties is to offer evidence-based (EB) parenting support programs (Chen & Chan, 2016; Gonzalez & MacMillan, 2008; Fortson et al., 2016). Among these, the Triple P – Positive Parenting Program (Blueprints for Healthy Youth Development, 2023; Sanders, 2012) is among the most studied and widely disseminated in the world. It has been demonstrated to enhance parental experience among diverse populations (de Graaf et al., 2008a; Sanders et al., 2014) and to reduce behavior problems among children (de Graaf et al., 2008b; Gagné et al., 2023). According to a cost-benefit analysis, every dollar invested in the Triple P system in the United States would save six dollars in government spending in sectors such as child welfare, juvenile justice, and mental health treatment for both children and adults (Lee et al., 2012). The present study emerges from the trial implementation of the multilevel Triple P system in two Quebec communities. It aims to assess the effectiveness of this program on parental stress and self-efficacy in the natural context of a community roll-out.
Parental stress is a psychological reaction generated by the many demands and tasks associated with parenting, which elicit unpleasant emotions in some parents (Deater-Deckard, 1998). Parental stress can manifest itself in feelings of distress, perception of having a difficult child, and perception that the parent-child relationship is dysfunctional (Abidin, 2012, 2013). This stress is attributable to various factors, including characteristics of the child (e.g., temperament, mood, behavior), of the parent (e.g., isolation, depression, marital conflicts, overload), and certain stressful life events (e.g., separation marriage, moving) (Criss et al., 2021; Johnson, 2015). As for the feeling of parental self-efficacy, this refers to one’s belief in his or her ability to integrate the knowledge and master the skills necessary to exercise the role of parent (Sims & Skarbek, 2019). This feeling is influenced by several factors, including the child’s behavior, the quality of parent-child interactions, and the parent’s beliefs about child behavior (Glatz & Buchanan, 2021). Studies have shown that parents who have a higher level of self-efficacy are better able to cope with difficult parenting situations (Bojczyk et al., 2018) and report a lower level of stress (Albanese et al., 2019). Albanese and colleagues’ (2019) literature review showed that parental self-efficacy is associated with multiple indicators of parents’ psychological health. Parents who feel more confident in their parenting skills may perceive difficult situations as challenges rather than problems. Reframing cognitions and alleviating parental stress could be beneficial for a constellation of parent and child outcomes, including the prevention of child maltreatment (Waid et al., 2022).
The combination of a good social support network and access to appropriate professional services appears to lead to a reduction in parenting stress, ineffective parenting practices, and child behavior problems (Gouin et al., 2016; Guralnick et al., 2008; McConnell et al., 2011). A meta-analysis by Amin et al. (2018), looking at ten studies from seven countries (n = 1504 participants), showed that universal parenting education programs for new parents increase their sense of self-efficacy, which may lead to improved mental health and more positive interactions with their child (Hefti et al., 2020). This meta-analysis considered various indicators of self-efficacy, including feeling of parental competence and confidence as well as parents’ knowledge of child development. Examples of effective programs include the Parenting Journey, a community-based parenting support program, which is associated with reduced parenting stress and parents’ awareness of the benefits of the program on their parenting skills (Kistin et al., 2020). Additionally, Thomas & Zimmer-Gembeck’s (2007) meta-analysis found that both the Triple P program and the Parent-Child Interaction Therapy (PCIT) improved parenting experience in terms of stress and self-efficacy. These findings suggest that programs focusing on the development of skills and positive attitudes in parents may reduce stress within families by promoting resilience (Singer et al., 2007).
Among these programs, Triple P has been recognized as a promising approach. Triple P is a population-based, multilevel intervention system (Sanders & Prinz, 2018), aimed at parents with at least one child aged between 0 and 12. Its approach is derived from social learning theory, applied behavior analysis, cognitive behavioral principles, developmental theory, and population health principles. Triple P promotes parental self-regulation by inviting all parents in the community to adopt “positive parenting practices”, that is reassuring, stimulating, warm, supportive, and consistent educational behaviors rooted in realistic expectations towards the child, as well as the ability to take care of oneself as a parent. Triple P differs from most parenting interventions in offering five levels of intervention of increasing intensity aimed at parents who show an increasing level of risk. The first two levels of intervention target all parents in a community. Level 1 takes the form of a media campaign (Metzler et al., 2012; Sanders, 2012) while level 2 consists of a series of three public seminars on the theme of positive parenting practices. Levels 3, 4, and 5 are aimed at parents who experience various levels of difficulty (Prinz & Sanders, 2007; Sanders, 2012). They are manualized interventions, structured in terms of content and duration, designed to help parents to apply positive parenting practices to achieve their own objectives with their children, using a variety of intervention strategies including presentations, reflections, role play, feedback, and homework. These levels can be offered in individual or group mode and in a wide variety of settings including social service agencies, schools, community organizations, and at home. For levels 2 through 5, Triple P promoters provide practitioners with a mandatory training and accreditation process, an intervention manual, and a full range of intervention and practice tools. Our study focused specifically on the effects of levels 3 and 4 of Triple P; the specific conditions of program implementation are described below.
The Triple P program has been evaluated in a number of studies. A meta-analysis by de Graaf et al. (2008a) carried out on 19 evaluative studies with a control group, including 18 with a randomized controlled design, showed that Triple P – level 4 has a moderate significant effect on the feeling of parental competence. Another meta-analysis by Nowak and Heinrichs (2008) covered 55 evaluative studies carried out on all program levels (1 to 5) and controlled for the methodological quality of the studies. Across all program levels, Triple P had a large, significant positive effect on parenting and parental well-being that was maintained over time. However, the longer the follow-up (mean = 6.3 months, range 3–36 months), the more the effect of the intervention on parenthood declined, although remaining significant. A more recent meta-analysis (Sanders et al., 2014) examined 101 evaluative studies carried out over a 33-year period, including 35 unpublished studies. The outcomes measured were grouped into categories, including: “sense of parental efficacy and satisfaction” and “parental adjustment.” For all levels of intervention combined, the results revealed significant effects of Triple P on these variables, in the short or long term. Overall effect sizes ranged from small to moderate and were larger for levels 3 to 5 of the program, compared to levels 1 and 2.
Recent studies supported the positive outcomes of Triple P in improving the parenting experience. Zhou et al. (2017) compared 64 parents who received level 4 and 43 parents who received a combination of levels 4 and 5 of Triple P. Their results showed significant improvements in both groups on parents’ psychological adjustment (symptoms of depression, anxiety, and stress), but not on self-efficacy. A quasi-experimental study by Criss and colleagues (2021), carried out with 68 parents who followed level 4 “Group” of Triple P and 23 control parents (without intervention), revealed that Triple P parents showed a significant, moderate reduction in parental stress. Qualitatively, parents who received Triple P perceived a better ability to manage their stress and an improvement in the parent-child relationship thanks to the program. Moreover, a qualitative study by Garcia et al. (2018), carried out with 34 parents who took Level 3 “Discussion Group” of the Triple P program, indicated that parental involvement in the program resulted in an increase in parents’ confidence to engage in positive discussions with their child.
The only recent evaluative study that included a follow-up is that of Nogueira et al. (2021). This randomized controlled study with an active control group (care as usual) assessed the effects of level 4 “Group” of Triple P on four measurement times (pretest, post-test, follow-up at 6 months and follow-up at 12 months) with 134 mothers. Parental stress was measured using the Parental Stress Index Short Form (PSI-SF) subscales: Parental distress, Parent-child dysfunctional interaction and Difficult child. When compared to care as usual, Triple P led to significant improvements in mothers’ parental sense of competence (self-efficacy and satisfaction). They were also less likely to perceive their child as difficult. However, both Triple P and control mothers equally reduced their parental distress, as well as parent-child dysfunctional interaction levels, immediately after the intervention. All positive changes were maintained after six and twelve months.
Despite the significant body of research documenting the effects of Triple P, evaluating the effects of the program is still relevant for several reasons. First, the variable methodological quality of previous evaluative research may have led to overestimating the program’s positive effects (Coyne & Kwakkenbos, 2013; Wilson et al., 2012). Samples in previous research were often small and when control groups were used, they were most often composed of parents who did not receive any services (e.g., waiting lists). Although current and valid, this type of research design cannot document the added value of Triple P compared to the usual services offered to parents. This type of information is essential for decision makers who must determine whether it is cost-effective to invest in a new program and the infrastructure necessary for its implementation. Moreover, studies on the long-term effects of Triple P are still scarce. While several studies have documented short term effects on parenting practices, few have demonstrated that effects endure over time. In addition, the vast majority of studies have been carried out in an Anglo-Saxon environment (Sanders et al., 2014). Given that Triple P aims for international dissemination, it is important to demonstrate its effectiveness in different cultural and linguistic contexts. Finally, there are few independent evaluations of Triple P, in which neither the author nor the promoters of the program are involved (Bussières et al., 2015). This can affect the credibility of the scientific evidence that supports program efficacy (Wilson et al., 2012). The present study addresses several of these limitations.
Study Context
The Quebec Triple P roll-out took place in two health catchment areas, one in an urban setting and the other in a semi-urban setting, serving respectively populations of 90,257 and 123,615 individuals and including respectively 7,765 and 10,110 families with children under 18. The entire Triple P system (levels 1 to 5) was implemented and monitored by a research team for a period of two years, from January 2015 to December 2016 inclusively. The implementation included a Quebec-specific promotional campaign (Charest et al., 2017; Gagné et al., 2018) corresponding to level 1, as well as the following components: Seminars and Brief Triple P (level 2), Primary care (level 3), Group (level 4), and Pathways (Level 5). This Triple P roll-out adopted a universal public health approach where, within both target communities, all parents could be exposed to the program to varying degrees.
On the whole, 51 trained practitioners offered at least one level 3 activity, 34 offered at least one level 4 activity, and 13 offered at least one activity from each of these levels. Staff were mostly women (92.4%) with at least a college diploma (94.4%) who had worked in the family and childhood field for an average of 13 years (Charest & Gagné, 2019a, 2019b). These practitioners were neither targeted nor selected: rather, our community approach was to train all staff of partner agencies likely to offer Triple P in their practice. Technical and clinical support was provided to practitioners and stakeholders through two implementation coordinators, especially by facilitating peer supervision meetings.
Program adherence was documented and analyzed for level 4 (Sheshko et al., 2020). About three-quarters of the practitioners completed reports after each group session to document the degree to which the intervention was delivered according to the manual. In addition, nearly half of the sessions (n = 66) were audio recorded for coding by an independent team. The practitioners reported that they offered the program in accordance with what was prescribed in the manual almost 80% of the time, while the coders estimated that adherence was closer to 50%. Sheshko and colleagues argued that this discrepancy between self-reports and coder ratings is consistent with past research in adherence (Hogue et al., 2015), with coders tending to rate more strictly than do practitioners. This should not necessarily be seen as a threat to the integrity of the program implementation. Indeed, independent coders have the sole task of assessing adherence, while practitioners must deliver content while managing group processes and respecting the principle of minimal sufficiency which is at the core of the Triple P approach (McWilliam et al., 2016). This is probably reflected in their more flexible stance on program adherence.
Objectives of the Study
The present study aims to evaluate the effectiveness of the Triple P program (levels 3 “Primary care” and 4 “Group”) on parental stress and self-efficacy, as reported by participating parents. These levels required parents to register for the program, making it possible to monitor participants over time. These two levels were considered together, since the objective was not to compare levels but to evaluate the Triple P approach, which is similar from one level to another, only adjusted in intensity to meet participants’ levels of need. A populational evaluation of Level 1 was reported in another study (Gagné et al., 2018). Level 2 was not evaluated per se because the public nature of the parenting seminars made it impossible to document program exposure or to follow parents over time. As for level 5, the 26 parents who had access to it also received level 3 and/or 4 beforehand; the added value of level 5 was assessed in another study (Filion et al., revised and resubmitted). Results presented here focus on parental experience for the sake of parsimony; additional results regarding children’s behavioral outcomes, parenting practices and family violence were reported elsewhere (Gagné et al., 2023; Gagné et al., 2023). Finally, as the program is aimed at parents of 0–12-year-old children, this age range was targeted for the study.
The first objective of this study was to evaluate the effectiveness of Triple P versus care as usual on four aspects of the parenting experience: three indicators of parental stress (parental distress, parent-child dysfunctional interaction, and perception of having a difficult child), and parental self-efficacy. It was expected that the parenting experience would improve significantly on all variables following the two forms of intervention. However, parents receiving Triple P were expected to report significantly greater improvements than those receiving care as usual. Given that effect sizes in previous research have varied from one study to another, no hypothesis was formulated in terms of effect size. A second objective was to verify whether the observed changes in parenting experience were maintained over time in the Triple P group. The hypothesis was that changes will be maintained two to four years after the intervention.
Method
The research hypotheses were verified through a quasi-experimental, pretest – post-test protocol with an active comparison group composed of parents receiving care as usual as offered by local health services. Parents received either Triple P or care as usual, depending on their residence in the two health-catchment territories offering the program or in two comparison territories. Comparison territories were adjacent to Triple P territories and similar in terms of the size of the population of underage children, the poverty rate of families, and the reporting rate to child protection services. All territories were selected following a structured decision-making process involving the research team and their partners in the field (Gagné et al., 2013). The research was conducted by a research team independent from the program developers and sponsors. A follow-up study was carried out two to four years after the post-test with parents who had benefited from the Triple P program.
Conditions of Intervention
Two levels of the Triple P system were the focus of this study: Primary Care (Markie-Dadds et al., 2011) and Group (Turner et al., 2010). Primary care (level 3) is an individualized parental coaching intervention consisting of four meetings lasting approximately 30 min. Session content is adapted to the specific problem identified by the parent. Activities include setting goals, developing an action plan to achieve the parent-identified goals, providing parental support, assigning homework, following-up, and reviewing progress. In the present study, 75.9% of level 3 interventions took place at home, 5.2% took place in the practitioner’s office, and 19.0% took place in both environments. Group Triple P (level 4) is offered to groups of 10 to 12 parents and takes the form of a series of four 120-minute sessions each focusing on teaching parenting skills and practicing through role play and other exercises. Group sessions are followed by three optional individualized telephone follow-ups of 15 to 30 min as well as a closing session (in group or by telephone) in which skill generalization and maintenance are discussed. In the present study, group sessions took place on the premises of a public health and social services agency or a community organization partner in the project.
Parents in the comparison condition received the usual parenting support services through the front line of the Québec public network of health and social services agencies. These services include any assistance of a psychosocial or educational nature, delivered in individual format, meeting one or more of the following needs: the parent is experiencing communication, relational, or supervision difficulties with their child; s/he complains about the difficult attitude or behavior of his/her child; s/he admits to feeling deprived and helpless in parenting; s/he feels s/he is losing control of him/herself with his/her child; s/he is worried about his/her child and seeks to protect him/her in a situation of adversity. Excluded from the comparison group are parents who participated in structured, evidence-based parenting programs similar to Triple P (e.g.: Incredible Years). In Quebec, this type of program does not represent the usual standard of care offered by parenting support services. Care as usual services most often take the form of weekly meetings with a social worker, either in a community service center or at home. The comparison group reflects this reality.
Recruitment Strategy
Triple P Group
Parents in this group were recruited from a pool of 1016 parents from 769 families who received Level 3 or 4 of Triple P between January 2015 and December 2016. Research recruitment began in May 2015, which allowed time for program implementation and mastery by staff. Recruitment ended in May 2016 for level 4 and in November 2016 for level 3, once the sample size objective was reached. In concrete terms, 368 parents from 504 families who benefited from the program during this period took part in the research, for a research participation rate of 73.0%. Parents were referred to the program or recruited by partners participating in the delivery of at least one level of Triple P, i.e., front-line public agencies or youth protection agencies, community organizations, schools, and childcare services. Participation in the research was proposed to parents as soon as they registered for the program. Parents who consented to having their contact details transmitted to the research team were contacted in order to verify their eligibility and to obtain their informed consent.
Comparison group
Recruitment for the comparison group took place from January 2016 to February 2017 in comparison territories described above. Parents were recruited with the collaboration of public front-line services agencies, through their practitioner or their immediate superior, thanks to continuous links with the research team. It was not possible to calculate research participation rate because the clinical settings did not provide information either on the total number of parents invited to participate in the research, nor on the number of refusals. Once parents received information about the opportunity to participate in the research, the remainder of recruitment process was similar to that of the Triple P group.
Sample
The sample included 384 participants (94.5% French-speaking) who answered the pre-test and the post-test. Sample attrition at post-test was 18.8%. These were parents or parental figures of at least one child between the ages of 0 and 12 – for research purposes, one child per family was targeted, that is, the one who caused the most concern to the responding parent. To participate in the study, the parent had to understand French and live with the target child full-time or part-time. If the child was in foster care, the parent was eligible provided that a process of reintegration of the child into his family had been initiated. Data from only one parent per family were collected to ensure the independence of observations. If two parents of the same child wished to participate, we preferred the participation of a male subject to counter the under-representation of men in this type of study (Fletcher et al., 2011).
Participants were in one of two groups: (1) Triple P (n = 291) and (2) Care as usual (n = 93). Within the Triple P group, 57 participants received level 3, 229 received level 4, and 5 received both levels. In terms of participation, 88.7% of parents enrolled in level 3 attended the four sessions scheduled in the program and 30.1% benefited from at least one additional session if they expressed the need (20.6%: 1 meeting; 7.9%: 2 meetings; 1.6%: 3 meetings). Of parents registered for level 4, 62.5% took part in all four group sessions presenting the program’s content, and 25.9% participated in three of them. In addition, 46.4% took part in the three telephone follow-ups provided for in the program, while 19.6% had two, 18.3% had one, and 15.6% had none.
A follow-up study was carried out two to four years later among parents in the Triple P group. The research team was able to contact 164 of the 256 parents who had authorized a follow-up contact. Table 1 describes the characteristics of the initial sample and the follow-up sample.
bThis variable was analyzed as a continuous variable: 1 = less than $15,000; 2 = $15,000 to $24,999; 3 = $25,000 to $34,999; 4 = $35,000 to $44,999; 5 = $45,000 to $54,999; 6 = $55,000 to $64,999; 7 = $65,000 to $74,999 $; 8 = $75,000 and more
*t(378) = −3.98, p = 0.000, d = 0.480
Measures
Most data were collected from parents by questionnaire administered at pre-test, post-test, and follow-up. The amount of intervention received by each parent was calculated from data provided by the practitioners.
Sociodemographic information
Information gathered about the characteristics of the target child included gender, age, and time spent at home (full-time, part-time). For parents, information was obtained about sex, age, highest level of education (high school and less, college, university), and employment status (employed or unemployed). Household information was also collected about family structure (two-parent, single-parent, stepfamily), the number of underage children living in the household, and the household income before taxes.
Parental stress
Parental stress was measured using the French version of the Parenting Stress Index-4, short form (PSI-4-SF – Abidin, 2012, 2013). This instrument consists of 36 questions forming three subscales: Parental Distress, Parent-Child Dysfunctional Interaction and Difficult Child. The PSI-4-SF is one of the most widely used and studied instruments for measuring parental stress (Holly et al., 2019). Using a cross-sectional design with four samples, Touchèque et al. (2016) studied the psychometric properties of the French version of the PSI-4-SF. The results showed good internal consistency for each of the three subscales and confirmed the validity of this instrument for assessing parental stress in a French-speaking population. Parents answer on a five-point Likert scale, ranging from (1) strongly agree to (5) strongly disagree. The instrument provides an average score for each subscale. The higher the score, the more parental stress. In the sample of the present study, the respective internal consistencies for each subscale (Cronbach’s α) were 0.86, 0.83, and 0.88.
Parental self-efficacy
The feeling of parental self-efficacy is measured using the five items of the Parent Self-Agency Measure (PSAM) (Dumka et al., 1996), translated into French for the purpose of this study. This brief instrument has been used in various socio-cultural contexts (e.g., Abarashi et al., 2014; Piedra et al., 2012). In an analysis of the psychometric qualities of 34 measures of parental self-efficacy, it ranks sixth among the best instruments identified (Wittkowski et al., 2017), its main strengths being its content and construct validity and internal consistency. Self-efficacy is defined by the parent’s perception of his or her parenting skills. For each item, parents provide the answer that best corresponds to their perception using a 7-point frequency scale (1: rarely to 7: always). An average score is calculated. The higher the score, the more the parent demonstrates self-efficacy. In the sample of the present study, the internal consistency (α) is 0.77.
Amount of intervention
The amount of intervention received by each parent was measured using the total number of intervention sessions each parent attended between pretest and post-test. For parents in the Triple P group, this information was entered on a continuous basis by the practitioners offering the intervention, using a computerized application designed for this purpose. After each of the individual or group meetings scheduled in the program, the practitioner indicated whether the parent was present or absent. This information was submitted electronically to the research team at the end of each session. For parents in the comparison group, this information was extracted from their file from the clinical-administrative database of the relevant agency. The number of sessions was determined from the session dates recorded in the files.
Procedures
Data collection (pretest and post-test) took place between May 2015 and May 2017. In the Triple P group, participants completed the questionnaire before and after receiving the level 3 or 4 interventions. In the comparison group, the objective was to administer the post-test in a similar time frame, even if services were still being delivered. Still, the delay between both measurement times was significantly longer in the control group (M = 11.1 weeks, SD = 4.0) than in the Triple P group (M = 9.5 weeks, SD = 4 0.8), t(382) = −2.99, p = 0.003, d = 0.357.
At pretest, parents participating in the individual intervention (i.e., all parents of level 3 and 12 parents of level 4) received a questionnaire at the first session that could be completed at home and returned at the next session. For those participating in a group intervention (i.e., most level 4 parents), a preliminary meeting was planned with the practitioner and a member of the research team during which parents were able to complete the questionnaire. If a parent did not show up for this session, the questionnaire was provided at the first group session with the request to fill it out at home and return it at the next session. At post-test, the questionnaire was given to the parents at the end of the last scheduled session to fill it out right away. All parents who participated in at least one session were asked to respond to the post-test. If a parent did not show up for the last scheduled session, the questionnaire was mailed to him/her to complete it at home and return in a postage-paid envelope. If necessary, telephone support was provided for parents completing their questionnaire at home.
The follow-up study among parents in the Triple P condition was carried out online between March and May 2019 (i.e., two to four years after participation in the program). Only parents of the Triple P group who had responded to the pre- and post-tests and who had previously consented to be recontacted by the research team were solicited. Of the 256 potential participants, 218 were traced and invited to participate, and 164 completed the follow-up questionnaire. Each participant received a personalized link to a LimeSurvey questionnaire. An information and consent form was presented when the link was opened, and parents checked an “I accept” button to access the questionnaire. The procedure ensured that the parent answered in reference to the same child as in the previous questionnaires. The data were submitted electronically and stored in a database hosted on a secure university server.
In order to respect confidentiality, a code assigned to each participant was used to identify all questionnaires. All participants were offered financial compensation of $30 for each measurement time in which they participated.
Analysis Strategy
Preliminary analyses
First, data were examined with regard to missing data, outliers, and score distributions. In order to detect any selection biases, parents who responded to both the pretest and post-test (N = 384) were compared to those who responded only to the pretest (N = 89) on the different socio-demographic variables (t-test, chi-2) and on parental outcomes (MANOVA). Next, the equivalence between the Triple P (N = 291) and comparison (N = 93) groups, comprising only the parents who responded to the two measurement times, was verified. Finally, the equivalence between respondents (N = 164) and non-respondents (N = 127) to the follow-up was verified on the sociodemographic and parental variables collected at pretest.
The Triple P and Care as usual groups were compared on the mean number of sessions they received (t-test). Parents in the Triple P group received a higher amount of intervention in the interval between pretest and post-test, t (352) = 7.98, p = 0.000, d = 1.03. They participated in an average of 5.9 sessions (SD = 1.85), while parents of the Care as usual group participated in 3.9 sessions (SD = 2.22).
Program effectiveness
Since the amount of intervention could act as a confounding variable in assessing the differential effects of Triple P, a path analysis was performed using the MPlus software to test whether the nature of the intervention (Triple P vs Care as usual) and the amount of intervention (number of sessions) independently contributed to changes in parenting experience between the two measurement times. As all variables were measured using standardized and validated instruments whose scores display satisfactory internal consistency, path analysis was preferred to structural equation modeling (SEM) for parsimony purposes. The dependent variables are the differential scores (Δ post-test - pretest) calculated for each of the four parenting variables under study. Following an extensive simulation study, Estrada et al. (2018) favor the use of individual-based change statistics to evaluate effect sizes because: (a) they allow the identification of cases that changed reliably; (b) they facilitate the interpretation and communication of results; and (c) they provide a straightforward evaluation of the magnitude of empirical effects while avoiding the problems of arbitrary general cutoffs. Path analysis is a recognized strategy (Hunter, 1987; Yang & Chung, 1992) and still used to this day (e.g., van den Brand et al., 2021) to assess the effects of a program. It makes it possible to model the variables that produce the changes and to estimate the percentage of variance of these changes that is explained by the model.
The long-term maintenance of the effects of the Triple P program on parenting experience was verified with the follow-up subsample (N = 164) through a repeated measures MANOVA (pre-test, post-test, and follow-up) using all the dependent variables studied. In the event of a significant result, post hoc univariate analyzes were carried out. These analyzes were performed using SPSS software.
Results
Preliminary Analyses
Analyses revealed that between 0 and 1.3% data were missing depending on the variable (multivariate: 2.3%). No imputation method was used since 5% or less of missing data is considered of little consequence for multivariate analyses (Dong & Pen, 2013). Data indicated that the postulates for path analysis and for multivariate analysis of variance (normality of the distributions, homogeneity of the variances) were respected.
It appeared that 18.8% of pretest respondents did not complete a post-test. Non-completers were significantly younger than completers, t (466) = −3.25, p = 0.001, d = 0.385, with an average age of 33.4 years (SD = 6.49) compared to 35.9 years (SD = 6.45). They also had a significantly lower family income, t(459) = −3.06, p = 0.002, d = 0.367, were proportionally less likely to live in a two-parent family (50.0% vs 59.1%), and more likely to live in a stepfamily (21.6% vs. 11.8%), χ2 (2) = 6.05, p = 0.049. The proportion of single-parent families were similar in both groups (28.4% vs. 29.1%), as were group means on parental stress and self-efficacy, F (4, 462) = 1.23, p = 0.299.
Parents in the Triple P group identified a younger target child than parents in the comparison group, t(378) = −3.98, p = 0.000, d = 0.480 (see Table 1). At pretest, multivariate analyses indicated that parents in the Triple P group also differed from the Care as usual group on the parenting variables, F(4, 375) = 2.92, p = 0.021 (power = 0.784), a difference that was due to perceived self-efficacy being slightly higher for the Care as usual group (Triple P: M = 4.79, SD = 0.95; Care as usual: M = 5.14, SD = 0.93), F (1, 387) = 9.58, p = 0.002, η2p = 0.025.1
Finally, two significant differences were detected between Triple P group parents who responded to the long-term follow up vs those who only participated in pre and post-test measurements. Parents who did not participate in the follow-up had lower levels of education (27.4% high school, 44.4% college, 28.2% university) than those who did complete the follow-up (respectively 15.4%, 44.4% and 40.1%), χ2 (2) = 7.7, p = 0.021, V = 0.16, as well as a lower income (respondents: M = 5.61, SD = 2.59 non-respondents: M = 4.87, SD = 2.75), t (283) = −2.32, p = 0.021, d = 0.279. However, these two groups were equivalent on parenting variables, F (4, 284) = 1.48, p = 0.209.
Modeling the Combined Effects of Intervention’s Nature and Amount
Path analysis was performed with the data from 342 parents, due to some missing data on the number of sessions in which the parent participated. Results are illustrated in Fig. 1. The nature of the intervention (Triple P vs Care as usual) significantly predicted the magnitude of the change on each of the dependent variables under study, with estimates (β) varying from 0.17 to −0.30. Parents who received Triple P reported greater improvements than those who received care as usual. In addition, the number of sessions independently contributed to explaining the changes in self-efficacy (β = 0.17) and in dysfunctional parent-child relationship (β = −0.17). The higher the amount of intervention, the greater the improvements on these variables. Taken together, the nature and amount of intervention explained between 4.0% and 12.6% of the change scores (R2). No fit index was reported for this model because there are as many estimated parameters as information available in the variance/covariance matrix of the variables included in the model (all parameters were estimated).
Fig. 1
Path analysis predicting various indicators of parenting experience (N = 342)
×
Maintenance of Changes Following the Triple P Program
For parents who received Triple P and participated in the follow-up study, a repeated measures MANOVA revealed significant changes between pretest, post-test, and follow-up, F (8, 153) = 29.4, p < 0.000, η2p = 0.606. Post hoc univariate analyzes revealed that significant change was observed on each of the dependent variables: self-efficacy, F (2) = 81.5, p < 0.000; parental distress, F (2) = 23.5, p < 0.000; dysfunctional parent-child relationship, F (2) = 19.0, p < 0.000; perception of having a difficult child, F (2) = 69.1, p < 0.000. Statistical power was 1.00 in all these analyses.
Table 2 presents the details of these analyses including effect sizes, while the changes are illustrated in Fig. 2. We first observe a significant long-term improvement in parental self-efficacy. Even if this improvement was significantly attenuated between the post-test and the follow-up, it remained significant. Significant long-term reductions were also observed on various indicators of parental stress. Regarding parental distress, the long-term reduction was moderate size and the level remained stable between post-test and follow-up. As for dysfunctional parent-child relationship, the improvement was also of moderate size in the long term, despite a slight attenuation between post-test and follow-up. The most notable change was in the perception of having a difficult child: the improvement was large in the long term and remained stable between post-test and follow-up.
Table 2
Descriptive statistics and effectiveness of the intervention between each measurement time
Pretest
Post-test
Follow-up
M
SD
M
SD
M
SD
F
df
p
η2p
Self-efficacy
4.89
0.92
5.78
0.71
5.48
0.75
Pre-follow
62.17
1
0.000
0.280
Post-follow
23.64
1
0.000
0.129
Par. distress
2.50
0.75
2.14
0.69
2.25
0.67
Pre-follow
17.62
1
0.000
0.099
Post-follow
3.72
1
0.056
–
Dysf. P-C rel.
2.18
0.60
1.90
0.58
2.00
0.59
Pre-follow
13.48
1
0.000
0.078
Post-follow
4.75
1
0.031
0.029
Diff. temper.
3.21
0.74
2.56
0.83
2.61
0.80
Pre-follow
77.15
1
0.000
0.325
Post-follow
0.77
1
0.380
–
Note: η2p < 0.06 : small effect size; 0.06 < η2p < 0.13: medium effect size; η2p ≥ 0.14: large effect size
Fig. 2
Illustration of long-term changes in the Triple P group
×
Discussion
The aim of this quasi-experimental study was to verify the short-term effectiveness of the Triple P parenting support program (levels 3 and 4) on the experience of parents of 0–12-year-old children, and the persistence of observed changes over time. Given that parents in the Triple P group participated on average in more intervention sessions than those in the Care as usual group, it was decided to consider the nature of the intervention (Triple P vs Care as usual) and the amount of intervention (number of sessions) as two separate independent variables in the analysis of program’s effectiveness. This approach avoids overestimating the effects of Triple P on improving parenting experience.
The results showed that both the nature and the amount of intervention helped to explain the changes observed between pretest and post-test on the parenting variables under study. On the one hand, belonging to the Triple P group was associated with more favorable outcomes in terms of increased self-efficacy and reduced parental stress (parental distress, dysfunctional parent-child relationship, and perception of having a difficult child). This result confirmed that this program proved to be more effective than the usual intervention on all of these aspects, and particularly for the perception of having a difficult child. On the other hand, the higher the amount of intervention, the more parents reported an improvement in their self-efficacy and their relationship with the target child, regardless of the type of intervention received. However, the amount of intervention did not contribute to explaining the reduction in parental distress, nor the reduction in the perception of having a difficult child, beyond the type of intervention received. Finally, a follow-up of parents in the Triple P group revealed significant long-term changes, two to four years after receiving the intervention. In the case of parental self-efficacy, the change between pretest and follow-up remained large despite a significant attenuation between post-test and follow-up. The change in the quality of the parent-child relationship also showed a slight attenuation between post-test and follow-up but remained of moderate size in the long term. Changes in parental distress and in the perception of having a difficult child remained stable in the long term, and were respectively moderate and large in size. Given the absence of a control group at follow-up, it should be noted, however, that the maintenance of these changes could be partly attributable to factors other than the Triple P program, or may have been comparable in the Care as usual group.
Overall, these results supported the hypotheses of the study: Triple P was effective in improving the parenting experience of Quebec parents, its effects surpassing those of the usual parenting support intervention offered by the front line of the public system. The strong attention paid to implementation processes in this particular Triple P roll-out (Charest & Gagné, 2019a, 2019b; Côté & Gagné, 2020; Delawarde-Saïas et al., 2018) and its inter-organizational collaboration approach (Gagné et al., 2022) might have contributed to these positive outcomes. The clinical significance of these outcomes must be assessed in the light of two indices: the R2 from the path analysis (which are modest) and the η2p from the repeated measures MANOVA (which are moderate to large). This means although the long-term changes observed in the Triple P group were quite important, only a small proportion of the variance of these changes was attributable to the intervention (type and amount combined) at post-test. This finding is not unique to Triple P. In general, four broad categories of factors can explain the effectiveness of a therapeutic intervention: (1) participant characteristics, (2) participant-provider relationship, (3) placebo effect, hope and expectation of the participant, and (4) intervention model or techniques. The latter would explain more or less 15% of the changes in participants, regardless of the intervention offered (Isebaert, 2017).
The positive outcomes observed in the present study are in line with the intervention principles of Triple P, which invite parents to take care of themselves and develop their parenting skills. They are also consistent with the results of meta-analyses on the subject (deGraaf et al., 2008a; Nowak & Heinrichs, 2008; Sanders et al., 2014), which have suggested that Triple P has a significant positive effect on various indicators of parenting experience. However, across meta-analyses and depending on how indicators of parenting are categorized, overall reported effect sizes vary considerably. Many effects seem to be maintained over time, but the longer the follow-up (up to 36 months), the more the effect of the intervention on parenthood declines (Nowak & Heinrichs, 2008). In the present study, such a decline was observed, but it was weak and did not occur for all of the indicators. In general, the changes observed lasted for up to two to four years after the program.
The results of this quasi-experimental study are generally consistent with those of recent studies, despite some differences. Zhou et al. (2017) compared parents from Singapore who received levels 4 and 5 of Triple P. Their results showed significant improvements in both groups on parental psychological adjustment (symptoms of depression, anxiety, and stress), but not on parental self-efficacy. However, they used measurement instruments different from those used in the current study, their groups were small (n = 37 and 24), and no follow-up was carried out, which did not make it possible to assess the durability of the effects. The Portuguese study by Nogueira et al. (2021) is more interesting as a comparable, because it also used a comparison group receiving care as usual, as well as the same parental stress indicators as the present study. However, it only assessed the effects of level 4 “Group” of Triple P, and the follow-ups carried out were shorter than in the present study (6 and 12 months). Their results showed that Triple P was more effective than care as usual in enhancing parental competence and reducing the perception of having a difficult child. The two interventions were equivalent in reducing parental distress and improving parent-child relationship, which differs from the results of the present study. As in the present study, however, the positive changes were maintained over time.
The results of this study showed that the Triple P program as implemented in Quebec made it possible to reach families with greater intensity than care as usual. Although the nature (Triple P vs care as usual) and the amount of intervention received were significantly associated, they still contributed independently to the changes observed in the parental experience between pretest and post-test. Thus, part of the success of Triple P seemed attributable to its more intensive nature than care as usual. As to why the Triple P program allowed this greater intensity, several hypotheses can be considered. First, the group format that characterizes level 4 allows practitioners to see up to 12 parents at a time, thus optimizing the use of their time and allowing for greater frequency of sessions. In addition, the structured character of the program and the predictability of its curriculum may have encouraged parents to attend regularly for fear of missing important material. Finally, for the comparison group, it is possible that there was a delay between the moment when the case was assigned to a practitioner and the moment when the latter held the first meeting with the family, thus reducing the number of sessions offered over a similar period of time. Regardless, the Triple P program provided more intervention to the parent, which is a result in and of itself and may well be an active ingredient of the program’s efficacy. For example, the study by Rivière et al. (in revision), carried out with the same sample of parents (Triple P group only), showed that the amount of intervention was more important than the sex of the parent in explaining changes in various parenting variables.
Finally, although the changes observed in parents of the Triple P Group were clinically significant, the path analysis showed that only a modest proportion of the pre-post change (4.0% to 12.6%) was explained by the type and amount of intervention received. This means that much of the change self-reported by parents were explained by other variables that were not considered in the model. These might include factors such as the parent’s personality, his/her motivation to solve his/her problems, his/her level of confidence and satisfaction with the intervention received, the quality of the relationship s/he has established with the practitioner (therapeutic alliance), and the events that may have occurred in his/her life between pretest and post-test. Given that it is now well established that the Triple P program is effective in reducing parental stress and enhancing parental self-efficacy, it seems opportune to question how these changes occur (Blase & Fixsen, 2013). This involves paying attention to the “drivers” of change that are at work when a parent engages in this type of intervention.
Strengths and Limitations
Strengths of this study include the large sample size, which lent power to the statistical analyses, the use of an active comparison group, and the fact that the amount of intervention was considered in the data analysis and in the interpretation of results. The main limitations of the study include the absence of randomization, research participation rate and attrition between the different measurement times, self-report measurement of parenting variables, and the absence of a long-term follow-up for the comparison group. In addition, practitioners showed mixed adherence to program’s content and processes, participants’ ethnic origin was not documented beyond the language spoken at home, and the fact that parents were offered financial compensation for agreeing to participate in the study could have led to a sampling bias by over-representing parents of low socio-economic level. If that were the case, however, that may not be a problem as such. Given that a low socio-economic status is a vulnerability factor for families, it is desirable to study the effects of parenting support programs on this clientele.
The Quebec Triple P roll-out adopted a universal public health approach where, within each of the “experimental” communities, all parents who felt the need could have access to the program. As this context did not lend itself to randomization, the comparison group was recruited from “control” communities not offering the program. Although the comparison group was similar to the Triple P group in several aspects, it differed in the age of the child and it may have differed in other variables not considered in this study, possibly influencing the results. Moreover, some unmeasured contextual differences related to the territories themselves, such as systems capacity, may have affected the findings. However, previous analyses showed that territories involved in this study were similar in terms of community readiness to prevent child maltreatment, with the exception of one comparison territory which had higher informal social resources (Gagné et al., 2020). It is also interesting to consider that the meta-analysis by Nowak and Heinrichs (2008) on the efficacy of Triple P suggests that non-randomized quasi-experimental studies produce results similar to randomized controlled trials and do not overestimate the size of program effects, nor lead to less reliable or less conclusive results, as also shown by other authors (Saunders et al., 2003).
Additional limitations of the present study are related to participation. First, the proportion of parents in the Triple P group who consented to participate in the research is 73.0%, whereas the participation rate within the comparison group was not documented. This is a limitation to the internal validity of the study. Another limitation is participant attrition, first at post-test (18.8% attrition - 473 to 384) and then at follow-up for the Triple P group (43.6% attrition - 291 to 164). At each measurement time, the socioeconomic characteristics of the sample improved slightly. This increases the risk of overestimating the effects of the program, as it is possible that less advantaged parents were more likely to drop out of the program because they benefited less from it. However, it is important to keep in mind that the differences between research participants and non-participants were small, which can help to reduce bias due to attrition.
A further limitation is that parental stress and self-efficacy were assessed by a single source of information (the parent) using self-report instruments. It is therefore possible that the present study somewhat overestimates the effects of the program on parenting. Given the type of experiential outcomes being assessed in this study, it would be difficult to assess these constructs with observational or third-party reporters. Future studies of parental stress could consider using measures such as cortisol levels or experience sampling measures to get alternative measures of stress and parenting experience.
Finally, only the Triple P group was followed to assess the persistence of change. The absence of follow-up data for the comparison group does not make it possible to know whether the persistence that is observed is solely attributable to Triple P, or to other factors as well such as receiving subsequent services or experiencing positive life events. This limitation diminishes the internal validity of the follow-up study.
Conclusion
The present study adds to the body of literature that evaluates the effectiveness of the Triple P parenting support program on parental stress and self-efficiency in a natural implementation context. With its active comparison group and its consideration of the amount of intervention received by the parents, it helps to further support the effectiveness of Triple P’s levels 3 and 4 on these aspects of the parenting experience. In addition, this study provides support for the effectiveness of the program in a sociocultural and linguistic context different from that in which it was developed. In the Quebec context, Triple P stands out as a major asset for public establishments in the health and social services network, since it makes it possible to intervene with more intensity than the usual intervention and with better, and persistent, outcomes. Despite its limitations, this study is one of the few to suggest positive long-term changes in parenting stress and perceived self-efficacy linked to participation in the Triple P program. It also contributes to the literature by estimating how much change in parental experience was specifically associated with the Triple P program versus care as usual, suggesting that the success of Triple P is not solely attributable to program’s content: other unmeasured drivers of change probably come into play. Future research should explore potential drivers of change, such as parents’ motivation and readiness to change, significant life events happening between pretest and post-test, or parents’ involvement in the intervention, beyond mere session attendance.
The authors of this manuscript certify that they have NO affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.
Ethical Approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (Canada) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Ethical approval was obtained under the Multi-Centre Research Ethics Review Mechanism established by the Quebec Ministry of Health and Social Services, from the Research Ethics Committee of the Centre jeunesse de Québec (#MP-CJQ-14-14-020).
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.
Given these significant groups differences, the propensity scores method was used to form equivalent groups, and subsequent analyzes were re-run with these groups. Results were generally in the same direction, so analyses with the original groups were preferred.
