Background
Mental illness is a world-wide problem associated with great suffering to individuals (Polanczyk et al.,
2015) and large costs to society (Seabury et al.,
2019). For many who suffer from mental illness in adulthood, the disorder debut during childhood (Jones,
2013). In the early development of children’s health or ill health, parents play a key role. The quality of the parent-child relationship has long-standing impacts on the child’s health (e.g., Stewart-Brown et al.,
2005). Interventions focusing on the parent-child relationship and on parental behavior are therefore suggested as a key to overcoming the large societal burden that mental illness presents today (Yap et al.,
2016).
Universal parenting programs can be implemented to prevent mental health and behavioral problems by reducing the occurrence of risk factors and enhancing protective factors around the child. The focus of such programs is to teach parents strategies to handle challenging situations and to improve parent-child relationships. Parenting programs have been evaluated in many trials and are disseminated across the world (Leijten et al.,
2019). One program that is widely adopted in Sweden is the universal parenting program the
All Children in Focus (ABC) program. ABC has been evaluated in an RCT with positive effects on child health and parent practices (Ulfsdotter et al.,
2014), and is today employed by clinics all over the country. Hundreds of parents participate in ABC every year. The current study is part of a larger ongoing nation-wide study where data on ABC is collected continuously to evaluate its dissemination and effects in regular services.
When the Covid-19-pandemic started, conducting parenting programs was suddenly difficult. Many programs, including ABC, are in the format of group meetings which were not possible to carry out due to pandemic restrictions. A demand for other ways to organize parenting programs therefore arose, not the least since the pandemic and its consequences, such as parental stress, loneliness, job loss, and income instability, increased rates of child maltreatment and abuse (Lawson et al.,
2020; Rodriguez et al.,
2020). One solution to this problem was to offer group parenting programs through remote delivery, instead of on-site meetings (Sullivan et al.,
2021).
There is a growing evidence base for the effectiveness of delivering parenting programs through digital solutions (Baumel et al.,
2016; Bausback & Bunge,
2021; Thongseiratch et al.,
2020). Even when modes of delivery were compared directly in the same study, a digital version was found to be non-inferior to the on-site version of a parenting program (Prinz et al.,
2022). However, most studies of remotely delivered parenting programs have relied on self-directed content (e.g., internet-based programs with text, videos, and online-based exercises) with therapist support through text messaging or telephone. Thus, most studies investigate the effects of programs that have been specifically developed for digital delivery (Baumel et al.,
2016; Bausback & Bunge,
2021; Thongseiratch et al.,
2020). However, at least in Sweden, the access to such programs in regular services is low. Instead, group-based parent training with on-site meetings are offered. During the Covid-19 pandemic, many services started to provide their regular parenting programs through video conferencing (VC), in lack of access to programs developed for digital or remote use. Despite the necessity for many services to transfer to VC tools during the pandemic (Sullivan et al.,
2021), the evidence base for this type of remote delivery is rather limited. Only a couple studies with few participants have investigated the use of VC to deliver group-based parenting programs (Canário et al.,
2021; Fogler et al.,
2020; Reese et al.,
2012; Reese et al.,
2015; Xie et al.,
2013). While the reported effects on symptoms and fidelity have been promising, the evidence is still uncertain.
The practice to offer a group based parenting programs through VC instead of on-site may be viewed as a variation in implementation rather than a novel type of program – i.e., it is not a new intervention, but an existing intervention implemented in a new way. And in the case it is conducted spontaneously by practitioners, rather than planned by program developers, it is important to identify if it should be considered as program drift and a threat to fidelity, or if it is a constructive innovation and necessary adaptation (Fixsen et al.,
2005). During the Covid-19 pandemic, it can clearly be argued that transferring existing programs to VC delivery was a necessary adaptation, given that the alternative often was no intervention at all. On that note, in one of the most common frameworks in implementation science (RE-AIM), the
reach of an intervention is defined as one of five core dimensions that is important to include in evaluations of interventions (Glasgow et al.,
1999). In other words, it does not matter how effective an intervention is, if it does not reach the intended target group. Potential reach is of course one of the general benefits of digital delivery of interventions (Kazdin,
2019: Sullivan et al.,
2021; Vigerland et al.,
2016). A prerequisite is however the second dimension of RE-AIM:
efficacy. The efficacy of delivering parenting programs through VC is still uncertain, given the limited number of studies specifically addressing that issue.
When testing a new way of delivering an intervention, the last three dimensions of the RE-AIM model are equally important to consider. To what extent is it
adopted by the relevant services and to what degree of fidelity is it
implemented? And will the new way of delivery be
maintained over time? Besides addressing these issues with quantitative methodology, it is also important to include qualitative methods to generate hypotheses of possible barriers and facilitators of implementation through different modes of delivery. The inclusion of qualitative methods in research can be especially helpful in novel and time-pressed contexts, like the Covid-19 pandemic (Vindrola-Padros et al.,
2020).
Results
Participant Characteristics
Demographics of parents and children are presented in Table
1. The number of siblings differed significantly between groups. Dunn’s test of pairwise comparisons showed that the difference was due to slightly more siblings in the on-site group compared to the VC (
p < 0.05). Analyses also showed that more parents in the blended and VC groups had a university education than parents in the on-site groups, in which more parents’ highest education level was high school or vocational school (
p < 0.01). Furthermore, One-Way ANOVA showed no differences between the three groups at baseline on the parent rated measures presented in Table 3 (PPP, NPP, and CPB;
p > 0.05).
In total, 178 of the group leaders who led the groups responded to a survey about their previous education and experience. Common professional backgrounds were preschool teachers (n = 78), social workers (n = 57), teachers of children in need of special support (n = 57), nurses (n = 7), and psychologists (n = 5). Most common workplaces were Family centers – a type of agency where health care for children and mothers, social services, and open playgroups for children are located in the same place (n = 53), social services (n = 41), school (n = 36), and preschool (n = 32). Group leaders who held on-site groups only (n = 101 with available data) had 1–45 (M = 11, SD = 11) years of professional experience in providing support or treatment for parents, not statistically different from group leaders who held groups with VC meetings (n = 74; 1 – 43 years, M = 11, SD = 10). The group leaders had varying experience with ABC; on-site only-group leaders had experience in holding 1-22 (M = 4, SD = 4) groups each, which was less than group leaders who held groups containing VC meetings (1–42, M = 6, SD = 7 groups each; p < 0.05).
A significant difference between groups in follow up time from T1-T2 was found (F = 27.33, p < 0.001). Days from T1-T2 in the three conditions: On-site: M = 91, SD = 84; Blended: M = 102, SD = 74; VC: M = 42, SD = 21. The follow-up time was in general longer than the 60 days that is the length of ABC when offered strictly as advised. One reason that came to our knowledge was that some groups starting during Spring 2020 were set on pause due to Covid-19 and continued during the autumn.
The Adjustment of ABC to a VC Format During the Covid-19-Pandemic
Group leaders had led 1–10 (M = 2.05, SD = 1.56) VC or blended ABC groups each. To arrange meetings, they had used Microsoft Teams (n = 61), Zoom (n = 8), Google Meet (n = 6), Skype (n = 4), Pexip (n = 1), and telephone (n = 1). Almost all of the group leaders (n = 72) reported that the majority of their digital ABC-meetings had been in groups (i.e., including parents from more than one family). The remaining two conducted the majority of their ABC-VC meetings individually (parents to children in the same family).
Effects on Children and Parents of ABC-VC
Mean values, standard deviations (SD) and statistics for repeated measures ANOVA with three levels (on-site, VC, and blended) on the grouping variable are presented in Table
3. Not taking the format of the parent group into account, PPP improved significantly over time, while NPP and CPB were significantly reduced (
p < 0.001 for all three measures). The change over time did not depend on the format of the parent group as shown by the lack of interaction effect. Results were the same for CPB employing robust ANOVA (main effect of group value = 0.24,
p = 0.79; main effect of time value = 50.91,
p < 0.001; interaction effect value = 1.06,
p = 0.35). As also shown in Table
3, parents in the three conditions were equally satisfied with ABC. Analyses were also carried out with two levels on the grouping variable (VC versus on-site). The results were the same as for analyses including the blended groups (i.e., a significant effect of time at
p < 0.001, no significant group nor group by time interaction effect). Regarding homework completion, there was a tendency (three level analysis:
p = 0.090; two level analysis:
p = 0.053) towards that parents in the VC condition completed homework slightly more than parents in the other conditions. Finally, since parents in the on-site groups had lower education levels, additional analyses were carried out for each outcome with education as a covariate. The analysis showed that controlling for educational level, the results remained the same with no significant differences between conditions on any of the outcome measures.
Table 3
Mean values (SD) and statistics at T1 and T2
PPP1 | 16.3 (4.38) | 17.2 (4.00) | 0.22 | 16.0 (4.08) | 17.3 (3.28) | 0.35 | 16.8 (3.85) | 18.1 (3.50) | 0.34 | 0.56 | 0.57 | 35.83 | <0.001 | 2.04 | 0.13 |
NPP | 7.72 (2.45) | 6.14 (2.11) | 0.69 | 7.96 (2.51) | 5.96 (1.89) | 0.89 | 7.93 (2.55) | 6.35 (2.02) | 0.68 | 1.00 | 0.37 | 192.80 | <0.001 | 0.53 | 0.59 |
CPB | 2.49 (1.96) | 1.70 (1.42) | 0.44 | 2.30 (1.77) | 1.64 (1.19) | 0.43 | 2.28 (1.81) | 1.72 (1.42) | 0.34 | 0.90 | 0.41 | 68.58 | <0.001 | 0.32 | 0.72 |
Sat1 | | 26.22 (2.36) | | | 26.49 (2.07) | | | 25.82 (2.56) | | | | | | 2.29 | 0.10 |
HC | | 3.04 (1.03) | | | 2.98 (1.11) | | | 3.25 (1.04) | | | | | | 2.46 | 0.09 |
Group Leaders’ Experiences of VC Parent Groups – Quantitative Measures
Among the 74 group leaders who answered the survey about arranging ABC-VC, 63 answered the questions about parents’ homework completion and dropouts. The remaining 11 did not answer these questions because they had never conducted an on-site ABC-group. Regarding how much group leaders experienced that parents had completed homework assignments during VC groups compared to groups on-site (0: much less – 10: as much as a physical group), the average rating was 8.65 (SD = 1.83). Further, 24 group leaders reported that there had been less drop outs during digital parent groups, 20 reported that dropout rates had been about the same as during on-site groups, and 19 experienced it as more dropouts during VC groups.
Regarding how the group leaders thought that the VC format had influenced their possibilities to guide parents in need of more help, n = 4 experienced it as easier/better in the VC format, n = 29 as no difference, and n = 25 as more difficult. The remaining 16 group leaders reported that they could not answer the question.
Group Leaders’ Experiences of VC Parent Groups – Qualitative Analyses
Qualitative analyses revealed four key themes: 1) More can participate, 2) Parents may attend and learn less, 3) Rethinking the group leader job, and 4) Less relationship building. Themes and sub-themes are presented below. The number of group leaders touching upon a theme or aspect of the analysis is presented in brackets.
More can participate
Group leaders experienced that a greater variety of parents, living in different life circumstances, can participate in VC than on-site groups. This advantage was mostly described regarding families’ life situation in general and not specifically concerning the pandemic circumstances.
Parents may attend and learn less
A second theme concerns that group leaders experienced a shift in how parents attended sessions in VC compared to on-site groups. Factors in the remote situation may influence the quality of parents’ attendance and concentration during sessions.
Rethinking the group leader job
The VC format changed group leaders’ work situation and working conditions. New challenges arose which needed time and practice to handle.
Less relationship building
Group leaders missed the social part of on-site groups. It was more difficult to build relationships when meeting through VC. The online groups focused mainly on the content of the program, while the social parts around the meetings were lost.
Discussion
The current study aimed to evaluate different ways of delivering ABC during the Covid-19-pandemic. The first aim was to evaluate if mode of delivery (VC, blended, or on-site) would affect parenting and child outcomes. The analyses showed that parent practices improved and CBP decreased in all modes of delivery. There were no statistically significant differences in change over time between the three conditions. The second aim was to investigate the group leaders’ experience of delivering ABC through VC. Group leaders experienced both advantages and disadvantages of VC compared to on-site groups. Some (38%) experienced less dropouts during VC groups and some (30%) more. Half (50%) of the group leaders experienced no difference in their ability to guide parents in need of more help during VC than on-site groups while 41% experienced this as more difficult in VC groups. Qualitative analyses revealed four key themes. Group leaders experienced VC to offer a greater flexibility that makes it easier for parents to attend, but could also negatively impact learning during sessions. In addition, the digital format made it more difficult to build relationships within the groups. To hold VC groups also brought new challenges for the group leader to handle during sessions. These results suggest potential positive and negative consequences of using VC that should be taken into account in the implementation of parenting programs in regular practices.
The lack of differences between the study conditions in parenting and child outcomes is an important addition to the evidence base of using VC as delivery method of group-based parenting programs. The results are in accordance with, to our knowledge, the only previous controlled study of VC-delivered parenting programs (Xie et al.,
2013), that also showed essentially no difference between the VC and on-site conditions. Our results are also in line with a couple of pre-post studies in which VC was used as way of delivery, where effect sizes were reported to be similar to other studies of on-site parenting programs (Canário et al.,
2021; Reese et al.,
2012; Reese et al.,
2015). Even if the evidence of the effectiveness for parenting programs that have been developed for digital delivery (e.g., internet-based parenting programs) is fairly strong (Baumel et al.,
2016; Bausback & Bunge,
2021; Thongseiratch et al.,
2020), it is important to build evidence for the use of VC as delivery method of parenting programs designed for on-site group meetings. This is not the least imperative in times of crises or in settings that lack recourses, in which access to platforms and programs designed to be delivered digitally may be limited. VC-delivery gives services an opportunity to instantly offer their existing group based parenting programs through remote delivery. Since parenting programs is suggested as a key in preventing mental ill-health (e.g. Yap et al.,
2016), the expansion of mental health services through novel models of delivery is crucial (Kazdin,
2019). In particular since rates of child abuse increased during the pandemic (Lawson et al.,
2020; Rodriguez et al.,
2020). Our results suggest that parenting programs such as ABC could be offered by VC under such circumstances.
In research and development of novel ways of intervention delivery, it is relevant to consider several aspects of implementation besides efficacy. In terms of the RE-AIM framework (Glasgow,
1999), the group leaders’ experience of using VC to deliver parenting programs offered some notable insights. First of all, several group leaders experienced that VC-delivery had an impact on
reach of the parenting program, in the sense that a more diverse group of parents could participate. This is a common argument for the dissemination of digital interventions in general (Kazdin,
2019; Sullivan et al.,
2021; Vigerland et al.,
2016), and with the addition of VC to other digital delivery methods the reach could become even wider. On the other hand, the demographics of the included participants suggest that the reach of the digital offering may be narrower in regard to education and larger families, since the parents who choose to be part of ABC-VC on average had higher education, and parents in the on-site condition had more children than those in the VC condition.
The qualitative analysis did not offer any specific insights related to the third dimension of the RE-AIM framework (adoption), but the natural experiment in itself shed some light on the matter. Of the 113 clinics that participated in the study, 63 (56%) started to offer ABC through VC or blended format when that opportunity was allowed by owners of the copyright of ABC. The true proportion would probably have been higher if all clinics would have been prohibited to arrange on-site group meetings (i.e., no variation in local pandemic restrictions).
How will VC impact the adherence and quality of the delivery of parenting programs – “Will it work as well on Zoom?” That question was of central interest in the study and several conclusions from the quantitative and qualitative analysis concerns
implementation, the fourth dimension of the RE-AIM model. First, according to the self-assessments, parents participating through VC reported at least as much homework completion as parents in on-site groups. The group leaders who used VC also experienced that the parents in VC-groups completed almost as many homework assignments as parents in regular on-site groups. In terms of completion rate, about as many group leaders reported that VC resulted in more dropout, as those who experienced a decrease in dropout compared to on-site delivery. One interpretation of this is that VC suits some groups of parents better than others, and a way to improve the general reach and attendance to parenting programs would be to offer VC as a complement to on-site meetings within the same service. That the dropout rate (on average) seemed to be unaffected by VC delivery was however encouraging, given that completion and retention has been expressed as one of the challenges with other methods of digital delivery in regular services. For example, in a recent meta-analysis of ICBT for depression and anxiety in routine care, only 30% of screened patients were offered ICBT. Of those, 73% started the program, and of those almost 40% dropped out (Etzelmueller et al.,
2020).
Despite the encouraging results regarding attendance and homework completion, the group leaders also experienced challenges with VC in terms of implementation. Some group leaders experienced difficulties in guiding parents in need of more help as well as more dropouts compared to on-site groups. In addition, group leaders thought that parents’ ability to concentrate could be negatively impacted by the remote situation, due to e.g., distractions in the surroundings and technological problems. It was also more difficult for parents to get to know each other. One consequence could be that while group VC could suit some parents better than on-site groups, some could benefit less. Our qualitative analyses indicate that parents who fit better in an
on-site group could be characterized by being in a greater need of social support or having problems concentrating, either due to individual difficulties or to family factors (e.g., living crowdedly). Significant predictors and moderators of parenting programs in previous research include factors that contribute to a more disadvantaged situation regarding e.g., socioeconomic status (e.g., Reyno & McGrath,
2006). It is possible that parents who experience more difficulties are also those in need of more social support, which can be more difficult to get in a VC group.
Group leaders experienced that the remote situation changed their own working conditions and the role of the group leader, which could have influenced fidelity to the manual and overall quality of ABC. Along with the perceived differences in parents’ abilities to benefit from ABC-VC, this could indicate that some adaptations of the program are needed. Some group leaders did not report as many challenges as others. It is possible that the variation could have been due to a varying degree of local adaptations of ABC when conducted by VC, that we have no knowledge of. Many also experienced that technological problems interrupted sessions and a number of group leaders did not feel comfortable in using the technology. Besides maintaining the quality of the program, addressing such issues can also be of importance regarding data confidentiality, which is a critical issue in digital health care delivery where sensitive information is continuously transferred. In addition, group leaders reported using many different softwares to deliver ABC, of which some are not high-security alternatives. When VC is employed in regular care, training in using the technology is needed as well as access to VC software that are easy to use, have functions to facilitate delivering the program with quality (e.g., to use whiteboard and have small group discussions), and secure enough to maintain confidentiality.
Taken together, our results indicate that VC can be a viable alternative to on-site meetings in terms of effectiveness. Meanwhile, there are both pros and cons of VC versus on-site groups that should be taken into account when planning a group and deciding what is the best fit for families. For families who are motivated to work independently, VC can be a good choice. Families needing more support could suit better in a physical group. When VC is used, clear guidelines for how to participate can be useful (e.g., how to create an undisturbed home-environment). Group leaders can also benefit from training in how to offer programs by VC (e.g., in handling technological problems, data security, and facilitating online communication between parents).
This study has several strengths regarding external validity. Families were recruited through regular services and no exclusion criteria were employed. We therefore expect the sample to be representative for the parents who actually attend ABC in Sweden. In addition, the study captures the experiences of group leaders who work in regular services with ABC, some of whom have for years. Since the pandemic was ongoing during the entire study period we also expect all groups to be exposed to about the same circumstances. Also, the combination of quantitative and qualitative methods applied in this study contribute with different perspectives and therefore a more comprehensive analysis.
Conclusions about effectiveness from this study are limited by the study design; we have only tested for superiority, we have limited knowledge of parents who dropped out or chose not to participate, and parents were not randomized to conditions. The lack of randomization does imply risk for selection bias, but it may have been countered by the fact that neither the parents nor group leaders were able to self-select condition. Instead, the decision of whether ABC was conducted by VC or on-site was mainly based on restrictions implemented at the municipal or regional level. To establish the effectiveness of ABC-VC compared to ABC on-site, a non-inferiority design would have been a superior research design. The achieved statistical power was however insufficient for a non-inferiority study, given that the non-inferiority margin would be rather small (universal parenting programs generally produce small to medium effects).
Another limitation concerns the lack of validated measurements. However, the experiences from the larger study that data was collected from had called for a short survey with questions relevant to the content of ABC to facilitate receiving responses from as many parents as possible, in order to achieve a representative sample. A further limitation concerns the lack of data on the fidelity of the intervention and the extent to which groups were adjusted to comply with the pandemic situation and VC format. The group leaders’ experiences indicate that some adjustments of ABC was most likely inevitable to cope with practical challenges and to better fit the needs of some parents. To have data on if, and if so how, group leaders adapted the program would have been useful. An adjustment that did come to our knowledge was that some groups starting Spring 2020 were set on pause and continued during the autumn. This, and possibly other reasons, influenced the follow up time to be longer than expected in two of the groups (on-site and blended). However, the aim was to evaluate ABC in the circumstances of the pandemic. Several of these limitations were therefore an inevitable part of this trial.
Limitations regarding the qualitative part of the study concerns the limited amount of qualitative data per group leader. However, we wanted the perspective of many group leaders and with a questionnaire we reached more than we would have had the opportunity to interview.
In future research, the effectiveness of VC as a delivery method of parenting programs needs to be investigated in RCT’s. Preferably, by comparing VC to golden standard face-to-face options using a non-inferiority design. Research on factors influencing parents’ ability to benefit from or participate in VC-delivered parenting programs is also needed. Our findings suggest that parents’ ability to benefit could be influenced by individual or family level factors which need further investigation in quantitative trails. Guidelines for how to tailor support to families and to decide whether a family should join on-site or by VC are also needed.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.