Pilot Randomized Controlled Trial of the WHO Caregiver Skills Training in Public Health Services in Italy

Joint engagement experiences with partners and shared activities are fundamental for the emergence of communication skills, the mutual regulation of affect, the development of problem solving abilities and the sharing of cultural meaning (Adamson et al., 2009). Children with autism spectrum disorder (ASD) spend significantly less time jointly engaged with communicative partners than typically developing children and children with developmental delay (Adamson et al., 2009), which likely has significant effects on their development (Bottema‐Beutel, 2016; Mundy et al., 1990). The reduced exposure to joint engagement states occurs as children with ASD display a reduced ability in both responding to joint attention and initiating joint attention (Mundy et al., 2007): children with ASD are more frequently unaware of or reject bids for joint engagement (Adamson et al., 2001) and display fewer communicative acts to share interests in objects or events (Landa et al., 2013; Watson et al., 2013). These behavioral patterns disrupt the expectations and strategies adults intuitively use to successfully sustain engagement, which often makes parents of children with ASD who have not received intervention less effective in establishing joint engagement. Indeed, prior to receiving intervention, parents of children with ASD display a lower ability to ‘scaffold’ (Bruner, 1985), or support, shared activities and to ‘follow-in’ (Tomasello & Farrar, 1986), or reference, the child’s focus of attention in play interactions, compared to parents of typically developing children or children with developmental delay (Adamson et al., 2019). The effects on parental psychological wellbeing of such repeated perceived ‘failures’ at attempts to engage the child are not negligible, as they may lead to self-doubt and reduced confidence in the ability to parent effectively. Lower parental self-efficacy, in turn, is associated with fatigue and lowered wellbeing which may further exacerbate parenting difficulties (Giallo et al., 2013).

To protect caregiver wellbeing and improve long-term child outcomes, caregivers of children with ASD therefore need sensitive and specific support to develop skills to create and sustain joint engagement experiences within every-day interactions with their children. There is evidence that parent-mediated interventions can change the way parents interact with their children (Oono et al., 2013). However, the majority of this evidence is derived from efficacy trials conducted within controlled university settings and with highly specialized clinical personnel (Hardan et al., 2015; Kasari et al., 2010; Pickles et al., 2016; Wetherby et al., 2014), whereas when these models are implemented in community settings the effectiveness is considerably reduced. A meta-analysis of community-based early intervention for cognitive, communication, social, and adaptive behavior outcomes found small effect sizes (0.21- 0.32) for children with ASD (Nahmias et al., 2019), while previous meta-analyses of university-based clinical trials report substantially higher effect sizes (0.42 – 0.76) for the same domains (Reichow, 2012).

Primary challenges of implementing parent-mediated interventions in community settings include: (a) the threats to fidelity and integrity of the intervention due to a reduced expertise of interventionists, (b) the heterogeneity of participants by socioeconomic status (SES) and clinical characteristics, which increases the variability in the sample and (c) the limited control over the design (such as amount and quality of usual care). However challenging, investigating these models under real-world conditions should nonetheless be pursued as it can accelerate the process of bridging the gap between research and clinical practice (Weisz et al., 2015). As the global burden of disease for neurodevelopmental disorders is predicted to gradually increase (Whiteford et al., 2013), there is indeed an urgent need to increase access to evidence-based intervention through public health services. This applies not only to low-and-middle income countries (LMIC), where notoriously the majority of children with developmental disorders do not have access to care, but also to high-income countries (HIC), where family characteristics such as race, ethnicity, and SES contribute to service disparities (Smith et al., 2020).

In response to the need for an open-access, evidence-based, feasible in low-resource contexts caregiver training program, the World Health Organization (WHO) promoted the development of a novel program for developmental disorders, the ‘WHO Caregiver Skills Training for Families of Children with Developmental Delays and Disabilities’ (hereafter: CST). The CST program was developed as part of the Mental Health Gap Action Programme (mhGAP), an initiative aiming to bridge the ‘treatment gap’ for priority mental, neurological and substance use conditions (Saxena, 2016). The development of the CST program, which is founded on principles of developmental science, social communication interventions, applied behavior analysis, positive parenting, and self-care methods, was informed by evidence reviews, meta-analyses and consultations with experts and users (Salomone et al., 2019). At the core of the intervention methodology in CST is the shaping of every-day activities into shared caregiver and child routines so that children have regular joint engagement experiences that provide opportunities for the development of new skills. Through modelling, coaching and group discussions, caregivers are shown strategies to improve their ability to scaffold the child’s activities, follow the child’s lead and use positive affect to establish and maintain routines.

The present study examined the acceptability, feasibility and indicators of effectiveness of community-implementation of the CST in Northern Italy. Italy is a HIC where the provision of evidence-based interventions through public Child Neuropsychiatry services is free, but with several areas significantly under-served. An Italian National Institute of Health survey reported that 50% of public child neuropsychiatry services do not offer any kind of ASD-specific interventions (Borgi et al., 2019), confirming earlier parent-report evidence that publicly provided ‘treatment as usual’ (TAU) for ASD mostly consists of either speech and language therapy or non-specific occupational therapy (‘psychomotor therapy’), with two-thirds of families not accessing parent training/education (Salomone et al., 2016). Personnel shortages and limited access to specific professional training are thought to be among the underlying reasons for the current state of public intervention provision (Borgi et al., 2019). To examine whether the CST could address these limitations, we first undertook a pre-pilot implementation of CST in Northern Italy. This pre-pilot was the first ‘test-run’ of the CST program globally. It showed good feasibility and acceptability of key intervention components and delivery methods and informed, together with early implementation data from other sites, such as Ethiopia (Tekola et al., 2020), the global field testing initiative (Salomone et al., 2019). We then conducted an effectiveness-implementation hybrid pilot randomized controlled trial (RCT) in public Child Neuropsychiatry services of the Piedmont region in Northern Italy. Reported here are the clinical outcomes of the pilot RCT; the formative adaptation process and acceptability and feasibility data from the trial are reported elsewhere (Salomone et al., 2021).

Figure 2 shows the CONSORT diagram of participant flow through the study. A total of 3/86 (3.5%) participants (2 from CST and 1 from eTAU) were lost from follow-up; primary outcome data (BOSCC and JERI ratings) were available for 83 (96%) subjects (41 from the treatment arm and 42 from the control arm).

We examined the effectiveness of the novel WHO Caregiver Skills Training program (Salomone et al., 2019) implemented for the first time in public outpatient child neuropsychiatry settings in Italy, through a pilot RCT. In summary, we observed high levels of feasibility of delivery by clinicians, excellent acceptability to caregivers and favorable effects, 3 months post-intervention, on dyadic fluency of the caregiver/child interaction, child non-verbal communication, caregiver skills supportive of the interaction, self-efficacy and stress. The effect size of the intervention was large for parent skills and moderate for parenting self-efficacy. Smaller effects were found for flow of the interaction, child gestures and parenting stress.

The feasibility of delivery, as reflected in high fidelity of implementation, high attendance rates and good self-reported viability of delivery, combined with the high acceptability to the target beneficiaries, provide evidence in support of the scalability of the CST in public health services in Italy. The user-facing intervention materials (participant booklets), the contents covered during the sessions and the training methodology were found to be well received and in line with caregivers’ needs, confirming the qualitative analysis of post-intervention feedback (Salomone et al., 2021). With respect to feasibility assessment, it is noteworthy that the fidelity of implementation of CST strategies in interaction with children during the home visits, while overall acceptable, was lower for some interventionists (Salomone et al., 2021). This likely reflects the variety of previous professional experiences of the clinicians, including receiving training and being directly involved in delivering ASD-specific behavioral interventions, rather than their seniority. As discussed more extensively in Salomone et al. (2021), the uptake of CST intervention strategies by part of specialist professionals who nonetheless have received limited training in developmental/behavioral methods (Roll-Pettersson et al., 2020) presents with specific challenges.

The estimation of the blinded primary outcomes derived from the free play caregiver/child interaction showed that, although change scores are in the hypothesized direction indicating the effects of the intervention, there were no significant differences between arms in child’s joint engagement, availability to interact and autism symptom severity. We report, however, strong treatment effects on the blinded parent outcome: parents in the treatment arm significantly improved in their ability to support the interaction by scaffolding the child’s actions, following in on the child’s focus, and appropriately modulating affective communication to enhance the interaction. With respect to the scaffolding skills, it is noteworthy that high scaffolding scores in the JERI coding system require that the parent successfully guides the child’s behaviors in positive ways that are observable within the interaction; as such, high quality scaffolding extends beyond the mere application of the ‘correct’ support strategies on the part of the parent and captures the child’s response. There was also a significant improvement in the dyadic flow of the free play interaction, which captures the parent and child ‘connectedness’, or reciprocal interest in interacting and reciprocity of shared topic, ‘balance’ between partners in their contribution to the interaction, and overall ‘fluency’, or flow and lack of rigidity in the play or conversational turns.

These findings of change 3 months after the end of treatment are of particular relevance as they reflect the role of both partners in the success of the interaction, and as such indirectly highlight a significant improvement in both parent and child behavior, in spite of a lack of a significant treatment effect on the child’s primary outcomes. This pattern of findings suggests that a longer follow-up may be necessary to detect changes on joint engagement or that changes might occur on specific aspects of joint engagement that were not measured here, such as how often the child coordinates attention between the partner and objects or integrates language into joint engagement.

To study early effects of the intervention, we conducted the same analyses performed on change scores 3 months post intervention also on data taken immediately after the end of the treatment, as a secondary analysis. Results indicate that the intervention had no significant effect on the study outcomes except for the JERI Parent support of interaction skills, which significantly increased in the treatment group with respect to the TAU condition. Since at 3 months post-intervention both a larger significant change on the parent skills outcome and a significant improvement in the flow of the interaction were found, it can be postulated that change in dyadic outcomes 3 months post intervention may be driven by earlier occurring changes in parents’ actions. Parents may need more time to practice skills that lead to greater flow of the interaction and in turn, potentially, to increased time spent in joint engagement. This interpretation is in keeping with the significant treatment effect found for use of spontaneous gestures, a core impairment in ASD (Mishra et al., 2020): increased gesture production may be a first, critical indicator of improvement in early communication as it predicts language development (Mundy et al., 1987) and later socio-communicative skills (Riva et al., 2021). The post-intervention qualitative feedback provided by interventionists and parents, who reported that mastery of CST strategies required more time than anticipated and advocated for enhancing the intensity of the intervention with booster sessions (Salomone et al., 2021), supports this understanding. There is indeed evidence that parent skills supporting the interaction are pivotal to promote developmental outcomes and reduction in symptoms, e.g. parental synchrony mediated change in autism symptom outcome in the PACT trial (Pickles et al., 2015).

We also found a treatment effect on self-report parent outcomes, including the autism-specific parental self-efficacy (but not the general parenting self-efficacy) and parental stress measures, indicating that CST is effective for proximal, rather than distal (Sandbank et al., 2021) wellbeing outcomes. Similarly, there was no indication of change on the child adaptive behavior domain, as reported in other caregiver mediated trials in both high- (Pickles et al., 2016) and low- resource (Divan et al., 2012) settings, perhaps reflecting poor sensitivity to change of the Vineland measure within a limited time-frame for interventions addressing, but not exclusively targeting, the child’s daily living skills. Overall, the pattern of findings on blind observational measures and self-report questionnaires is mirrored by the qualitative evidence derived from focus groups with interventionists and caregivers.

There are several methodological strengths to the present study. Firstly, the sample size for this community-implemented caregiver mediated intervention evaluated through a randomized controlled design, the first of this kind conducted in public outpatient health settings in Italy, was relatively large – the median sample size in a recent review of interventions for preschoolers which included 48 studies was n = 49 (French & Kennedy, 2018). This, combined with the rigorous clinical characterization of participants through standardized assessments and the use of primary outcomes that were blind-rated, to a high level of reliability, allows for a better understanding of the effectiveness of the intervention in a community setting.

Nevertheless, the results should be interpreted in the context of some design features that may limit the interpretation of findings. First, while the CST program was developed to cater for a range of neurodevelopmental disorders and delays, recruitment was restricted to children with ASD to reduce the sample’s heterogeneity, which restricts the generalizability of our findings, particularly with respect to child outcomes. Similarly, as the CST was originally designed to be implemented by non-specialists, the relevance and applicability of some of our feasibility and acceptability findings may be limited to settings employing interventionists with similar levels of qualification and supervision.

A number of conclusions can be drawn from this pilot RCT of the CST in Italy. The CST was found to be acceptable and feasible. While balanced across groups, the sample was representative of the clinical population of children with ASD and as such characterized by high heterogeneity in clinical profiles, which indicates that our effectiveness trial could well be generalizable. We report treatment effects at family level outcomes, albeit proximal, which are important as they support the effective implementation of interventions in real-world contexts, maximizing the adoption and sustainability of caregiver-mediated interventions within community settings (Wainer et al., 2017). The pattern of change in parent, child and dyad outcomes suggest that strategies to increase the intensity of the intervention, such as the addition of booster group sessions and individual coaching to parents, may be critical to improve its effectiveness. Future research steps should examine treatment mediators and moderators to identify patterns of response to treatment by child and parent characteristics (Smith et al., 2007) and build on the present evidence of the theorized mechanism of intervention effect on proximal outcomes derived from caregiver/child interaction to examine intervention outcomes that may be less prone to correlated measurement error (Crank et al., 2021).