“The Videos Made Me Feel Less Alone”: Feasible Executive Function Supports For Families
Auteurs:
Jessica V. Smith, Tennyson Dahlman, Jessica Holmes, Chelsea A. Armour, Alyssa Verbalis, Allison B. Ratto, Kristina K. Hardy, Meredith Gunn, Kaitlyn Decker, Dennard Brown, Te’Andis Elliott, Monica A. Werner, Katie C. Alexander, Lynn Cannon, Bruno J. Anthony, Lauren Kenworthy, Laura G. Anthony
Executive function challenges are linked to important outcomes in adulthood, including mental and physical health and employment. Executive function skills can be taught but require support for generalization into home and school settings. The present study examined the feasibility of a series of brief videos (e.g., animations, expert and caregiver interviews) for caregivers of children with symptoms consistent with a neurodevelopmental disorder and flexibility or planning/organization problems. Caregivers (n = 102) viewed 12 videos over a four-month time period that describe ways to support and model executive function skills for their children. Quantitative and qualitative feedback suggest caregivers found the videos acceptable, feasible, and that they changed their view of their child’s EF challenges. In addition, caregiver knowledge of key executive function strategies and concepts improved. Results suggested these videos show promise to reduce caregiver-reported sense of strain and child executive function problems; however, caregiver competence did not improve from pre- to post-video review. These findings suggest even brief, online, and asynchronous caregiver-oriented instructional videos on executive function supports may improve caregiver and child outcomes. Moreover, they raise the possibility that low-cost, highly accessible caregiver support tools may yield important benefits for a common and potent childhood difficulty. Given the present study is a preliminary feasibility study of the caregiver support videos, future randomized controlled trials are necessary to fully examine caregiver- and child-related outcomes.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Both Autism Spectrum Disorder (ASD) and Attention-Deficit/Hyperactivity Disorder (ADHD) are common neurodevelopmental conditions (Antshel & Russo, 2019; Pennington & Ozonoff, 1996). Though differing diagnoses, each with separate defining characteristics, there is considerable overlap between the two (Antshel & Russo, 2019). One such overlap relates to challenges with executive functions (EFs). EFs are a set of interrelated, but distinct self-regulatory abilities that include working memory, inhibition, flexibility, organization, and planning (Gioia et al., 2002; Miyake et al., 2000; Stuss & Alexander, 2000). EFs enable the regulation of thoughts, feelings, and behaviors in order to reach a goal or carry out a multi-step task. EF challenges predict lower quality of life (de Vries & Geurts, 2015, Snyder et al., 2015), co-occurring psychiatric conditions or symptoms (Lawson et al., 2015; Wallace et al. 2016), and poorer adaptive functioning (Pugliese et al., 2014, 2016). Considering these clinical outcomes, the development of treatments targeting EFs is critical (e.g., Diamond & Lee, 2011).
COVID-19 and Child EFs
Never was there a time that EFs were needed more than during the COVID-19 pandemic, during which time most people were asked to be flexible, regulate their emotions and thinking, and stay calm when faced with challenges or unexpected changes to routines. Broadly in the general population, children’s EF challenges increased because of pandemic-imposed shifts to remote learning and confinement in the home (Hanno et al., 2022; Lavigne-Cerván et al. 2021). Caregivers experiencing acute COVID-induced distress also appraised their children’s EF as less competent (Polizzi et al., 2021). Caregivers of youth with both ASD and ADHD expressed that behavioral challenges or concerns appeared to be “worsening” during the pandemic (White et al., 2021; Zhang et al., 2020). Additionally, a comparison study of autistic children before and during the pandemic highlighted autistic children experienced more acute behavioral challenges during the pandemic compared to prior (Mutluer et al., 2020).
COVID-19 and Service Access
Despite a high need for services, as evidenced by increased EF challenges and related difficulties, access to public services was incredibly limited due to the pandemic. Many schools transitioned to virtual platforms due to the public health emergency. Consequently, families were placed under great stress (Russell et al., 2020) and unable to receive physical or mental healthcare in-person due to social isolation (Brunt et al., 2023). As the world adapted to lockdown measures and restrictions, individuals in the home navigated new routines, new conditions in which to support their children, and unexpected responsibilities. Compared to pre-pandemic service delivery, high intensity or high frequency services such as special education, occupational therapy, or applied behavior analysis were most disrupted (White et al., 2021). With approximately 14% of all public-school students in the United States receiving federally supported special education services in school, this transition greatly limited access to necessary services for those with support needs and placed the onus on caregivers to support their children at home (U.S. Department of Education National Center for Education Statistics, 2021). In fact, the theme of “coaching families” emerged from a qualitative study of the remote special education service delivery system. In this study, early childhood special educators commented on how service delivery changed from direct in-person service provision to coaching families remotely on how to use intervention strategies at home, in the wake of the pandemic (Steed et al., 2022). Despite shifts to telehealth for some services, caregivers still felt that services were lacking in meeting their children’s needs during the pandemic (Neece et al., 2020).
COVID-19 and Effect on Caregivers
Prior to COVID-19, many researchers examined the linkage between caregiver strain and caring for children with ASD and/or ADHD (Cadman et al., 2012; Fridman et al. 2017; Kirby et al., 2015; Miranda et al., 2015; Theule et al., 2013). Such research supports that the relationship between caregiver strain and child characteristics is bidirectional. Caregiver strain influences caregiving behaviors, which can in turn impact children (Rodriguez et al., 2019); conversely, child characteristics, such as the clinical signatures of ASD, differentially predict both objective and subjective caregiver strain (Bradshaw et al., 2021). Caregivers of children with EF challenges experience higher strain (Craig, et al., 2016; McLuckie, et al., 2021), which may negatively impact mental health, employment, and coping with negative events (Brannan et al., 2012; Sales et al., 2004; Seymour et al., 2013).
While more research is needed to better understand the multidimensional nature of the relationship between caregiver strain and child characteristics, this relationship grew even stronger during the COVID-19 pandemic. Due to the pandemic, levels of anxious and depressive symptoms were higher in the general caregiving population (Geweniger et al., 2022; Radomski et al., 2022; Russell et al. 2020), but caregivers of children with neurodevelopmental and intellectual disabilities experienced stress and anxiety/depression at rates nearly 2–3 times higher than caregivers of children without these conditions (Willner et al., 2020). A high instance of depressive symptoms was especially present in those tending to children with high support needs during COVID-19 (Dhiman et al., 2020; Geweniger et al., 2022), likely as a result of increased caregiving demands. Such demands involved helping children navigate a new day-to-day life, including virtual learning, a new service delivery landscape, and the other consequential changes from COVID-19 (Tseng et al., 2021).
Caregivers’ Role in Child Interventions
While caregiver-mediated intervention models routinely aim to increase caregiver knowledge and efficacy related to the intervention concepts, many also seek to reduce caregiver strain or sense of burden. When engaged in treatment with their child, caregivers can serve as powerful agents of change and generalization because of their increased presence in the child’s life; therefore, positive outcomes may be seen for both children and their caregivers, as caregivers gain new skillsets to support their children. Caregiver-mediated models are particularly effective for children with neurodevelopmental disorders who often struggle to generalize skills learned in treatment to new contexts (Brown & Bebko, 2012; Solomon et al., 2008; Turner-Brown et al., 2019; Webster-Stratton et al., 2013). In addition to caregiver-mediated interventions, caregiver psychoeducation programs are related to better outcomes for caregivers and their children by improving caregiver implementation of strategies and decreasing children’s challenging behaviors (Gross et al., 2019; Sanders et al., 2014; Kenworthy et al., 2023).
More recent research has focused on remote caregiver-mediated and psychoeducation-based interventions, which is especially timely in the wake of COVID-19. Online caregiver-directed support allows caregivers to learn about valuable tools to help their children without requiring them to travel to in-person programs that only run during limited, specified times, or wait to set up regular appointments with an available trained professional (Symon, 2001; Holzman et al., 2023). Online training is also an effective service delivery forum for caregiver-mediated interventions that improve caregiver outcomes (Ingersoll & Berger, 2015). Remote caregiver-mediated intervention trainings can improve caregiver knowledge and social behavior/skills in children with ASD (Parsons et al., 2017). With minimal assistance from a therapist, caregiver knowledge increased as a result of a video program (Dai et al., 2018). Further, relatively brief interactions with online materials related to ASD may be related to positive impacts on caregiver strain, knowledge, acceptance, and the reduction of implicit bias towards children with ASD (Anthony et al., 2020b; Dickter et al., 2021). Another randomized trial has shown that an asynchronous, online caregiver training helped caregivers learn to support and improve their children’s EF abilities (Kenworthy et al., 2023). More research is necessary to evaluate the effectiveness and impact of at-home, caregiver-directed supports, particularly those which do not involve therapist mediation.
Present Study
Born out of the COVID-19 pandemic, the present study was designed to support families as many transitioned to online work and school, leaving children without access to their usual school supports. To equip caregivers with tools to support their children and gain a better understanding of child EF difficulties, the project team created free, brief videos, which demonstrate EF support strategies, such as how to stay calm when faced with a challenge (emotion regulation) or how to support children as they navigate unexpected changes (cognitive flexibility)—situations made only more common during the COVID-19 pandemic. Additionally, the project aimed to evaluate the feasibility, acceptability, and efficacy of such videos for caregivers of youth with EF challenges, many of whom experienced significant shifts in their caregiving roles during the COVID-19 pandemic.
Unstuck and On Target!Unstuck and On Target! (Unstuck; Cannon et al., 2021), the intervention on which this project was based, is a cognitive-behavioral EF intervention designed for delivery in schools for children in third to fifth grade. Unstuck incorporates skill-building activities to promote cognitive flexibility, aid in planning and organization, and help children regulate their emotions through the use of a common language. Although the first edition was originally designed for school-aged children with ASD, positive caregiver and child outcomes for children with symptoms consistent with ASD and/or ADHD and EF challenges were found in two school-based trials and one clinic-based trial (Anthony et al., 2020a; Elias et al., 2019; Kenworthy et al., 2014; Troxel et al., in press). Moreover, Unstuck was effective in reducing caregiver strain and improving some child EFs in a school-based randomized controlled trial with an added caregiver training component (Swain et al., 2021). In addition to the school-based intervention curriculum, the Unstuck curriculum authors developed and tested an online training program (e-Unstuck) for caregivers of children with ASD, of which the findings support the utility of online caregiver training supports to reduce caregiver strain and improve child EFs (Kenworthy et al., 2023).
In the present study, the Unstuck author team, in conjunction with vested community collaborators, chose the most salient topics from the school-based curriculum and presented the topics in a series of distilled, brief videos. Though the core concepts presented are the same as those presented in other Unstuck resources, the present video suite differs in 3 key ways: the setting, the diagnoses of children, and the depth of content. The Unstuck curriculum was designed for delivery in schools or clinics by a staff member knowledgeable in the 21-lesson small group curriculum (about 20 h of content). While children with ASD, ADHD, or EF challenges broadly show benefits through the Unstuck curriculum, the four-hour caregiver training (e-Unstuck) was only tested in caregivers of children with ASD. Finally, both the school- and caregiver-curriculums were designed to provide depth of information through 21 lessons or 10 modules respectively. In contrast, the present suite of videos is meant to give a rudimentary introduction of EF supports to caregivers of children with EF difficulties through viewing of only 60 min of content presented in brief, freely accessible videos that incorporate the Unstuck vocabulary consistent across other Unstuck resources. The video content centers around EFs specifically to help caregivers of any child who may be struggling to develop EF skills, not only those with a diagnosis.
The present study aims to describe the development of these videos and evaluate the feasibility of using this brief suite of videos to support caregivers of elementary school-age children with EF challenges. We hypothesized that (a) caregivers would find the videos helpful and informative, acceptable, feasible, and efficacious; (b) the videos would increase caregiver knowledge of EF principles and tools; (c) caregiver strain would decrease as a result of the videos; (d) caregiver sense of competency would increase; and (e) caregiver-report of their child’s EF challenges, both frequency and interference in daily life, would decrease. The final hypothesis (e) was meant to be an exploratory hypothesis, as the scope of the project is not directly targeting child EF outcomes explicitly. Rather, the present study targets caregivers’ perceptions of themselves and their own abilities to support their children’s EF challenges.
Methods
The study design was in accordance with and approved by the Colorado Multiple Institutional Review Board (COMIRB), protocol number 20-2449. Participation was voluntary, and all participants consented to participation. Participants were compensated for their review of the videos, receiving up to $205 for participation in the study.
Participants
To meet inclusion for the study, caregivers met the following criteria: 18 years or older, fluent in English, lived in the greater Washington, DC area (DC, Maryland, Virginia: DMV) or the state of Colorado (CO), and had no prior participation in an Unstuck research study due to the “basic” nature of video content and to understand feasibility of videos without prior content exposure. Caregivers’ children were required to: be enrolled in 3rd–5th grade, be identified as needing supports in school, have symptoms of (but not necessarily a formal diagnosis of) ADHD and/or ASD and EF challenges at home, verbal language abilities of a 7-year-old or higher, and whose EF challenges interfered in daily life in the home. Child characteristics necessary for inclusion mirror those in the school-based comparative effectiveness trial of Unstuck, which demonstrated the efficacy of Unstuck for children in third to fifth grade who experienced EF challenges and required supports in school (Anthony et al., 2020a). In order to guarantee the presence of caregivers who could review a subset of videos that were produced in Spanish, we conducted purposive recruitment of bilingual (Spanish-English) caregivers. As not all videos were available yet for review in Spanish, monolingual Spanish-speaking individuals were not eligible to review the video suite because of the large proportion of videos to review in English. Inclusion was based solely on caregiver-report to screening questions. For more details regarding inclusionary criteria and recruitment methods, see Supplement A.
A total of 102 caregivers were enrolled into the study and completed baseline measures. 12 participants reviewed six or fewer videos, five of whom did not complete any review. Using the intention-to-treat method to avoid analytic bias, no participants were excluded from the study or analyzes, regardless of the number of videos they reviewed (McCoy, 2017). The study sample was broadly similar to the census-designated representation of the United States. See Table 1 for detailed demographic information and Fig. 1 for the Consolidated Standards of Reporting Trials (CONSORT) diagram.
Table 1
Sample demographics
Total N
102
Caregiver age,M (SD)
41.33 (6.15)
Child age,M (SD)
9.75 (0.93)
Caregiver gender,n
Male
8
Female
93
Non-binary
1
Child gender,n
Male
77
Female
24
Gender fluid
1
Child EF frequency,M (SD)a
7.77 (1.79)
Child EF Interference,M (SD)a
6.73 (1.92)
Caregiver ethnoracial identity,n
Hispanic/Latino
18
Asian, non-Hispanic/Latino
2
Black, non-Hispanic/Latino
14
Pacific Islander, non-Hispanic/Latino
1
White, non-Hispanic/Latino
64
Multiracial, non-Hispanic/Latino
3
Child ethnoracial identity,n
Hispanic/Latino
17
Native American, non-Hispanic/Latino
1
Asian, non-Hispanic/Latino
2
Black, non-Hispanic/Latino
14
Pacific Islander, non-Hispanic/Latino
1
White, non-Hispanic/Latino
55
Multiracial, non-Hispanic/Latino
11
Chose not to respond
1
Languages spoken,n
Only English
86
English and Spanish
14
English and another language
2
Caregiver education in years of schooling completed,M (SD)(n = 99)
16.63 (2.68)
Caregiver gross household income (In Thousands of US dollars),n
<30
11
30–60
13
61–90
19
91–120
22
121–150
11
151–180
6
181–210
7
<210
13
aFrequency and interference of child executive functioning challenges were queried through a Likert scale on a scale of 1(no/never) to 10
Fig. 1
Consort flow diagram
×
Procedures
The videos translated the core components of Unstuck into home-based EF supports in partnership with: neurodivergent self-advocates, a diversity, equity, and inclusion consultant, caregivers of children with ADHD, a teacher, an EF counselor, an occupational therapist, clinical psychologists, and neuropsychologists-all with experience treating child EF challenges. Additionally, the development team members participated in the most recent comparative effectiveness trial on Unstuck, and five are authors of the Unstuck school curriculum (Cannon et al., 2021). The development team identified the following topics based on core components of Unstuck: (1) education on the biological basis of EF; (2) the importance of supporting EF in children with neurodevelopmental disorders; (3) coping with overwhelming feelings; and (4) the development of a simple shared vocabulary, or common language, or self-regulatory phrases (e.g., “Big Deal/Little Deal” and “Compromise”; Cannon et al., 2021). The content of each video aimed to provide psychoeducation of EF and effective strategies to support children EF at home, including common child behavior management strategies that can help reinforce emerging child EF skills, such as the use of praise and rewards (Kazdin, 1997; Patterson, 1982). Subgroups of the team wrote the scripts for each video, which were then reviewed by the entirety of the development team as well as a Community Advisory Board (CAB) that was established alongside the video development team. Details about the CAB can be found in the Community Involvement Statement below. Content was revised iteratively after each review and then delivered to the video production team, which included a Curriculum Developer, Multimedia Specialist/Videographer, and Voice Actors.
After initial animation or filming, content was returned to the development team to review the videos, suggest edits, consult with the CAB, and ultimately approve the final product. 12 videos were produced for this project in total, five of which were also in Spanish; the duration of the videos ranged from two minutes and fifty-seven seconds to seven minutes and forty-five seconds, with an average video length of four minutes and forty-three seconds. The video suite consists of nine animations (six in English, three in Spanish) and eight interviews (six in English, two in Spanish). Two of the interviews were with caregivers of autistic children; all other interviews were with authors of the Unstuck school curriculum (see Supplement B for more information).
Participant enrollment began in January 2021 and ended in April 2021, during the presence of the COVID-19 delta variant and continued virtual/hybrid learning in some parts of the country. Videos were distributed to participants in four groups of three videos each, with three weeks between groups, beginning 4/26/2021. The distribution of videos in groups at this time interval was a product of the development timeline, as video content continued being developed while earlier videos were reviewed by participants. Videos ready for review were emailed to participants via a REDCap survey link, which first asked a content knowledge question related to the video that the participants had not yet watched to capture baseline knowledge of video content. Upon participant response to the knowledge question, participants were able to watch the video and advance to the following form page, which asked the same content knowledge question again to capture their knowledge post-video review. Each video to be reviewed by the participants followed this format: video content-related knowledge question to be completed before watching the video for the first time, embedded link to watch video, post-video review of the same knowledge question. Participant video viewing was not monitored, but the REDCap survey and database platform does track survey form completion. In this way, study staff could monitor which surveys were completed, and by proxy, which videos were viewed. Participants were not required to view the current set of videos to receive the next set. However, in an effort to have as many videos reviewed as possible, at the time the next set of videos were sent, participants were encouraged to watch the videos and complete the surveys from the previous group, if they had not done so already. Monolingual English-speaking participants viewed all of the English videos. The five videos translated into Spanish replaced the English versions for our Spanish-English bilingual participants, but they also viewed the remaining videos only available in English, thus all participants were asked to view a total of 12 videos. The order that videos were viewed was not randomized, so every participant received the videos in the same order. Post-measures were sent three weeks after video group four to follow the cadence of video distribution. The videos are now available on an open access playlist on YouTube (Unstuck And On Target 2021a; Unstuck And On Target 2021b).
Community involvement statement
The project included a community advisory board (CAB), which was made up of teachers, school administrators, caregivers of children with ASD and/or ADHD, and an autistic caregiver and self-advocate. Each played a significant role in the development of the videos that were produced for this study. The CAB was exclusively female, but included Black, Latina, Indigenous, and White women. The CAB was developed to include vested community members’ voices in the content of the videos. Video content did not proceed past the scripting phase until at least two members had seen and either approved of the content or provided their feedback. All of their feedback was addressed. A CAB chair was appointed to organize the CAB’s content review and liaise between the development team and vested community members, though the development team met directly with CAB members to obtain feedback and edit material for individual videos.
Another important aspect of this project was obtaining feedback directly from the caregiver participants themselves. After completing each video group that the research team distributed, participants provided their own feedback that was later considered as final edits were made before widely distributing the videos. In recognition of the value of multiple perspectives (Mitroff & Linstone, 1993), particularly when investigating EF challenges which are transdiagnostic and influence a wide range of people, the project team followed up with an individual who expressed continued dissatisfaction with the content to understand their perspective. The detailed results of caregiver feedback are provided in the Results section below. Based on feedback and suggestions from the caregivers participating in this project, additional videos and tip sheets for each video were made after this initial evaluation.
Measures
Content-related measures
Caregiver knowledge and self-report of helpfulness
Caregiver knowledge was assessed before and after each video in order to capture immediate change in knowledge, and at four set timepoints throughout the study, coinciding with the group of videos distributed for review (see Procedures). Knowledge questions were created by the video content development team and targeted key themes from the corresponding video, see Supplement C for the knowledge questions and response options. Questions were coded as either correct (1) or incorrect (0) and a final score for baseline and post was calculated by summing the number of correct answers (maximum score of 12). After viewing of each video and completion of the corresponding knowledge question, caregivers were also asked, “How informative or helpful was this video” and gave their rating on a five-point Likert scale from 1 “Not at all” to 5 “Very much”.
Feasibility, efficacy, and acceptability
Feasibility and efficacy questions were asked at the beginning of video groups starting with group two and ending with post. Instructions specified that caregivers answer in relation to the previous group of videos. For example, the feasibility questions sent with group two asked about the feasibility and efficacy of applying concepts and strategies presented in the first group of videos. These questions were presented in this manner to allow caregivers time to use concepts and strategies presented in the previous videos and attest to any change they observed. To measure feasibility of the video content and assess if caregivers started using strategies as they were introduced, caregivers reported the key EF strategies, if any, they used with their child, phrased as, “Have you used the following strategies or words with your child?” Caregivers checked the strategies they used from a list of 14 options (see Supplement D). As new strategies were introduced with each video group, caregivers may not have had knowledge of all 14 strategies prior to completion of all video review. To measure efficacy, caregivers answered the question, “How much did these videos change the way you think about your child’s difficulties?” using a five-point Likert scale from 1 “Not at all” to 5 “Very much”.
The acceptability of video content was measured only at post-video review, such that caregivers could assess the video content as a whole. Caregivers reported the acceptability of the videos through two five-point Likert scale questions, including: the degree to which they found the videos valuable on a five-point Likert scale from 1 “Not at all” to 5 “Very much,” and the degree to which they would endorse the videos to other caregivers on a five-point Likert scale from 1 “Strongly disagree” to 5 “Strongly agree”.
Caregivers were also given the opportunity to provide us with open-ended feedback. Before viewing a new video group, caregivers were asked, “Do you have any comments, suggestions, or other feedback for the previous videos you watched”?
Caregiver–related measures
Caregiver strain
The Caregiver Strain Questionnaire-Short Form 7 (CSQ-SF7; Brannan et al., 2012) is a seven-item questionnaire in which caregivers rate their child-related caregiver strain, using a five-point Likert scale ranging from “not at all” to “very much.” Composite scores are calculated from the sum of raw scores; higher scores indicate greater strain, with a maximum score of 35. Though not originally designed for caregivers of children with ASD or ADHD, the long version of the CSQ has been validated in caregivers of children with ASD and deemed a reliable, valid measure (Cronbach’s α = 0.93; Khanna et al. 2012). In the present sample, psychometric properties of the CSQ-7 showed good internal consistency at both timepoints (Cronbach’s αBaseline = 0.81; Cronbach’s αPost = 0.88). The CSQ-SF7 was collected at baseline and post.
Caregiver competence
Caregiver sense of empowerment was measured through the competence subscale of the Family Empowerment Scale (FES; Koren et al., 1992). This subscale consists of eight questions in which caregivers rate their perceived abilities using a five-point Likert scale ranging from 1 “Never” to 5 “Very often.” The sum of raw scores from the competence subscale yields a composite score in which higher scores indicate greater competence (maximum score of 40). Psychometric properties of the scale have been evaluated and are acceptable, including strong test-retest reliability and validity (Cronbach’s α = 0.85; Singh et al., 1995). In the present sample, the measure showed strong internal consistency at both timepoints (Cronbach’s αBaseline = 0.82; Cronbach’s αPost = 0.84). The competence subscale of the FES was collected at baseline and post-.
Caregiver-report of child behavior measures
Caregivers reported their child’s observed EF through two global EF questions which assess the degree of interference from child EF challenges at home and frequency of EF challenges on a 10-point Likert scale; higher scores (maximum 10) indicate greater EF challenges, both in interference of and frequency of EF challenges. These items were adapted from the Flexibility Interference Scale. The Flexibility Interference Scale was previously developed by the research team, inspired loosely off of the Interests Scale (Bodfish, 2003), which is a caregiver-report measure of the presence and intensity of circumscribed interests.
The Flexibility Interference Scale was sensitive to change in a comparative effectiveness trial of Unstuck & On Target! (Anthony et al., 2020a). In order to ensure participants understood the construct of EFs, as we defined it from empirically derived conceptualizations, we provided participants with a definition of EFs. This stated: “Executive functions help your child follow directions, set goals, stay organized, follow plans, manage transitions and disappointment, and stay calm when faced with a challenge.” The full Flexibility Interference Scale is available in Supplement E.
Data Analysis
Data were analyzed using IBM SPSS Statistics (Version 27) and R Statistical Software (Version 4.1.1). Independent samples t-tests were used to evaluate potential differences between those who completed more or less than half of the video review across continuous variables, such as caregiver strain, caregiver empowerment, and child EF. For categorical variables (i.e., demographic variables) chi-squared tests were used. Due to the disparate sample sizes between those who viewed only English videos and those who viewed both English and Spanish, Welch t-tests were used to examine group differences across key variables (caregiver strain, caregiver empowerment, child EF frequency, and child EF interference). Paired samples t-tests were used to evaluate the magnitude of the videos’ effect from baseline to post-video review for all key outcome variables, including caregiver strain, caregiver competence, child EF, and content efficacy. A post hoc one-way repeated measures analysis of variance (ANOVA) was used to better understand the significant difference in efficacy from pre- to post-video review. Missing data were pairwise removed, and the sample size for each analysis is provided.
Qualitative data, namely the caregiver feedback provided through open response, was thematically coded by members of the research staff as either: (a) positive, (b) negative, or (c) suggestive of ways to improve the videos through specific advice or changes. Feedback was coded as positive if the content included praise for the program or expressions of gratitude or preference for the content provided. Feedback was coded as negative if caregivers expressed negative feelings towards the content without any suggestions, including feeling they did not learn from the videos or did not enjoy them. Finally, feedback was coded as suggestive if it included things that could be improved about the videos (e.g., decreasing the volume level). Initial coding was conducted by one member of the research staff (JH), followed by secondary coding by another research staff member (JS). In circumstances in which the two coders did not agree, a third coder (TD) was engaged in discussion to determine a consensus code. Feedback was most often coded as the caregiver’s full open response; however, in cases where feedback from caregivers contained information that could equally fit into more than one category, comments were separated into their respective excerpts and coded accordingly.
Results
Baseline Comparisons
Overall, a majority of caregivers completed the review of more than half of the videos (87.25% of the total sample). Of those caregivers that did not review most, or all of the videos, those missing were spread throughout the video sets. For a more complete breakdown of the percentage of missing reviews per video, see Supplement F. Caregivers who completed the review of more than half of the videos were compared to those who completed less than half (12.74%) to assess for group differences. We also compared those who reviewed only English videos (84.31%) and those who reviewed both Spanish and English videos (13.73%). No significant baseline differences between those who completed more or less than half of the video review emerged from analyses of categorical demographic variables and continuous variables (caregiver strain, caregiver empowerment, or child EF; all ps > 0.05); however, the groups did differ on baseline knowledge based only on whether they completed the review of more or less than half of the videos (t(8) = −4.72, p = 0.001, 95% CIs [−3.06, −1.06]), such that those who completed more than half of the review of the videos had higher baseline knowledge. There was no difference in baseline knowledge based on languages spoken. Those who reviewed only English videos did not differ from those who reviewed both Spanish and English videos on baseline strain, empowerment, or child EF frequency; however, these groups differed on child EF interference (Welch’s t (122.92) = 5.21, p < 0.05).
Caregiver Ratings on Helpfulness, Acceptability, Efficacy, and Feasibility
Caregivers found the videos helpful and informative (M = 4.01, SD = 0.14, range: 1–5). They also indicated that they would recommend the videos to other caregivers (M = 4.61, SD = 0.57) and found them valuable (M = 4.16, SD = 0.79). After reviewing the videos, caregivers reported that the videos changed the way they viewed their child’s difficulties (M = 3.81, SD = 0.84). Furthermore, their views of their child’s EF difficulties significantly changed between video group one and post, t(69) = −7.31, p < 0.001, 95% CI [−0.75, −0.43], with a strong effect size (Cohen’s d = 0.87). Exploratory post hoc analysis of the change in efficacy suggests that caregivers’ views varied between each video group (see Supplement G for results and Supplemental Fig. 2 for the mean efficacy ratings at each timepoint).
Finally, after reviewing all videos, caregivers indicated using an average of seven key EF strategies with their child; the three most widely used were “taking a deep breath,” “giving praise and rewards,” and using the word “flexible.” No caregivers endorsed using “whiteboard” or “unstuck” after review of all videos. See Supplement H for a summary of caregiver ratings by timepoints.
Caregiver Knowledge, Strain, and Sense of Competency
Caregiver knowledge significantly increased from baseline to post, t(88) = −14.21, p < 0.001, with a large effect size (see Fig. 2), showing promise that the videos taught EF concepts and skills to caregivers. To test if caregiver income may have been related to knowledge increase, a Spearman rank-order correlation was calculated, the results of which suggested income is not significantly related to caregiver change in knowledge (Spearman’s ρ = 0.18, p = 0.08) with a small effect size (Cohen’s d = 0.08). Caregiver strain significantly reduced from baseline to post (t(73) = 3.80, p < 0.01) with a small to moderate effect size (see Fig. 2). Removing one outlier, results remain significant: t(73) = 2.59. Caregiver competency did not significantly change from baseline to post, t(73) = −0.20, p > 0.05.
Fig. 2
Caregiver and Child Outcomes of the Video Suite. Note. This figure displays the mean scores of the variables of interest (caregiver strain, caregiver sense of competence, frequency of child EF problems, interference of child EF problems, and caregiver knowledge) at baseline and post-video review. The figure displays effect sizes for all paired samples t-tests. *** p < 0.001, ** p < 0.01, * p < 0.05
×
Child EF
The frequency of caregiver-reported child EF challenges significantly decreased from baseline to post (t(74) = 4.48, p < 0.001) with a moderate effect size (see Fig. 2). Additionally, there was a significant decrease in the interference of child EF challenges from baseline to post (t(74) = 2.51, p < 0.05) with a small effect size (see Fig. 2). Change scores were calculated between baseline- and post-video review (Baseline – Post) for child EF interference, child EF frequency, and caregiver strain. Change in caregiver strain was significantly correlated to caregiver-reported change in child EF interference (r = 0.43, p < 0.001) and to change in child EF frequency (r = 0.38, p < 0.001).
Feedback from Caregivers
Caregivers’ comments were generally positive (75% of comments); however, some caregivers offered negative (7% of comments) or suggestive feedback (18% of comments). One example of a positive comment from a caregiver stated: “The videos made me feel less alone. They reminded me that my son’s behavior is ‘normal’ within the spectrum and hopeful that there are more things I can try to help him.” Using the project’s iterative approach, the team directly addressed 68% of any negative or suggestive feedback, including whether comments were unique to just one caregiver (see Table 2). Comments were not addressed if they were not feasible for the scope of the project or appeared in direct conflict with comments from the majority of other caregivers (these are also presented in Table 2).
Table 2
Sample caregiver feedback and solutions
Feedback addressed
Solution
1a. Background noise in the videos was too distracting
→
1b. Volume lowered or removed.
2a. Request for a written summary following each video
→
2b. Guides written to accompany each video
3a. Concern that videos indicated caregivers should use specific vocabulary (e.g., “Plan A/Plan B”) with their children without explicitly stating that a child should not be forced to use/hear words that may be aversive
→
3b. Addition of specific guidance stating caregivers should not feel compelled to use language their child finds aversive.
3c. Guidance also underscored the importance of collaborative efforts with children to identify preferred words.
4a. Content in the video was not new, and suggested strategies had been tried unsuccessfully
→
4b. Explicit addition of the term “basic” in the title of the video library (“Executive Function Basics for Parents”) to signal the material is meant to serve as an introduction to EF supports
4c. Addition of new video entitled “I Need More Help!” to help caregivers troubleshoot strategies
5a. Request to watch videos multiple times or have videos after completion of the study
→
5b. Dissemination pages were provided to all research participants with links to video suite
Feedback not addressed
Request for more specific (“in action”) feedback added to each video
Request to remove closed captioning
Feedback on content/style changes (changing of visuals, themes)
Request for answers to questions that were specific to their child/specific circumstances
Spanish language translation changes
This table highlights sample key themes from caregivers’ comments that were either addressed or not addressed by the research team. The table also displays the solutions from feedback addressed by the team
After preliminary review of caregiver feedback, the research team identified one caregiver who provided consistently lower ratings than all others in the project. A member of the research staff (JS), CAB chair (TE), and one of the principal investigators (LK) directly met and interviewed this caregiver just before post- surveys were sent (7/11/2021) to better understand their perspective and address their needs appropriately.
Discussion
Results from the current study provide preliminary evidence of the feasibility of a collection of 12 brief, online videos (five of which are also presented in Spanish) that teach caregivers key skills and concepts. These videos provide simple “how to” guidance related to understanding their child’s brain-based EF differences, supporting their children in the context of daily life, teaching coping skills, and using self-regulatory scripts like “compromise” to help caregivers support their child’s EF at home. Results highlighted caregivers’ high appraisal of the videos, finding them helpful and informative while showing promise to change the way they view their child’s EF difficulties, and reduce caregiver strain and caregivers’ report of their child’s EF challenges; results also indicate that video content enhanced caregiver knowledge, though self-report of caregiver competency did not change. Qualitative feedback from caregivers illustrated the quantitative findings, as caregivers remarked that the videos made them feel less alone, helped them to understand “it’s not defiance but more EF,” and that they learned both how their children process things and corresponding tools to prevent child overload. The collection of feedback regarding the subset of Spanish videos from bilingual individuals served to gather preliminary feasibility of developing the videos in another language—a necessary step prior to robust engagement with multilingual participants. Finally, the present study lays a blueprint for participatory community-based work through its iterative inclusion of both vested community members’ and caregivers’ feedback. While the study design was limited in its ability to draw conclusions regarding efficacy or outcomes, its timeliness allowed caregivers to receive educational tools at a moment of critical need. Due to the intent to fill an imminent service gap, a control group was not immediately appropriate for this feasibility study; however, the results show promise for change, meriting future randomized controlled trials.
Research such as this project helps to increase generalizability of learned skills, as caregivers are in the best position to support children across multiple settings, and they engage with their children more than clinical staff. Very few evidence-based trainings have been developed using an online, asynchronous modality for caregivers. Of the few for caregivers of autistic children, the present findings that caregivers like self-directed video supports to support their children and increase their content knowledge are consistent with that of previous work (Anthony et al., 2020a; Anthony et al., 2020b; Dickter et al., 2021; Kenworthy et al., 2023; Nefdt et al., 2010; Wainer & Ingersoll, 2013). Differing from studies only focused on knowledge or acceptance (Anthony et al., 2020a & Anthony et al. 2020b), the present findings also highlight the feasibility, acceptability, and preliminary efficacy of self-directed content.
While other self-directed programs (e.g., Nefdt et al., 2010; Wainer & Ingersoll, 2013) aimed to teach specific interventions that required fidelity to implement, the present study covered general concepts and strategies for a transdiagnostic issue. After reviewing all videos, caregivers in the present study reported using some strategies more often than others. The popularity of certain strategies increases our understanding of what caregivers may find most feasible or important to implement. Most caregivers endorsed the use of self-regulatory scripts (e.g., “flexible”), emotion regulation strategies (e.g., “taking a deep breath”), and positive reinforcement. Conversely, few caregivers reported using a “whiteboard” or “unstuck.” One possible explanation may be that caregivers adapted strategies to fit their family needs. For example, caregivers may have found the word “flexible” to be a more accessible way of talking to their children about getting “unstuck,” or may have used the concept of “talk less, write it down” without the implementation of a whiteboard. Future qualitative research is needed to better understand what self-regulatory scripts and strategies are most feasible and acceptable for caregivers to implement.
Despite direct implementation of learned strategies and increased knowledge, caregivers reported their level of competence or self-efficacy in caregiving for their child did not significantly change from pre- to post-video review. The present finding is consistent with previous Unstuck research (Kenworthy et al., 2023; Swain et al., 2021), in which caregiver-reported competence did not change, or showed more modest change despite gains in other child- and caregiver-domains (e.g., reduction in self-reported caregiver strain, reduction in caregiver-reported child behavior challenges), after participation in an Unstuck parent program. One explanation may be that caregivers in the present study reported high levels of competence at baseline, thus a ceiling effect may have made it more difficult to detect meaningful change with the chosen measure. However, it may also be that these videos taught caregivers new skills that require time to build self-efficacy or competence in the implementation. Nonetheless, future qualitative research would be informative to disentangle the complex relationship between knowledge, implementation, strain and competence and identify additional intervention targets to bolster caregiver competence.
Though the study included only caregivers of children with symptoms of ASD or ADHD, these diagnoses are never explicitly mentioned in the videos, and content surrounds EF broadly. Given EF challenges are not exclusive to these neurodevelopmental disorders and have been found to be transdiagnostic markers of psychopathology (Bausela-Herreras et al., 2019; Snyder et al., 2015), we suggest content from these videos can be utilized by a breadth of caregivers. Moreover, the capacity to reach many caregivers is not just a product of the transdiagnostic nature of the content; the videos being web-based but optimized for viewing on handheld devices, asynchronous, free, brief, and 508 compliant allows for greater reach compared to other models of caregiver support, such as those that require caregivers to travel, have access to a device with internet at home, or a myriad of other factors that limit generalizability.
In addition to creating content that is accessible, our results serve as a testament to the caliber of content that can be created through the use of iterative feedback and content evaluation. Participatory community research has been proposed as a way to narrow the translational gap between research and practice, which is estimated to be as high as 17 years (Green, 2008; Morris et al., 2011). Using iterative feedback to ensure the content was reflective of the community’s needs and priorities, we strove to create videos that filled an immediate service gap for families. Our two-pronged approach to video development may be a useful blueprint for future development and feasibility studies. The CAB diligently reviewed and provided suggestions for the videos before they finished production, and then research participants provided critical end-user feedback to refine the content, presentation and supports further. More than ensuring the content is of the highest quality, this approach also creates an environment in which every member of the project (research team, CAB, research participants) feels as though they have a voice in the final product, which is atypical of many academic research designs. The final video suite’s high feasibility and acceptability ratings are likely a direct byproduct of the ground-up research design and centering community member’s voices within content development.
While the results provide preliminary support for the feasibility and acceptability of these accessible resources, even supports designed to be accessible are obsolete if the targeted audience does not access them.
Following the completion of data collection, the video suite was made available to the public, and the research team began disseminating all videos as a package via flyers and media announcements, including school-, clinic-, and community-based dissemination. We now face a new hurdle: how do we engage caregivers to consume video content on their own without the prompting of a research team? Following the initial dissemination, close monitoring of video views suggests that caregiver engagement tapers off after the first few videos in the playlist, with public views dropping from multiple thousand to a few hundred, which highlights a key barrier to dissemination. While caregivers may be excited to learn strategies, their excitement does not appear to lead to sustained engagement with the resources, which warrants further exploration. Future dissemination strategies could include staged releases, in which the content may appear more palatable because it is presented as a shorter list of videos. Furthermore, previous research in self-directed trainings for caregivers of children with a neurodevelopmental disorder suggests caregivers benefit from additional coaching or feedback following a training (Wainer & Ingersoll, 2013). For those who do not complete the entire video suite, perhaps the inability to ask questions regarding content or assess their own fidelity made it more difficult to continue learning new content.
Limitations
Future studies could benefit from addressing the following limitations. First, while the videos were designed to be viewed in any order, the research team prompted caregivers to watch the previous set of videos at the time the next one was sent, which may have implied a viewing order or that the program was not completely asynchronous.
The present study was limited in the ability to draw even preliminary conclusions regarding videos in Spanish, given the small sample of bilingual reviewers and number of Spanish videos included in the suite. Additionally, all reviewers of the Spanish videos were bilingual (Spanish-English) speakers, so they may provide a different perspective than monolingual individuals. As dissemination continues to be monitored through public playlist views, Spanish video views are not climbing parallel to the English videos, which suggests a dissemination barrier to consider moving forward; thus, we are currently translating the remaining videos into Spanish. Though matching the US census, our sample was also predominately White, well-educated, with high income; future research should include a more diverse sample, to ensure generalizability of findings.
All findings were measured based on caregiver-report, and there was no control condition; thus, these should be considered preliminary findings to examine the feasibility of the video suite. Additionally, the caregiver-report child behavior measure was modified from its previous version specifically for this study and had not been rigorously validated. Measurement error may have impacted both participant inclusion (as the child EF measure was part of inclusion criteria) and the study results. Furthermore, the question of whether caregiver views of their child’s difficulties had changed did not indicate a direction of change (positive or negative); consequently, the findings cannot directly show a direction for the change seen in these results; however, one could reasonably infer that the views changed in a more understanding and accepting direction, given the qualitative feedback. Finally, caregiver knowledge of video content was assessed using a single question, which was asked before and after each video. While these questions were created around the thesis of each video, they may fail to assess other ancillary content from the video.
Additionally, caregivers were compensated for their review of the videos, making them more motivated to continue watching the videos. To assess this limitation, the project team continually monitors the video playlist to assess real-world generalizability (e.g., number of views per video) of the high appraisal caregivers provided, where we see high numbers of views for the first video, but lower numbers for the following videos, which is typical of accessible online supports. Caregivers included in the study were also motivated to seek support, as evidenced by their research participation. In attempting to generalize these findings, one should consider the presence of treatment-seeking behaviors, which could moderate the effectiveness of the videos. Additionally, the study took place at a period of time of great distress for many families, which may play a latent role in caregivers’ responses to the receipt of free, supportive resources. Interpretation of the present findings would have benefited from the collection of COVID-19 specific scales to better understand the impact of the pandemic on participating caregivers; however, many of these scales (e.g., COVID-19 Exposure and Family Impact Scale; Kazak et al., 2021) were designed after the onset of the research study. In interpretation of the present findings, one should consider whether there is something about the particular moment at which we reached caregivers, who were seeking support for themselves during a global pandemic, that could explain these results. Future, more robust and randomized studies with a control group, as well as validated measures, are necessary to fully understand: the efficacy of these support videos to improve caregiver- and child-related outcomes, whether they are effective when viewed in any order, and whether these videos can be used as a standalone support.
Nonetheless, these results demonstrate promise, considering the caregivers watched a total of only 60 min of content, indicating caregiver support tools presented in brief videos are welcomed supports. As waitlists for evidence-based treatment continue to grow, these videos may serve as an initial support while caregivers wait for more intensive, personalized treatment. Moreover, clinicians could consider the utility of these or similar videos as part of a stepped-care model, in which receipt of these or similar videos could be the first step in treatment sequencing. Use of these videos may alleviate a key challenge faced by caregivers: the high cost and wait time to receive treatment. As the videos are open access and freely available, caregivers can readily watch them, increasing both access and reach of supports linked to preliminary positive outcomes for both caregiver and child. Ultimately, these findings raise the possibility that low-cost, highly accessible caregiver support tools may yield important benefits for a common and potent childhood difficulty, though more research is necessary to fully evaluate this potential.
Authors M.A.W., K.C.A., L.C., L.K., and L.G.A. receive royalties from Paul H. Brookes Publishing, Inc. for Unstuck & On Target! No other authors have any conflicts of interest to disclose.
Ethics Approval
The study design was in accordance with and approved by the Colorado Multiple Institutional Review Board (COMIRB), protocol number 20-2449.
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Met BSL Psychologie Totaal blijf je als professional steeds op de hoogte van de nieuwste ontwikkelingen binnen jouw vak. Met het online abonnement heb je toegang tot een groot aantal boeken, protocollen, vaktijdschriften en e-learnings op het gebied van psychologie en psychiatrie. Zo kun je op je gemak en wanneer het jou het beste uitkomt verdiepen in jouw vakgebied.
“The Videos Made Me Feel Less Alone”: Feasible Executive Function Supports For Families
Auteurs
Jessica V. Smith Tennyson Dahlman Jessica Holmes Chelsea A. Armour Alyssa Verbalis Allison B. Ratto Kristina K. Hardy Meredith Gunn Kaitlyn Decker Dennard Brown Te’Andis Elliott Monica A. Werner Katie C. Alexander Lynn Cannon Bruno J. Anthony Lauren Kenworthy Laura G. Anthony