Digitally-Mediated Social Stories Support Children on the Autism Spectrum Adapting to a Change in a ‘Real-World’ Context

Recent estimates suggest that approximately 1 in 59 children are diagnosed with autism, a lifelong neurodevelopmental condition characterised by impairments in social communication and interactions, combined with repetitive and restricting patterns of behaviour, activities and/or interests (RRBs) (American Psychiatric Association 2013; Baio et al. 2018). Males are diagnosed more than females with a ratio of around 4:1 (Baio et al. 2018). As children on the autism spectrum typically have impaired social skills they often lack the ability to understand social cues and protocols. Social skills are essential for both academic and social success, as they facilitate peer acceptance and student–teacher relationships (Garwood and van Loan 2019). Such impairments can be a cause of distress and frustration, and using parental/carer report, Brereton et al. (2006) identified greater disruptive, inattentive and anxious behaviour in children on the autism spectrum compared to children with intellectual disability. Research suggests that approximately two thirds of children on the autism spectrum display such behaviours (Hartley et al. 2008), which can have a negative impact upon daily activities (Brereton et al. 2006), create life-long barriers to inclusion (Rhodes 2014), and increase caregiver/family stress (Ludlow et al. 2011; Tomanik et al. 2004; Yacoub et al. 2018).

A popular intervention for children on the autism spectrum aimed at supporting social skills is Social Stories™ (SS). SS, first developed by Carol Gray in the 1990′s (Gray and Garand 1993), provide social information in a simple visual format that explains what to expect and what constitutes appropriate behaviour (Hutchins and Prelock 2013). They are highly structured, personalised social narratives that are developed and delivered in according to a set of specific criteria to ensure that the content has an emphasis on being descriptive rather than directive (Gray 2010, 2018). They have proved to be an acceptable intervention within the autism community and are widely used both within school and home settings (Green et al. 2006; Hess et al. 2008; Reynhout and Carter 2009).

However, despite their popularity, and an increase in the number of published studies within this area, the evidence relating to the effectiveness of SS remains questionable. Case studies have reported positive effects of using SS interventions (e.g. increasing play skills—Barry and Burlew 2004; appropriate behaviour at lunchtime—Bledsoe et al. 2003; or reducing disruptive behaviours—Crozier and Tincani 2005; Scattone et al. 2002). However, a large number of reviews and meta-analyses have now been conducted and these reveal a less positive picture overall (Ali and Frederickson 2006; Garwood and Van Loan 2019; Kokina and Kern 2010; Leaf et al. 2015; Mayton et al. 2013; McGill et al. 2015; Qi et al. 2018; Test et al. 2011). Such reviews yielded broadly similar conclusions, highlighting mixed findings in terms of effectiveness and raising a series of methodological weaknesses and reliance on single case study methodology. Kokina and Kern’s findings, for example, highlight the large variation in effectiveness across both studies and participants. The authors found that whilst 51% of included SS interventions were classified as “highly effective”, almost all the remaining stories (44%) were deemed “ineffective”. Similarly, McGill et al. (2015) found effect sizes to vary from small to large.

When determining the effectiveness of SS, a key factor to consider is what the SS is addressing. Kokina and Kern (2010) identify four categories that SS interventions are used for, namely: reducing negative behaviour, increasing positive behaviour, managing transitions/novel situations/anxiety, and teaching new skills (academic/functional). This is consistent with Reynhout and Carter (2009), who found that within an educational context, teachers report using SS to target a wide range of behaviours including decreasing challenging behaviours, teaching social skills, and assisting with changes and new routines. Within the research literature, reviews have shown that SS are predominantly used to decrease negative behaviours or increase positive behaviours (95%; Garwood and van Loan 2019; 86%; Kokina and Kern 2010) with some evidence suggesting that SS are most effective when targeted towards reducing negative behaviours (Hutchins and Prelock 2013; Kokina and Kern 2010; Qi et al. 2018).

There is less research on the third category of using SS to target managing transitions/novel situations/reducing anxiety, which makes up 9% of the SS literature¹ (Kokina and Kern 2010; and no studies in this category were identified by Garwood and van Loan 2019). Qi et al.’s (2018) systematic review of SS identified studies that only fell under the categories of increasing positive behaviours/social communication skills and decreasing negative behaviours, and Wahman et al.’s (2019) systematic review only included the categories of increasing positive behaviour and reducing negative behaviour. Whilst the effectiveness of SS is largely evaluated upon research assessing increasing/decreasing behaviour, there is evidence that in practice SS are used for managing transitions/novel situations/anxiety reduction (Briody and McGarry 2005; Daly et al. 2010; Ivey et al. 2004; Marion et al. 2016; Morrison and Gullón-Rivera 2016). Reynhout and Carter (2009) reported that 87% of the teachers they surveyed used SS to support the introduction of changes/new routines (such as transition to school, or visits to the dentist). This is pertinent as difficulties in this area may relate to features such as resistance to change/insistence on sameness, which are RRB diagnostic features that are specific to autism (APA 2013; see Russell and Brosnan 2018). Previously reported studies within this area have been largely anecdotal. For example, Daly et al. (2010) describe the use of SS in preparing a child for an upcoming allergy test and Morrison et al. (2016) describe how SS could be used to help support siblings of patients in Neonatal Intensive Care Units. Similarly, a case study approach has been used to explore how SS can reduce levels of anxiety (O'Connor 2009), which is pertinent as co-occurring clinical levels of anxiety are also reported to be extremely high in autism (30–40%, Simonoff et al. 2008; Uljarevic et al. 2016). In addition, adapting to change has been proposed to be a separable aspect of RRBs that positively relates to anxiety in autism (Eisenberg et al. 2015; Uljarevic et al. 2016).

Thus within ‘real-world’ settings, there is evidence that SS are being widely used to support children on the autism spectrum to adapt to change (i.e. manage transitions/novel situations/reduce anxiety). However, little research on effectiveness focuses upon this. One challenge for research is how to assess the impact of the SS on supporting a child for an upcoming change. When SS are used to increase a positive behaviour or reduce a negative behaviour, the frequency of this behaviour can be compared pre- and post-intervention. This methodology, however, is less amenable to SS when the goal is to support the child manage a transition or adapt to a novel situation. In addition, increasing social skills or reducing challenging behaviours may be addressing behaviours that occur frequently and over an extended time period, whereas adapting to a specific change or event is likely to be relevant to a specific time point (such as visiting the dentist/allergy testing) and opportunities for comparison data can be limited. Teacher/parent ratings of perceived anxiety and closeness to the goal of the SS can be reliable measures within real world settings (Marshall et al. 2016).

In addition, it has been reported that researchers conform to Gray’s criteria for SS intervention more than teachers, when developing and delivering interventions (Mayton et al. 2013; McGill et al. 2015; Styles 2011) and little is known regarding intervention fidelity for SS within community settings (Mandell et al. 2013). Whilst evidence for efficacy (under ideal/controlled circumstances; see Singal et al. 2014) may be developed within a research-led context (Smith et al. 2020), any changes made within the real-world setting may compromise fidelity and effectiveness (Cohen et al. 2008; Gearing et al. 2011; Smith et al. 2007). This is pertinent as the high training requirements to ensure fidelity within real-world settings make this an unlikely solution to reduce disparities between research and practice (Stahmer et al. 2014). Significant variability in the implementation of evidence-based practices for children on the autism spectrum specifically have been identified, which highlights the need to address challenging issues related to fidelity in real-world settings (Mandell et al. 2013).

Children on the autism spectrum can show a preference for interventions being delivered through digital devices (such as iPads), compared to traditional methods, such as one sentence being written per page in a paper-based book (Bouck et al. 2014; Mancil et al. 2009; for systematic reviews of the benefits of iPads for autism interventions see Alzrayer et al. 2014; Kagohara et al. 2013). SS for children on the autism spectrum may be particularly amenable to delivery through digital technology (e.g. Kennedy et al. 2019; Ghanouni et al. 2019) as there are benefits of digitally-mediated SS for those who have difficulties with social interaction, enabling greater intensity of interaction with the content of the story. Digital technology can provide a more consistent and structured environment for the story, enabling repetition and direct feedback, and can offer the child more control over the learning experience. Digital technology can also enhance visual support, self-monitoring, and rewards, all of which can be personalised to the child (Constantin et al. 2017; Hanrahan et al. 2020; Moore 2008; Odom et al. 2003; Ozdemir 2010; Segers and Verhoeven 2005; Smith et al. 2020; Yildirim et al. 2001). Tablets, such as the iPad, have been used as a SS intervention for children on the autism spectrum through showing ‘social movies’ based upon the criteria for SS (Flores et al. 2014; Mowling et al. 2018). Almutlaq and Martella (2018) also used the ‘Kid in Story Book Maker’ app to successfully write a SS for three children on the autism spectrum (to give compliments).

Whilst digitally-mediated interventions have great promise for autism, evidence for best practice is yet to be established (Zervogianni et al. 2020a, b). Kim et al. (2018) reviewed nearly 700 mobile device apps listed under “Autism Apps” and found the 0.6% has direct evidence to support their usage. With respect to SS apps specifically, an online survey identified 22 features that parents and practitioners found desirable in a SS app (such as a library of stories, guidance of criteria for writing stories). A subsequent survey of app stores identified 19 SS apps and, of these, only one had more than half of these desired features (StoryMaker had 12/22; see Smith et al. 2020). SOFA-app was co-developed with the autism community (Constantin et al. 2017; see Parsons et al. 2019) and addresses all 22 desired features, and a prototype was used for the present study. The SOFA-app runs on mobile devices (smartphones and tablets) with Android and IOS operating systems (SOFA-app.org, released free-of-charge Autumn, 2020).

Thus the Stories Online for Autism (SOFA-app) application has been co-developed with the autistic community to enhance fidelity for community-delivered SS interventions through providing appropriate support for practitioners and parents/caregivers (Constantin et al. 2017; Hanrahan et al. 2020; Smith et al. 2020). An opportunity arose to use the SOFA-app with a group of children on the autism spectrum who needed support adapting to the same upcoming change. A school summer camp trip was planned for the class and the aim of the present study was therefore to explore the use of digitally-mediated SS for supporting children on the autism spectrum adapt to this upcoming change. Prior to the study, to put the research in a broader context, a preliminary survey of practitioners (who were familiar with using SS) was conducted to confirm the extent to which SS were used for adapting to change (i.e. managing transitions/novel situations/reducing anxiety) during their usual practice working with children on the autism spectrum in schools.

Initially pre- and post- intervention scores were compared, revealing that that the children were significantly closer to reaching their goal at the post-intervention time point (M = 7.30, SD = 1.42), compared to pre-intervention (M = 3.90, SD = 1.91, W(10) = 2.68, p = 0.007; η² = 0.60, see Fig. 1a) with 9 (out of 10) children responding positively. The children’s perceived level of understanding was also rated significantly higher post-intervention (M = 7.10, SD = 1.52), compared to pre-intervention (M = 4.70, SD = 2.16, W(10) = 2.09, p = 0.036; η² = 0.47, see Fig. 1b) with 8 (out of 10) children responding positively. Perceived anxiety scores were also significantly lower post intervention (M = 3.10, SD = 1.37), compared to pre-intervention (M = 5.30; SD = 1.25, W(10) = 2.68, p = 0.007; η² = 0.60, see Fig. 1c) with 9 (out of 10) children responding positively.

Secondly the post-intervention ratings were compared with the post-event ratings to identify any relationships between teacher ratings at the end of the intervention (but before the event) with teacher ratings of the behaviour during the event itself. During the event (i.e. the summer camp) the mean perceived difficulty the child was having with their SS goal (problem severity) was 4.0 (SD = 2.67, range = 1–8) and the mean perceived anxiety was 2.8 (SD = 1.48, range = 1–6). Correlations between each of the three post-intervention scores (closeness to goal, perceived understanding, perceived anxiety) and the two post-event scores (problem severity, anxiety) were conducted. Results revealed a significant negative relationship between the ratings for both ‘closeness to goal’ and ‘perceived understanding’ at post-intervention with problem severity (r_s = − .82, p = .004; r_s = − .67, p = .03; respectively) and perceived anxiety (r_s = − .77, p = .01; r_s = − .74, p = .02; respectively) during the summer camp. This indicated that the closer the child was perceived to be towards understanding and reaching and their goal, the less severe problems and anxiety they experienced during the summer camp. Perceived anxiety at post-intervention did not significantly correlate with problem severity or anxiety during the event (both p > .05). Importantly too, problem severity or anxiety during the event did not correlate with any of the pre-intervention measures (closeness to goal, perceived understanding, perceived anxiety, all p > .05).

Children were also asked to reflect on how they felt about the focus topic of their SS, identified by their Teacher from the Summer Camp Questionnaire (e.g. mealtimes). Based on their smileyometer scores (1 = awful, 2 = not very good, 3 = good, 4 = really good, 5 = Brilliant), the pre-intervention mean rating was 3.45 (SD = 1.01, range = 2–5) and the post-event mean was 4.15 (SD = 1.13, range = 2–5). Although this was a positive increase with a medium effect size, it was not significantly different (W(10) = 1.45, p = .15; η² = 0.32). Finally, children completed an evaluation of the SOFA-app (see Table 2, which also has the children’s daily ratings as the top item).

SS is a widely used intervention for children on the autism spectrum, although the evidence for their effectiveness is mixed. In line with previous research (Reynhout and Carter 2009; Varnava et al. 2019), our preliminary survey confirmed that SS are often used to support adapting to change, that is managing transitions, novel situations, reducing anxiety, within primary education (ages 5–11). This is pertinent as the evidence for the effectiveness of SS largely draws upon research focussed upon reducing negative behaviour or increasing positive behaviour (Garwood and van Loan 2019; Kokina and Kern 2010; Qi et al. 2018). Difficulties adapting to change are part of the RRB diagnostic criteria for autism (APA 2013) and high levels of anxiety co-occur in many people on the autism spectrum (O’Nions et al. 2018; Simonoff et al. 2008; Uljarević et al. 2017). Adapting to change has been proposed to be a separable aspect of RRBs that positively relates to anxiety in autism (Eisenberg et al. 2015; Uljarević et al. 2017). This was the first study to examine the effectiveness of digitally-mediated SS in supporting a group of children on the autism spectrum adapting to change, as previous research has comprised of single case studies (Briody and McGarry 2005; Daly et al. 2010; Morrison and Gullón-Rivera 2016).

Findings indicated that a daily SS intervention, delivered over a 1-week period, significantly reduced perceived anxiety levels and increased understanding in the children prior to the change (attending a school summer camp). The children were also rated as being significantly closer to their target goal by the end of the intervention. The comparison of the pre- and post-teacher measures were therefore all positive, which was consistent with the child-based ratings. The effect sizes for the intervention were found to be medium to large. In addition, the relationships between the post-intervention ratings (before the event itself) and the child’s subsequent perceived experience during the summer camp indicated that these positive effects extended beyond the intervention phase, to the event itself. The greater a child’s understanding and reaching the social story goal prior to the event, the less problem severity and anxiety were displayed by the child during the event. Thus, despite the apparent dearth of SS research within the category of ‘managing transitions/novel situations/reducing anxiety’ (Kokina and Kern 2010; see also Garwood and van Loan 2019; Qi et al. 2018), this study indicates that SS are beneficial within this context. This is particularly pertinent as the preliminary survey (above) identified this as the most frequent category for SS interventions by practitioners.

Supporting the child’s understanding of the situation is central to a SS which should ‘provide a student with autism an accurate understanding of the situation in which the targeted behaviour occurs’ as they may be ‘more impaired in their access to accurate social information than their ability to understand and respond appropriately to it’ (Gray and Garand 1993: 2). A theoretical understanding of why SS can be effective needs to be developed and one potential is the Dual Process Theory of Autism which proposes that difficulties with rapid, implicit, spontaneous processing can be ameliorated by building upon strengths associated with slower, explicit, prompted processing (Brosnan et al. 2016, 2017; Lewton et al. 2019; see also Callenmark et al. 2014; Senju 2013; see Happé et al. 2017, for review).

Previously, a short protocol of less than ten sessions had been highlighted as impacting upon effectiveness (Kokina and Kern 2010). Our findings suggest that an intervention period of 1-week, consisting five sessions only, was sufficient to elicit a positive change. This is also of notable benefit considering time and budget constraints often reported by teachers (Lang et al. 2010; Machalicek et al. 2007). Within the SS literature previous studies have yielded small to large effect sizes (see McGill et al. 2015). The large effect sizes found within the current study may be due in part to the high level of fidelity as the Researcher developed and delivered the intervention (see McGill et al. 2015). In addition, they may also relate to the measures employed, which were kept deliberately short and simple for teachers to complete, in line with feedback from the largest SS evaluation feasibility study, and included the ‘goal based measure’ that was found to be most reliable (Marshall et al. 2016).

The findings suggest that digital SS can be effective in supporting children on the autism spectrum adapting to change through identifying the specific goal of the intervention and increasing understanding of the issue. This is of particular interest to autism, as the point of any intervention is to affect change, yet a diagnostic feature of autism RRBs is resistance to change/ insistence on sameness.² Whatever the focus of the intervention, resistance to change may represent an obstacle to engagement for autistic participants (e.g. Isenberg et al. 2019). This would suggest that supporting the adaptation to change could be an invaluable addition to any intervention protocol. SS may therefore be a useful addition in preparing children on the autism spectrum for an upcoming intervention, as well as changes such as holidays or events such as visits to medical professionals.

It is also pertinent that children perceived the SS intervention to be a positive experience. All the child ratings were good to brilliant, and there were no negative responses to the other items, though it should be noted that the smileyometer has three positive and two negative options. Consistent with previous research (Bouck et al. 2014; Mancil et al. 2009), all the children indicated that they would prefer digitally-mediated SS intervention compared to traditional book formats, which may also have impacted upon the success of the intervention. A limitation is that the study did not index the children’s experience with traditional SS formats and this may reflect a general preference for using digital technology rather than a specific preference for digitally-mediated SS interventions. Digital technology has been argued to support a consistent, structured, personalised learning environment enabling repetition, direct feedback and control over the learning experience (Constantin et al. 2017; Hanrahan et al. 2020; Moore 2008; Odom et al. 2003; Ozdemir, 2010; Segers and Verhoeven 2005; Smith et al. 2020; Yildirim et al. 2001). The extent to which these features support a preference for slower, explicit, prompted processing in autism (see Brosnan et al. 2016, 2017), provides a framework for future research to identify whether digitally-mediated SS interventions are particularly effective for children on the autism spectrum. The SOFA-app has been co-developed with the autistic community with the aim of supporting the fidelity of the intervention. Whilst the present study followed the progress of children in a real-world context of a school summer camp, fidelity of the intervention was ensured by a Researcher developing and delivering the intervention. Other research is needed to identify the extent to which fidelity can be supported by the SOFA-app when the SS are developed and delivered by parents and teachers (see Smith et al. 2020).

When considering the findings from this study however, the following should be borne in mind. A major limitation is the small sample size and the number of comparisons, indicating the findings should be considered exploratory. The present study, however, is one of the largest studies of SS to support adapting to change in children on the autism spectrum, expanding upon previous research which has largely been based on case studies (see Qi et al. 2018). All the children had formal diagnoses from clinicians employing established criteria for Autism Spectrum Disorder (APA 2013). ADOS scores, were not available to the Researchers yet diagnosis was consistent with the SCQ assessment, though it should be noted that teachers were not able to reflect on the child’s behaviour before the age of 4.

As noted above, there are methodological issues which make this a challenging function of SS interventions to research. Firstly, there are no objective measures of behaviour change, the present study relied upon teacher ratings and it is possible that these may be susceptible to bias. It is not possible within this context to obtain objective pre- and post-measures of behavioural frequency, for example. Importantly too, there was no comparison data, we do not know what the problem severity or anxiety at the event would have been like without the intervention. In addition, without comparison data, firm conclusions cannot be drawn as to what impacted upon effectiveness. Whilst the findings are consistent with children on the autism spectrum having a preference for using digital technology and SS being effective, this study is limited in causal inferences that can be made. It may be the personal attention from the Researcher related to effectiveness for example. In their review of evidence for SS, Wright et al. (2016) identified seven between-groups SS published studies, all of which have methodological limitations. Importantly too the teachers and children were not blinded and were aware of the nature of the study. Recently, Hanrahan et al. (2020) identified benefits of SS when compared to a control condition of reading the child a poem.

Whilst post-intervention measures correlated with the summer camp assessments, the pre-intervention measures did not. This is potentially consistent with the intervention impacting upon the summer camp, although clearly post-intervention ratings were closer in time to the summer camp than the pre-intervention ratings. Larger numbers of participants would enable randomised control trials of the intervention, although larger numbers of children on the autism spectrum adapting to the same change at the same time may be difficult to identify. The present study addressed this through identifying a range of potential goals for the SS that may arise during the summer camp. Transitioning to school may represent a context that this approach would be useful for in future research.

Finally, the SS literature has focussed on increasing positive behaviour and reducing negative behaviour (e.g. Wahman et al. 2019) and this study has highlighted the effective use of SS for supporting adapting to change. The use of SS for teaching new skills (academic/functional) is an under-researched area (Garwood and van Loan 2019; Kokina and Kern 2010). As 29.1% of survey respondents in the present study indicated that they used SS for this purpose, future research can address the effectiveness of SS for teaching new skills.