Measuring Changes in Social Skills Throughout an Intervention Program for Children with ASD, Contributions from Polar Coordinate Analysis

In order to confirm the diagnoses of autism, a clinical interview based on DSM 5 criteria (APA, 2013) was done. Additionally, the ADOS-2 (Lord et al., 2000) was administered to all participants. To assess Verbal Comprehension, cognitive abilities were measured using the Wechsler intelligence scale for children and adolescents (Wechsler, 2007; Wechsler & Kaplan, 2015).

Sessions were recorded with two different cameras located at different angles. Cameras were positioned discreetly to avoid inconvenience to participants and followed all the ethics aspects.

To codify social behaviors, we reorganized and evolved an observational instrument (Alcover et al., 2019) which had been inspired by an original observational instrument of Bauminger (2002) and by the approach of social difficulties by the authors of ADOS-2 and Autism Diagnostic Interview-Revised (Lord et al., 2000; Rutter et al., 2003). Certain behaviors were not included in the analysis because the observation frequency was very low such as proximity, sharing objects, affection, talk that reflects an interest in another child's hobbies, giving help, peer or therapist imitation, idiosyncratic language and repetitive behavior. We reorganized the observational instrument in order to adjust it to the behavior of our participants (see Table 1). The final instrument had 6 dimensions. For each dimension, we built a category system, following the requirements of exhaustivity and mutual exclusivity. Dimensions and categories are shown in Table 2.

As aforementioned, our program was an adaptation of the Social Skills program of the UC Davis Mind Institute (Solomon et al., 2004). We implemented 10 sessions of 1:30 hours each. Solomon’s program is less structured than most traditional programs. It includes semi-structured instructions, positive reinforcement, motivation for social interactions, and free-time play as an opportunity to practice. Regarding the intervention, the structure of each session was: salutation with a little talk about the week and introduction of the topic (allowing participants to talk and share thoughts or problems), free time to play (were children play games in an unstructured time-lapse), didactic activity (structured part were pedagogic information and activities of social competence are performed) and closing (structured part that include joke-telling time and the optional homework, called “social experiment for the week”). The specific contents of each didactic session are detailed in Table 1. Clinicians developed strategies and activities based on theory of mind and social competence (Garcia Winner, 2007) in order to practice the didactic components.

The sample was divided in four groups in order to facilitate group intervention. Each group was formed by 7, 8 or 9 participants and each group received the same intervention with similar conditions. As aforementioned, the clinical team that conducted the groups included a clinical psychologist, a psychiatrist and a mental-health nurse with large experience in social skills interventions with ASD children. After obtaining approval from the authors (Solomon and colleagues), our clinicians were trained in the program by a member of the team who had previously collaborated with the developing team of the program at the UC MIND Institute. The clinical psychologist was the main therapist of the intervention, while the psychiatrist and the mental-health nurse were co-therapists. Additionally, two master’s degree students participated as co-therapists in order to support children in a more individual way.

In order to ensure fidelity of implementation of the intervention, before each session, clinicians had a meeting with the member of the team who had previously collaborated with the developing team of the program. In these meetings, specific instructions for each session were provided to the team and a protocol with a complete description of steps was reviewed. By the end of each session, clinicians reviewed again the specific protocol and its checklist, in order to ensure that the intervention was provided consistently and accurately as designed.

From the qualitative research perspective, a systematic observation was used to obtain data that we managed as a code matrix. Two psychologists who did not participate in the intervention were trained on the observation instrument in order to analyze and code data. These observers were aware of the research hypotheses, but they did not have information about the session number (session 2 or 10) or the students' IQ groups. The degree of interobserver agreement calculated with Cohen’s Kappa () ranged between 0.76 and 0.89. To obtain this value, 20% of the material was coded and the Kappa coefficient of the 2 and 10 sessions (which were randomized) was obtained of the 10% of the participants. This inter-observer agreement was conducted between the observers and the main therapist of the intervention, who had collaborated with them in the adaptation of the observational instrument. Once we had confirmed the reliability of the data, we codified 14 min during the free play activity of session 2 (as a pre-evaluation) and 10 (as a post-evaluation) of the participants that participated in all sessions and appeared in all videos, to observe differences between sessions. Social conditions of both sessions were identical. The same group of children and therapists were present and the same materials were offered to the participants.

The software LINCE (Gabin et al., 2012) was used to codify social behaviors during the free time of the selected sessions. After the codification, the analysis strategy was developed in two stages.

First, in a descriptive level, the absolute frequencies of each of the behaviors recorded were analyzed and compared according to the groups of participants and the session (2 and 10 session).

In the second stage, mixed methods methodology was used in order to integrate qualitative and quantitative elements. This approach implies the integration of different types of data, which includes the transformation of quantitative data into qualitative data (Sandelowski et al., 2009), of qualitative data into quantitative data (Arias-Pujol & Anguera, 2017, 2020), or any other type of information (Onwuegbuzie & Teddlie, 2003) in a comprehensive way (“crossover”, according to Onwuegbuzie & Dickinson, 2008). We selected this methodology due to the lack of objective measures that reflect the real progress of social skills abilities. The majority of studies use questionnaires to assess the evolution of participants, but these methods tend to be subjective and incomplete (McMahon et al., 2013; McMahon et al., 2013; Moody et al., 2020). Therefore, additional methodologies that complement questionnaires have been proposed, such as social cognitive assessments and behavioral observations (Alcover et al., 2019; Solomon et al., 2004). These methodologies have demonstrated to be effective in assessing evolution and changes in social behaviors, including micro-conducts like gestures. In addition, they provide greater knowledge and specificity about social behaviors, which can offer information for professionals to improve their interventions and focus them on their participants (Alcover et al., 2019). As aforementioned, mixed methods methodology allow to transform qualitative data into quantitative data.

Specifically, in our study, we first used an observation instrument, which is a qualitative procedure. We registered categorical variables (behaviors) which were then transformed into quantitative data, in order to obtain parameters such a frequency, order and duration. Finally, the polar coordinate technique (Cochran, 1954) was applied to compare the social interactions generated in sessions 2 and 10. We selected this technique because it offers information about the relationship between a focal behavior and other conditional behaviors. Polar coordinate technique, applied by Sackett (1980) and further optimized with the genuine retrospective technique proposed by Anguera (1997), allows the reduction of data by using the Zsum statistic (Zsum = Σz/√n), where Z represents the independent values obtained from the adjusted residuals found for the respective delays of − 5 to − 1 and 1 to 5, and n represents the number of delays considered. Previous literature suggests that conduct patterns tend to get dissolved after 5 delays (Bakeman & Gottman, 1997). The sequential analysis of delays allows us to control the random effect. The goal is to identify sequences that appear in a higher frequency than expected by random. Thus, the Zsum values allow us to estimate the type of relationships established between the selected focal behavior and the other behaviors (conditioned behaviors) that constitute the instrument of observation. The type of relationship between focal and conditioned behaviors is shown qualitatively (Quadrant I, II, III or IV) and quantitatively (vector length). Quadrants indicate whether the focal and conditional conducts activate or inhibit each other. Activation involves that when a conduct occurs, another conduct is elicited. In contrast, inhibition means that one conduct might inhibit another. Vectors in Quadrant I indicate mutual excitation between focal and conditional behavior. Quadrant II informs about inhibitory focal behavior and excitatory conditional behavior. Vectors in Quadrant III indicate mutual inhibition between focal and conditional behavior. Quadrant IV informs about excitatory focal behavior and inhibitory conditional behavior.

To perform these analyses, we included 20 participants from the sample, because data from VIQ from one participant was missing. Coordinate analyses were performed in two subgroups: Verbal IQ (VIQ)> 90 (n = 14) and VIQ < 90 (n = 6).

In session 2 (see Table 4 and Fig. 1), a significant activation relationship was observed between focal behavior response to an interaction and low-level interaction conditioned behavior, in 9 of the 14 participants with VIQ > 90. Simultaneously, low-level interaction (LLI) inhibited the response to interactions, since the vector with a significant radius representing LLI is located in quadrant IV (Quadrant IV, radius = 2.29, SD_radius = 0.59, angle = 346°, SD_angle = 8.91, p < 0.05). On the other hand, for the rest of the participants with VIQ > 90 (5/14), there was a relationship of mutual activation between response and low-level interaction (LLI), but it was not statistically significant (Quadrant I, radius = 1.69, p > 0.05). In two of these participants, functional communication (FUNC) significantly activated the response to interactions (participant 13, Quadrant I, radius = 2.29, p < 0.05; participant 1, Quadrant II, radius = 2, p < 0.05). However, while for the first participant, the response to interactions in turn activated functional communication, for the second, these responses inhibited functional communication. Finally, we observed again in one participant from VIQ > 90 group that high-level interaction (HLI) activated the response to interactions (participant 21, Quadrant II, radius = 2.05, p < 0.05) and initiations of interaction (IN) inhibited the appearance of new social responses for another (participant 13, Quadrant III, radius = 2.5, p < 0.05).

Coinciding with the other group, we observed that low-level interaction (LLI) inhibited the response to interactions observed in 5 of the 6 participants with VIQ < 90 (Quadrant IV, radius = 2.34, SD_radius = 0.19, angle = 340º, SD_angle = 8.98, p < 0.05). Other findings obtained in the VIQ > 90 were not observed in this group, but a significant relationship of mutual activation between look without eye contact (NOEC) and response to interactions in one of the members of this group was found (participant 15, Quadrant I, radius = 2.34, p < 0.05).

In session 10, we observed again, in fewer participants, the previous relationship of significant activation between focal behavior response to an interaction and conditioned behavior low-level interaction (5 of the 14 participants with VIQ> 90) (Fig. 1) (Quadrant IV, radius = 2.58, SD_radius = .47, angle = 345.73°, SD_angle = 8.18, p < 0.01; in 2 of the 6 participants with VIQ < 90 (Quadrant IV, radius = 2.47, SD_radius = 0.66, angle = 355.8°, SD_angle = 0.23, p < 0.05).

In session 10, regarding to the VIQ < 90 group, we observed in one participant how conventional gestures (CONVG) activated responses to interactions while they inhibited these types of gestures (participant 18, Quadrant II, radius = 2.28, angle = 131.72°, p < 0.05). No other significant relationships between behaviors were observed in this subgroup.

The set of associated behaviors was significantly greater for this focal behavior than for the response to interaction.

In session 2 (see Table 5 and Fig. 2), the group of participants with VIQ> 90 had a greater initiation of interactions (11 of 14 participants initiated social interactions in the initial session (M = 5.64, SD = 3.58, Range [1–12]), compared to the other group (only 3 of 6 participants, M = 2.67, SD = 2.08, Range [1–5]). In two of the participants in this group, there was a significant relationship of mutual activation between high level interaction (HLI) and focal behavior (initiation of interaction) (Quadrant I, radius = 3.90, SD_radius = 1.39, angle = 10.11°, SD_angle = 5.18, p < 0.01). Also, we observed a relationship of mutual inhibition between focal behavior (initiation of interaction) and deictic gesture behavior (POINTG) in 4 of the participants (Quadrant III, radius = 2.12, SD_radius = 0.15, angle = 227.98°, SD_angle = 8.09, p < 0.05). Mutual activation relationships between focal behavior (initiation of interaction) and verbal social communication (SOVERC) were also observed in 2 participants (participants 3 and 13: Quadrant I, radius = 3.20, SD_radius= 0.04, angle = 24.37°, SD_angle = 3.94, p < 0.01) and activation of eye contact (EC) by focal behavior (initiation of interaction) (participant 3, Quadrant IV, radius = 4.57, angle = 354.75°, p < 0.01) (participant 13, Quadrant I, radius = 3.2, angle = 48.59°, p < 0.01).

In the group of participants with VIQ < 90, only 3 of the 6 participants-initiated interactions in session 2 (M = 2.67, SD = 2.08, Range [1–5]). As it can be seen in Figure 2, results show a significant relationship between deictic gestures (POINTG) that activated focal behavior (initiation of interaction) in 2 of the 6 participants (participant 15, Quadrant I, radius = 2.77, angle = 56.59°, p <0.01) (participant 10, Quadrant II, radius = 2.68, angle = 97.39°, p <0.01).

In session 10 a lower number of significant relationships was compared to session 2 and in a smaller number of participants in both groups (Figure 2). However, both eye contact (EC) (Quadrant II, radius = 3.55, SD_radius= 0.19, angle = 95.41°, SD_angle= 1.94, p < 0.01) and smile (SMIL) (Quadrant II, radius = 3.04, SD_radius = 0.93, angle = 96.87°, SD_angle = 4.06, p < 0.01) activated the initiation of social interactions in two participants from the VIQ > 90 group (participants 3 and 13). This type of relationship did not occur among participants in the VIQ < 90 group.