Brain–computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum
Elisabeth V. C. Friedrich1* 
Neil Suttie2 
Aparajithan Sivanathan3 
Theodore Lim3 
Sandy Louchart2 
Jaime A. Pineda1- 1Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
- 2School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, UK
- 3School of Engineering and Physical Science, Heriot-Watt University, Edinburgh, UK
Individuals with autism spectrum disorder (ASD) show deficits in social and communicative skills, including imitation, empathy, and shared attention, as well as restricted interests and repetitive patterns of behaviors. Evidence for and against the idea that dysfunctions in the mirror neuron system are involved in imitation and could be one underlying cause for ASD is discussed in this review. Neurofeedback interventions have reduced symptoms in children with ASD by self-regulation of brain rhythms. However, cortical deficiencies are not the only cause of these symptoms. Peripheral physiological activity, such as the heart rate and its variability, is closely linked to neurophysiological signals and associated with social engagement. Therefore, a combined approach targeting the interplay between brain, body, and behavior could be more effective. Brain–computer interface applications for combined neurofeedback and biofeedback treatment for children with ASD are currently nonexistent. To facilitate their use, we have designed an innovative game that includes social interactions and provides neural- and body-based feedback that corresponds directly to the underlying significance of the trained signals as well as to the behavior that is reinforced.
Neuroetiology of Autism Spectrum Disorder (ASD)
Autism spectrum disorder (ASD) is an increasingly prevalent condition in the U.S. with core deficits in the unique domain of human social behaviors (American Psychiatric Association, 2000; Hansen et al., 2008; Rice, 2011). Individuals with high functioning ASD show deficits primarily in social and communicative skills such as imitation, empathy, and shared attention, as well as restricted interests and repetitive patterns of behaviors. These deficits substantially impair satisfactory social interactions and prevent children from establishing adequate relations with their family or friends from their early years.
To date, no single explanation can account for the broad and varied profile of the deficits in ASD. Nonetheless, exploring the neuroetiology of this disorder is a focus of our research which was prompted by the discovery of mirror neurons. The discovery of these visuomotor cells in monkey prefrontal cortex (di Pellegrino et al., 1992) and the description of a similar network of areas in the human brain, or mirror neuron system (MNS, Figure 1; Rizzolatti and Craighero, 2004), has provided a testable neurobiological substrate for understanding many key concepts in human social and emotional cognition directly relevant to the behavioral and cognitive deficits observed in children with ASD (Williams et al., 2001). ASD is marked by impairments in social skills - from joint attention and the ability to comprehend actions, to learning through imitation to understanding the intentions of others (Carpenter et al., 1998; Baron-Cohen, 2009). An increasing amount of studies suggest that a dysfunction in the human MNS contributes to these kinds of social deficits (Nishitani et al., 2004; Oberman et al., 2005; Théoret et al., 2005; Dapretto et al., 2006; Hadjikhani et al., 2006; Bernier et al., 2007). Specifically, deficits are likely to arise from an inability to “form and coordinate social representations of self and others” “via amodal or cross-modal representation processes” (Rogers and Pennington, 1991), the type of function ascribed to mirror neurons. However, the theory of MNS is the object of critical debates (Enticott et al., 2013). An alternative explanation, for example, is that dyspraxia rather than the MNS could account for imitation deficits in children with ASD (Mostofsky et al., 2006; Stieglitz Ham et al., 2011). Moreover, questions have been raised as to whether the discovery of mirror neurons in monkeys can be translated to explaining human social behavior (Hickok, 2009; Turella et al., 2009).
FIGURE 1. The mirror neuron system and the social engagement system. The mirror neuron system (MNS) is represented by the white boxes with the black letters (adapted from Pineda, 2008). The core MNS (i.e., parietal frontal in the rostral cortical convexity of the inferior parietal lobule, ventral premotor area of the inferior frontal gyrus, and the superior temporal sulcus) is extended by the sensorimotor cortex as well as the insula and the middle temporal gyrus. The MNS is involved in perceiving sensory input as well as in motor output through various processes. The social engagement system is represented by the black boxes with the white letters and shows how social communication is regulated by the cortex (adapted from Porges, 2007, 2003). The solid lines indicate the somatomotor components that control the muscles of the face and the head. The dashed lines represent the visceromotor component which consists of the myelinated vagus that controls the heart and bronchi.
From an anatomical perspective, an underconnectivity hypothesis has been proposed by Just et al. (2004), which posits that “autism is a cognitive and neurobiological disorder marked and caused by underfunctioning integrative circuitry that results in a deficit of integration of information at the neural and cognitive levels.” Reduced connectivity, especially in ASD individuals, is consistent across studies using various cognitive, emotional, and social tasks (Villalobos et al., 2005; Welchew et al., 2005; Just and Varma, 2007) and in both default mode and task-related functional connectivity magnetic resonance imaging (fcMRI) studies. While a general theory of disordered connectivity has emerged, the nature of the disorder is not yet clear. To bring some level of reconciliation along various observations, several investigators have proposed a compromise solution that focuses on both local overconnectivity and long range underconnectivity (Anderson et al., 2011). This is not inconsistent with the MNS hypothesis since over- and under connectivity likely characterizes this specific network.
From electrophysiological studies of ASD, there is an equally emergent framework. Using phase coherence in multiple frequency bands as a measure of functional connectivity, evidence shows both global hypoconnectivity and local hyperconnectivity (Murias et al., 2007). Specifically, locally elevated coherence in the theta (3–6 Hz) frequency range in ASD subjects, particularly over left frontal and temporal regions, as well as globally lower coherence in the lower alpha range (8–10 Hz) within frontal regions was found (Murias et al., 2007). In contrast, decreased local and decreased, as well as increased, long range spectral coherences for the ASD-group in comparison to controls was reported recently (Duffy and Als, 2012). Furthermore, the coherence patterns in the ASD group were unusually stable across a wide spectral range, which was interpreted as “over-damped neural networks.” Other studies have reported lower delta and theta coherences within as well as between hemispheres across the frontal region, with delta, theta, and alpha hypocoherences over temporal regions while in posterior regions, low delta, theta, and beta coherences were observed (Coben et al., 2008). Moreover, increased gamma activity over parietal cortex (Brown et al., 2005), decreased left hemispheric gamma power (Wilson et al., 2007) and increased connectivity of temporal lobes with other lobes in the gamma frequency band (Sheikhani et al., 2012) have been reported for individuals with autism. Based on the neuroanatomical, functional, and electrophysiological evidence, we hypothesize that a range of over- and underconnectivity in children with ASD, particularly in the MNS system, correlates with levels of performance in cognitive, emotional, and behavioral outcomes.
Rationale for Brain–Computer Interface (BCI) and Neurofeedback Training (NFT) for ASD
We have previously hypothesized that BCI-based neurofeedback using specific electroencephalographic (EEG) frequency bands should induce neuroplastic changes and lead to normalization of the MNS (Pineda et al., 2012). A BCI allows real-time information of brain activity to be fed back to a user by means of a computer in a closed loop (Figure 2) enabling control and natural operation of brain oscillations across cortical networks in vivo and in near real time (Nowlis and Kamiya, 1970; Wolpaw et al., 2002; Friedrich et al., 2009, 2013; Neuper et al., 2009; McFarland et al., 2010). The possibility of volitional control of these oscillations suggests – provided that they play a causal role in specific cognitive functions – that it is theoretically plausible that their modulation can have a functional impact.
FIGURE 2. Closed feedback loop of the Social Mirroring Game. The user’s EEG and peripheral physiological measures are recorded (Thought Technology Ltd., Canada) and fed into the Social Mirroring Game which gives the user visual feedback. For positive feedback (i.e., indicated in green), the child’s avatar must first approach the non-player character (NPC) and while facing him, the player has to show appropriate brain and/or peripheral physiological activity. The rewarding feedback involves the child’s avatar imitating the facial emotions of the NPC. The negative feedback (i.e., indicated in red) involves the child’s avatar being not responsive to the NPC. By means of the feedback, the user can learn to change his/her brain activity voluntarily and thus can control the game.
The gold standard of neurofeedback training (NFT) is based on quantitative electroencephalography (QEEG). This approach is able to identify unique electrophysiological phenotypes (Coben et al., 2010), which makes the possibility of a QEEG-based NFT as a personalized therapeutic approach viable. That approach improves the likelihood that the intervention will be effective by first identifying activity at specific electrode sites that are outside the norm, i.e., comparing the data to already existing normative databases, and then targeting the sites of greatest difference for NFT (Cantor and Chabot, 2009; Coben and Myers, 2010; Thompson et al., 2010). Recent QEEG guided studies have reported behavioral improvements on a number of measures and it has been used to achieve behavioral and neuroregulatory improvements, primarily in children with attention deficit hyperactivity disorder, but also in those with ASD (Coben and Myers, 2010; Thompson et al., 2010). More specifically, assessment guided NFT was used to reduce hyperconnectivity in posterior-frontal to anterior-temporal regions (Coben and Padolsky, 2007). Following NFT, parents reported symptom improvement in 89% of the experimental group, with very little change in the control group. Improvement also occurred in the areas of attention, visual perceptual functioning, language, and executive functioning, with a 40% reduction in core ASD symptoms as assessed by the Autism Treatment Evaluation Checklist. There was also decreased hypercoherence in 76% of the experimental group as measured by a post-training QEEG. Kouijzer et al. (2009b) reported improved executive functions for attention control, cognitive flexibility, and planning as well as improved social behavior after a theta/beta-based NFT training in children with ASD compared to a waiting list group. The linear decrease in theta power and the increase in low beta power were hypothesized to enhance activation of the anterior cingulate cortex, which has been found to show reduced connectivity in ASD individuals (Cherkassky et al., 2006). A follow-up after twelve months revealed maintenance of the described outcomes on both executive functioning and social behavior, suggesting that NFT treatment can have long-term effects (Kouijzer et al., 2009a). The examination of physiological and behavioral data from the children themselves as well as the use of a control group and the comparison between different NFT paradigms (i.e., increase/decrease of different EEG rhythms) or between different electrode sites (i.e., occipital versus central) is crucial as parents’ evaluations could be biased.
In addition to the above discussed promising NFT paradigms, research in our laboratory focus on training children on the spectrum to modulate their mu rhythm. Pineda et al. (2008, 2014) reported improvements in symptoms of autism evaluated by the parents as well as normal mu suppression after a mu-based NFT in contrast to a control group. Several studies from different laboratories have shown that mu rhythm phenomenology (alpha range: 8–13 Hz; beta range: 15–25 Hz) is linked to mirror neuron activity in that both are sensitive to movement, as well as to motor, affective, and cognitive imagery (Hari et al., 1997; Klimesch et al., 1997; Pfurtscheller et al., 1997, 2000; Muthukumaraswamy et al., 2004; Oberman et al., 2005; Pineda et al., 2008; Keuken et al., 2011). Furthermore, it was reported that mu rhythms, like mirror neurons, are modulated by object-directed actions (Muthukumaraswamy and Johnson, 2004; Muthukumaraswamy et al., 2004) and that during self-initiated, observed, and even imagined movement, mirror neuron asynchrony results in mu rhythm suppression (Pineda et al., 2000; Pineda, 2005; Neuper et al., 2009). Recently, it was demonstrated that mu rhythm suppression to movement observation is dependent on whether someone wants to be socially involved with another person and on the kind of movement observed (i.e., kinematic or goal-relevant; Aragón et al., 2013).
Both, functional magnetic resonance imaging (fMRI) and (EEG) techniques have demonstrated that mu rhythm suppression occurs in human MNS regions during tasks that activate this system, namely the inferior parietal lobe, dorsal premotor cortex, and primary somatosensory cortex (Arnstein et al., 2011). In individuals with autism, this mu rhythm suppression is not observed compared to typically developing children, supporting the role of an altered MNS (Oberman et al., 2005, 2008; Bernier et al., 2007; Oberman and Ramachandran, 2007). In contrast, Raymaekers et al. (2009) did not find a difference in mu suppression to self-executed or observed movement in autistic individuals in comparison to controls. Braadbaart et al. (2013), Arnstein et al. (2011) explained the reduced mu rhythm suppression in ASD as a more general deficit in visuomotor integration although they confirmed the relationship between mu rhythm suppression and the activation of mirror neuron areas described. In summary, although there is a lack of consensus, the majority of the literature provides enough evidence to speculate that training children to control mu rhythms may lead to functional improvements.
The Polyvagal Theory: A Rationale for Combining NFT and Biofeedback for ASD
Cortical deficiencies might not be the only cause of ASD symptoms. Individuals with ASD show deficits in emotional responsiveness (Scambler et al., 2007). This phenomenon cannot be solely explained by specific cortical deficiencies but likely involves peripheral physiological reactions of the autonomous nervous system (Thompson and Thompson, 2009; Thompson et al., 2010). The Polyvagal Theory proposed by Porges (2003, 2007) links cortical and peripheral physiological components in the social engagement system, which is responsible for facial expression, head turning, vocalization, listening, and other socially relevant behaviors that are atypical in individuals with ASD (Figure 1). According to this theory, autism is associated with autonomic states that foster the misinterpretation of a neutral environment as being threatening, and consequently can change normal vagal activity and result in withdrawal from social interaction. Thus, individuals with ASD show deficits in cardiac vagal tone regulation and impaired heart rate reactivity to external stimuli (i.e., heart rate variability, HRV), which are linked to the social engagement system (Porges, 2003). Consistent with this, Thayer and Lane (2000) suggested a model of neurovisceral integration, which proposes that HRV is an index of individual differences in regulated emotional responding (Appelhans and Luecken, 2006). Moreover, recent publications argue that heart rate and its variability play an important role in emotion recognition (Quintana et al., 2012) as well as for BCI control (Kaufmann et al., 2012; Pfurtscheller et al., 2013). This suggests that training children on the spectrum to increase their vagal tone via biofeedback (Lehrer, 2007; Gevirtz, 2010, 2007) should lead to additional improvements in the social engagement system, including emotional responsiveness.
In contrast to vagal tone, which is an indicator of parasympathetic activity (Task, 1996), skin conductance is a reliable index for sympathetic arousal of the autonomous nervous system (Bach et al., 2010). While different patterns of skin conductance in individuals with ASD have been shown (Schoen et al., 2008), it is not yet clear what kind of differences occur in skin conductance and heart rate between individuals with ASD and controls (Levine et al., 2012; Mathersul et al., 2013). Therefore, more research including peripheral physiological parameters in individuals with ASD is crucial to develop a more comprehensive model of the disorder and thus produce better treatment approaches.
Game Applications to Combine NFT and Biofeedback Introducing a Novel Game Platform for Children on the Spectrum
One treatment approach is to combine biofeedback of peripheral physiological reactions with neurofeedback of cortical electrophysiology and to do this in the context of play. Play is an ideal medium to engage children and help develop their motor skills, communication, problem solving and social skills (Oden and Asher, 1977; Hughes, 1998; MacDonald et al., 2013). There are many challenges in creating NFT and biofeedback games, not least the application must maintain player interest (Tan and Jansz, 2008) and secondly the limited genres available for ASD. The visualization of the feedback in NFT and biofeedback paradigms ranges from controlling a simple bar graph to more sophisticated visual renditions. However, the feedback typically is not related to the specific significance of the signals being trained or the anticipated behavioral changes. For example, the feedback might be the speed or response of a race car – indicating the level of control of the mu rhythm – while the anticipated outcome is that training the mu rhythm will lead to better imitation behavior. However, a specific feedback (i.e., showing the control of imitation behavior instead of a race car on the screen) for specific signals being trained (i.e., training the mu rhythm to improve imitation behavior) might be more effective in linking brain activation and anticipated behavior. Accordingly, training the EEG mu rhythm as well as training HRV should increase positive social behavior in children with ASD. Investigating this research question requires the development and implementation of a game platform that includes social interactions and specific feedback based on imitation behavior and emotional responsiveness. Therefore, we propose games such as the newly developed Social Mirroring Game (Figure 2), which requires children with ASD to modulate their brain activity (i.e., mu power) and/or peripheral physiological activation (e.g., increase in vagal tone) in gaming parts as well as in social situations between the child’s avatar and his friend (i.e., a non-player character, NPC) in order to get rewarded. The rewarding feedback involves the child’s avatar imitating the facial emotions of the NPC. The role-playing game mechanics allow the temporal dynamics of the player to be recorded to track behavior changes, accommodate game mechanic changes and to help direct the player.
For a game with the goal of improving social interactions, it is important to address the following questions: (1) is playing a social game without modulating physiological activity able to enhance appropriate social interactions? (2) is a single-person game rather antisocial than promoting social behavior? and (3) can the learned behavior be transferred from the gaming situations to the real-world?
First, it has been shown that role-play mechanism is a powerful tool towards assessing and intervening on social behaviors. Without actually manipulating brain or peripheral physiological activity, the Fearnot! social agent demonstrator (Aylett et al., 2006; Enz et al., 2008) was successful in proving that game-based platforms could have significant effect on a children population in domains related to social behavior (i.e., anti-bullying). Moreover, playing a cooperative computer game was shown to reinforce social interactions and appropriate social communicative behavior in children with ASD (Piper et al., 2006). MOSOCO (Escobedo et al., 2012) is a mobile augmented reality application based on the Social Compass curriculum (Tentori and Hayes, 2010) that facilitates practicing and learning social skills in children with ASD in social groups of neurotypical children. The results indicate that such assistive technologies with game-like interactions and role-play where points and rewards are earned improve the learning experience.
Second, the Social Motivation Adaptive Reality Treatment Games (SMART-Games; Gotsis et al., 2010) address the issue of single-player versus multiplayer games by using an avatar that exhibits different moods as an interface to a computer game which can be played as single-player, virtual or co-located multiplayer. The ECHOES project (Bernardini et al., 2014, 2012) dispels the myth that single-person games are inherently antisocial as it increases social interactions in the real world for some children with ASD.
Third, the ECHOES project also illustrates how role-play mechanics transferred to a virtual agent can be used to increase learned social skills from the game to the real world for children with ASD. The ECHOES game world, however, has no capability to adapt as its behavioral agent does not take psychophysiological inputs from a player. It is likely that brain and peripheral physiological activity is different in the video-game scenario compared to face-to-face interaction with a peer in real-life and the generalization has yet to be shown. However, like Fearnot!, ECHOES demonstrated the benefits of a game-based intervention towards social understanding, mechanisms and behavioral regulation in social situations.
In summary, games such as the Social Mirror Game are moving in the right direction and are promising tools to examine and improve the effects of training physiological measurements during social and emotional imitation behavior and interactions.
Conclusion
This review highlights the importance of using BCI, NFT, and biofeedback to provide novel insights about the physiological correlates of ASD, as well as the need to design innovative treatment approaches for such individuals. To date, the complex mechanisms underlying autism are not entirely understood. We propose that combining NFT and biofeedback may prove to be more effective than traditional approaches and describe a new game interface designed specifically for this purpose, i.e., to link appropriate behavior, neurophysiological and peripheral physiological reactions in social situations. As the rewarding feedback corresponds directly to the underlying significance of the signals we train as well as to the behavior we aim to reinforce and through the reinforcement of all facets of social interactions, substantial improvements in behavior, cognition and emotion can be expected for children with ASD.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This research was supported by a fellowship provided by the Max Kade Foundation to the Department of Cognitive Science, University of California San Diego, by grant funding from the ISNR Research Foundation, the European Community Seventh Framework Programme (FP7/2007 2013, nr. 258169) and the EPSRC/IMRC grant 113946. This paper only reflects the authors’ views and funding agencies are not liable for any use that may be made of the information contained herein.
References
American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders DSM-IV-TR, 4th Edn. Washington, DC: American Psychiatric Association. doi: 10.1007/SpringerReference-69770
Anderson, J. S., Nielsen, J. A., Froehlich, A. L., DuBray, M. B., Druzgal, T. J., Cariello, A. N., et al. (2011). Functional connectivity magnetic resonance imaging classification of autism. Brain J. Neurol. 134, 3742–54. doi: 10.1093/brain/awr263
Appelhans, B. M., and Luecken, L. J. (2006). Heart rate variability as an index of regulated emotional responding. Rev. Gen. Psychol. 10, 229–240. doi: 10.1037/1089-2680.10.3.229
Aragón, O. R., Sharer, E. A., Bargh, J. A., and Pineda, J. A. (2013). Modulations of mirroring activity by desire for social connection and relevance of movement. Soc. Cogn. Affect. Neurosci. doi: 10.1093/scan/nst172 [Epub ahead of print].
Arnstein, D., Cui, F., Keysers, C., Maurits, N. M., and Gazzola, V. (2011). μ-suppression during action observation and execution correlates with BOLD in dorsal premotor, inferior parietal, and SI cortices. J. Neurosci. 31, 14243–14249. doi: 10.1523/JNEUROSCI.0963-11.2011
Aylett, R., Louchart, S., Dias, J., Paiva, A., Vala, M., Woods, S., et al. (2006). Unscripted narrative for affectively driven characters. IEEE Comput. Graph. Appl. 26, 42–52. doi: 10.1109/MCG.2006.71
Bach, D. R., Friston, K. J., and Dolan, R. J. (2010). Analytic measures for quantification of arousal from spontaneous skin conductance fluctuations. Int. J. Psychophysiol. 76, 52–55. doi: 10.1016/j.ijpsycho.2010.01.011
Baron-Cohen, S. (2009). Autism: the empathizing-systemizing (E-S) theory. Ann. N. Y. Acad. Sci. 1156, 68–80. doi: 10.1111/j.1749-6632.2009.04467.x
Bernardini, S., Porayska-Pomsta, K., and Smith, T. J. (2014). ECHOES: an intelligent serious game for fostering social communication in children with autism. Inform. Sci. 264, 41–60. doi: 10.1016/j.ins.2013.10.027
Bernardini, S., Porayska-Pomsta, K., Smith, T. J., and Avramides, K. (2012). “Building autonomous social partners for autistic children,” in Intelligent Virtual Agents, eds Y. Nakano, M. Neff, A. Paiva, and M. Walker (Berlin-Heidelberg: Springer), 46–52. doi: 10.1007/978-3-642-33197-8_5
Bernier, R., Dawson, G., Webb, S., and Murias, M. (2007). EEG mu rhythm and imitation impairments in individuals with autism spectrum disorder. Brain Cogn. 64, 228–237. doi: 10.1016/j.bandc.2007.03.004
Braadbaart, L., Williams, J. H. G., and Waiter, G. D. (2013). Do mirror neuron areas mediate mu rhythm suppression during imitation and action observation? Int. J. Psychophysiol. 89, 99–105. doi: 10.1016/j.ijpsycho.2013.05.019
Brown, C., Gruber, T., Boucher, J., Rippon, G., and Brock, J. (2005). Gamma abnormalities during perception of illusory figures in autism. Cortex 41, 364–376. doi: 10.1016/S0010-9452(08)70273-9
Cantor, D. S., and Chabot, R. (2009). QEEG Studies in the assessment and treatment of childhood disorders. Clin. EEG Neurosci. 40, 113–121. doi: 10.1177/155005940904000209
Carpenter, M., Nagell, K., and Tomasello, M. (1998). Social cognition, joint attention, and communicative competence from 9 to 15 months of age. Monogr. Soc. Res. Child Dev. 4, 1–143.
Cherkassky, V. L., Kana, R. K., Keller, T. A., and Just, M. A. (2006). Functional connectivity in a baseline resting-state network in autism. Neuroreport 17, 1687–1690. doi: 10.1097/01.wnr.0000239956.45448.4c
Coben, R., and Myers, T. E. (2010). The relative efficacy of connectivity guided and symptom based EEG biofeedback for autistic disorders. Appl. Psychophysiol. Biofeedback 35, 13–23. doi: 10.1007/s10484-009-9102-5
Coben, R., and Padolsky, I. (2007). Assessment-guided neurofeedback for autistic spectrum disorder. J. Neurother. 11, 5–23. doi: 10.1300/J184v11n01_02
Coben, R., Clarke, A. R., Hudspeth, W., and Barry, R. J. (2008). EEG power and coherence in autistic spectrum disorder. Clin. Neurophysiol. 119, 1002–1009. doi: 10.1016/j.clinph.2008.01.013
Coben, R., Linden, M., and Myers, T. E. (2010). Neurofeedback for autistic spectrum disorder: a review of the literature. Appl. Psychophysiol. Biofeedback 35, 83–105. doi: 10.1007/s10484-009-9117-y
Dapretto, M., Davies, M. S., Pfeifer, J. H., Scott, A. A., Sigman, M., Bookheimer, S. Y., et al. (2006). Understanding emotions in others: mirror neuron dysfunction in children with autism spectrum disorders. Nat. Neurosci. 9, 28–30. doi: 10.1038/nn1611
di Pellegrino, G., Fadiga, L., and Fogassi, L. (1992). Understanding motor events: a neurophysiological study. Exp. Brain Res. 91, 176–180. doi: 10.1007/BF00230027
Duffy, F. H., and Als, H. (2012). A stable pattern of EEG spectral coherence distinguishes children with autism from neuro-typical controls - a large case control study. BMC Med. 10:64. doi: 10.1186/1741-7015-10-64
Enticott, P. G., Kennedy, H. A., Rinehart, N. J., Bradshaw, J. L., Tonge, B. J., Daskalakis, Z. J., et al. (2013). Interpersonal motor resonance in autism spectrum disorder: evidence against a global “mirror system” deficit. Front. Hum. Neurosci. 7:218. doi: 10.3389/fnhum.2013.00218
Enz, S., Zoll, C., Vannini, N., Schneider, W., Hall, L., Paiva, A., et al. (2008). e – motional learning in primary schools: fearnot! an anti- bullying intervention based on virtual role-play with intelligent synthetic characters. Electron. J. E Learn. 6, 111–118.
Escobedo, L., Nguyen, D. H., Boyd, L., Hirano, S. H., Rangel, A., García-rosas, D., et al. (2012). “MOSOCO: a mobile assistive tool to support children with autism practicing social skills in real-life situations,” in Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems, Austin, 2589–2598. doi: 10.1145/2207676.2208649
Friedrich, E. V. C., McFarland, D. J., Neuper, C., Vaughan, T. M., Brunner, P., and Wolpaw, J. R. (2009). A scanning protocol for a sensorimotor rhythm-based brain-computer interface. Biol. Psychol. 80, 169–75. doi: 10.1016/j.biopsycho.2008.08.004
Friedrich, E. V. C., Neuper, C., and Scherer, R. (2013). Whatever works: a systematic user-centered training protocol to optimize brain-computer interfacing individually. PLoS ONE 8:e76214. doi: 10.1371/journal.pone.0076214
Gevirtz, R. (2007). “Psychophysiological perspectives on stress-related and anxiety disorders,” in Principles and Practice of Stress Management, eds P. M. Lehrer, R. L. Woolfolk, and W. E. Sime (New York, NY: Guilford Press), 209–226.
Gevirtz, R. (2010). “Autonomic nervous system markers for psychophysiological, anxiety, and physical disorders,” in Integrative Neuroscience and Personalized Medicine, eds E. Gordon and S. H. Koslow (New York, NY: Oxford), 164–180.
Gotsis, M., Piggot, J., Hughes, D., and Stone, W. (2010). “SMART-games: a video game intervention for children with autism spectrum disorders,” Proceedings of the 9th International Conference on Interaction Design and Children, Barcelona, 194–197. doi: 10.1145/1810543.1810569
Hadjikhani, N., Joseph, R. M., Snyder, J., and Tager-Flusberg, H. (2006). Anatomical differences in the mirror neuron system and social cognition network in autism. Cereb. Cortex 16, 1276–1282. doi: 10.1093/cercor/bhj069
Hansen, R., Ozonoff, S., Krakowiak, P., Angkustsiri, K., Jones, C., Deprey, L., et al. (2008). Regression in autism: prevalence and associated factors in the CHARGE Study. Ambul. Pediatr. 8, 25–31. doi: 10.1016/j.ambp.2007.08.006
Hari, R., Salmelin, R., Mäkelä, J. P., Salenius, S., and Helle, M. (1997). Magnetoencephalographic cortical rhythms. Int. J. Psychophysiol. 26, 51–62. doi: 10.1016/S0167-8760(97)00755-1
Hickok, G. (2009). Eight problems for the mirror neuron theory of action understanding in monkeys and humans. J. Cogn. Neurosci. 21, 1229–1243. doi: 10.1162/jocn.2009.21189
Just, M. A., Cherkassky, V. L., Keller, T. A., and Minshew, N. J. (2004). Cortical activation and synchronization during sentence comprehension in high-functioning autism: evidence of underconnectivity. Brain J. Neurol. 127, 1811–1821. doi: 10.1093/brain/awh199
Just, M. A., and Varma, S. (2007). The organization of thinking: what functional brain imaging reveals about the neuroarchitecture of complex cognition. Cogn. Affect. Behav. Neurosci. 7, 153–191. doi: 10.3758/CABN.7.3.153
Kaufmann, T., Vögele, C., Sütterlin, S., Lukito, S., and Kübler, A. (2012). Effects of resting heart rate variability on performance in the P300 brain-computer interface. Int. J. Psychophysiol. 83, 336–341. doi: 10.1016/j.ijpsycho.2011.11.018
Keuken, M. C., Hardie, A., Dorn, B. T., Dev, S., Paulus, M. P., Jonas, K. J., et al. (2011). The role of the left inferior frontal gyrus in social perception: an rTMS study. Brain Res. 1383, 196–205. doi: 10.1016/j.brainres.2011.01.073
Klimesch, W., Doppelmayr, M., Pachinger, T., and Russegger, H. (1997). Event-related desynchronization in the alpha band and the processing of semantic information. Brain Res. Cogn. Brain Res. 6, 83–94. doi: 10.1016/S0926-6410(97)00018-9
Kouijzer, M. E. J., de Moor, J. M. H., Gerrits, B. J. L., Buitelaar, J. K., and van Schie, H. T. (2009a). Long-term effects of neurofeedback treatment in autism. Res. Autism Spectr. Disord. 3, 496–501. doi: 10.1016/j.rasd.2008.10.003
Kouijzer, M. E. J., de Moor, J. M. H., Gerrits, B. J. L., Congedo, M., and van Schie, H. T. (2009b). Neurofeedback improves executive functioning in children with autism spectrum disorders. Res. Autism Spectr. Disord. 3, 145–162. doi: 10.1016/j.rasd.2008.05.001
Lehrer, M. P. (2007). “Biofeedback training to increase heart rate variability,” in Principles and Practice of Stress Management, eds P. M. Lehrer, R. L. Woolfolk, and W. E. Sime (New York, NY: Guilford Press), 227–248.
Levine, T. P., Sheinkopf, S. J., Pescosolido, M., Rodino, A., Elia, G., and Lester, B. (2012). Physiologic arousal to social stress in children with autism spectrum disorders: a pilot study. Res. Autism Spectr. Disord. 6, 177–183. doi: 10.1016/j.rasd.2011.04.003
MacDonald, M., Lord, C., and Ulrich, D. A. (2013). The relationship of motor skills and social communicative skills in school-aged children with autism spectrum disorder. Adapt. Phys. Activ. Q. 30, 271–282.
Mathersul, D., McDonald, S., and Rushby, J. A. (2013). Automatic facial responses to affective stimuli in high-functioning adults with autism spectrum disorder. Physiol. Behav. 109, 14–22. doi: 10.1016/j.physbeh.2012.10.008
McFarland, D. J., Sarnacki, W. A., and Wolpaw, J. R. (2010). Electroencephalographic (EEG) control of three-dimensional movement. J. Neural Eng. 7, 1–9. doi: 10.1088/1741-2560/7/3/036007
Mostofsky, S. H., Dubey, P., Jerath, V. K., Jansiewicz, E. M., Goldberg, M. C., and Denckla, M. B. (2006). Developmental dyspraxia is not limited to imitation in children with autism spectrum disorders. J. Int. Neuropsychol. Soc. 12, 314–326. doi: 10.1017/S1355617706060437
Murias, M., Webb, S. J., Greenson, J., and Dawson, G. (2007). Resting state cortical connectivity reflected in EEG coherence in individuals with autism. Biol. Psychiatry 62, 270–273. doi: 10.1016/j.biopsych.2006.11.012
Muthukumaraswamy, S. D., and Johnson, B. W. (2004). Changes in rolandic mu rhythm during observation of a precision grip. Psychophysiology 41, 152–156. doi: 10.1046/j.1469-8986.2003.00129.x
Muthukumaraswamy, S. D., Johnson, B. W., and McNair, N. A. (2004). Mu rhythm modulation during observation of an object-directed grasp. Brain Res. Cogn. Brain Res. 19, 195–201. doi: 10.1016/j.cogbrainres.2003.12.001
Neuper, C., Scherer, R., Wriessnegger, S., and Pfurtscheller, G. (2009). Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain-computer interface. Clin. Neurophysiol. 120, 239–247. doi: 10.1016/j.clinph.2008.11.015
Nishitani, N., Avikainen, S., and Hari, R. (2004). Abnormal imitation-related cortical activation sequences in Asperger’s syndrome. Ann. Neurol. 55, 558–562. doi: 10.1002/ana.20031
Nowlis, D., and Kamiya, J. (1970). The control of electroencephalographic alpha rhythms through auditory feedback and the associated mental activity. Psychophysiology 6, 476–484. doi: 10.1111/j.1469-8986.1970.tb01756.x
Oberman, L. M., and Ramachandran, V. S. (2007). The simulating social mind: the role of the mirror neuron system and simulation in the social and communicative deficits of autism spectrum disorders. Psychol. Bull. 133, 310–327. doi: 10.1037/0033-2909.133.2.310
Oberman, L. M., Hubbard, E. M., McCleery, J. P., Altschuler, E. L., Ramachandran, V. S., and Pineda, J. A. (2005). EEG evidence for mirror neuron dysfunction in autism spectrum disorders. Brain Res. Cogn. Brain Res. 24, 190–198. doi: 10.1016/j.cogbrainres.2005.01.014
Oberman, L. M., Ramachandran, V. S., and Pineda, J. A. (2008). Modulation of mu suppression in children with autism spectrum disorders in response to familiar or unfamiliar stimuli: the mirror neuron hypothesis. Neuropsychologia 46, 1558–1565. doi: 10.1016/j.neuropsychologia.2008.01.010
Oden, S., and Asher, S. (1977). Coaching children in social skills for friendship making. Child Dev. 48, 495–506. doi: 10.2307/1128645
Pfurtscheller, G., Neuper, C., Andrew, C., and Edlinger, G. (1997). Foot and hand area mu rhythms. Int. J. Psychophysiol. 26, 121–135. doi: 10.1016/S0167-8760(97)00760-5
Pfurtscheller, G., Neuper, C., and Krausz, G. (2000). Functional dissociation of lower and upper frequency mu rhythms in relation to voluntary limb movement. Clin. Neurophysiol. 111, 1873–1879. doi: 10.1016/S1388-2457(00)00428-4
Pfurtscheller, G., Solis-Escalante, T., Barry, R. J., Klobassa, D. S., Neuper, C., and Müller-Putz, G. R. (2013). Brisk heart rate and EEG changes during execution and withholding of cue-paced foot motor imagery. Front. Hum. Neurosci. 7:379. doi: 10.3389/fnhum.2013.00379
Pineda, J. A. (2005). The functional significance of mu rhythms: translating “seeing” and “hearing” into “doing”. Brain Res. Brain Res. Rev. 50, 57–68. doi: 10.1016/j.brainresrev.2005.04.005
Pineda, J. A. (2008). Sensorimotor cortex as a critical component of an “extended” mirror neuron system: does it solve the development, correspondence, and control problems in mirroring? Behav. Brain Funct. 4:47. doi: 10.1186/1744-9081-4-47
Pineda, J. A., Allison, B. Z., and Vankov, A. (2000). The effects of self-movement, observation, and imagination on rhythms and readiness potentials. IEEE Trans. Rehabil. Eng. 8, 219–222. doi: 10.1109/86.847822
Pineda, J. A., Brang, D., Hecht, E., Edwards, L., Carey, S., Bacon, M., et al. (2008). Positive behavioral and electrophysiological changes following neurofeedback training in children with autism. Res. Autism Spectr. Disord. 2, 557–581. doi: 10.1016/j.rasd.2007.12.003
Pineda, J. A., Carrasco, K., Datko, M., Pillen, S., and Schalles, M. (2014). “Neurofeedback training produces normalization in behavioral and electrophysiological measures of high functioning autism,” in Mirror Neurons: Fundamental Discoveries, Theoretical Perspectives and Clinical Implications, eds P. F. Ferrari and G. Rizzolatti (London: Royal Society Publishing).
Pineda, J. A., Juavinett, A., and Datko, M. (2012). Self-regulation of brain oscillations as a treatment for aberrant brain connections in children with autism. Med. Hypotheses 79, 790–798. doi: 10.1016/j.mehy.2012.08.031
Piper, A. M., O’Brien, E., Morris, M. R., and Winograd, T. (2006). “SIDES: a cooperative tabletop computer game for social skills development,” in Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative work, Alberta, 1–10. doi: 10.1145/1180875.1180877
Porges, S. W. (2003). The Polyvagal Theory: phylogenetic contributions to social behavior. Physiol. Behav. 79, 503–513. doi: 10.1016/S0031-9384(03)00156-2
Porges, S. W. (2007). The polyvagal perspective. Biol. Psychol. 74, 116–143. doi: 10.1016/j.biopsycho.2006.06.009
Quintana, D. S., Guastella, A. J., Outhred, T., Hickie, I. B., and Kemp, A. H. (2012). Heart rate variability is associated with emotion recognition: direct evidence for a relationship between the autonomic nervous system and social cognition. Int. J. Psychophysiol. 86, 168–172. doi: 10.1016/j.ijpsycho.2012.08.012
Raymaekers, R., Wiersema, J. R., and Roeyers, H. (2009). EEG study of the mirror neuron system in children with high functioning autism. Brain Res. 1304, 113–121. doi: 10.1016/j.brainres.2009.09.068
Rice, C. E. (2011). The changing prevalence of the autism spectrum disorders. Am. Fam. Physician 83, 515–520.
Rizzolatti, G., and Craighero, L. (2004). The mirror-neuron system. Annu. Rev. Neurosci. 27, 169–192. doi: 10.1146/annurev.neuro.27.070203.144230
Rogers, S. J., and Pennington, B. F. (1991). A theoretical approach to the deficits in infantile autism. Dev. Psychopathol. 3, 137–162. doi: 10.1017/S0954579400000043
Scambler, D. J., Hepburn, S., Rutherford, M. D., Wehner, E. A., and Rogers, S. J. (2007). Emotional responsivity in children with autism, children with other developmental disabilities, and children with typical development. J. Autism. Dev. Disord. 37, 553–563. doi: 10.1007/s10803-006-0186-y
Schoen, S. A., Miller, L. J., Brett-Green, B., and Hepburn, S. L. (2008). Psychophysiology of children with autism spectrum disorder. Res. Autism Spectr. Disord. 2, 417–429. doi: 10.1016/j.rasd.2007.09.002
Sheikhani, A., Behnam, H., Mohammadi, M. R., Noroozian, M., and Mohammadi, M. (2012). Detection of abnormalities for diagnosing of children with autism disorders using of quantitative electroencephalography analysis. J. Med. Syst. 36, 957–963. doi: 10.1007/s10916-010-9560-6
Stieglitz Ham, H., Bartolo, A., Corley, M., Rajendran, G., Szabo, A., and Swanson, S. (2011). Exploring the relationship between gestural recognition and imitation: evidence of dyspraxia in autism spectrum disorders. J. Autism. Dev. Disord. 41, 1–12. doi: 10.1007/s10803-010-1011-1
Tan, E. S., and Jansz, J. (2008). “The game experience,” in Product Experience, eds H. N. J. Schifferstein and P. Hekkert (Oxford: Elsevier Ltd), 531–556. doi: 10.1016/B978-008045089-6.50026-5
Tentori, M., and Hayes, G. R. (2010). “Designing for interaction immediacy to enhance social skills of children with autism,” in Proceedings of the 12th ACM international conference on Ubiquitous computing - Ubicomp ’10, Copenhagen, 51. doi: 10.1145/1864349.1864359
Thayer, J. F., and Lane, R. D. (2000). A model of neurovisceral integration in emotion regulation and dysregulation. J. Affect. Disord. 61, 201–216. doi: 10.1016/S0165-0327(00)00338-4
Théoret, H., Halligan, E., Kobayashi, M., Fregni, F., Tager-Flusberg, H., and Pascual-Leone, A. (2005). Impaired motor facilitation during action observation in individuals with autism spectrum disorder. Curr. Biol. 15, R84–R85. doi: 10.1016/j.cub.2005.01.022
Thompson, L., Thompson, M., and Reid, A. (2010). Functional neuroanatomy and the rationale for using EEG biofeedback for clients with Asperger’s syndrome. Appl. Psychophysiol. Biofeedback 35, 39–61. doi: 10.1007/s10484-009-9095-0
Thompson, M., and Thompson, L. (2009). “Asperger’s syndrome intervention: combining neurofeedback, biofeedback and metacognition,” in Introduction to Quantitative EEG and Neurofeedback, 2nd Edn, eds T. H. Budzynski, H. K. Budzynski, J. R. Evans, and A. Abarbanel (Oxford: Elsevier Ltd), 365–415. doi: 10.1016/B978-0-12-374534-7.00015-0
Turella, L., Pierno, A. C., Tubaldi, F., and Castiello, U. (2009). Mirror neurons in humans: consisting or confounding evidence? Brain lang. 108, 10–21. doi: 10.1016/j.bandl.2007.11.002
Villalobos, M. E., Mizuno, A., Dahl, B. C., Kemmotsu, N., and Müller, R.-A. (2005). Reduced functional connectivity between V1 and inferior frontal cortex associated with visuomotor performance in autism. Neuroimage 25, 916–925. doi: 10.1016/j.neuroimage.2004.12.022
Welchew, D. E., Ashwin, C., Berkouk, K., Salvador, R., Suckling, J., Baron-Cohen, S., et al. (2005). Functional disconnectivity of the medial temporal lobe in Asperger’s syndrome. Biol. Psychiatry 57, 991–998. doi: 10.1016/j.biopsych.2005.01.028
Williams, J. H. G., Whiten, A., Suddendorf, T., and Perrett, D. I. (2001). Imitation, mirror neurons and autism. Neurosci. Biobehav. Rev. 25, 287–295. doi: 10.1016/S0149-7634(01)00014-8
Wilson, T. W., Rojas, D. C., Reite, M. L., Teale, P. D., and Rogers, S. J. (2007). Children and adolescents with autism exhibit reduced MEG steady-state gamma responses. Biol. Psychiatry 62, 192–197. doi: 10.1016/j.biopsych.2006.07.002
Keywords: autism spectrum disorder (ASD), brain–computer interface (BCI), neurofeedback and biofeedback training, games, mirror neuron system, mu rhythm, heart rate variability, social engagement system
Citation: Friedrich EVC, Suttie N, Sivanathan A, Lim T, Louchart S and Pineda JA (2014) Brain–computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum. Front. Neuroeng. 7:21. doi: 10.3389/fneng.2014.00021
Received: 03 February 2014; Accepted: 13 June 2014;
Published online: 03 July 2014.
Edited by:
Disha Gupta, Burke-Cornell Medical Research Institute, USAReviewed by:
Danny Eytan, Technion – Israel Institute of Technology, IsraelDean Krusienski, Old Dominion University, USA
Copyright © 2014 Friedrich, Suttie, Sivanathan, Lim, Louchart and Pineda. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Elisabeth V. C. Friedrich, Department of Cognitive Science, University of California, San Diego, 9500 Gilman Drive, La Jolla, CA 92093-0515, USA e-mail: efriedrich@ucsd.edu