Skip to main content
SpringerLink
Log in

Inclusion of third-person perspective in CAVE-like immersive 3D virtual reality role-playing games for social reciprocity training of children with an autism spectrum disorder

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

The present study aimed to improve the ability of children with autism to recognize emotions correctly. We used our third-person perspective role-playing game (TPP-RPG) method to teach social skills and help develop an improved understanding of the six basic emotions. The experiment was divided into two phases: The first involved working with traditional figure card emotional recognition and the second involved a subject entering a 3D cave automatic virtual environment (CAVE) to engage with interactive games. While the traditional graphic card is a static picture that represents one of the six basic human emotions, the virtual reality of CAVE-like immersive 3D role-playing games enables the use of actual picture scene syntheses plus the animation of 3D characters to express emotions. The participating children were instructed to role-play with (1) three-dimensional (3D) virtual role animations and observe (2) two different real-time switchable role-play animations of themselves and their counterpart socially interacting. This single-subject study was based on multiple-baseline, across-subject design and involved 5 weeks of TPP-RPG training intervention. From this research activity, we found that the role-play performance of all three participants rose substantially during the intervention phase and remained significantly higher in the maintenance phase compared to their baseline levels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Adjorlu, A., Hussain, A., Mødekjær, C., Austad, N.W.: Head-mounted display-based virtual reality social story as a tool to teach social skills to children diagnosed with autism spectrum disorder. In: 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning Through Virtual & Augmented Reality (KELVAR). IEEE (2018)

  2. Ahlers, K.P., Gabrielsen, T.P., Lewis, D., Brady, A.M., Litchford, A.: Supporting individuals with autism spectrum disorder in understanding and coping with complex social emotional issues. Sch. Psychol. Int. 38(6), 586–607 (2017)

    Google Scholar 

  3. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Pub, Philadelphia (2013)

    Google Scholar 

  4. Anderson, P.L., Rothbaum, B.O., Hodges, L.: Virtual reality: using the virtual world to improve quality of life in the real world. Bull. Menn. Clin. 65(1: Special issue), 78–91 (2001)

    Google Scholar 

  5. Axe, J.B., Evans, C.J.: Using video modeling to teach children with PDD-NOS to respond to facial expressions. Res. Autism Spect. Disord. 6(3), 1176–1185 (2012)

    Google Scholar 

  6. Ayres, K.M., Langone, J.: Intervention and instruction with video for students with autism: a review of the literature. Educ. Train. Dev. Disabil. 40, 183–196 (2005)

    Google Scholar 

  7. Baron-Cohen, S.E., Tager-Flusberg, H.E., Cohen, D.J.: Understanding Other Minds: Perspectives from Developmental Cognitive Neuroscience. Oxford University Press, Oxford (2000)

    Google Scholar 

  8. Baron-Cohen, S., Belmonte, M.K.: Autism: a window onto the development of the social and the analytic brain. Annu. Rev. Neurosci. 28, 109–126 (2005)

    Google Scholar 

  9. Baron-Cohen, S., Leslie, A.M., Frith, U.: Does the autistic child have a “theory of mind”. Cognition 21(1), 37–46 (1985)

    Google Scholar 

  10. Baker, J.: The Social Skills Picture Book: Teaching Communication, Play and Emotion. Future Horizons, Arlington (2017)

    Google Scholar 

  11. Becker-Weidman, A.: Effects of early maltreatment on development: a descriptive study using the Vineland Adaptive Behavior Scales-II. Child Welf. 88(2), 137–161 (2009)

    Google Scholar 

  12. Bellini, S., Akullian, J.: A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Except. Child. 73(3), 264–287 (2007)

    Google Scholar 

  13. Bierman, K.L., Greenberg, M.T., Coie, J.D., Dodge, K.A., Lochman, J.E., McMahon, R.J.: Social and Emotional Skills Training for Children: The Fast Track Friendship Group Manual. Guilford Publications, New York (2017)

    Google Scholar 

  14. Bölte, S., Hallmayer, J. (eds.): Autism Spectrum Conditions: FAQs on Autism, Asperger Syndrome, and Atypical Autism Answered by International Experts. Hogrefe Publishing, Göttingen (2011)

    Google Scholar 

  15. Cameirão, M.S., i Badia, S.B., Oller, E.D., Verschure, P.F.: Neurorehabilitation using the virtual reality based rehabilitation gaming system: methodology, design, psychometrics, usability and validation. J. Neuroeng. Rehabil. 7(1), 48 (2010)

    Google Scholar 

  16. Charlop-Christy, M.H., Daneshvar, S.: Using video modeling to teach perspective taking to children with autism. J. Posit. Behav. Interv. 5(1), 12–21 (2003)

    Google Scholar 

  17. Charlop, M.H., Lang, R., Rispoli, M.: Lights, camera, action! Teaching play and social skills to children with autism spectrum disorder through video modeling. In: Charlop, M.H., Lang, R., Rispoli, M. (eds.) Play and Social Skills for Children with Autism Spectrum disorder, pp. 71–94. Springer, Cham (2018)

    Google Scholar 

  18. Chen, C.H., Lee, I.J., Lin, L.Y.: Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Res. Dev. Disabil. 36, 396–403 (2015)

    Google Scholar 

  19. Chen, C.H., Lee, I.J., Lin, L.Y.: Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions. Comput. Hum. Behav. 55, 477–485 (2016)

    Google Scholar 

  20. Cohn, J.F., Ambadar, Z., Ekman, P.: Observer-based measurement of facial expression with the facial action coding system. In: Coan, J.A., Allen, J.J.B. (eds.) The Handbook of Emotion Elicitation and Assessment, vol. 1, no. 3, pp. 203–221. Oxford University Press, New York (2007)

    Google Scholar 

  21. Davies, R.R., Rogers, E.S.: Social skills training with persons who are mentally retarded. Ment. Retard. 23(4), 186 (1985)

    Google Scholar 

  22. Ekman, P.: Emotional and conversational nonverbal signals. In: Larrazabal, J.M., Miranda, L.A.P. (eds.) Language, Knowledge, and Representation, pp. 39–50. Springer, Dordrecht (2004)

    Google Scholar 

  23. Ghanouni, P., Jarus, T., Zwicker, J.G., Lucyshyn, J., Mow, K., Ledingham, A.: Social stories for children with autism spectrum disorder: validating the content of a virtual reality program. J. Autism Dev. Disord. 49(2), 660–668 (2019)

    Google Scholar 

  24. Goldstein, S., Ozonoff, S. (eds.): Assessment of Autism Spectrum Disorder. Guilford Publications, New York (2018)

    Google Scholar 

  25. Gray, C.: Social Stories. Future Horizons, Arlington, TX (1994)

    Google Scholar 

  26. Hillier, A., Fish, T., Cloppert, P., Beversdorf, D.Q.: Outcomes of a social and vocational skills support group for adolescents and young adults on the autism spectrum. Focus Autism Other Dev. Disabil. 22(2), 107–115 (2007)

    Google Scholar 

  27. Howlin, P., Yates, P.: The potential effectiveness of social skills groups for adults with autism. Autism 3(3), 299–307 (1999)

    Google Scholar 

  28. Ip, H.H., Wong, S.W., Chan, D.F., Byrne, J., Li, C., Yuan, V.S., Lau, K.S.Y., Wong, J.Y.: Enhance emotional and social adaptation skills for children with autism spectrum disorder: a virtual reality enabled approach. Comput. Educ. 117, 1–15 (2018)

    Google Scholar 

  29. Karal, M.A., Wolfe, P.S.: Social story effectiveness on social interaction for students with autism: a review of the literature. Educ. Train. Autism Dev. Disabil. 53(1), 44–58 (2018)

    Google Scholar 

  30. Lee, S.Y., Lo, Y.Y., Lo, Y.: Teaching functional play skills to a young child with autism spectrum disorder through video self-modeling. J. Autism Dev. Disord. 47(8), 2295–2306 (2017)

    Google Scholar 

  31. Lee, I.J., Chen, C.H., Wang, C.P., Chung, C.H.: Augmented reality plus concept map technique to teach children with ASD to use social cues when meeting and greeting. Asia Pac. Educ. Res. 27(3), 227–243 (2018)

    Google Scholar 

  32. Lee, I.J., Lin, L.Y., Chen, C.H., Chung, C.H.: How to create suitable augmented reality application to teach social skills for children with ASD. In: Mohamudally, N. (eds.) State of the Art Virtual Reality and Augmented Reality Knowhow, vol. 8, pp. 119–138. IntechOpen, London (2018)

    Google Scholar 

  33. Lillard, A.S.: Why do the children (pretend) play? Trends Cogn. Sci. 21(11), 826–834 (2017)

    MathSciNet  Google Scholar 

  34. Nikopoulos, C.K., Keenan, M.: Using video modeling to teach complex social sequences to children with autism. J. Autism Dev. Disord. 37(4), 678–693 (2007)

    Google Scholar 

  35. Parsons, S., Leonard, A., Mitchell, P.: Virtual environments for social skills training: comments from two adolescents with autistic spectrum disorder. Comput. Educ. 47(2), 186–206 (2006)

    Google Scholar 

  36. Pearson, A., Ropar, D., Hamilton, A.F.D.: A review of visual perspective taking in autism spectrum disorder. Front. Hum. Neurosci. 7, 652 (2013)

    Google Scholar 

  37. Reynhout, G., Carter, M.: Social Stories™ for children with disabilities. J. Autism Dev. Disord. 36(4), 445–469 (2006)

    Google Scholar 

  38. Richards, S.B.: Single Subject Research: Applications in Educational Settings. Cengage Learning, Boston (2018)

    Google Scholar 

  39. Riva, G.: Virtual reality in psychotherapy. Cyberpsychol. Behav. 8(3), 220–230 (2005)

    MathSciNet  Google Scholar 

  40. Sharples, S., Cobb, S., Moody, A., Wilson, J.R.: Virtual reality induced symptoms and effects (VRISE): comparison of head mounted display (HMD), desktop and projection display systems. Displays 29(2), 58–69 (2008)

    Google Scholar 

  41. Shillingsburg, M.A., Cariveau, T.: Responding to conversational partners’ behavior: teaching children with autism strategies to recapture the interest of others. Evid. Based Commun. Assess. Interv. 10(3–4), 143–147 (2016)

    Google Scholar 

  42. Smyth, J.M.: Beyond self-selection in video game play: an experimental examination of the consequences of massively multiplayer online role-playing game play. CyberPsychol. Behav. 10(5), 717–721 (2007)

    MathSciNet  Google Scholar 

  43. Tustin, F.: Autistic Barriers in Neurotic Patients. Routledge, London (2018)

    Google Scholar 

  44. Van Wermeskerken, M., Grimmius, B., van Gog, T.: Attention to the model’s face when learning from video modeling examples in adolescents with and without autism spectrum disorder. J. Comput. Assist. Learn. 34(1), 32–41 (2018)

    Google Scholar 

  45. Wallace, S., Parsons, S., Westbury, A., White, K., White, K., Bailey, A.: Sense of presence and atypical social judgments in immersive virtual environments: responses of adolescents with autism spectrum disorders. Autism 14(3), 199–213 (2010)

    Google Scholar 

  46. Yuan, S.N.V., Ip, H.H.S.: Using virtual reality to train emotional and social skills in children with autism spectrum disorder. Lond. J. Prim. Care 10(4), 110–112 (2018)

    Google Scholar 

Download references

Acknowledgements

We are grateful to the participants, therapists, and family members who participated in the study as well as the participants who assisted in the various phases of the study. We would also like to thank the individuals who participated in this research and the Autism and Developmental Research Center in Taiwan.

Funding

This work is supported by the Ministry of Science and Technology of Taiwan (MOST 107-2218-E-027 -013 -MY2). The authors thank the referees very much for their valuable comments and suggestions on this paper.

Author information

Authors and Affiliations

  1. Ergonomics and Interaction Design Lab, Department of Industrial Design, National Cheng Kung University, No. 1 University Rd., East District, Tainan, Taiwan

    Wei-Te Tsai & Chien-Hsu Chen

  2. Ergonomics Design and Technology Lab, Department of Industrial Design, National Taipei University of Technology, 1, Sec. 3, Zhongxiao E. Rd., Taipei, 10608, Taiwan

    I-Jui Lee

Authors
  1. Wei-Te Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I-Jui Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chien-Hsu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to I-Jui Lee.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1

Appendix 1

Parts of the social scripts used in the experiments

 

Scenes

Related character

Social scripts

Related character feelings

Respond to each other’s social actions

1

Park

Friends

Joe borrowed a toy car from Tom. Joe accidentally broke a wheel while playing with the car. Tom saw it and said: “Hum! You broke my toy car. I don’t want to be nice to you! “

Anger

Apologize (bow)

2

Park

Friends

When Tina was playing in the park, she met her new friend Allen. Allen said to Tina: “Hi, we will play together in the future, we will be good friends.”

Happiness

Hold hands

3

Living room

Mother

Peter was at home because his mother had gone shopping. She was gone a long time and he had to spend a long time waiting for her. “Why didn’t my mother come back yet? Why is she so late?”

Fear

Ask hugs

4

Living room

Mother

Today, my mother cooked John’s favorite food for dinner. John said at dinner: “Awesome, I am getting my favorite—half-boiled eggs tonight!”

Happiness

Ask hugs

5

Park

Friends

On the way home, when Lion was going through an alley, he saw his good friend Ali suddenly appears. “Wow! Why are you hiding here? You scared me!” he exclaimed.

Surprise

Express surprise (action)

6

Living room

Mother

When Lina came home from practicing riding a bicycle, her mother saw her palm was injured. Her mother asked what happened to Lina and she said: “I accidentally fell and it hurts!”

Sadness

Ask hugs

7

Class room

Friends

Gino and Archie are good friends and both like animations. At school, Gino said to Archie: “Did you watch that animation yesterday? Super funny!”

Happiness

High five

8

Class room

Friends

Bob borrowed a pencil with Joanna, but Bob lost the pencil. Joanna complained to the teacher: “Teacher, he lost my pencil!”

Disgust

Raise hand

9

Park

Friends

Danny was lost in a hide-and-seek game with friends and said: “Where am I? I have accidentally run into a strange place”

Fear

Ask for help

10

Class room

Classmate

Bill made an appointment with his classmates to play after class, but when he was about to go out he saw that it was raining: “Oh! Why is it raining ~ Now I can’t go out to play!”

Sadness

Talk to classmate (action)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsai, WT., Lee, IJ. & Chen, CH. Inclusion of third-person perspective in CAVE-like immersive 3D virtual reality role-playing games for social reciprocity training of children with an autism spectrum disorder. Univ Access Inf Soc 20, 375–389 (2021). https://doi.org/10.1007/s10209-020-00724-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-020-00724-9

Keywords

Search

Navigation