Skip to main content
SpringerLink
Log in

Augmented Reality Plus Concept Map Technique to Teach Children with ASD to Use Social Cues When Meeting and Greeting

  • Regular Article
  • Published:
The Asia-Pacific Education Researcher Aims and scope Submit manuscript

Abstract

Autism spectrum disorders (ASD) are characterized by a reduced ability to appropriately express social greetings. Studies have indicated that individuals with ASD might not recognize the crucial nonverbal cues that usually aid social interaction. Social reciprocity depends on the ability to empathize with others, to be aware of emotional and interpersonal cues, and to respond appropriately; it requires joint attention and nonverbal social skills. Fortunately, there is evidence-based research which shows that augmented reality (AR) attracts the attention of children with ASD and allows them to focus on social cues. AR has also been proved effective for teaching social skills. However, there is a lack of appropriate instructional scaffolds in AR applications to help students organize learning materials. Therefore, in this study, we use AR combined with concept map (CM) strategy as a training tool to focus on the standard nonverbal social cues to teach children with ASD how to appropriately reciprocate when they greet others. The learner can integrate the AR with CM strategy to visually conceptualize the social scenarios in a tabletop role-play training platform. Single-subject research with a multiple-baselines across-subject design was used in this study. Our results showed substantial increases in the children’s target responses during the intervention phases compared with the baseline phases. Generalization probes were administered during baseline (4–8 sessions for 0.5–1 month), intervention (10 sessions for 1.2 months), and maintenance phases (4–8 sessions for 0.5–1 month) to assess the generalization and maintenance of learned skills. The three-phase test data suggest that the AR with CM intervention was moderately effective in teaching the target greeting responses to children with ASD. The practical and developmental implications of the findings are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5 ® ). Washington, DC: American Psychiatric Pub.

    Book  Google Scholar 

  • Attwood, T. (2006). The complete guide to Asperger’s syndrome. Philadelphia: Jessica Kingsley Publishers.

    Google Scholar 

  • Axe, J. B., & Evans, C. J. (2012). Using video modeling to teach children with PDD-NOS to respond to facial expressions. Research in Autism Spectrum Disorders, 6(3), 1176–1185.

    Article  Google Scholar 

  • Ayres, K. M., & Langone, J. (2005). Intervention and instruction with video for students with autism: A review of the literature. Education and Training in Developmental Disabilities, 40(2), 183–196.

    Google Scholar 

  • Bai, Z., Blackwell, A. F., & Coulouris, G. (2015a). Using augmented reality to elicit pretend play for children with autism. IEEE Transactions on Visualization and Computer Graphics, 21(5), 598–610.

    Article  Google Scholar 

  • Bai, Z., Blackwell, A. F., & Coulouris, G. (2015b, April). Exploring expressive augmented reality: The FingAR puppet system for social pretend play. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1035–1044). ACM.

  • Baker, J. (2001). Social skills picture book: Teaching communication, play and emotion. Arlington, TX: Future Horizons.

    Google Scholar 

  • Bellini, S., & Akullian, J. (2007). A meta-analysis of video modeling and video self-modeling interventions for children and adolescents with autism spectrum disorders. Exceptional children, 73(3), 264–287.

    Article  Google Scholar 

  • Billinghurst, M. (2002). Augmented reality in education. Seattle WA: New Horizons for Learning.

    Google Scholar 

  • Boelte, S., & Hallmayer, J. (2013). Autism spectrum conditions: FAQs on autism, Asperger syndrome, and atypical autism answered by international experts. Ashland, OH: Hogrefe Publishing.

    Google Scholar 

  • Broadhead, P. (2006). Developing an understanding of young children’s learning through play: The place of observation, interaction and reflection. British Educational Research Journal, 32(2), 191–207.

    Article  Google Scholar 

  • Castelli, F. (2005). Understanding emotions from standardized facial expressions in autism and normal development. Autism, 9(4), 428–449.

    Article  Google Scholar 

  • Charlop-Christy, M. H., & Daneshvar, S. (2003). Using video modeling to teach perspective taking to children with autism. Journal of Positive Behavior Interventions, 5(1), 12–21.

    Article  Google Scholar 

  • Charlop-Christy, M. H., Le, L., & Freeman, K. A. (2000). A comparison of video modeling with in vivo modeling for teaching children with autism. Journal of Autism and Developmental Disorders, 30(6), 537–552.

    Article  Google Scholar 

  • Charsky, D., & Ressler, W. (2011). “Games are made for fun”: Lessons on the effects of concept maps in the classroom use of computer games. Computers & Education, 56(3), 604–615.

    Article  Google Scholar 

  • Chen, C. H., Chou, Y. Y., & Huang, C. Y. (2016). An augmented-reality-based concept map to support mobile learning for science. The Asia-Pacific Education Researcher, 25(4), 567–578.

    Article  Google Scholar 

  • Chen, C. H., Lee, I. J., & Lin, L. Y. (2015). Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Research in Developmental Disabilities, 36, 396–403.

    Article  Google Scholar 

  • Chen, C. H., Wu, F. G., & Zhung, S. K. (2006). Design an augmented reality teaching system with concept mapping technique. In Proceedings of the Annual Conference of JSSD the 53rd Annual Conference of JSSD (pp. 250–251). Japanese Society for the Science of Design.

  • Cheng, K. H., & Tsai, C. C. (2013). Affordances of augmented reality in science learning: Suggestions for future research. Journal of Science Education and Technology, 22(4), 449–462.

    Article  Google Scholar 

  • Di Serio, Á., Ibáñez, M. B., & Kloos, C. D. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Computers & Education, 68, 586–596.

    Article  Google Scholar 

  • Dube, W. V., MacDonald, R. P., Mansfield, R. C., Holcomb, W. L., & Ahearn, W. H. (2004). Toward a behavioral analysis of joint attention. The Behavior Analyst, 27(2), 197–207.

    Article  Google Scholar 

  • Elsabbagh, M., Volein, A., Holmboe, K., Tucker, L., Csibra, G., Baron-Cohen, S., et al. (2009). Visual orienting in the early broader autism phenotype: Disengagement and facilitation. Journal of Child Psychology and Psychiatry, 50(5), 637–642.

    Article  Google Scholar 

  • Escobedo, L., Tentori, M., Quintana, E., Favela, J., & Garcia-Rosas, D. (2014). Using augmented reality to help children with autism stay focused. IEEE Pervasive Computing, 13(1), 38–46.

    Article  Google Scholar 

  • Facoetti, A., Ruffino, M., Peru, A., Paganoni, P., & Chelazzi, L. (2008). Sluggish engagement and disengagement of non-spatial attention in dyslexic children. Cortex, 44(9), 1221–1233.

    Article  Google Scholar 

  • Farr, W., Yuill, N., & Hinske, S. (2012). An augmented toy and social interaction in children with autism. International Journal of Arts and Technology, 5(2–4), 104–125.

    Article  Google Scholar 

  • Farr, W., Yuill, N., & Raffle, H. (2009, April). Collaborative benefits of a tangible interface for autistic children. In Proceedings of the CHI.

  • Fodstad, J. C., Matson, J. L., Hess, J., & Neal, D. (2009). Social and communication behaviours in infants and toddlers with autism and pervasive developmental disorder-not otherwise specified. Developmental Neurorehabilitation, 12(3), 152–157.

    Article  Google Scholar 

  • Gardner, S., & Wolfe, P. (2013). Use of video modeling and video prompting interventions for teaching daily living skills to individuals with autism spectrum disorders: A review. Research and Practice for Persons with Severe Disabilities, 38(2), 73–87.

    Article  Google Scholar 

  • Graham, M., Zook, M., & Boulton, A. (2013). Augmented reality in urban places: Contested content and the duplicity of code. Transactions of the Institute of British Geographers, 38(3), 464–479.

    Article  Google Scholar 

  • Grandin, T. (2008). Thinking in pictures, expanded edition: My life with autism. New York: Vintage.

    Google Scholar 

  • Gray, C. A. (1998). Social stories and comic strip conversations with students with Asperger syndrome and high-functioning autism. In E. Schopler, G. Mesibov, & L. Kunce (Eds.), Asperger syndrome or high-functioning autism? (pp. 167–198). New York: Plenum Press.

    Chapter  Google Scholar 

  • Heath, M. A., Sheen, D., Leavy, D., Young, E., & Money, K. (2005). Bibliotherapy: A resource to facilitate emotional healing and growth. School Psychology International, 26(5), 563–580.

    Article  Google Scholar 

  • Hedley, N. R. (2003, August). Empirical evidence for advanced geographic visualization interface use. In Proceedings of the International Cartographic Congress, Durban, South Africa.

  • Heinze-Fry, J. A., & Novak, J. D. (1990). Concept mapping brings long-term movement toward meaningful learning. Science Education, 74(4), 461–472.

    Article  Google Scholar 

  • Hobson, R. P., & Lee, A. (1998). Hello and goodbye: A study of social engagement in autism. Journal of Autism and Developmental Disorders, 28(2), 117–127.

    Article  Google Scholar 

  • Hornecker, E., & Buur, J. (2006, April). Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 437–446). ACM.

  • Hoyek, N., Di Rienzo, F., Collet, C., Creveaux, T., & Guillot, A. (2014). Hand mental rotation is not systematically altered by actual body position: Laterality judgment versus same–different comparison tasks. Attention, Perception, & Psychophysics, 76(2), 519–526.

    Article  Google Scholar 

  • Hwang, G. J., Sung, H. Y., Hung, C. M., Huang, I., & Tsai, C. C. (2012). Development of a personalized educational computer game based on students’ learning styles. Educational Technology Research and Development, 60(4), 623–638.

    Article  Google Scholar 

  • Ibanez, L. V., Messinger, D. S., Newell, L., Lambert, B., & Sheskin, M. (2008). Visual disengagement in the infant siblings of children with an autism spectrum disorder (ASD). Autism, 12(5), 473–485.

    Article  Google Scholar 

  • Jarrold, C. (2003). A review of research into pretend play in autism. Autism, 7(4), 379–390.

    Article  Google Scholar 

  • Kaufmann, H., & Schmalstieg, D. (2003). Mathematics and geometry education with collaborative augmented reality. Computers & Graphics, 27(3), 339–345.

    Article  Google Scholar 

  • Kikuchi, Y., Senju, A., Akechi, H., Tojo, Y., Osanai, H., & Hasegawa, T. (2011). Atypical disengagement from faces and its modulation by the control of eye fixation in children with autism spectrum disorder. Journal of Autism and Developmental Disorders, 41(5), 629–645.

    Article  Google Scholar 

  • Kinchin, I. M. (2012). Concept mapping and the fundamental problem of moving between knowledge structures. Journal for Educators, Teachers and Trainers, 4(1), 96–106.

    Google Scholar 

  • Klatzky, R. L., Wu, B., Shelton, D., & Stetten, G. (2008). Effectiveness of augmented-reality visualization versus cognitive mediation for learning actions in near space. ACM Transactions on Applied Perception (TAP). https://doi.org/10.1145/1279640.1279641.

    Google Scholar 

  • Landry, R., & Bryson, S. E. (2004). Impaired disengagement of attention in young children with autism. Journal of Child Psychology and Psychiatry, 45(6), 1115–1122.

    Article  Google Scholar 

  • LeGoff, D. B. (2004). Use of LEGO© as a therapeutic medium for improving social competence. Journal of Autism and Developmental Disorders, 34(5), 557–571.

    Article  Google Scholar 

  • Lillard, A. S., Lerner, M. D., Hopkins, E. J., Dore, R. A., Smith, E. D., & Palmquist, C. M. (2013). The impact of pretend play on children’s development: A review of the evidence. Psychological Bulletin, 139(1), 1.

    Article  Google Scholar 

  • Lo, C.-Y. (2016). Incorporating concept map to design augmented reality learning system in inquiry teaching for natural science courses. National Taichung University of Science and Technology.

  • Marshall, P. (2007, February). Do tangible interfaces enhance learning? In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (pp. 163–170). ACM.

  • Matson, J. L. (2009). Applied behavior analysis for children with autism spectrum disorders. New York: Springer.

    Book  Google Scholar 

  • Nikopoulos, C. K., & Keenan, M. (2007). Using video modeling to teach complex social sequences to children with autism. Journal of Autism and Developmental Disorders, 37(4), 678–693.

    Article  Google Scholar 

  • Novak, J. D., & Gowin, D. B. (1984). Learning how to learn. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Reynhout, G., & Carter, M. (2006). Social Stories™ for children with disabilities. Journal of Autism and Developmental Disorders, 36(4), 445–469.

    Article  Google Scholar 

  • Roberts, V., & Joiner, R. (2007). Investigating the efficacy of concept mapping with pupils with autistic spectrum disorder. British Journal of Special Education, 34(3), 127–135.

    Article  Google Scholar 

  • Schwendimann, B. A. (2015). Concept maps as versatile tools to integrate complex ideas: From kindergarten to higher and professional education. Knowledge Management & E-Learning: An International Journal (KM&EL), 7(1), 73–99.

    Google Scholar 

  • Shelton, B. E. (2003). How augmented reality helps students learn dynamic spatial relationships. Doctoral dissertation. University of Washington.

  • Shelton, B. E., & Hedley, N. R. (2002). Using augmented reality for teaching earth-sun relationships to undergraduate geography students. In Augmented Reality Toolkit, The First IEEE International Workshop (8 pp.). IEEE.

  • Shelton, B. E., & Hedley, N. R. (2004). Exploring a cognitive basis for learning spatial relationships with augmented reality. Technology, Instruction, Cognition and Learning, 1(4), 323.

    Google Scholar 

  • Sherer, M., Pierce, K. L., Paredes, S., Kisacky, K. L., Ingersoll, B., & Schreibman, L. (2001). Enhancing conversation skills in children with autism via video technology: Which is better, “self” or “other” as a model? Behavior Modification, 25(1), 140–158.

    Article  Google Scholar 

  • Shipley-Benamou, R., Lutzker, J. R., & Taubman, M. (2002). Teaching daily living skills to children with autism through instructional video modeling. Journal of Positive Behavior Interventions, 4(3), 166–177.

    Article  Google Scholar 

  • Sparrow, S. S. (2011). Vineland adaptive behavior scales. In J. S. Kreutzer, J. DeLuca, & B. Caplan (Eds.), Encyclopedia of clinical neuropsychology (pp. 2618–2621). New York: Springer.

    Chapter  Google Scholar 

  • Sucksmith, E., Allison, C., Baron-Cohen, S., Chakrabarti, B., & Hoekstra, R. A. (2013). Empathy and emotion recognition in people with autism, first-degree relatives, and controls. Neuropsychologia, 51(1), 98–105.

    Article  Google Scholar 

  • Sun, J. C. Y., & Chen, A. Y. Z. (2016). Effects of integrating dynamic concept maps with interactive response system on elementary school students’ motivation and learning outcome: The case of anti-phishing education. Computers & Education, 102, 117–127.

    Article  Google Scholar 

  • Tajeddin, Z., & Tabatabaei, S. (2016). Concept mapping as a reading strategy: Does it scaffold comprehension and recall. The Reading Matrix: An International Online Journal, 16(1), 194–208.

    Google Scholar 

  • Taylor, B. A., Hoch, H., Potter, B., Rodriguez, A., Spinnato, D., & Kalaigian, M. (2005). Manipulating establishing operations to promote initiations toward peers in children with autism. Research in Developmental Disabilities, 26(4), 385–392.

    Article  Google Scholar 

  • van der Geest, J. N., Kemner, C., Camfferman, G., Verbaten, M. N., & van Engeland, H. (2001). Eye movements, visual attention, and autism: A saccadic reaction time study using the gap and overlap paradigm. Biological Psychiatry, 50(8), 614–619.

    Article  Google Scholar 

  • Weiss, M. J., & Harris, S. L. (2001). Reaching out, joining. Teaching social skills to young children with autism. Bethesda, MD: Woodbine House.

    Google Scholar 

  • Wexler, M., Kosslyn, S. M., & Berthoz, A. (1998). Motor processes in mental rotation. Cognition, 68(1), 77–94.

    Article  Google Scholar 

  • Wing, L., & Gould, J. (1979). Severe impairments of social interaction and associated abnormalities in children: Epidemiology and classification. Journal of Autism and Developmental Disorders, 9(1), 11–29.

    Article  Google Scholar 

  • Wraga, M., Thompson, W. L., Alpert, N. M., & Kosslyn, S. M. (2003). Implicit transfer of motor strategies in mental rotation. Brain and Cognition, 52(2), 135–143.

    Article  Google Scholar 

  • Wu, H. K., Lee, S. W. Y., Chang, H. Y., & Liang, J. C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49.

    Article  Google Scholar 

  • Yuill, N., Strieth, S., Roake, C., Aspden, R., & Todd, B. (2007). Brief report: Designing a playground for children with autistic spectrum disorders—Effects on playful peer interactions. Journal of Autism and Developmental Disorders, 37(6), 1192–1196.

    Article  Google Scholar 

  • Zaman, B., Abeele, V. V., Markopoulos, P., & Marshall, P. (2012). The evolving field of tangible interaction for children: The challenge of empirical validation. Personal and Ubiquitous Computing, 16(4), 367–378.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ergonomics and Interaction Design Lab, Department of Industrial Design, National Taipei University of Technology, 1, Sec. 3, Zhongxiao E. Rd., Taipei, 10608, Taiwan, ROC

    I-Jui Lee

  2. Ergonomics and Interaction Design Lab, Department of Industrial Design, National Cheng Kung University, Tainan, Taiwan, ROC

    Chien-Hsu Chen & Chi-Hsuan Chung

  3. Hierarchical Green-Energy Materials (Hi-GEM) Research Center, National Cheng Kung University, Tainan, Taiwan

    Chien-Hsu Chen

  4. Department of Innovative Living Design, Overseas Chinese University, Taichung, Taiwan

    Chuan-Po Wang

Authors
  1. I-Jui Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chien-Hsu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chuan-Po Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chi-Hsuan Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to I-Jui Lee.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 29 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, IJ., Chen, CH., Wang, CP. et al. Augmented Reality Plus Concept Map Technique to Teach Children with ASD to Use Social Cues When Meeting and Greeting. Asia-Pacific Edu Res 27, 227–243 (2018). https://doi.org/10.1007/s40299-018-0382-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40299-018-0382-5

Keywords

Search

Navigation