Erin Treacy Solovey
Audrey Girouard
Krysta Chauncey
Robert J.K. Jacob
ABSTRACT
Because functional near-infrared spectroscopy (fNIRS) eases many of the restrictions of other brain sensors, it has potential to open up new possibilities for HCI research. From our experience using fNIRS technology for HCI, we identify several considerations and provide guidelines for using fNIRS in realistic HCI laboratory settings. We empirically examine whether typical human behavior (e.g. head and facial movement) or computer interaction (e.g. keyboard and mouse usage) interfere with brain measurement using fNIRS. Based on the results of our study, we establish which physical behaviors inherent in computer usage interfere with accurate fNIRS sensing of cognitive state information, which can be corrected in data analysis, and which are acceptable. With these findings, we hope to facilitate further adoption of fNIRS brain sensing technology in HCI research.
- Bunce, S.C., Izzetoglu, M., Izzetoglu, K., Onaral, B. and Pourrezaei, K. Functional Near Infrared Spectroscopy: An Emerging Neuroimaging Modality. IEEE Engineering in Medicine and Biology Magazine, Special issue on Clinical Neuroengineering, 25 (4). 2006, 54--62.Google Scholar
- Chen, D., Hart, J. and Vertegaal, R. Towards a Physiological Model of User Interruptability. in INTERACT'07, 2008, 439--451. Google ScholarDigital Library
- Chenier, F. and Sawan, M., A New Brain Imaging Device Based on fNIRS. in BIOCAS 2007, (2007), 1--4.Google ScholarCross Ref
- Coyle, S., Ward, T. and Markham, C., Physiological Noise in Near-infrared Spectroscopy: Implications for Optical Brain Computer Interfacing. in Proc. EMBS, (2004).Google Scholar
- Devaraj, A., Izzetoglu, M., Izzetoglu, K. and Onaral, B. Motion Artifact Removal for fNIR Spectroscopy for Real World Application Areas. Proc. SPIE, 5588. 2004, 224--229.Google Scholar
- Ehlis, A.-C., Bähne, C.G., Jacob, C.P., Herrmann, M.J. and Fallgatter, A.J. Reduced lateral prefrontal activation in adult patients with attention-deficit/hyperactivity disorder (ADHD) during a working memory task: A functional near-infrared spectroscopy (fNIRS) study. Journal of Psychiatric Research, 42 (13). 2008, 1060--1067.Google ScholarCross Ref
- Fink, A., Benedek, M., Grabner, R.H., Staudt, B. and Neubauer, A.C. Creativity meets neuroscience: Experimental tasks for the neuroscientific study of creative thinking. Methods, 42 (1). 2007, 68--76.Google ScholarCross Ref
- Folley, B.S. and Park, S. Verbal creativity and schizotypal personality in relation to prefrontal hemispheric laterality: A behavioral and near-infrared optical imaging study. Schizophrenia Research, 80 (2--3). 2005, 271--282.Google ScholarCross Ref
- Girouard, A., Solovey, E.T., Hirshfield, L.M., Chauncey, K., Sassaroli, A., Fantini, S. and Jacob, R.J.K., Distinguishing Difficulty Levels with Non-invasive Brain Activity Measurements. in INTERACT 2009, (2009), Springer. Google ScholarDigital Library
- Grimes, D., Tan, D.S., Hudson, S.E., Shenoy, P. and Rao, R.P.N. Feasibility and pragmatics of classifying working memory load with an electroencephalograph Proc. CHI'08, ACM, 2008. Google ScholarDigital Library
- Herrmann, M.J., Huter, T., Plichta, M.M., Ehlis, A.-C., Alpers, G.W., Mühlberger, A. and Fallgatter, A.J. Enhancement of activity of the primary visual cortex during processing of emotional stimuli as measured with event-related functional near-infrared spectroscopy and event-related potentials. Human Brain Mapping, 29 (1). 2008, 28--35.Google ScholarCross Ref
- Hirshfield, L., Chauncey, K., Gulotta, R., Girouard, A., Solovey, E.T., Jacob, R., Sassaroli, A. and Fantini, S., Combining Electroencephalograph and Near Infrared Spectroscopy to Explore Users' Mental Workload. in HCI International 2009 (2009). Google ScholarDigital Library
- Hirshfield, L.M., Solovey, E.T., Girouard, A., Kebinger, J., Sassaroli, A., Tong, Y., Fantini, S. and Jacob, R.J.K., Brain Measurement for Usability Testing and Adaptive Interfaces: An Example of Uncovering Syntactic Workload with Functional Near Infrared Spectroscopy. in Proc. CHI'09, (2009). Google ScholarDigital Library
- Hoshi, Y. Near-Infrared Spectroscopy for Studying Higher Cognition. in Neural Correlates of Thinking, 2009, 83--93.Google Scholar
- Huppert, T. and Boas, D.A. (2005). Hemodynamic Evoked Response NIRS data analysis GUI Program User's Guide, 2009, from http://www.nmr.mgh.harvard.edu/PMI/resources/homer/HomER_UsersGuide_05-07-28.pdfGoogle Scholar
- Izzetoglu, K., Bunce, S., Onaral, B., Pourrezaei, K. and Chance, B. Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks. IJHCI, 17 (2). 2004, 211--231.Google Scholar
- Izzetoglu, M., Devaraj, A., Bunce, S. and Onaral, B. Motion Artifact Cancellation in NIR spectroscopy Using Wiener Filtering. IEEE Transactions on Biomedical Engineering, 52 (5). 2005, 934--938.Google ScholarCross Ref
- Lee, J.C. and Tan, D.S., Using a Low-Cost Electroencephalograph for Task Classification in HCI Research. in Proc. ACM Symposium on User Interface Software and Technology 2006, (2006). Google ScholarDigital Library
- Mappus, R.L., Venkatesh, G.R., Shastry, C., Israeli, A. and Jackson, M.M., An fNIR Based BMI for Letter Construction Using Continuous Control. in Proc. CHI 2009 Extended Abstracts, (2009). Google ScholarDigital Library
- Matthews, F., Pearlmutter, B.A., Ward, T.E., Soraghan, C. and Markham, C. Hemodynamics for Brain-Computer Interfaces. Signal Processing Magazine, IEEE, 25 (1). 2008, 87--94.Google ScholarCross Ref
- Millán, J.d.R., Renkens, F., Mouriño, J. and Gerstner, W. Brain-actuated interaction. Artif. Intell., 159 (1--2). 2004, 241--259. Google ScholarDigital Library
- Morioka, S., Yamada, M. and Komori, T. Frontal Lobe Activity during the Performance of Spatial Tasks: fNIRS Study. J Phys Ther Sci, 20. 2008, 135--139.Google Scholar
- Ramnani, N. and Owen, A.M. Anterior prefrontal cortex: insights into function from anatomy and neuroimaging. Nat Rev Neurosci, 5 (3). 2004, 184.Google ScholarCross Ref
- Sitaram, R., Zhang, H., Guan, C., Thulasidas, M., Hoshi, Y., Ishikawa, A., Shimizu, K. and Birbaumer, N. Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NEUROIMAGE, 34 (4). 2007, 1416--1427.Google ScholarCross Ref
- Villringer, A. and Chance, B. Non-Invasive Optical Spectroscopy and Imaging of Human Brain Function. Trends in Neuroscience, 20. 1997, 435--442.Google Scholar
Index Terms
- Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines
Recommendations
Examining the Reliability of Using fNIRS in Realistic HCI Settings for Spatial and Verbal Tasks
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsRecent efforts have shown that functional near-infrared spectroscopy (fNIRS) has potential value for brain sensing in HCI user studies. Research has shown that, although large head movement significantly affects fNIRS data, typical keyboard use, mouse ...
Measuring the effect of think aloud protocols on workload using fNIRS
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThe Think Aloud Protocol (TAP) is a verbalisation technique widely employed in HCI user studies to give insight into user experience, yet little work has explored the impact that TAPs have on participants during user studies. This paper utilises a brain ...
Using fNIRS brain sensing to evaluate information visualization interfaces
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsWe show how brain sensing can lend insight to the evaluation of visual interfaces and establish a role for fNIRS in visualization. Research suggests that the evaluation of visual design benefits by going beyond performance measures or questionnaires to ...
Comments