Distinct visuo-motor brain dynamics for real-world objects versus planar images

Abstract

Ultimately, we aim to generalize and translate scientific knowledge to the real world, yet current understanding of human visual perception is based predominantly on studies of two-dimensional (2-D) images. Recent cognitive-behavioral evidence shows that real objects are processed differently to images, although the neural processes that underlie these differences are unknown. Because real objects (unlike images) afford actions, they may trigger stronger or more prolonged activation in neural populations for visuo-motor action planning. Here, we recorded electroencephalography (EEG) when human observers viewed real-world three-dimensional (3-D) objects or closely matched 2-D images of the same items. Although responses to real objects and images were similar overall, there were critical differences. Compared to images, viewing real objects triggered stronger and more sustained event-related desynchronization (ERD) in the μ frequency band (8–13 Hz) – a neural signature of automatic motor preparation. Event-related potentials (ERPs) revealed a transient, early occipital negativity for real objects (versus images), likely reflecting 3-D stereoscopic differences, and a late sustained parietal amplitude modulation consistent with an ‘old-new’ memory advantage for real objects over images. Together, these findings demonstrate that real-world objects trigger stronger and more sustained action-related brain responses than images do. The results highlight important similarities and differences between brain responses to images and richer, more ecologically relevant, real-world objects.

Introduction

Current knowledge of the cognitive and neural basis of human visual perception has been established predominantly by studies that have used stimuli in the form of planar images. Although this approach has yielded important insights into image vision, the human brain presumably has evolved to allow us to perceive and interact with real objects in naturalistic environments (Gibson, 1979). Despite the fundamental differences between real objects and images, the overarching assumption in cognitive neuroscience research has been that images are equivalent to their real-world counterparts. This basic assumption is rarely recognized or acknowledged. For example, many report studying real-world or graspable objects (Brady et al., 2008, 2016; Handy et al., 2003; Konkle et al., 2010; Konkle and Oliva, 2011, 2012; Lee et al., 2012; McNair et al., 2017; Nako et al., 2015; Khaligh-Razavi et al., 2018), yet representations of objects are neither real, nor do they offer genuine affordances.

Emerging evidence from cognitive psychology has begun to challenge the assumption of equivalence between real objects and images. Compared to 2-D images of objects, real-world objects elicit different gaze patterns in infants (Gerhard et al., 2016), facilitate object recognition (Chainay and Humphreys, 2001; Humphrey et al., 1994), enhance memory (Snow et al., 2014), increase attentional capture (Gomez et al., 2017), and bias valuation and decision-making (Romero et al., 2018). These unique effects of real objects on behavior are thought to be driven at the neural level by format-specific increases in the strength and/or duration of activation in visuo-motor networks involved in automatic planning of motor actions (Cisek and Kalaska, 2010; Gallivan et al., 2009; Gomez et al., 2017). However, no studies to date have tested this hypothesis. Although evidence from fMRI suggests that the format in which a stimulus is displayed influences neural responses across successive object presentations (Snow et al., 2011), this leaves open the critical question of whether, and how, real objects modulate cortical brain dynamics at the level of individual occurrences, independently of previous presentations (Cisek and Kalaska, 2010; Gallivan et al., 2009, 2011a; Gomez et al., 2017).

Unlike fMRI, in which blood oxygenation level dependent (BOLD) contrast detects vascular responses that lag the underlying neural events by seconds (Logothetis et al., 2001), electroencephalography (EEG) measures electrical changes at the surface of the scalp with millisecond-precision and can therefore provide fine-grained information about the time-course of cortical dynamics (e.g., Makeig et al., 2002). The EEG signal can be decomposed to reveal frequency-specific changes associated with cognitive processes (Basar et al., 1999; Klimesch, 1999). One such process is the transformation of visual object information into action representations, which is reflected by desynchronization of the μ (‘mu’) rhythm (8–13Hz) (Pineda, 2005). Desynchronization (including α, μ and β rhythms) is a reliable correlate of activated cortical networks (Pfurtscheller, 2001) and is directly related to fMRI BOLD response amplitude (Laufs et al., 2003). The rolandic μ rhythm originates in primary sensorimotor and premotor cortex and is recorded over central electrodes (Pfurtscheller et al., 1997). Typically, μ desynchronization occurs during both preparation and execution of self-initiated hand actions, as well as when hand actions are visually observed or imagined (Hari, 2006; Muthukumaraswamy and Johnson, 2004; Muthukumaraswamy et al., 2004; Pfurtscheller et al., 1997; Pineda, 2005). Observation of images of manipulable objects, such as tools, also elicits desynchronization of the μ rhythm over sensorimotor networks (Proverbio, 2012; Suzuki et al., 2014).

Here, we used EEG to contrast how cortical brain dynamics unfold when right-handed human observers view everyday real-world graspable objects versus 2-D images of the same items. Previous studies have shown that viewing images of graspable objects can automatically trigger motor preparation responses (Proverbio et al., 2011; Proverbio, 2012; Wamain et al., 2016). Given that real objects afford genuine motor actions, whereas images do not, we predicted that real objects would trigger stronger and more prolonged motor preparation signatures compared to 2-D images of the same items. We also predicted that the motor preparation signals for real objects would be stronger in the left hemisphere, contralateral to the dominant (right) hand. To pre-empt the results, we found that although there were similarities in overall neural responses to both stimulus formats, real objects elicited perceptual and neural responses that were distinct from those elicited by 2-D images. Specifically, real objects elicited a stronger and longer desynchronization in the μ frequency band, particularly over the left hemisphere. Real objects (versus 2-D images) also elicited differences in early and late event-related potential (ERP) amplitudes over occipital and parietal electrodes, corresponding to known signatures of stereoscopic disparity (Pegna et al., 2017) and memory (Donaldson and Rugg, 1999; Friedman and Johnson, 2000; Harris and Wilcox, 2009; Rugg and Curran, 2007; Rugg et al., 1998; Schendan and Kutas, 2003; Voss and Paller, 2008), respectively. Importantly, we show that the early difference in ERP amplitudes over occipital areas are dissociable from subsequent amplitude and frequency effects recorded over dorsal cortex. Together, our results confirm that real-world objects trigger neural signatures that are distinct from those of planar images.