Abstract
Brain event-related potentials are a useful tool for investigating visual processing and action planning. This technique requires extremely accurate synchronization of stimulus delivery with recordings. The precision of the onset time of visual stimulus delivery is a major challenge when attempting to use real, three-dimensional objects as stimuli. Here, we present an innovative device, the “box for interaction with objects” (BIO), that is designed to synchronize the presentation of objects with electroencephalographic (EEG) recordings. To reach the required resolution of stimulus-onset timing, the BIO system features an interface with reflective glass and light-emitting diodes (LEDs). When the LEDs inside the BIO are turned on, the object inside becomes visible, and a synchronizing pulse is sent to the recording systems. The BIO was tested in a motivational study that focused on visual and motor event-related potentials. EEG signals were recorded during the presentation of an emotion-laden object that could be grasped and brought close to the participant’s chest. BIO successfully synchronized the appearance of a three-dimensional object with EEG recordings, which would allow for an analysis of visual and motor event-related potentials in the same experiment. The BIO device, through a high-quality psychophysiological approach, offers a new perspective for the study of the motivational factors that drive actions toward relevant stimuli.
Similar content being viewed by others
The neural substrates of the motivational factors involved in action planning and execution are a current topic of interest. Motivation is the driving force by which we guide our behavior, and it is closely related to emotion. Although the motivation to act is often considered a core function of emotion, few studies have investigated the relationship between emotion and action planning and execution. Emotion-laden pictures have been employed to modulate somatic and behavioral reactions in humans (Bradley, Codispoti, Cuthbert, & Lang, 2001; Coombes, Cauraugh, & Janelle, 2006, 2007; Souza, Pereira, Vila, Oliveira, & Volchan, 2012; Facchinetti, Imbiriba, Azevedo, Vargas, & Volchan, 2006; Pereira et al., 2006). However, the aesthetic distance inherent to a pictorial stimulus creates a barrier that prevents interaction between the stimulus and the observer. The use of veritable stimuli, such as real, three-dimensional objects, has the benefit of allowing the observer access to the intrinsic and extrinsic motivational aspects that guide advantageous interactions with the stimuli.
From an integrative perspective, motivation to interact with three-dimensional objects stands at the intersection of emotion and action processing. At present, methodological approaches to explore the neural substrates of this intersection are still lacking. Event-related potentials have been used in the attempt to capture brain processes during the execution of behavioral tasks. These potentials can be reliably measured using electroencephalography (EEG). However, an event of interest is not observable in a single trial; the experimenter must conduct many trials and average the signal segments, causing random brain activity to be averaged out and the relevant event-related potentials to remain. Most importantly, markers time-locked to the event of interest must define the segments that will be averaged (Fabiani, Gratton, & Federmeier, 2007; Luck, 2005).
The components of event-related potentials are studied by being time-locked to stimulus and/or movement onset. A recorded event-related potential time-locked with voluntary movement is called a readiness potential (Bereitschaftspotential). This motor event-related potential presents itself as a slow negative deflection beginning up to 1 s before movement onset (Jahanshahi & Hallett, 2003). As the analysis is time-locked to movement onset, this component is a useful tool for investigating motor predispositions related to interaction with an object (van Schie & Bekkering, 2007). A growing body of work utilizing event-related potentials has begun to shed light on the neural dynamics involved in processing visual emotional stimuli. Such stimuli seem to reflexively engage both corticolimbic appetitive and defensive systems to facilitate perception and, ultimately, to promote adaptive behavior. In this line of research, event-related potentials time-locked to stimulus onset have been demonstrated to be sensitive to visual emotional stimuli, including early (N1, P1, P2, and N2) and late (P300 and LPP) components of the potential (Olofsson, Nordin, Sequeira, & Polich, 2008). Because event-related potential waveforms are derived from averaged segments that are time-locked to stimulus onset, the precise timing of stimulus presentation and trigger delivery is a common concern in visual event-related potential paradigms. The time control of stimulus presentation is largely facilitated when digital images are used. Difficulties in precision time control emerge, however, when using three-dimensional objects as stimuli, and therefore the use of such stimuli has been limited.
The development of a device that allows for the simultaneous analysis of visual and motor event-related potentials, in a biologically relevant context, has the potential to shed light on the time course of motivational processing that drives overt interaction with three-dimensional objects. We present a method for extracting, with high temporal resolution, the brain processing related to interactions with three-dimensional affective stimuli: the “box for interaction with objects” (BIO). The BIO is an novel tool for presenting three-dimensional objects and triggering the onset of their presentation with high temporal resolution, making it possible to analyze event-related potentials time-locked to both stimulus and movement onsets.
Method
The BIO is a prism-shaped hollow box. The base, the back, and the trapezoidal sides are made of wood. The inside of the base is lined with soft rubber, and there is a gap at the bottom of the front side of the box. A large rectangular sheet of glass coated with a reflective film (5-mm thickness) is inclined at 45º over the front of the box, leaving the gap at the bottom open. A wooden beam rests underneath the glass. A centralized aperture is also present at the bottom of the box’s back side. The gap and the aperture are each covered with a black curtain (see Supplemental Fig. 1, in the supplemental materials, for an illustration).
Six white light-emitting diodes (LEDs) are fixed on the wooden beam. The LEDs are electrically connected to a control box that contains resistors to limit the current of the LEDs, a switch to turn the LEDs on, and an optocoupler to synchronize the onset of the LEDs with the recording systems. Two additional white LEDs are fixed on the outside of the back of the BIO. Switch Position 1 turns on the LEDs on the inside of the BIO and turns off the LEDs on the back. Switch Position 2, on the other hand, turns off the LEDs on the inside of the box and turns on the LEDs on the back. The switch position is coupled as a trigger for the recording systems. A 40-V/500-mA source powers the system. (See Supplemental Fig. 2 in the supplemental materials for an illustration of the circuit.)
The device rests on a table with dimensions similar to those of the box. The participant is seated facing the front of the BIO, and the experimenter is seated facing the back (Fig. 1). The participant inserts his or her arms into the BIO through the gap and rests them on the rubber surface that lines the base of the box.
Experimental testing of the BIO
The BIO was tested in a motivational study focused on visual and motor event-related potentials. One of the authors (I.A.D.), a right-handed woman, participated in the experimental session. Her task was to look at and grasp an unpleasant object. This choice of stimulus was based on previous studies that had found enhanced activity in sensory–motor circuits when there was a discrepancy between the instructed action and preset networks engaged in avoiding unpleasant stimuli (Nogueira-Campos, Oliveira, Esteves, Rodrigues, & Vargas, 2010; Oliveira et al., 2007; Pereira et al., 2010).
Experimental design
The experiment was conducted in a dimly lit room. The participant sat in a comfortable chair facing the BIO (Fig. 2). The BIO was designed to allow the use of different categories of objects in the same experimental session, which could be inserted by the experimenter through the rear aperture. Because the back-side LEDs stay on during the intertrial, the experimenter can see the objects to be replaced without being noticed by the subject. For purposes of experimental testing, we used a single object to be grasped: a transparent plastic cup (sealed with plastic film) containing cockroaches. The reflective film that covered the glass on the front side of the BIO prevented the participant from seeing inside it. When the experimenter turned on the inside LEDs, the object became visible, and a synchronizing pulse was sent to the recording systems (Fig. 3). Therefore, this was the trigger that signaled stimulus presentation.
The participant was instructed to fix her gaze on the center of the reflective glass where the stimulus would appear. Upon illumination of the inside LEDs (stimulus onset), she was instructed to look at the stimulus for a few seconds, then to grasp it with her right hand and bring it close to her body. She should then put the glass down and move her hand to its initial position (Fig. 4). The session was composed of 90 trials. During the intertrial intervals, the switch was positioned so that the inside LEDs were turned off and the LEDs on the back of the BIO were turned on. The total duration of the experiment was about 30 min.
Physiological recording and analysis
Electromyographic (EMG) recordings were collected via two Ag/AgCl electrodes (8-mm diameter, 2-cm interelectrode distance) coupled to an MP150 amplifier (BIOPAC Systems Inc., Santa Barbara, CA, USA). After skin preparation, these electrodes were positioned on the long head of the biceps brachii muscle according to SENIAM recommendations (Freriks, Hermes, Disselhorst-Klug, & Rau, 1999). A reference electrode was fixed on the left lateral epicondylus (Fig. 2). The sample rate was 1000 Hz. The EMG signal was segmented offline from 500 ms before to 12 s after stimulus onset. Those segments were rectified and digitally filtered (low pass, 3 Hz).
We recorded electroencephalograms using a BrainNet BNT-36 (EMSA Equipamentos Médicos Ltda, Rio de Janeiro, Brazil) recording system with 23 electrodes positioned in accordance with electrode sites defined by the 10–20 system. All electrodes were referenced to Cz and re-referenced to the averaged mastoids obtained during the recording session. The sample rate was 400 Hz during data acquisition, and the impedance was kept below 5 kΩ for all electrodes. The data were filtered offline using 0.1-Hz high-pass and 30-Hz low-pass digital filters. Offline analysis of the data, including the removal of eye movement artifacts, was performed using the EEGLAB version 7.2.9 toolbox (Delorme & Makeig, 2004) with MATLAB version 7.5 (The Mathworks, Natick, MA). Eye movement artifacts were removed from the data using independent-components analysis (Jung et al., 2000).
For visual event-related potentials, the EEG data were epoched from 200 ms pre-stimulus-onset (i.e., onset of the inside LEDs) to 1,500 ms post-stimulus-onset, and averaged. The state 200 ms before stimulus onset was used as the baseline. To obtain the readiness potential, EEG segments were time-locked to the onset of the EMG signal that indicated the beginning of the movement. Movement onset was attributed to the time point at which 5% of the maximal EMG value was reached. This automatic estimation was also checked by visual inspection. For more details, researchers can access Supplemental Fig. 3 in the supplemental materials. EEG data were obtained from 2,500 ms pre-movement-onset to 500 ms post-movement-onset and were then averaged across trials. The baseline was set between 2,500 and 2,300 ms pre-movement-onset. Epochs containing voltage deviations larger than 100 μV relative to baseline for any of the electrodes were rejected.
Results and discussion
The use of the BIO allowed for high temporal resolution in the synchronization of object presentation with the EEG recordings. The onset of the visual stimulus was almost instantaneous, on the order of microseconds, as compared to the milliseconds required for screen refreshing (a commonly used method for stimulus presentation). The grand-average EEG of epochs time-locked to stimulus onset revealed a positive waveform in the 200- to 400-ms window. This waveform increased in magnitude from the frontal to the parietal electrodes and was compatible with previous findings for the P300 (Polich & Kok, 1995), a wave previously described that can be modulated by emotion (Carretié, Iglesias, & García, 1997; Carretié, Iglesias, García, & Ballesteros, 1997; Olofsson et al., 2008; Olofsson & Polich, 2007; Rozenkrants, Olofsson, & Polich, 2008; Schupp, Cuthbert, Bradley, Birbaumer, & Lang, 1997). Figure 5 depicts the grand averages obtained at the frontal, central, and parietal midline electrodes (Fz, Cz, and Pz).
The grand average taken from the same EEG signal, now time-locked to movement onset, revealed a negative waveform compatible with the readiness potential found in the premovement window (Jahanshahi & Hallett, 2003). Figure 6 illustrates the readiness potential preceding movement onset that was obtained at the central midline electrode (Cz).
For the first time, visual and motor event-related potentials were obtained in the same task using a real, three-dimensional stimulus. The BIO device permits a more naturalistic experimental paradigm to be used, involving the use of real objects instead of images as stimuli. Any real object, with a weight and size that allow it to be fit inside the BIO, can be used as a stimulus. The size of the rear aperture is 20 × 20 cm, and it can be increased to twice this size without compromising the operation of the BIO.
This new device will be useful for researchers wishing to study the brain activity involved in processing the onset of three-dimensional stimuli and in the preparation of actions toward the stimuli. The BIO can be used in experiments that involve the manipulation of neutral objects, or in a more motivational and challenging environment by using objects with emotional content.
The BIO was designed to allow for the use of different categories of objects, any that can be inserted by the experimenter through the rear aperture, in the same experimental session. The experimenter can see when the participant returns the object to the initial position through the rear aperture. The experimenter then turns on the back-side LEDs and chooses which object will replace the previous one. The objects can be organized in a blocked or in a randomized, previously sorted sequence.
The illumination of the LEDs can be easily customized to different experimental paradigms. In addition to the EMG triggering of the movement, trigger signals indicating the beginning and ending of object manipulation can be implemented. To this end, a weight sensor (a small plaque with a switch attached) to detect changes in weight triggered by the participant’s hand can be installed. Additionally, an LED located behind the BIO could indicate to the experimenter the start and the end of the subject’s movement.
In conclusion, the BIO successfully synchronizes the presentation of a real, three-dimensional object with EEG recordings, allowing for analysis of visual and motor event-related potentials in the same experiment. The BIO device, through a high-quality psychophysiological approach, offers a new perspective for the study of the motivational factors that drive actions toward relevant stimuli.
References
Bradley, M. M., Codispoti, M., Cuthbert, B. N., & Lang, P. J. (2001). Emotion and motivation I: Defensive and appetitive reactions in picture processing. Emotion, 1, 276–298. doi:10.1037/1528-3542.1.3.276
Carretié, L., Iglesias, J., & García, T. (1997a). A study on the emotional-processing of visual stimuli through event-related potentials. Brain and Cognition, 34, 207–217. doi:10.1006/brcg.1997.0895
Carretié, L., Iglesias, J., García, T., & Ballesteros, M. (1997b). N300, P300 and the emotional processing of visual stimuli. Electroencephalography and Clinical Neurophysiology, 103, 298–303. doi:10.1016/S0013-4694(96)96565-7
Coombes, S. A., Cauraugh, J. H., & Janelle, C. M. (2006). Emotion and movement: Activation of defensive circuitry alters the magnitude of a sustained muscle contraction. Neuroscience Letters, 396, 192–196. doi:10.1016/j.neulet.2005.11.048
Coombes, S. A., Cauraugh, J. H., & Janelle, C. M. (2007). Dissociating motivational direction and affective valence: Specific emotions alter central motor processes. Psychological Science, 18, 938–942. doi:10.1111/j.1467-9280.2007.02005.x
Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134, 9–21. doi:10.1016/j.jneumeth.2003.10.009
Fabiani, M., Gratton, G., & Federmeier, K. D. (2007). Event-related brain potentials: Methods, theory, and applications. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (3rd ed., pp. 85–119). New York, NY: Cambridge University Press.
Facchinetti, L. D., Imbiriba, L. A., Azevedo, T. M., Vargas, C. D., & Volchan, E. (2006). Postural modulation induced by pictures depicting prosocial or dangerous contexts. Neuroscience Letters, 410, 52–56. doi:10.1016/j.neulet.2006.09.063
Freriks, B., Hermes, H. J., Disselhorst-Klug, C., & Rau, G. (1999). The recommendations for signal processing methods for surface electromyography. In European recommendations for surface electromyography-SENIAM. Enschede, The Netherlands: Roessingh Research and Development.
Jahanshahi, M., & Hallett, M. (2003). The Bereitschaftspotential movement-related cortical potentials. New York, NY: Kluwer/Plenum.
Jung, T. P., Makeig, S., Humphries, C., Lee, T. W., McKeown, M. J., Iragui, V., & Sejnowski, T. J. (2000). Removing electroencephalographic artifacts by blind source separation. Psychophysiology, 37, 163–178. doi:10.1111/1469-8986.3720163
Luck, S. J. (2005). An introduction to the event-related potential technique. Cambridge, MA: MIT Press.
Nogueira-Campos, A., Oliveira, L. A. S., Esteves, P., Rodrigues, E. C., & Vargas, C. D. (2010). Preparing to grasp an emotional-laden object: A TMS study. Psychophysiology, 47, S84.
Oliveira, L. A., Furtado-Martins, E., Imbiriba, L. A., Rodrigues, E. C., Volchan, E., & Vargas, C. D. (2007). Grasping disgusting stimuli enhances motor readiness potential. Psychophysiology, 44, S57.
Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective picture processing: An integrative review of ERP findings. Biological Psychology, 77, 247–265. doi:10.1016/j.biopsycho.2007.11.006
Olofsson, J. K., & Polich, J. (2007). Affective visual event-related potentials: arousal, repetition, and time-on-task. Biological Psychology, 75, 101–108. doi:10.1016/j.biopsycho.2006.12.006
Pereira, M. G., de Oliveira, L., Erthal, F. S., Joffily, M., Mocaiber, I. F., Volchan, E., & Pessoa, L. (2010). Emotion affects action: Midcingulate cortex as a pivotal node of interaction between negative emotion and motor signals. Cognitive, Affective, & Behavioral Neuroscience, 10, 94–106. doi:10.3758/CABN.10.1.94
Pereira, M. G., Volchan, E., de Souza, G. G. L., Oliveira, L., Campagnoli, R. R., Pinheiro, W. M., & Pessoa, L. (2006). Sustained and transient modulation of performance induced by emotional picture viewing. Emotion, 6, 622–634. doi:10.1037/1528-3542.6.4.622
Polich, J., & Kok, A. (1995). Cognitive and biological determinants of P300: An integrative review. Biological Psychology, 41, 103–146. doi:10.1016/0301-0511(95)05130-9
Rozenkrants, B., Olofsson, J. K., & Polich, J. (2008). Affective visual event-related potentials: arousal, valence, and repetition effects for normal and distorted pictures. International Journal of Psychophysiology, 67, 114–123. doi:10.1016/j.ijpsycho.2007.10.010
Souza, G. G., Pereira, M. G., Vila, J., Oliveira, L., & Volchan, E. (2012). Affiliative stimuli as primers to prosocial predispositions. Spanish Journal of Psychology, 15(1), 237–243. doi:10.5209/rev_SJOP.2012.v15.n1.37315
Schupp, H. T., Cuthbert, B. N., Bradley, M. M., Birbaumer, N., & Lang, P. J. (1997). Probe P3 and blinks: two measures of affective startle modulation. Psychophysiology, 34, 1–6. doi:10.1111/j.1469-8986.1997.tb02409.x
van Schie, H. T., & Bekkering, H. (2007). Neural mechanisms underlying immediate and final action goals in object use reflected by slow wave brain potentials. Brain Research, 1148, 183–197. doi:10.1016/j.brainres.2007.02.085
Author note
The authors thank Gisella Tavares Barata for her assistance during the experimental testing. This work was supported by Fundação do Câncer, FAPERJ, and CNPq.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplemental Figure 1
The dimensions of the BIO structure in centimeters. Left: There is a porthole (100 X 20 cm) on the bottom of the front side. Middle: A glass plate coated with a reflective film is inclined at 45 degrees at the front of the box, leaving the porthole at the bottom. Right: A centralized aperture (20 X 20 cm) is present at the bottom of the back of the BIO. The sides fit together to form a prism-shaped hollow box. (PDF 79.0 kb)
Supplemental Figure 2
The control circuit for the operation of the BIO. The circuit contains the following components: three resistors (R1, R2 and R3), one switch (S1), one opto-coupler (U1), eight white LEDs (LED1 to LED8), one 40-volt power supply and four connectors (CN1, CN2, CN3 and CN4). The switch is used to select which set of LEDs stays on: the inside LEDs (S1, position 1) or the backside LEDs (S1, position 2). When the inside LEDs are on, the trigger output stays shorted by the opto-coupler; when the inside LEDs are off, the trigger output stays in high impedance. A recording channel plugged into the trigger correspondent connector (CN2) is used to register the trigger. The resistors are used at the current limit, and the other connectors are used to plug in the wires of the LEDs and the power supply. (PDF 85.8 kb)
Supplemental Figure 3
The figure exemplifies a segment of the electromyographic signal recorded during the task of grasping a cup with the right hand and bringing it close to the body. The EMG signal was segmented offline from 500 ms before to 12 s after stimulus onset. When the participant began to execute the task, the contraction of the right biceps muscle elicited an amplitude change that can be seen in the electromyographic signal. The raw signal (green line) was rectified and digitally filtered (black line). Movement onset was attributed to the point in time where the filtered EMG reached 5% of the maximal value. This automatic estimation was also checked by visual inspection. The EMG latency represents the time between the stimulus (blue line) and movement (red line) onsets. (PDF 120 kb)
Rights and permissions
About this article
Cite this article
Oliveira, J.M., Volchan, E., Vargas, C.D. et al. Box for interaction with objects (BIO): A new device to synchronize the presentation of objects with electrophysiological recordings. Behav Res 44, 1115–1120 (2012). https://doi.org/10.3758/s13428-012-0197-x
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13428-012-0197-x