Rewarding properties of visual stimuli

Abstract

The behavioral functions of rewards comprise the induction of learning and approach behavior. Rewards are not only related to vegetative states of hunger, thirst and reproduction but may also consist of visual stimuli. The present experiment tested the reward potential of different types of still and moving pictures in three operant tasks involving key press, touch of computer monitor and choice behavior in a laboratory environment. We found that all tested visual stimuli induced approach behavior in all three tasks, and that action movies sustained consistently higher rates of responding compared to changing still pictures, which were more effective than constant still pictures. These results demonstrate that visual stimuli can serve as positive reinforcers for operant reactions of animals in controlled laboratory settings. In particular, the coherently animated visual stimuli of movies have considerable reward potential. These observations would allow similar forms of visual rewards to be used for neurophysiological investigations of mechanisms related to non-vegetative rewards.

Introduction

Environmental events and objects can possess rewarding functions on behavior without having any nutrient value and without being related to vegetative needs of organisms or drive states. The presentation of interesting and engaging visual stimuli can serve as efficient rewards for animals, including the viewing of conspecifics and their social interaction (Thompson 1963; 1964; Rnic 1977). Such objects fulfill the defining criteria of reward and positive reinforcement by inducing learning and approach behavior (Kish 1966). Visual stimuli, in particular with social content, can comptete with nutrient rewards as reinforcers in primates (Deaner et al. 2005). The rewarding visual stimuli do not constitute specifically conditioned positive reinforcers, as their reinforcing functions are not based on prior pairings with nutrient or sexual rewards, suggesting that they may constitute rewards in their own right.

Various types of visual stimuli can serve as efficient reinforcers in primates (Butler and Woolpy 1963; Butler 1954; Humphrey 1972; Swartz and Rosenblum 1980; Fujita 1987; Andrews and Rosenblum 1993). However, previous studies were conducted in homecages equipped with video monitors and response levers (Fujita and Matzuzawa 1986; Washburn et al. 1991; Washburn and Hopkins 1994), with the exception of an earlier social reward study that used operant cages for viewing conspecifics (Butler 1954). The absence of rigorously controlled laboratory situations precludes the specific behavioral analysis required for neurobiological investigations of brain processes underlying sensory rewards.

The present study assessed the positive reinforcing potential of various visual stimuli in a variety of operantly conditioned behavioral reactions in a controlled laboratory setting. We define a reward operatively as any object or event that leads to approach behavior and learning in a way that will increase the frequency of receiving the reward (’come back for more’). We used three types of visual rewards, namely constant still pictures, changing still pictures and action movies. We tested each reward type in three different operant tasks in which animals were presented with a visual reward for the duration of touching a key, for a target-directed movement to a computer touch monitor, or as the outcome of a choice between two simultaneously presented types of reward.

Materials and methods

Subjects

Two monkeys served for the experiments (A: Macaca mulatta, male, 6.5 kg; B: Macaca fascicularis, female, 3.8 kg). They were cared for according to the Guiding Principles in the Care and Use of Animals of the American Physiological Society. Experiments were conducted at the University of Fribourg and approved by the animal ethics committee of the Canton of Fribourg (Switzerland). Animals were brought to the laboratory for about one hour on each working day and seated in a primate chair facing a computer touch screen in a closed room separated from experimenters. Animals were returned after each daily session to their home-cage, where they had access to water and food ad lib.

Behavioral procedures

Animals were trained for several weeks in three visual reward tasks, namely the Movement task, Choice task and Continuous touch task. Data were collected sequentially once performance had been established in all tasks.

Visual rewards

Each task employed three types of still or moving pictures that were presented as rewards on a computer touch screen for correct behavioral reactions: (a) an identical Constant Picture in every trial of a given block, (b) Changing Pictures that were randomly selected from 20 pictures and changed in every trial, (c) a short clip of action Movie that advanced in every trial. The three reward types occurred in separate trial blocks, following the sequence of Movie, Changing Pictures and Constant Picture in the Movement and Continuous Touch tasks. Trial durations varied depending on the task, and intertrial intervals were 3.0 s (from end of visual reward to first event of next trial). Each block of trials lasted 15 min (900 s) and was followed by a 10 min break. The Constant and Changing Pictures were scanned from advertisements and journals. They showed children’s toys, tools, cars, houses, plants, people, country scenes and nonsensical items. Each trial block used a new Constant Picture or a new set of 20 Changing Pictures, thus employing hundreds of pictures during the experiment. Movies were randomly selected each day from four action titles (King Kong and Indiana Jones 1–3) and shown via a computer-controlled 30“ Sony Laser Disk player. No attempt was made to differentiate movie scenes according to novelty, speed of picture change, types of objects, species displayed and social interaction.

Continuous touch task

After the animal touched a resting key with its right hand for >1.0 s, a Constant Picture, a Changing Picture or a Movie appeared and stayed on until the key was released again. We used each reward type in a separate trial block, for 6 days in animal A and 9 days in animal B. We measured touch durations >1.0 s during 900 s to assess the reward value of the stimuli.

Movement task

A colored target spot appeared in the center of the touch screen. The animal touched briefly the spot with its right hand and received a visual reward for 4, 8 or 16 s after a 2.0 s delay. The color of the square indicated the reward type (green for Constant Picture, blue for Changing Pictures, red for Movie) (Fig. 1). We used each reward type in one trial block per day, and used the three reward durations of 4, 8 and 16 s for five consecutive days each. As a control for presentation order, we reversed the sequence to Constant Picture, Changing Pictures and Movie with reward durations of 4 and 8 s, for three consecutive days. We measured the total durations of reward presentation and the total number of target acquisitions per time (touch frequency) during 900 s trial blocks to assess the reward value of the stimuli.

Fig. 1

Movement task. A colored target spot appeared in the center of the monitor and predicted the visual reward type (green for Constant Picture, blue for Changing Pictures, red for Movie). Touching the spot with the hand produced the reward for 4 s, 8 s or 16 s after a 2 s delay. Example shows Constant Picture

Choice task

Two color squares of same size but different color appeared simultaneously to the left and right of center on the monitor and indicated which of two visual rewards could be chosen by touching a square with the right hand (green versus blue for Constant Picture versus Changing Pictures, blue versus red for Changing Pictures versus Movie, green versus red for Constant Picture versus Movie). Left and right stimulus positions alternated pseudo-randomly, with maximally three consecutive identical positions. Durations of visual reward presentation were fixed at 4 s. Each comparison was run in one block, for a total of three blocks per day, on six and seven days in animals A and B, respectively. We assessed reward preference as the choice of a particular reward in percent of trials during 900 s trial blocks.

Data collection and evaluation

We recorded the touch of the resting key via the parallel port of a laboratory computer, and the touch of the monitor in x–y coordinates. Because of occasionally skewed distributions, we employed nonparametric measures (median=fiftieth percentile) and tests, separately for the two animals, namely the Friedman test for multiple two-way comparisons, the Kruskal–Wallis test for multiple one-way comparisons and post-hoc analysis after Friedman test, and the Mann-Whitney U-Test for two-sample and post-hoc comparisons of tasks, reward types and animals.

Results

Continuous touch task

Visual reward was presented for the duration of continuous key touch. Both animals held the key for about 50% of the possible time for Movies (medians of 480 and 465 s per 900 s trial block), but kept it for much shorter periods for Changing Pictures and Constant Pictures (Fig. 2). Overall differences across reward types were significant in both animals (A: χ²=10.398, P<0.01; B: χ²=18.78, P<0.001) and were located to the Movies (Movie versus Changing Pictures, P<0.02; Movie versus Constant Picture, P<0.01; Changing versus Constant Pictures, p>0.05).

Fig. 2

Viewing durations of visual reward in the Continuous touch task. Columns show medians (fiftieth percentile), with twenty-fifth and seventy-fifth percentiles from six and nine trial blocks of 900 s each in animals A and B, respectively. Both animals viewed Movies significantly longer than Changing or Constant Pictures

Movement task

Touching the target spot produced a visual reward for 4, 8 or 16 s. Median viewing durations of visual reward ranged from 40 to > 300 s during the 900 s test periods and varied significantly across reward types and reward durations in both animals (Fig. 3) (animal A: χ²=19.730, P=0.011; animal B: χ²=22.4, P=0.004). Median touch frequencies were 9–49 movements per 900 s and varied significantly (animal A: χ²=19.388, P<0.02; animal B: χ²=22.173, P<0.01) (Fig. 4).

Fig. 3

Viewing durations of visual reward in the Movement Task. Each screen touch resulted in single visual presentation of 4, 8 or 16 s. Both animals viewed Movies significantly longer than Changing and Constant Pictures. Columns show medians from five trial blocks of 900 s each, and fiftieth and seventy-fifth percentiles

Fig. 4

Touch frequencies for visual rewards in the Movement Task. Each screen touch resulted in single visual presentation of 4, 8 or 16 s. Both animals touched more frequently to view Movies compared to Changing and Constant Pictures. Columns show medians from five trial blocks of 900 s each, and twenty-fifth and seventy-fifth percentiles

Reward type

The two animals viewed Movies for 224 s and 234 s, Changing Pictures for 96 s and 112 s, and Constant Pictures for 80 s and 88 s, respectively (data pooled across reward durations) (Fig. 3). These differences were significant in both animals (animal A: χ²=11.79; animal B: χ²=14.506; both P<0.01) and occurred in all paired post-hoc comparisons (P<0.05), except for animal A for Changing versus Constant Pictures (p>0.05).

For animals touched the spot, the median was 37–49 times per 90 s for Movies, 7–23 times for Changing Pictures and 9–16 times for Constant Pictures (Fig. 4). The differences were significant in both animals (animal A: χ²=16.988; animal B: χ²=16.76; both P<0.05) and all post-hoc comparisons (P<0.05), except in animal A for Changing versus Constant Pictures.

Reward duration

The two animals viewed all visual rewards longer when reward durations were increased from 4 s to 8 s and 16 s per touch (animal A: χ²=8.175, P=.017; animal B: χ²=8.776, P=.012; data pooled across reward type) (Fig. 3). For example, viewing times of Movies were increased from 150–195 s to 155–270 s and 305–325 s per 900 s test periods for reward durations of 4, 8 and 16 s, respectively (animal A: χ²=3.317, P=0.19; animal B: χ²=5.647, P=0.059). Similar increases, although with lower overall viewing times, occurred for Changing Pictures (A: χ²=6.108, P=0.047; B: χ²=6.543, P<0.04) and Constant Picture (animal A: χ²=6.252, P=0.44; animal B: χ²=7.2, P<0.03 Fig. 3).

The shorter reward durations were met by higher touch frequencies for all three stimulus types in animal B (χ²=6.112, P<0.05), but not only for Movies in animal A (Fig. 4). For Movies, animal B touched 37–49 times per 90 s for 4 s reward duration, 19–34 times for 8 s and 19–20 times for 16 s. For Changing Pictures, animal B touched 10–23 times per 90 s for 4 s reward duration, 12–18 times for 8 s and 7–15 times for 16 s (all P<0.05). There were no significant differences for Constant Picture. These data suggest a partial compensation for shorter reward durations by higher touch frequencies.

Presentation sequence

To assess the potential influence of reward sequence, we reversed the test order to Constant Pictures, Changing Pictures and Movie, using reward durations of 4 and 8 s. Two-way analysis using presentation sequence and reward type revealed significant differences to normal sequence (for 4 s: animal A: χ²=12.212, P=0.032; animal B: χ²=12.981, P=0.024; similar values for 8 s). Post-hoc analysis confirmed the longer viewing times seen in the normal sequence according to Movies > Changing Pictures > Constant Pictures for both reward durations (e.g. for 4 s: animal A: χ²=3.586, P<0.2, animal B: χ²=7.2, P=0.027; Fig. 5). However, overall viewing times were somewhat shorter in the reversed compared to the normal sequence (animal A: Constant Picture P<0.005; animal B: Movie P<0.03, Changing Pictures P<0.005). Nevertheless, these data suggest that Movies are stronger rewards compared to Changing and Constant Pictures irrespective of test sequence.

Fig. 5

Influence of test sequence in the Movement Task. Animals touched the spot for visual reward presentation less frequently when the inverse sequence was run (Constant Picture, Changing Pictures, Movie), compared to the normal sequence of Movie, Changing pictures, Constant Picture, for both 4 s and 8 s reward durations. Movies were the preferred reward with inverse and normal sequences. Columns from reverse and normal sequences show medians from three and five trial blocks of 900 s each, respectively. A and B denote the two animals

Comparison with continuous touch task

To assess potential influences of physical effort, we compared reward viewing durations between the Movement Task and the physically less-demanding and more static Continuous Touch Task. Overall reward-viewing durations varied significantly between the two tasks and three reward types (animal A: χ²=21.25, P=0.001; animal B: χ²=39.413, P=0.000; using viewing durations of 4, 8 and 16 s in the Movement Task with equal proportions). Viewing durations were about twice as long for Movies in the Continuous Touch Task compared to the Movement Task (animal A: P<0.03, animal B: P<0.001), but varied inconsistently for Changing and Constant Pictures between the two tasks. Thus performance for the more effective of the visual rewards was sensitive to the effort involved in obtaining the reward.

Choice task

Animals chose between two visual rewards by touching the appropriate color spot. Animals A and B initiated 48–50 and 33–34 trials per 900 s trial block, respectively, whenever Movie was one alternative, but only 15–23 trials for choices between Constant and Changing pictures. Animals showed overall significant preferences among the three rewards (animal A: χ²=19.029, P<0.001; animal B: χ²=20.666, P<0.001). They chose the Movies in 59–74% of trials and thus significantly more frequently than Changing or Constant Pictures (both P<0.01), but showed less significant preferences between Constant and Changing Pictures (Fig. 6). Choices differed insignficantly between the four different Movies used (p>0.6; Kruskal–Wallis test on six and seven trial blocks in animals A and B, respectively).

Fig. 6

Preferences between different reward types in the Choice Task. Columns show medians and twenty-fifth and seventy-fifth percentiles from six and seven trial blocks in animals A and B, respectively. Both animals chose Movies significantly more frequently than Changing or Constant Pictures (**P<0.01; * P<0.02; Mann-Whitney test)

Discussion

This study shows that visual stimuli can serve as positive reinforcers for operant reactions in controlled primate laboratory situations. The increasing efficacy for Constant, Changing and Movie rewards observed consistently in all three tasks suggests a higher reinforcement potential for more animated stimuli in monkeys, a finding that intuitively relates to daily observable human behavior. Although the current behavioral response rates for visual stimuli were below those usually obtained for nutrient rewards in the laboratory, the observation that simple visual stimuli can have rewarding functions encourages the development of more specific visual reward tasks for neurophysiological experiments in behaving primates.

Comparisons between visual rewards

The three types of visual reward varied primarily across two aspects, novelty and coherent animation. The Movies contained more novelty and coherent animation compared to Changing and Constant Pictures, as they involved not only continuously new pictures but also their presentation in a coherent sequence. The Changing Pictures involved more novelty compared to the Constant Pictures, as they were different in every trial, but their presentation did not follow a natural or logical sequence. Thus the Constant Pictures were intuitively the least interesting ones to an animal.

The differences in stimulus attributes were reflected in the reward potential. The Movie reward was the most efficient of the three reinforcers in all three tasks in both animals. Compared to the still Pictures, animals viewed the Movies three to four times longer when maintaining key press and about two times longer when touching the monitor, and they preferred the Movies two to three times more than the still Pictures in the choice task. These data are in general agreement with the previously reported reinforcing effects of motion pictures in primates (Butler 1961; Swartz and Rosembaum 1980; Washburn and Hopkins 1994) and demonstrate their superior efficacy over still pictures. Thus continuing animation with coherent motion sequences appears to contribute particularly well to the reward potential of visual stimuli.

The Changing Pictures and the Constant Pictures showed definite albeit limited reward potential in all three tasks. Changing pictures were usually more effective than Constant Pictures, but the differences between them were often slight and reached statistical significance only in the Movement and Choice Tasks with animal B. Compared to these results, earlier experiments had shown more substantial reinforcing effects of pictures that contained social gestures of conspecifics (Butler 1961; Butler and Woolpy 1963; Humphrey 1972). Video clips of conspecifics appear to constitute particularly efficient rewards (Andrews and Rosenblum 1993, 2002), producing 205–1,210 behavioral reactions per 12 h period. Our animation Movies may have a comparable reinforcement potential, inducing up to 50 behavioral reactions per 15 min, although the differences in testing durations (12 h versus 15 min periods) preclude more direct comparisons. Taken together, the efficacy of visual rewards appears to increase from simple still pictures via changing still pictures and social pictures of other monkeys to engaging motion sequences.

Motivating factors

Both animals touched the monitor in the Movement Task more frequently for reward durations of 4 s, compared to durations of 8 and 16 s, and did so consistently for all three visual reward types. This result should not reflect a higher reward potential of shorter compared to longer reward durations. Rather it suggests that the animals were motivated to view the visual stimuli for a certain period with whatever energy it took to do so, within the confines of the experiment. This interpretation is supported by the observation that the animals’ behavior resulted in longer overall viewing times, for longer compared to shorter reward durations. These considerations underline the rewarding functions of the visual stimuli employed. Experiments involving electrophysiological recordings in controlled behavioral settings usually require several hundreds of trials in a session lasting several hours. The currently obtained response rates with Movies are similar to those for video clips, which in turn are similar or somewhat inferior to those involving food pellets in monkeys with ad lib access to food (Andrews and Rosenblum 1993, Brannon et al. 2004), and higher response rates are obtained with more controlled food or liquid intake. The current maximal rates of 50 responses per 15 min with Movies would be at the lower end of the rates necessary for electrophysiologcal experiments.

Several measures can be taken to increase the efficacy of visual rewards and obtain higher ressponse rates. These include prolonged training over several months (Andrews and Rosenblum 2002), viewing videos of other groups of the same species (Brannon et al. 2004), and employing variable ratio or interval reinforcement schedules to enhance motivation through attention. Furthermore, previous experiments have suggested perceived self-control as a motivating factor when using visual rewards in monkeys (Washburn et al. 1991). Considering also the presently observed substantial responses in the choice task, experimental situations involving choices and decisions between variable visual outcomes might provide favorable conditions for investigating neural processes underlying non-nutrient rewards in behaving monkeys. Another possibility for using visual stimuli as rewards in electrophysiological studies is to present them in simultaneous conjunction with nutrient rewards while comparing behavioral response rates and preferences between conjointly and separately presented rewards (Deaner et al. 2005).