Driving simulator validation with hazard perception

Abstract

How should we assess the comparability of driving on a road and “driving” in a simulator? If similar patterns of behaviour are observed, with similar differences between individuals, then we can conclude that driving in the simulator will deliver representative results and the advantages of simulators (controlled environments, hazardous situations) can be appreciated. To evaluate a driving simulator here we compare hazard detection while driving on roads, while watching short film clips recorded from a vehicle moving through traffic, and while driving through a simulated city in a fully instrumented fixed-base simulator with a 90 degree forward view (plus mirrors) that is under the speed/direction control of the driver. In all three situations we find increased scanning by more experienced and especially professional drivers, and earlier eye fixations on hazardous objects for experienced drivers. This comparability encourages the use of simulators in drivers training and testing.

Introduction

Practical driving can be assessed by a range of component-task laboratory tools – reaction time tests, spatial-ability tests, and judgement tests, for example, as well as with more inclusive tasks involving driving simulators. Simulators are essential tools of driver assessment, for ethical reasons above all others, in any task where drivers may be exposed to actual driving hazards such as high probability of collision. When we place inexperienced novice drivers in a roadway situation with other, unpredictable road users we are putting them at risk, and simulators eliminate the consequences of these risks. Drivers may behave in similar ways in simulators and on real roads, but questions have been raised about the validity of the measures taken, with Kemeny and Panerai (2003) pointing out that simulators do not present all of the most relevant visual cues for drivers (especially binocular cues and motion parallax), and with Owsley and McGwin (2010), for example, pointing to crude visual display with poor fidelity that cannot represent the visual complexity or range of lighting conditions experienced in actual driving. Simulator validation studies have tended to compare driving on a road against driving in a simulator, assessing speed and speed adaptation, and lane-keeping (e.g., Bella, 2008, Godley et al., 2002, Lee et al., 2003, Törnros, 1998). Results have generally shown good correspondence, but Godley et al. (2002) distinguished between relative validity (similar patterns of behaviour), which they did establish, and absolute validity (similar speeds), which was not established. Speed and lane-keeping are undoubtedly importance measures when validating a simulator, but they should be regarded as necessary conditions rather than sufficient conditions. They measure relatively low-level vehicle control, being perceptual-motor measures of driving, and given our current knowledge of the factors that influence performance it is now appropriate to included higher-level cognitive measures in the assessment. As well as controlling the vehicle we can also assess the comparability of the driver’s situation awareness by looking for behavioural change in roads and in specific situations associated by heightened levels of visual search. If an experienced driver is aware that a situation is likely to present difficulties from other road users, then their search of the roadway changes (Crundall and Underwood, 1998, Underwood et al., 2003). Also, when watching movies filmed from a driver’s perspective, their behaviour towards a potential hazard is distinctive. Differences between drivers with different abilities can be used as a measure for the assessment of simulator validity. This paper asks whether behaviour towards potential hazards is comparable in a simulator and in other driving tasks.

Early studies with simulated driving tasks were very promising in demonstrating a relationship between self-reported accident history and laboratory behaviour. Currie (1969) recruited 26 pairs of volunteers composed of a safe driver and an “accident repeater” (at least three accidents per 100,000 miles) who were matched for age, occupation, and driving experience. Their control of an electric model car (1:32 scale) was recorded as it travelled around a circuit while another car converged on a collision course during overtaking, junction-crossing or when pulling across the driver’s path. To make the study important to the drivers they appeared to be wired up to receive an electric shock in case of any “inappropriate action”. Currie commented on the efficacy of this threat, pointing out that many of the participants were seen to flinch when collisions occurred, and some reported mild nausea. The results suggested that safe drivers recognised the dangers from other cars earlier than accident-repeaters, by braking when a collision was likely, and they had fewer collisions overall. Even in simple driving-related tasks then, drivers exhibit patterns of behaviour that are consistent with their on-road behaviour. To validate the measures taken from simulators, however, we need to know how drivers behave in the situations that are simulated. A number of studies have looked for such comparisons, recording individual differences in measures including driver-selected road speed, braking, traffic sign compliance, non-signed rule compliance, steering, and use of vehicle controls (e.g., Behr et al., 2010, Godley et al., 2002, Lee et al., 2003, Reed and Green, 1999), but direct comparisons between in-simulator and in-car driving are relatively rare. In the present review we compare hazard perception responses in a driving simulator, with hazard responses while driving and while participating in a conventional hazard perception test.

Hazard perception is regarded here as a driver’s situation awareness for a dangerous configuration of roadway and road users, and will be used as the test-bed for comparing behaviour in different environments. Situations that require a driver to adapt their behaviour by changes of speed or direction are hazardous, and safer drivers will anticipate these situations before extreme braking or swerving is necessary to avoid a collision. For example, if driving along an otherwise unoccupied urban street with a group of children playing with a ball on the footpath ahead, there is a chance of the ball and possibly one of the children running into the road. The children therefore present a potential hazard well before there is a risk of a collision, and a driver may adjust the car’s speed to allow gentle braking in case the ball and child do appear on the roadway. In Endsley’s (1995) three-level model of situation awareness there is a basis for distinguishing between drivers with different skill, and for identifying the causes of differences in hazard perception. In this model the lower two levels of situation awareness correspond to perception of the current environment and knowledge of how the current situation has arisen (see also Endsley, 2004, Horswill and McKenna, 2004, Underwood, 2007). Drivers who are able to predict the behaviour of other road users, anticipating how the current situation might develop as other vehicles manoeuvre around them, or what a group of children on the footpath ahead might do, would correspond to awareness at the third and highest level in Endsley’s model. Hazard perception tests that are used for driver evaluation ideally test these anticipation skills and are now used for driver training and assessment. Typical hazard perception tests involve movies filmed from a driver’s perspective in a car that travels along a range of roadways. Events occur that would require braking or steering changes, such as the car in front of the camera car slowing sharply, or another road user moving into the path of the car. The participant is required to press a response button whenever one of these events would require a driving response, or in some cases a continuous recording is taken by the participant moving a lever between settings marked “safe” to “dangerous” (e.g. Crundall, Chapman, Phelps, & Underwood, 2003; Pelz & Krupat, 1974).

Results with the hazard perception test have shown sensitivity to individual driving ability, effects of sleepiness, and age-related decrements. In Pelz and Krupat’s (1974) early study 60 drivers were shown a 5 min movie with 10 hazardous events, and differences in the driver’s accident record were associated with selected settings on the continuous recordings of the “apprehension meter”. Drivers with fewer accidents tended to be more cautious overall and to respond faster to the onset of a hazard. More recent studies with discrete button-press responses have confirmed the tendency of inexperienced or novice drivers to respond slower to hazards than older more experienced drivers (Borowsky et al., 2010, Wallis and Horswill, 2007, Wetton et al., 2010), and that sleepiness slows the detection of hazards, especially in novice drivers (Smith, Horswill, Chambers & Wetton, 2009). There have been reports of an insensitivity of hazard perception tests, and the cause of this inconsistency is unclear. Chapman and Underwood, 1998a, Chapman and Underwood, 1998b, Sagberg and Bjørnskau, 2006 found weak relationships between driving experience and hazard perception responses, and one possibility for the discrepancy with other results might lie with the types of hazards shown. Some hazards are abrupt and attention-capturing, as when a pedestrian steps into the roadway from behind a parked vehicle. These types of hazards, which were certainly used in the Chapman and Underwood studies, are potentially unavoidable and do not necessarily discriminate between good and bad drivers because they capture attention whatever is the driving experience of the observer. These abrupt or exogenous hazards differ from anticipated hazards that call upon level-three situation awareness, which require the driver to understand what might happen in the immediate future if other road users behave in hypothetical ways. These gradual onset hazards are more sensitive to driving experience, as when we notice that an oncoming car might move into our path in order to manoeuvre past a stationery obstacle, for example. It is possible that failures to report differences in hazard perception responses between novice and experienced drivers stems from the selection of the types of hazards for inclusion in the study. We will use responses to hazards and hazardous situations here as a method of comparing driving and driving-related behaviour on the road, with hazard perception tests, and in a driving simulator.