Driving simulator validation for speed research

https://doi.org/10.1016/S0001-4575(01)00056-2Get rights and content

Abstract

The behavioral validation of an advanced driving simulator for its use in evaluating speeding countermeasures was performed for mean speed. Using mature drivers, 24 participants drove an instrumented car and 20 participants drove the simulator in two separate experiments. Participants drove on roads which contained transverse rumble strips at three sites, as well as three equivalent control sites. The three pairs of sites involved deceleration, and were the approaches to stop sign intersections, right curves, and left curves. Numerical correspondence (absolute validity), relative correspondence (or validity), and interactive (or dynamic) relative validity were analyzed, the latter using correlations developed from canonical correlation. Participants reacted to the rumble strips, in relation to their deceleration pattern on the control road, in very similar ways in both the instrumented car and simulator experiments, establishing the relative validities. However, participants generally drove faster in the instrumented car than the simulator, resulting in absolute validity not being established.

Introduction

The use of a modern advanced driving simulator for human factors research has many advantages over similar real world or on-road driving research. These advantages include experimental control, efficiency, expense, safety, and ease of data collection (see Nilsson, 1993). As such, simulators are becoming increasingly attractive for this purpose. However, there are some possible disadvantages, including simulator sickness, accurate replication of physical sensations, and most importantly, validity.

Simulators must have appropriate validity to be useful human factors research tools. Blaauw (1982) has proposed two levels of validity. The first is the physical correspondence of the simulator's components, layout, and dynamics with its real world counterpart. Blaauw labeled this physical validity, but it is often referred to as a simulator's fidelity. The second level concerns correspondence between the simulator and the real world in the way the human operator behaves, which he called behavioral validity, although it is commonly referred to as predictive validity. It is often presumed that fidelity incorporates behavioral validity. Thus, simulator studies often report the physical correspondence, and usually do not mention, let alone analyze, the behavioral correspondence. In reality, however, the two levels are not always related (Blaauw, 1982).

Researchers often account for physical validity through a description of their driving simulator, citing its many aspects that reproduce real life driving. The closer a simulator is to real driving in the way the vehicle is driven, in the way stimuli are presented, and in the way it physically reacts to that stimuli, the greater the fidelity it is considered to have (Triggs, 1996). Therefore, a moving-base driving simulator is often assumed to have greater physical validity than a fixed-base simulator. Another reported measure of fidelity is lack of simulator sickness (e.g. McLane and Wierwille, 1975, Harms, 1996), although, whether this measure is meaningful is arguable.

Although fidelity of a driving simulator is attractive, often too much importance is placed on it. In the search for simulators with ever greater fidelity, it should be remembered that, ultimately, no level of physical validity is useful to human factors research if behavioral validity cannot be established. Accordingly, a more sophisticated (and therefore greater physically valid) simulator may not have more behavioral validity than a less sophisticated and expensive one. As such, it will not be more useful for behavioral research (Triggs, 1996).

If new research apparatus is used for conducting experiments whose findings will influence decisions made for real life situations, then it is important to know that the apparatus is eliciting similar responses as the normal real life situation. For driving simulator research, it needs to be shown that the particular simulator appropriately reproduces driving responses as they occur on the road.

As most advanced driving simulators are developed independently of each other, validity information is required for individual simulators. This is because different simulators have distinct parameters, including the time delay between driver action and simulator response, the amount of physical movement available, and the size and quality of the visual display (Nilsson, 1993). In addition, different tasks using a simulator obviously can have different levels of validity. Moreover, validation of an individual simulator using a single type of task is not adequate at all to argue the validity of a simulator and different simulator task.

That said, however, the accumulated evidence from different driving simulators and a range of driving tasks does add weight to the validity of simulator research. However, the number of published driving simulator validation studies is quite limited to date (Blaauw, 1982, Harms, 1996, Riemersma et al., 1990, Törnros, 1998, Carsten et al., 1997).

Blaauw (1982) argued that the most comprehensive method of undertaking behavioral validation research for simulators is a comparison between driving in the simulator and a real car, using tasks that are as similar as possible in the two environments. If the numerical values between the two systems are the same, then absolute validity can be claimed. A more pragmatic modification of this technique is to compare performance differences between experimental conditions in the simulator and a real car. Here, relative validity is established when the differences found between experimental conditions are in the same direction, and have a similar or identical magnitude on both systems. The difference between these two types of validities is illustrated in Fig. 1. In Panel A, it can be seen that the dependent variable being assessed is of the same magnitude for the simulator and car for the control condition, so absolute validity would be achieved. For the treatment condition, and for both conditions in Panel B, there are differences between the simulator and car, so absolute validity fails. Despite this, in Panel B, relative validity is attained because the relationship between the treatment and control conditions are the same for the simulator and car. In contrast, in Panel A these relationships are different so relative validity fails. Törnros (1998) observed that for a simulator to be a useful research tool, relative validity is necessary, but absolute validity is not essential. This is because research questions usually deal with matters relating to the effects of independent variables, with experiments investigating the difference between a control and treatment(s), rather than aiming to determine numerical measurements.

The current experiments involved the behavioral validation of the Monash University Accident Research Centre (MUARC) driving simulator. Based on Blaauw's (1982) two tiered approach, a three tiered approach was developed. This included an evaluation of absolute validity, relative validity (hereafter referred to as average relative validity), and interactive relative validity, which examined the similarity of drivers’ dynamic reactions to stimuli between experimental conditions. The study examined drivers’ speed responses to transverse rumble strips. These were chosen as experimental stimuli because the simulator was to be used for a research project examining similar speeding countermeasures (reported in Godley et al., 1999). Rumble strips are lines placed across a driving lane on the approach to a hazard in an attempt to slow driving speeds more than usual before the hazard is reached.

Driving speeds in response to rumble strips in an instrumented car were compared to driving speeds on the MUARC driving simulator. The instrumented car recorded driving performance through specified routes that included rumble strips at three sites, and three separate but equivalent control sites. These pairs of sites were a stop sign approach, a right curve approach, and a left curve approach. This study is referred to as the instrumented car experiment. The simulator responses were recorded, in a separate experiment (simulator experiment), by replicating the rumble strip sites and control sites used in the instrumented car experiment on the simulator.

Section snippets

Participants

There were 24 participants, 12 males and 12 females, ranging in age from 22 to 52 years, with an average age of 29.8 years. All participants were post-graduate students or staff from Monash University and were recruited through personal contact. Every participant had a full Victorian driving license with a minimum of 3 years driving experience. None had previous experience driving an instrumented car. They were paid $10 for their participation, which involved 3 h from the time they left the

Participants

Twenty participants, 12 males and 8 females, where involved in the simulator experiment. Their average age was 26.4 years, ranging from 22 to 40 years. All possessed a full Victorian driver's license. Participants were staff and post-graduate students at Monash University, and were recruited through electronic mail and internet news group postings. None of the participants had previous experience driving the simulator. They were not paid for their involvement, which lasted 1 h.

No participants

Data organization

The data for the two experiments were collected at a rate of 30 Hz, but were converted to an average speed for each meter of the track. Measurements for the data analysis were recorded during the rumble strip area up to the intersection or the commencement of the curve, as well as approximately the same distance of road immediately before the rumble strips started. A schematic illustration is displayed in Fig. 2.

Validation approach and data analysis

The analyses of the experiments examined simulator validation for both relative and

Results

It should be noted that the data from the instrumented car, displayed in Fig. 4a, Fig. 5a, and Fig. 6a, fluctuate to an extent between subsequent averages per meter. This was a result of less than perfect data recording in the instrumented car. In comparison, the simulator data, displayed in Fig. 4b, Fig. 5b, and Fig. 6b, was recorded very accurately and thus resulted in the smooth curves present in these figures.

Discussion

The aim of the above two experiments was to validate the MUARC driving simulator for research on speeding countermeasures. This validation consisted of three levels, averaged relative validation, interactive relative validation, and absolute validation, for the dependent variable speed. It was considered, a priori, that relative validation was more important than absolute validation to justify using the simulator for future experiments. A summary of the validation findings are presented in

Conclusions

There is evidence to conclude that speed is a valid measure to use for experiments on the MUARC driving simulator involving road based speeding countermeasures. The most important indicator of an effective countermeasure is that the speed profiles found indicate a speed reduction relative to control roads or other roads. For this purpose, speed has been clearly validated as a dependent variable for research using the simulator. The validity of speed only covers research investigating the

Acknowledgements

This study was supported by the Federal Office of Road Safety of Australia and the Roads and Traffic Authority of the state of New South Wales, Australia. This paper is based on part of Stuart Godley's Doctoral Dissertation. The instrumented car was arranged and operated by ARRB Transport Research Ltd. The authors would like to acknowledge Simon Moss for his statistics advice.

References (16)

  • S. Yagar et al.

    Geometric and environmental effects on speeds of 2-lane highways

    Transportation Research Part A

    (1983)
  • G.J. Blaauw

    Driving experience and task demands in simulator and instrumented car: a validation study

    Human Factors

    (1982)
  • Carsten, O.M.J., Groeger, J.A., Blana, E., Jamson, A.H. 1997. Driver Performance in the EPSRC Driving Simulator [LADS]:...
  • J. Cohen

    Statistical Power Analysis for the Behavioural Sciences

    (1988)
  • B.N. Fildes et al.

    Speed Perception 1: Drivers' Judgement of Safety and Speed on Urban and Rural Straight Roads, Report CR 54

    (1987)
  • S.T. Godley et al.

    Perceptual Countermeasures: Experimental Research, Report CR 182

    (1999)
  • Harms, L., 1996. Driving performance on a real road and in a driving simulator: Results of a validation study. In:...
  • H.J.W. Leong

    The distribution and trend of free speeds on two-lane two-way rural highways in New South Wales

    Proceedings of the Fourth Australian Road Research Board Conference

    (1968)
There are more references available in the full text version of this article.

Cited by (362)

View all citing articles on Scopus
View full text