Predicting transfer from training performance
Introduction
Although the overriding goal of research in skill acquisition and transfer of training has been to understand how humans acquire and perform a variety of skills, more applied goals also exist. One of these is to determine the relationship between training and transfer tasks and performance on these tasks with the ultimate aim of developing a set of principles whereby training can be used to predict transfer. Given that this endeavour has occupied at least a century, it is appropriate to consider the extent to which current understanding of skill acquisition enables transfer to be predicted on the basis of training performance.
One approach to the issue of transfer that has been successful in recent years has been to assume that skill acquisition involves the acquisition of production rules (e.g., Anderson, 1982; Newell, 1989). These are rules that associate particular stimulus conditions with appropriate actions (e.g., IF X-ray contains feature A, feature F and feature M, THEN consider diagnosis X). In addition, these productions form the basis of transfer. The extent of transfer between two tasks is determined by the extent to which productions developed in the context of one task can be used in performance of the other task. The greater the production overlap, the greater the transfer. This theoretical perspective has received some empirical support in studies where transfer is predicted on the basis of an analysis of the production rules acquired during training and the production rules necessary for performance of the transfer task (e.g., Elio, 1986; Frensch, 1991; Kieras & Bovair, 1986; Singley & Anderson, 1989).
The view that transfer between tasks is a function of production overlap suggests that skill components develop as encapsulated wholes. That is, regardless of the situation, they will apply whenever stimulus conditions are appropriate, much like sub-routines in a computer program that apply whenever they are called and particular parameters possess appropriate values. This view has provided a useful method of predicting transfer in many situations. However, not all transfer situations can be handled by this rather simple view. In essence this view holds that fluent task performance involves fluent execution of component steps. Effective training, then, involves the acquisition of component knowledge. The problem with this view is that it appears to be contradicted by research into the part–whole problem. This problem is one of the most intriguing issues in the transfer of training literature, not least because it has perplexed researchers for the better part of this century. The issue concerns the effectiveness of whole-task training versus part-task training. Whole-task training involves practice with the whole task from the beginning of training. Part-task training involves practicing parts of a task in isolation and then combining these when performance has reached a particular level of proficiency. The rationale for partitioning a task for training purposes is usually obvious. For example, a task may be so complex as to overwhelm the cognitive resources of a novice, and this may result in little progress with practice. Reducing the complexity of such a task by dividing it into parts and providing separate training on each part may allow the novice to gain sufficient expertise at these sub-tasks to be able to attempt and practice the whole task (Gopher, Weil, & Siegel, 1989; Mané, Adams, & Donchin, 1989). The part–whole problem concerns which form of training is best.
Unfortunately generalisations are not straightforward when comparing part- and whole-task training (Adams, 1987; McGeoch, 1952), although whole-task training is generally thought to be superior (Adams, 1987; Gopher et al., 1989; Wightman & Lintern, 1985). Consensus is qualified however because the value of part- and whole-task training may be task dependent. For example, Welford (1968) suggested that whole-task training is the most efficient method for tasks which involve interrelated activities, such as flying an aircraft, whereas tasks which consist of independent components performed in a fixed order generally benefit most from part training.
The view that transfer is a function of production overlap suggests that part- and whole-task training should be equivalent in preparation for target task performance. In other words, it predicts that training with task components individually or in the context of the whole task will result in equivalent component knowledge (i.e., productions) and therefore equivalent transfer to the target task. The fact that part- and whole-task training are rarely equivalent in terms of transfer demonstrates that this view is too simplistic.
Anderson's (1982) ACT* theory avoids this problem by proposing that the context in which component knowledge is acquired is also important in determining transfer. According to ACT*, skill acquisition involves a composition mechanism that takes into account training context. Composition describes the process whereby several productions are collapsed into a single production. These productions must occur in a sequence and share the same overall goal. The new production now does the work of the sequence, but in fewer steps, which therefore results in faster performance. On the basis of this mechanism it can be predicted that where task components are to be performed in a fixed sequence, there will be an advantage in training with the sequence rather than training with the components individually or in some other sequence. Research supporting this prediction has been reported by Elio (1986) and Frensch (1991). At first glance, this more developed view of transfer suggests that whole-task training should be superior to part-task training because it will typically preserve the sequence of operations involved in a target task. However, if important task sequences can be identified and used as the basis for partitioning a task into components, then part-task training may not be so disadvantaged by the lack of opportunity for composition.
Given that transfer has been successfully predicted in some situations on the basis of a production system analysis of training and transfer tasks, it is important to consider the degree of success involved in this prediction. In other words, to what extent is it possible to predict transfer performance based on training performance? In all of the research cited here, prediction of transfer performance has been relative. That is, prediction has only been at the level of stating which conditions will result in the best transfer performance. Explicit prediction of performance has typically been post hoc – transfer performance is measured and regression equations are constructed to `describe' performance on the basis of features of training and transfer tasks.
There have been some attempts to predict the absolute level of transfer performance on the basis of training performance. In work by Speelman and Kirsner (1993), Speelman (1995), participants practised a training task, and then performed a transfer task that was basically a modification of the training task. The transfer task was constructed such that the skills developed to perform the training task could be executed in the transfer task in the same way as in the training task, at least logically. It was assumed that performance of the transfer task would require execution of all of the component skills of the training task, plus some new skills that would need to be acquired. The aim was to predict the improvement in performance time on the transfer task that resulted from practice on the basis of a power function that described training performance. There were two major assumptions in this approach: (a) new skills would be acquired at the same rate as the old skills were acquired; (b) old skills would continue to improve in the context of the new task according to the power function that described their improvement during the training task. This approach was able to predict the overall pattern observed: a dramatic slowing of performance followed by a gradual improvement with practice. However, it was only minimally successful at predicting transfer performance, requiring several ad hoc modifications to better approximate observed performance. In general these modifications were attempts to account for performance times during transfer that were typically slower than initially predicted.
The aim of this paper is to examine the reasons for the failure of Speelman and Kirsner's approach to predicting transfer, by addressing one of the assumptions underlying their approach. The assumption addressed here is that when old skills are executed in the context of new tasks, they continue to improve as if stimulus conditions had not changed (i.e., their improvement will continue to follow the training power function). This assumption was borrowed from Anderson's (1982) ACT* theory. According to this theory, the composition mechanism results in skill units representing component knowledge. These units are the basis of transfer to new tasks. The more units that are useful for performance of the target task, the greater the transfer. Furthermore, improvement on these task components is described by power functions (Anderson, 1982; Newell & Rosenbloom, 1981). Presumably if the same productions can be executed to perform these task components in both the training and transfer tasks, the speed with which they will be executed in the transfer task should be predicted by extrapolating the training power functions. In other words, given that practice determines the speed with which productions are executed, and stimulus conditions determine whether a production is executed, if a transfer task involves the right stimulus conditions for a production to be executed, the speed with which it does so will be determined by its previous execution history, as described by the training power function.
The prediction concerning the behaviour of old skills in new tasks relies on the assumption that component task knowledge behaves as encapsulated wholes, or sub-routines that can be applied in many different situations. However, recent research has demonstrated that composed component knowledge does not always behave in this manner. Elio (1986) reported that task performance was more dependent on knowledge of how various task components were to be integrated than on component knowledge. In fact, in Elio's experiments, where participants performed sequences of simple calculations, participants were faster at executing new component steps in an old integrative structure than they were at performing old, well-practised component steps in a new integrative structure. Other research by Thomas (1974) has demonstrated that participants trained to solve the second half of a complex problem enjoyed no performance benefit over naive participants when it came to solving the whole task. Furthermore, this same study demonstrated that, in the context of the whole task, the trained participants were no better than the untrained participants on the task component with which they were trained. Carlson, Sullivan, and Schneider (1989) trained participants to perform judgements with electronic logic gates. They found that practice with individual gates provided little transfer to more complex problems that involved making the same types of judgements about the same gates. Participants were slower at making the component judgements in the context of the problem-solving task than when they were made in isolation during training.
In summary, composed skills appear not to function as encapsulated routines that can be executed in any task context. Rather, composed skills, to some extent, appear to be tied to the context in which they are acquired. This suggests that when old skills are performed in new tasks, improvement on these skills will not follow the power functions that describe their original improvement. This issue was not examined in the above research, but forms the focus of the research reported in this paper. Thus the overall aim of the current research was to examine the extent to which transfer performance can be predicted from training performance. This aim was achieved by attempting to answer the empirical question of whether or not old skills continue to improve in the context of new tasks according to the power functions that describe their original improvement.
Three experiments were performed to achieve the aim of testing the assumption regarding old skills in new tasks. Experiment 1 was designed to provide a simple and clear test of this assumption. The results suggested that, contrary to the assumption, old skills performed in a new task do not continue to improve as if nothing changed. Experiment 2 was designed to examine whether this disruption was due to an increase in complexity in the task from training to transfer, or simply due to any change in task. Experiment 3 was designed to examine two variables that may affect the magnitude of this effect: the relative change in task complexity from training to transfer, and the amount of practice on a task prior to a change in task.
Section snippets
Experiment 1
A task was designed that involved several discrete components. The structure of the task was such that performance on each component was, at least logically, independent of performance on other components. That is, task components were arranged sequentially so that performance of one component began when performance on the previous component was complete. Two versions of the task were developed that involved different combinations of these components. One version involved three major
Participants
Twenty-four volunteers from the first-year Psychology course at the University of Western Australia participated in the experiment. The participants were paid $5 per session.
Apparatus
Presentation of the experimental task and collection of data was controlled by Hypercard software run on either a Macintosh IIci computer or a Macintosh Quadra 700 computer.
Task and procedure
The task involved a fictional water analysis procedure and was based on one described in Elio (1986). Participants were required to perform simple
Measures
For the purposes of analysis the task was divided into sub-tasks corresponding to the individual calculations. For each calculation response time and accuracy were recorded. Performance was analysed in blocks of ten trials. Accuracy is defined here as the number of correct calculations in a block. Five accuracy measures were analysed: Particulate Rating accuracy (PRAcc), Mineral Rating Accuracy (MRAcc), Content Index Accuracy (CIAcc), Marine Hazard Accuracy and Overall Quality Accuracy. Several
Discussion
In this experiment, the main aim was to examine the effect on performance of performing old skills in the context of a new task. During the transfer phase the experimental group performed the same three calculations per trial that they had performed in the training phase. However, this group also had to perform two additional calculations on each trial during this second phase. The results indicate, that in comparison with a group who performed the same three calculations in both phases of the
Experiment 2
The design of Experiment 2 was basically the same as Experiment 1. Two groups of participants performed the water analysis task for several sessions. The control group performed the three-calculation version throughout the experiment. The experimental group performed the five-calculation version during training, and then performed the three-calculation version during transfer. In this way, the change in task involved a reduction in complexity, from five calculations per trial to three. If the
Participants
Twenty-four undergraduate students from the University of New England participated in this experiment. The participants were paid $5 per session. Participants were randomly assigned into one of two groups of 12.
Apparatus
Presentation of the experimental task and collection of data were controlled by Hypercard software run on a Power Macintosh 6100/66 AV computer.
Procedure, task and design
The procedure, task and design of Experiment 2 were identical to Experiment 1, except for two features. The first difference concerned the tasks
Accuracy
Three 2 (group)20 (block) mixed design analyses of variance were performed on the three accuracy measures representing performance on the first three calculations. These analyses indicated that the experimental group (M=9.0) was less accurate than the control group (M=9.7) on PRAcc (F(1,22)=5.649, P<0.05, MSe=8.079), but that the other accuracy differences between the groups were not significant (MRAcc: Exp M=9.4, Cont M=9.7, F(1,22)=1.194, P>0.05, MSe=4.537; CIAcc: Exp M=9.2, Cont M=9.6, F
Discussion
Although the change in task in this experiment resulted in slower performance by the experimental group, the disruptive effect of a change in task was not as obvious as in Experiment 1. The analyses examining performance before and after the change in task indicated clearly that the transition resulted in slower performance. However, comparisons with the control group, and attempts to predict transfer performance by extrapolating from training performance revealed that the disruption in
Experiment 3
Experiment 3 was designed to examine the possibility that the amount by which old skills are disrupted by task changes is related to the amount by which a task is changed. Previous research suggests that this may be the case. For example, Carlson et al. (1989) found that performance on task components previously practised individually was slowed when the components were combined in a target task. This result is similar to the effect observed in Experiments 1 and 2. However, they also found that
Participants
Forty-eight volunteers from the first-year Psychology course at the University of Western Australia participated in the experiment. The participants were paid $5 per session.
Procedure and apparatus
These were identical to Experiment 1.
Task
The two versions of the water analysis task used in Experiment 1 were used in this experiment. Additional versions were also used. Some of these involved six calculations, five of which were those involved in the transfer task of Experiment 1. The sixth calculation involved deriving a
Accuracy
Two sets of analyses were conducted on the Accuracy data from the training phase. The first analysis examined all groups together on the first 15 blocks of trials. The second set compared those groups who were exposed to equivalent amounts of practice during this phase.
Three 2 (length of training)3 (difference in number of calculations between training and transfer tasks)15 (block) mixed design analyses of variance were performed on the three accuracy measures common to all conditions. That
Discussion
Two factors were manipulated in this experiment to examine their effect on the amount by which performance is disrupted by a change in task. One of these factors was the relative number of calculations in the training and transfer tasks. The results suggest that when the difference between training and transfer tasks was only one calculation, performance of old skills was unaffected. However, when the difference between training and transfer tasks was two or three calculations, performance of
General discussion
The main aim of the experiments described in this paper was to examine the extent to which transfer performance could be predicted on the basis of training performance. A basic assumption underlying this effort was that when old skills are executed in the context of new tasks, they continue to improve as if stimulus conditions have not changed. That is, power functions that describe improvement on old skills during their initial acquisition should predict further improvement on these skills
Acknowledgements
We thank two anonymous reviewers for providing useful comments on an earlier version of this paper. All correspondence regarding this article can be directed to the first author at the School of Psychology, Edith Cowan University, 100 Joondalup Drive, Joondalup 6163, Western Australia, [email protected]. This research was supported by funding from the Australian Research Council.
References (27)
Representation of similar well-learned cognitive procedures
Cognitive Science
(1986)- et al.
The role of information reduction in skill acquisition
Cognitive Psychology
(1996) - et al.
Practice under changing priorities: an approach to the training of complex skills
Acta Psychologica
(1989) - et al.
The acquisition of procedures from text: a production-system analysis of transfer of training
Journal of Memory and Language
(1986) - et al.
Adaptive and part–whole training in the acquisition of a complex perceptual-motor skill
Acta Psychologica
(1989) - et al.
Concurrence costs in double stimulation tasks
Acta Psychologica
(1981) Historical review and appraisal of research on the learning, retention, and transfer of human motor skills
Psychological Bulletin
(1987)Acquisition of cognitive skill
Psychological Review
(1982)- et al.
Practice and working memory effects in building procedural skill
Journal of Experimental Psychology: Learning, Memory and Cognition
(1989) - et al.
Practice schedules and subgoal instantiation in cascaded problem solving
Journal of Experimental Psychology: Learning, Memory and Cognition
(1996)
Transfer of composed knowledge in a multistep serial task
Journal of Experimental Psychology: Learning, Memory and Cognition
Shapes of reaction-time distributions and shapes of learning curves: a test of the instance theory of automaticity
Journal of Experimental Psychology: Learning, Memory and Cognition
The psychology of human learning
Cited by (15)
The effect of constant versus varied training on transfer in a cognitive skill learning task: The case of the Tower of Hanoi Puzzle
2016, Learning and Individual DifferencesHow drivers adapt to drive in driving simulator, and what is the impact of practice scenario on the research?
2013, Transportation Research Part F: Traffic Psychology and BehaviourCitation Excerpt :The disruptive effect of combining new and old tasks has been observed previously (e.g. Elio, 1986; Rickard, 1997) and is known as the context effect on learning and transfer. As suggested by Miller and Paredes (1990) and Speelman and Kirsner (2001), learning a skill in a new context forces the subject to rethink existing strategies and therefore, to reorganize previous knowledge, which in turn results in a temporary disruption of performance. Such disruption usually does not last long, and in most cases, practice of the new task will result in a quick transfer to the new task.
Health-Training Intervention for Community-Dwelling Elderly in the SeniorWISE Study
2010, Archives of Psychiatric NursingIndividual Differences in the Evolution of Counting: Evidence from Transfer in Alphabet Arithmetic
2022, Experimental PsychologyA Decay-Based Account of Learning and Adaptation in Complex Skills
2021, Journal of Experimental Psychology: Learning Memory and Cognition