Randomized Controlled Trial of the Effects of Order of Item Administration on Parent Ratings of Child and Adolescent Dispositions and Psychopathology

The effects of serial order of presentation on parent ratings of items referring to the behavior of their children were examined using two scales measuring dispositions and psychopathology that were administered to a representative sample of 1,358 4–17 year olds. Items within each scale were pseudo-randomized instead of being grouped according to the constructs they were written to measure. Each scale was presented in either forward or reverse serial order, with parents who were unaware of all hypotheses being randomly selected for forward or reverse order of administration. In both the disposition and psychopathology scales, the same items were rated significantly differently depending on the order of presentation to the raters; ratings declined in magnitude as the parents rated increasing numbers of items. Although parent ratings presumably partly reflect the behavior of their children, they are biased by robust order effects. The unbiased counter-balancing of items in scale construction and the use of randomized forward versus reverse orders of scale administrations can be used to minimize extraneous order effects.