An exploration of the psychology of the examinee: Can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance?

https://doi.org/10.1016/S0361-476X(02)00063-2Get rights and content

Abstract

This study examined the predictive power of self-regulated strategies and test-taking motivation on achievement performances under consequential and non-consequential test conditions. Sixty-two undergraduate students were asked to take two parallel classroom tests: one that counted towards their class grade (consequential) and one that did not (non-consequential). Each test consisted of a multiple-choice section and an essay. It was hypothesized that self-regulated strategies would predict both multiple-choice and essay performances in consequential and non-consequential contexts. Self-regulation strategy use significantly predicted both multiple-choice conditions; however it only predicted consequential essay performance. As was expected test-taking motivation did not predict test performances in consequential conditions but emerged in both non-consequential conditions. Study findings support the social cognitive self-regulation perspective and expectancy–value motivation theory.

Introduction

Academic achievement, performances, and learning are major constructs of interest to all educators and educational researchers. Demonstration of learning relies heavily on a variety of achievement tests, performance based assessments, and occasionally in-class assignments. For some testing conditions, the tests are administered in consequential test conditions, and the students are assumed to be highly motivated to perform well. These tests that have direct consequences for individuals completing them are termed ‘consequential,’ and are often referred to as ‘high stakes.’ However, for many important assessments of student achievement and learning, the tests are conducted in conditions for which no consequences or stakes are present. Non-consequential test conditions are those for which the test results have no bearing for individuals taking the test. These types of tests are often referred to as ‘low-stakes.’ For example, the tests administered for the National Assessment of Educational Progress (NAEP) and those administered for the International Education Association require students to participate in tests that may be of great importance to educators and policy makers world-wide, yet may have no importance to individual examinees. Similarly, many post-secondary institutions employ testing in conditions that bear no consequences for students; however, these test results are used to prepare reports for external stakeholders and accrediting agencies that inform judgments concerning institutional effectiveness (Gabbin, 2002; Olsen & Wilson, 1991; Sundre, 1997, Sundre, 1999; Wyatt, 2002). Many authors, such as Wainer (1993) and Ewell (1991) have raised concerns associated with the validity of inferences drawn from such test conditions, which would include many mandated national, state, and district educational assessments at both compulsory educational levels and those conducted within institutions of higher education.

Numerous research studies have been conducted to identify factors (e.g., rewards, motivation, dispositions) that may influence student test performances in non-consequential conditions across compulsory (DeMars, 2000; O’Neil, Sugrue, & Baker, 1996; Wolf, Smith, & Birnbaum, 1995) and post-secondary educational levels (Banta & Pike, 1989; O’Donnell, 1997; Sundre, 1997, Sundre, 1999; Wolf & Smith, 1995; Wolf, Smith, & DiPaolo, 1996). A construct that has emerged consistently in the literature as a factor that influences test performance in such conditions is examinee motivation. Examinee motivation in standardized tests has been shown to be positively associated with test performances in a variety of settings (Arvey, Strickland, Drauden, & Martin, 1990; Burke, 1991; Olsen & Wilson, 1991; Paris, Turner, Lawton, & Roth, 1991; Wolf & Smith, 1995; Wolf et al., 1995). The scope of the present study is to examine the impact of an additional construct commonly viewed as interdependent with motivation, self-regulation (Pintrich, 1988, Pintrich, 1989; Pintrich & De Groot, 1990; Zimmerman, 1989, Zimmerman, 2000). Given the findings of many researchers describing systematic differences in examinee dispositions toward tests (Jakwerth, Stancavage, & Reed, 1999; Olsen & Wilson, 1991; Zeidner, 1993), this study will also investigate the impact that test modality (multiple-choice vs. essay) has on test performances in non-consequential test conditions. To our knowledge this is the first attempt to investigate the role of self-regulation, motivation, and test modality in differential test conditions. This is an important area of study, because the major participant in assessment activities has been largely ignored. The psychology of the examinee is a critical factor in all assessment results and may assist in explaining discrepancies in performances emanating from different consequential test conditions and modalities.

Considerable theory and research has been generated pertaining to self-regulated learning, motivation, and academic achievement (Bandura, 1986; Pintrich, 1988, Pintrich, 1989; Pintrich & De Groot, 1990; Schunk & Zimmerman, 1994), providing support that (a) self-regulation and motivation are viewed as two interdependent processes (Zimmerman, 2000); and (b) that both are associated with high levels of academic achievement (Schunk & Zimmerman, 1994).

The interdependence of self-regulation and motivation is clearly illustrated in Zimmerman’s (1989) definition of self-regulated learning, which portrays self-regulated students as being metacognitively, motivationally, and behaviorally active participants of their learning process. Further, Zimmerman (2000) also emphasized the importance of the role of motivation in his recent three-phase model of self-regulation. Specifically in the first phase, the forethought phase, an individual’s goal attainment is highly related to one’s strategy choice and motivational beliefs. In the performance phase, self-monitoring of strategy sustains motivation and leads to accurate self-judgments of performance. Finally, in the self-reflection phase learner motivation is influenced by the attributions made for failure. Self-regulated learners attribute outcomes to strategy use thus, facilitating maintenance of self-motivation following failure (Zimmerman & Kitsantas, 1997).

The interdependence between self-regulation and motivation is also clearly illuminated by the work of Pintrich and De Groot (1990) in which the general expectancy–value model of motivation (Eccles, 1983; Pintrich, 1988, Pintrich, 1989) was adapted to suggest that cognitive and metacognitive strategies alone are not sufficient to promote student achievement. Students must be motivated to use self-regulation strategies. Pintrich and De Groot’s model was comprised of three motivational components that may be linked to self-regulated learning: an expectancy component, a value component, and an affective component. The expectancy component involves students’ beliefs about their ability to succeed at a task. The value component refers to students’ goals and consideration of the importance and their interest in the task. The affective component relates to students’ emotional reaction to the task.

In concluding, theories suggest that in the ideal condition both self-regulation and motivation are essential components for academic learning and achievement. Neither component alone is sufficient for successful demonstration of learning outcomes. For example, a student who exhibits a high degree of motivation and puts forward considerable effort toward study, but lacks self-regulated strategies may not be able to accomplish his/her academic goals. On the other hand, a student who possesses a great number of self-regulatory strategies but lacks motivation may elect not to employ them. Taking this a step further, a student may possess the ability to implement self-regulation strategies and the motivation to employ and sustain them to facilitate a high degree of learning, but when confronted with a testing situation, particularly one that has no direct consequences for them, a distinctly different set of motivational decisions may be made. All of the previous self-regulation and motivation phases described earlier can either be used to fuel a test performance that adequately represents the true level of learning achievement, or they can be dismissed. This represents a distinct, and we contend emotional, reaction to a given task. Testing conditions have provided a context for exploring the interdependence of self-regulation and motivation. As we shall see, researchers have consistently shown that whether or not a test is linked to consequences for individual examinees (e.g., course grades, SAT, employee selection tests) systematically impacts test-taking motivation and subsequent test performance. Exploration of non-consequential test conditions in relation to the constructs of self-regulation and motivation may increase our understanding of how these processes impact many ‘non-consequential’ situations that all of us, including our students, respond to on a daily basis.

The effect of differential self-regulation and examinee motivation levels on test performance is a concern for many testing conditions. The research to date in the area of self-regulation has been quite consistent showing that self-regulating students achieve higher test performances than low self-regulating students in consequential test conditions (Kitsantas, 2002; Schunk & Zimmerman, 1994). However, very little is known about the role of self-regulation in non-consequential test performances (Sundre & Kitsantas, 2001).

In regards to motivation, research studies have demonstrated the positive impact of test-taking motivation in consequential conditions; however, the negative impact of low motivation in non-consequential conditions on test performances has also been supported (Arvey et al., 1991; Burke, 1991; DeMars, 2000; Jakwerth et al., 1999; Olsen & Wilson, 1991; Sundre, 1997, Sundre, 1999; Wolf & Smith, 1995; Wolf et al., 1995; Wolf et al., 1996; Wyatt, 2002). This concern represents a recognized unresolved issue in testing (Wainer, 1993), particularly for non-consequential tests (Ewell, 1991), calling into question the validity of inferences and actions made on the basis of test scores (Messick, 1995). As O’Donnell (1997) pointed out cognitive theory suggests that individuals learn quite differently and performance is influenced by a multitude of factors; however, little recognition has been evidenced that students also respond to testing situations quite differently. Many researchers, administrators, and policy makers have expressed dismay in reviewing test results from non-consequential test administrations. Low motivation appears to be a factor. For example, Olsen and Wilson (1991) used follow-up interviews with ‘suspect’ scorers and identified low motivation as a key factor in explaining lower than expected ACT COMP scores among college sophomores. Banta and Pike (1989) reported student self-reports of low effort on state mandated higher education assessment activities. In a study investigating NAEP test scores, Jakwerth et al. (1999) reported that for low achievers, motivation was a pervasive problem. Indeed, Wolf and Smith (1995) demonstrated in an experimental study using college undergraduates that both the motivation to perform well on examinations and actual test performance are directly affected by the presence of test consequences. Further, O’Neil et al. (1996) in studies with 8th and 12th graders found that a lack of consequences or “stakes” on assessment tests results in a lower degree of motivation. Consequently, test takers demonstrate less effort, which in turn produces poorer performance. In addition, the nature of the task (selected response vs. constructed response) required has been shown to differentially impact student attitudes, motivation, and performance (DeMars, 2000; Sundre, 1996, 1999; Wolf et al., 1995; Zeidner, 1993). Hence, not only do individual consequences for examinees affect test performances, task demand has been shown to further impact examinee affect and diminishes test performances.

The application of the Pintrich and De Groot (1990) expanded expectancy–value model of motivation provides a clear theoretical basis by which consequential vs. non-consequential test conditions and tasks with distinctly different cognitive demand can be compared. Namely, non-consequential test conditions present individuals with expectancy, value, and affect considerations that may undermine willingness to perform optimally. A non-consequential test condition, particularly one requiring students to construct responses in an essay format, may present individuals with tasks deemed of little importance, or requiring greater effort than they may be willing to exert. Therefore, having the skills and knowledge necessary to successfully complete a task are not sufficient; an affective component that assesses the learner’s emotional reaction to the task must be added to the learning model (Pintrich & De Groot, 1990).

Considerable research has described student affect and attitudes toward tests and their modalities. The literature is replete with quite negative affective descriptors of student reactions to tests. Olsen and Wilson (1991) found many college students expressed considerable “dislike” for standardized tests. Paris et al. (1991) used the terms “disillusionment,” “cynicism,” “suspicious,” “try half-heartedly,” and “skepticism” to describe student views toward tests and cautioned that a “psychological perspective” is necessary to appropriately frame test results. Paris and colleagues also found that the attitudes of school children towards standardized tests appear to become more negative over the school years. Kellaghan et al. (1982) reported similar results in Ireland where they found significant numbers of “disaffected” students who “did not care,” “felt nervous,” or were “bored.” Eggleston (1988) referred to college student attitudes toward standardized tests as tending toward “apathy.” Burke (1991) advised after observation and interviews with teachers and students that NAEP scores should be considered as “displays” rather than ability or grasp given the high item omission rates and negative affect reported. Clearly, many students have reactions to tests that are affective in nature and with great potential for negative impact on performance.

A review of the literature consistently indicates that students prefer multiple-choice tests to constructed response test formats (Jakwerth et al., 1999; Zeidner, 1993). These findings are clarified by additional studies that demonstrate that not only do students dislike examinations with great task demand, performances drop significantly when they are employed in non-consequential test contexts. For example, Wolf et al. (1995) and Sundre (1999) provided evidence that the difficulty and effort necessary to complete tasks impacted both motivation and performance. DeMars (2000) demonstrated significant decline in test performances for non-consequential test conditions across both multiple-choice and constructed response format test items, but also revealed an interaction effect yielding a greater decline for constructed response format item performance. These results suggest strongly that (a) both test consequences and test modality have a systematic impact on both student attitudes and actual test performance; and that (b) are also consistent with theoretical expectations based on self-regulation and expectancy–value motivation theories. However, not all students elect not to exert effort on tasks for which no consequences are present. Many students put forth consistent effort and perform well regardless of the presence or lack of consequences. In light of current public policies, it is important to try to explore the explanatory power of current self-regulation and motivation theory in these test conditions. Perhaps inconsequential for the student, the results of these assessments are of major consequence for educators, the general public, and policy makers.

To test the theorized roles of self-regulation and motivation, this study employed context specific measures of self-regulation strategies and examinee motivation across a series of experimental conditions in which test consequences (counts toward grade, does not count toward grade) and test modality (multiple-choice and essay) were manipulated. Measures of self-regulation and test-taking motivation were collected to predict performance across experimentally manipulated testing conditions. All three components of the expectancy–value motivation model were included. Expectancy for success on the tasks was held constant, because all students had equal preparation for the two parallel form in-class examinations. Value was experimentally manipulated by presenting one of the parallel examination forms as consequential and the other as non-consequential. Collecting test-taking motivation responses upon completion of each test section assessed task affect, student reactions to each task. The interdependent roles of self-regulation and test-taking motivation were tested through the formulation of the following four hypotheses. First, since self-regulation has been shown to predict achievement and all students would be uniformly motivated to perform well, it was anticipated in the consequential multiple-choice test condition that self-regulated strategies would explain significant test performance variability but motivation would not. Second, it was postulated that in the non-consequential multiple-choice test condition both self-regulation strategy use and motivation would explain significant variability in test performance. The task is easy to perform, and all are prepared. Those with greater self-regulation should find the task easier, but in a non-consequential test condition, a positive reaction to the task is critical to trigger the willingness to display performance. Moving to the essay test conditions, it was predicted in the consequential essay condition that self-regulated strategies would predict significant variance but motivation would not. Previous research indicates that a complex task, such as writing an essay benefits greatly from the use of self-regulation strategies. Test-taking motivation is not expected to predict performance in the consequential condition, because all students would be highly motivated and would react positively to the task. The fourth hypothesis indicated that in the non-consequential essay test performance, self-regulation strategies and test-taking motivation would explain significant variability in test performances. In this condition, despite the greater task demand of an essay performance, self-regulated learners would have greater opportunity to demonstrate learning, and those with higher test-taking motivation would react positively to the task triggering willingness to display learning.

Section snippets

Participants

Participants were 62 college students (50 females and 12 males) enrolled in an undergraduate psychology of personality course at a comprehensive state institution. All students were university psychology majors. A total of 77 students were invited to participate in the study; however, 11 dropped the course and 4 others did not complete all aspects of data collection. Two of the students (3%) were sophomores, 40 (65%) were juniors, and 20 (32%) were seniors. The participant ages ranged from 18

Results

Descriptive statistics for the coded self-regulation interview data, all motivation scores and reliabilities, and the achievement test scores are presented in Table 2. The alpha reliability estimates for the total test-taking motivation scores ranged from a low of .81 in the consequential multiple-choice condition to a high of .93 in the non-consequential essay task condition. Table 3 provides the intercorrelations for all variables used in the regression analyses.

The four sets of regression

Discussion

The purpose of the present study was to assess the roles of self-regulation and test-taking motivation within the framework of academic achievement in differing test-taking contexts and modalities. Overall, the results provide considerable support for the interdependence of the self-regulation and motivation theory as outlined by Zimmerman (2000), and Pintrich and De Groot (1990) showing that self-regulation strategy use is predictive of higher achievement across task demands and consequences.

References (37)

  • B.J. Zimmerman

    Attaining self-regulation: A social cognitive perspective

  • R.D. Arvey et al.

    Motivational components of test taking

    Personnel Psychology

    (1990)
  • A. Bandura

    Social foundations of thought and action: A social cognitive theory

    (1986)
  • T.W. Banta et al.

    Methods of comparing outcomes assessment instruments

    Research in Higher Education

    (1989)
  • Burke, P. (1991). You can lead adolescents to a test you can’t make them try: Final report (Contract No....
  • C.E. DeMars

    Test stakes and item format interactions

    Applied Measurement in Education

    (2000)
  • J. Eccles

    Expectancies, values & academic behaviors

  • P.T. Ewell

    To capture the ineffable: New forms of assessment in higher education

    Review of Research in Education

    (1991)
  • A. Gabbin

    Experience with using samples to assess students in the major

    Assessment Update

    (2002)
  • Jakwerth, P. R., Stancavage, F. B., & Reed, E. D. (1999). An investigation of why students do not respond to questions....
  • A. Kitsantas

    Test preparation and test performance: A self-regulatory analysis

    Journal of Experimental Education

    (2002)
  • S. Messick

    Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning

    American Psychologist

    (1995)
  • O’Donnell, A. M. (1997). Self-adapted testing: Fitting the test to oneself. Paper presented at AERA national...
  • Olsen, S. A., & Wilson, K. (1991). A follow-up of suspect sophomore scores on the COMP test. Kirkville, Missouri:...
  • J.H.P. O’Neil et al.

    Effects of motivational interventions on the National Assessment of Educational Progress mathematics performance

    Educational Assessment

    (1996)
  • S.G. Paris et al.

    A developmental perspective on standardized achievement testing

    Educational Researcher

    (1991)
  • E.J. Pedhazur et al.

    Measurement, design, and analysis

    (1991)
  • P.R. Pintrich

    A process-oriented view of student motivation and cognition

  • Cited by (0)

    View full text