Exploring how relevance instructions affect personal reading intentions, reading goals and text processing: A mixed methods study

https://doi.org/10.1016/j.cedpsych.2009.12.001Get rights and content

Abstract

The purpose of this mixed methods study was to investigate how relevance instructions influence readers’ personal reading intentions, reading goals, text processing, and memory for text. Undergraduates (n = 52) were randomly assigned to one of three pre-reading relevance instruction conditions that asked them to read from a perspective or to read for understanding. Experimental results showed that information was read slower and remembered better when it was relevant. However, some readers spent more time reading irrelevant information, whereas others spent less time reading this information. Post-reading interviews were analyzed to explain these reading time differences. The interview data indicated that relevance instructions influenced readers’ goals and the strategies they used to meet those goals. The data sets were complementary: the quantitative data indicated differences in reading time and recall, and the qualitative data allowed us to explain why these differences occurred. These data revealed three distinct reader profiles within and across conditions, and demonstrate how relevance instructions affect reader goals, processing, and comprehension.

Introduction

Educators routinely assign readings to students to promote learning. An assigned reading’s value may be suboptimal if teachers do not adequately cue readers to the purpose for reading a particular text. Similarly, readers may not be capable of discerning subtle cues meant to guide their processing of text, even if cues are provided (Lorch & van den Broek, 1997), or their general approach to a reading task may be maladaptive or non-responsive to the instructional guidance. Thus, while assigned readings can be a powerful tool for enhancing academic performance, teachers may not provide cues for the purpose for reading a text as effectively as they could. In this article we further explicate and examine McCrudden and Schraw’s (2007) goal-focusing model of relevance which provides an explanation for how relevance instructions (i.e., cues that provide criteria for determining information’s relevance to a particular reading task) affect reading goals, processing, and learning. This model informed the present study and afforded predictions about how readers use relevance instructions to learn from text, and about the effects of more or less explicit relevance instructions within a text-based environment. We situated the present study within the context of reading to learn.

The present study adds to the literature in two ways. First, we provided insights into how relevance instructions affect readers’ personal intentions, goals, processing, and memory. Previous research indicates that relevance instructions affect readers’ attention toward relevant and irrelevant information and their memory for this information (e.g., Goetz et al., 1983, Kaakinen et al., 2002, McCrudden et al., 2005). However, there is little research assessing how readers use relevance instructions to form reading goals and enact strategies to meet these goals. The present study replicated attention and memory outcomes from previous studies, but added a qualitative component that enabled us to examine in detail how readers described using relevance instructions to form reading goals and their strategies for processing the text. Thus, the present study not only examines how individuals read text and remembered information, but also why they read text as they did.

Second, this study demonstrates the value of a mixed methods design in examining how relevance instructions influence text learning in particular and its role in cognitive research in general. The central premise of mixed methods research is that “the use of quantitative and qualitative approaches in combination provides a better understanding of research problems than either approach alone” (Creswell & Plano Clark, 2007, p. 5). We used a mixed methods design to triangulate measures of attention and memory with interview data. By mixing the quantitative data from the experiment with the qualitative data from the interviews, we provided comprehensive evidence to describe how relevance instructions affect text learning. For example, we discovered that readers who received the same relevance instructions described enacting different strategies for meeting their goals, which was corroborated by processing and comprehension data. Thus, by using a mixed methods design we were able to identify important differences in readers’ goals and strategies that otherwise would have been hidden. Next we describe the goal-focusing model in greater detail (see Fig. 1).

Section snippets

Goal-focusing model of relevance

Reading is an intentional act (Kulikowich & Alexander, 2010) and reading in educational settings involves both personal intentions and given intentions (Graesser et al., 1994, van den Broek et al., 1995). Personal intentions are internally-generated standards that readers hold for understanding text, such as standards of coherence. Standards of coherence “reflect a reader’s knowledge and beliefs about what constitutes good comprehension as well as the reader’s specific goals for reading the

The present study

The purpose of the present study was to investigate how relevance instructions influence readers’ personal intentions, goals, processing, and memory for text by using online and self-report data from the same readers. We used an embedded, sequential mixed methods design to examine both quantitative and qualitative aspects of text processing (Creswell & Plano Clark, 2007). We selected this type of design because it is ideally-suited to provide insights into experimental findings (Creswell and

Quantitative phase

Participants in the experimental conditions received perspective relevance instructions (i.e., Imagine you will be moving to Pitcairn/Honduras for several years, determine the good and bad sides of your new home country from the perspective of a research scientist) and then read a text that described four remote countries. The relevance instructions typify perspective instructions because readers were asked to examine the text from a designated reference point (i.e., moving to a designated

Participants

Fifty-two undergraduates from a southeastern university in the United States participated for extra credit for their introduction to educational psychology course. The students were juniors (69%) and seniors (29%), with 2% sophomores. Approximately 81% of the participants were female. Ages ranged from 18 to 45 years (M = 24.5, SD = 6.5). The students were primarily European–American 69%, 8% were African–American, 2% were Latino, 12% of the students indicated that they were other ethnicities, and 10%

Results: quantitative phase

We conducted a preliminary analysis to determine whether there were differences in vocabulary knowledge among the three conditions. A one-way analysis of variance (ANOVA) with relevance instructions (Pitcairn, Honduras, or control) as the independent variable and vocabulary test score as the dependent variable showed no significant differences on the vocabulary test scores across the three conditions, F (2, 49) = .313, p = .733. Thus, vocabulary knowledge scores were not included in subsequent

Analysis of reading time data and formation of qualitative groups

Extreme-case sampling is a form of purposive sampling used in mixed methods research for comparing different types of cases along a specified dimension to provide information about a topic of interest (Teddlie & Yu, 2007). This type of comparison involves determining a dimension of interest, identifying a distribution of cases or individuals on that dimension, and then locating extreme cases.

Our primary dimension of interest was reading time for irrelevant sentences. Although students in the

Qualitative phase

Researchers often use qualitative procedures to explain or follow-up initial experimental results (Creswell and Plano Clark, 2007, Igo et al., 2008). In the context of the present study, we used the qualitative data to further explore how readers responded to the relevance instructions (Igo et al., 2008, Miles and Huberman, 1994, Tashakkori and Teddlie, 1998, Teddlie et al., 1989). In keeping with this method, we first analyzed the qualitative data set prior to mixing the quantitative and

Mixing of qualitative and quantitative data

The initial quantitative analyses revealed that relevance instructions affected resource allocation and learning. We were interested in explaining why readers spent more or less time reading different types of segments, and irrelevant information in particular. Follow-up qualitative interviews revealed three general goal-focusing strategies, which included narrowing, broadening, and familiarity. We combined the results of the quantitative (i.e., reading time and recall) and qualitative (i.e.,

Discussion

The purpose of the present study was to investigate how relevance instructions affect goals, processing, and learning. Although there is considerable evidence that relevance instructions affect processing and memory outcomes, little is known about how personal intentions interact with given intentions or how readers use relevance instructions to set goals and implement strategies to meet those goals. Our focus on the impact of goal setting and strategy implementation provided a basis for

References (67)

  • M.G. Cataldo et al.

    Why are poor comprehenders inefficient searchers? An investigation into the effects of text representation and spatial memory on ability to locate information in a text

    Journal of Educational Psychology

    (2000)
  • R. Cerdán et al.

    The effects of tasks on integrating information from multiple documents

    Journal of Educational Psychology

    (2008)
  • A.S. Chulef et al.

    A hierarchical taxonomy of human goals

    Motivation and Emotion

    (2001)
  • N. Coté et al.

    Building representations of informational text: Evidence from children’s think-aloud protocols

  • J.W. Creswell et al.

    Designing and conducting mixed methods research

    (2007)
  • A.E. Cunningham et al.

    Early reading acquisition and its relation to reading experience and ability 10 years later

    Developmental Psychology

    (1997)
  • P.D. Eggen et al.

    Educational psychology: Windows on classrooms

    (2010)
  • R.B. Ekstrom et al.

    Kit of factor referenced tests

    (1976)
  • E. Fox

    The role of reader characteristics in processing and learning from informational text

    Review of Educational Research

    (2009)
  • M.A. Gernsbacher

    Language comprehension as structure building

    (1990)
  • E.T. Goetz et al.

    Reading in perspective: What real cops and pretend burglars look for in a story

    Journal of Educational Psychology

    (1983)
  • A.C. Graesser et al.

    Constructing inferences during narrative text comprehension

    Psychological Review

    (1994)
  • J.T. Guthrie et al.

    Increasing reading comprehension and engagement through concept-oriented reading instruction

    Journal of Educational Psychology

    (2004)
  • J. Hyönä et al.

    Individual differences in reading to summarize expository text: Evidence from eye fixation patterns

    Journal of Educational Psychology

    (2002)
  • L.B. Igo et al.

    Exploring differences in students’ copy and paste decision-making and processing: A mixed methods study

    Journal of Educational Psychology

    (2005)
  • L.B. Igo et al.

    Should middle school students with learning problems copy and paste notes from the Internet? Mixed methods evidence of study barriers

    Research on Middle Level Education Online

    (2009)
  • L.B. Igo et al.

    Individual differences and intervention flaws: A sequential explanatory study of college students’ copy-and-paste note taking

    Journal of Mixed Methods Research

    (2008)
  • J.K. Kaakinen et al.

    Perspective effects in repeated reading: An eye movement study

    Memory & Cognition

    (2007)
  • J.K. Kaakinen et al.

    Perspective effects on online text processing

    Discourse Processes

    (2002)
  • W. Kintsch

    The role of knowledge in discourse comprehension: A construction–integration model

    Psychological Review

    (1988)
  • W. Kintsch

    Comprehension: A paradigm for cognition

    (1998)
  • I. Kirsch et al.

    Reading for change: Performance and engagement across countries. Results from PISA 2000

    (2002)
  • P.A. Kirschner et al.

    Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching

    Educational Psychologist

    (2006)
  • Cited by (0)

    An earlier version of this article was presented at the annual conference of the American Educational Research Association, San Diego, California, April 2009.

    View full text