Text difficulty effects on mind wandering
Mind wandering refers to attention drifting away from the current task, towards task-unrelated or stimulus-independent mental content. Several prior experiments have studied the impact of text difficulty on mind wandering (Feng et al.
2013; Forrin et al.
2019; Fulmer et al.
2015; Giambra and Grodsky
1989; Mills et al.
2013, Mills et al.
2015; Soemer et al.
2019; Soemer and Schiefele
2019), but the issue is not yet resolved. On the one hand, research suggests that easy (compared to difficult) texts are more susceptive to mind wandering. For example, in one experiment (Fulmer et al.
2015), college students read educational texts that were tuned to be either easy or difficult to read. Students mind wandered more when reading easy texts, at least when they expected the texts to be uninteresting. This finding can be explained by the theory that mind wandering and reading compete for the allocation of cognitive control (Smallwood and Schooler
2006). As easy texts demand the allocation of less cognitive control for task-related processes, there should be greater opportunities to simultaneously process task-unrelated content. So, during easy (but not difficult) texts, the mind has the opportunity to wander. This view, sometimes called the
executive resource hypothesis, is well-supported by studies outside the reading domain, e.g., by research that used working memory tasks (for a review, see Smallwood and Schooler
2015).
On the other hand, and in contrast to the research addressed above, several studies suggest that difficult (vs. easy) texts are more conductive to mind wandering (e.g., Feng et al.
2013; Mills et al.
2013,
2015; Soemer et al.
2019; Soemer and Schiefele
2019). To explain this finding, it helps to consider that readers need to build a
situational model to be fully engaged in reading, and to comprehend a text. A situational model refers to an extensive mental representation of the meaning of the concepts and events described in the text, their implied context, and their connection to pre-existing knowledge (Feng et al.
2013; Smallwood
2011). The situational model also helps the reader to understand how different events explained in the text are connected to each other, which is crucial to understand a text as an entity. With increasing text difficulty, building a situational model is increasingly likely to fail. When this happens, people are thought to disengage from the text and mind wander.
Traditionally, mind wandering is assumed to be initiated mostly unintentionally (e.g., some theories conceptualize mind wandering as a by-product of attentional failure, Kane and McVay
2012). However, recent theories suggest that mind wandering can also happen intentionally, in that people voluntarily choose to disengage from their current task (Seli et al.
2016a,
b). Research has shown that unintentional mind wandering tends to happen especially during difficult tasks, whereas intentional mind wandering tends to happen during easy tasks (Seli et al.
2016a,
b). Although this dissociation seems to be clear-cut during cognitive tasks, such as the sustained attention to response task (SART), this dissociation does not clearly extend to reading. Specifically, one study showed that reading difficult texts was associated with more unintentional
and more intentional mind wandering (Soemer and Schiefele
2019). Thus, the distinction between unintentional and intentional mind wandering cannot a priori explain the mixed findings in the reading domain. Nevertheless, theories on the role of intentionality in mind wandering may provide a useful starting point for explaining the mixed findings concerning text difficulty and mind wandering. Specifically, the insight that mind wandering may be voluntary suggests that a motivational approach to studying mind wandering may be viable (see also Seli et al.
2015; Seli et al.
2019; Soemer & Schiefele,
2019).
In the current study, we draw from the models of motivation–cognition interactions to make sense of the mixed findings concerning text difficulty and mind wandering (Shenhav et al.
2017). These models are based on the classic assumption that humans (and other organisms) avoid effort that is not proportional to the reward expected to result from it—in other words, people avoid effort that is not “worth it”. This behavioral principle has first been applied to behavioral effort (e.g., Hull
1943), and has since been extended to cognitive effort (e.g., Aridan et al.
2019; Dora et al.
2020; Kool et al.
2010). Phenomenologically, effort feels aversive (e.g., Dunn et al.
2016; Inzlicht et al.
2018; Saunders et al.
2017); however, exerting more effort is often associated with greater reward. Therefore, to decide whether to invest effort, people make cost–benefit analyses. When faced with a cognitively demanding task, people weigh its potential benefits (How rewarding will it be?) against its potential direct costs (How much control is needed for this task?) and opportunity costs (What else could I do instead?).
We suggest that such cost–benefit analyses can be applied to the context of mind wandering and reading. First, we assume that the more difficult a text is, the more cognitive effort is required to build a situational model. Hence, with increasing text difficulty, the potential benefits of reading become less and less likely to be worth the increasing cognitive costs, triggering people to process other mental content. Thus, we expect mind wandering to occur when people read (very) difficult texts. By contrast, for (very) easy texts, however, people need to allocate only little cognitive effort to reading. Thus, when people read easy texts, cognitive effort can also be allocated to processing and experiencing other mental content. Based on this line of reasoning, our main prediction is that the relationship between text difficulty and mind wandering should be U-shaped. That is, we predict that people should be most likely to mind wander when they are reading (very) easy and (very) difficult texts, compared to when they are reading texts that are moderately difficult.
In addition, we assume that people assess the potential benefits of reading a text as higher when they experience a text as more interesting. As potential benefits can compensate for potential costs, we further predicted that the U-shaped effect of text difficulty should be less pronounced (i.e., flatter) when people experience the text as more interesting.
The present research
Going beyond prior work (see Feng et al.
2013; Forrin et al.
2019; Mills et al.
2013,
2015), we designed an experiment in which participants were exposed to texts of five—instead of only two—difficulty levels. We collected reading materials that naturally vary in difficulty on a continuum, rather than using a binary manipulation that enabled us to test the hypothesis that the relation between text difficulty and mind wandering is U-shaped. As in previous work, participants were sometimes interrupted by probes. In response to these probes, participants indicated whether they were mind wandering.
In making our design choices, we prioritized high ecological validity. Therefore, we selected non-fiction texts about themes that could well feature in college-level courses. Rather than modifying these texts to be more or less difficult (as prior work did), we carefully selected texts from similar sources, about similar topics, but that varied in difficulty. To mirror natural reading, we presented the texts page by page instead of sentence by sentence.
We also examined the role of interest in the text as a motivational factor. Previously, interest was found to predict mind wandering regardless of text difficulty (Fulmer et al.
2015; Unsworth and McMillan
2013). Furthermore, some studies suggested that interest mediates the relationship between text difficulty and mind wandering (Giambra and Grodsky
1989; Soemer and Schiefele
2019). In the current study, we reasoned as follows: if (a) the allocation of cognitive control results from cost–benefit analysis, and if (b) people’s interest in a text inputs in such cost–benefit analysis, the effect of text difficulty should be suppressed when people find a text more interesting. After all, when they read less interesting texts, people should allocate less control effort to reading regardless of difficulty, thus leaving more room to process unrelated mental content (see Fulmer et al.
2015).
Finally, we aimed to replicate the well-established finding that mind wandering is associated with decrements in reading comprehension (Feng et al.
2013; Mrazek et al.
2013; Soemer and Schiefele
2019) using our newly developed stimulus materials.
Methods
Participants and design
We originally planned to recruit 80 participants, as power simulations suggested that 80 participants would be sufficient to detect an effect size of OR 1.24 (from Feng et al.
2013) with ~ 80% power. However, 17 additional people expressed interest in participating, and we decided to allow them. Thus, 97 participants completed the study (
Mage = 22.4, SD = 2.9; 76 women, 21 men), all university students. Of these participants, we excluded 7 before data analyses using pre-registered criteria (no variance in self-reported mind wandering, 4; no variance in perceived text difficulty, 2; age over 30, 1). Since earlier research has shown that the prevalence of mind wandering differs between age groups (Jordaõ et al.
2019), and since we aimed to recruit a homogenous sample of university students, we decided to include participants between 18 and 30 years of age. For one additional participant, no data were stored due to a software error. Thus, data from 89 participants were included in our analyses.
The main independent variable in the experiment was text difficulty, which we manipulated within-subjects. As an additional independent variable, we measured text interest after each text. The main dependent variable was the occurrence of mind wandering during reading. We pre-registered our planned sample size, exclusion criteria, hypotheses, and analysis plan on
https://aspredicted.org/bk6a6.pdf.
Materials
We used ten text passages. We gathered these text passages by searching in popular-science magazines and scientific journals with different target groups (children, interested lay audience, college students, academics). Specifically, we searched for articles about either of two topics,
animals and
politics, using search terms, such as
animals,
politics,
bees,
penguins,
cold war, and
civil war. We found 67 articles that were potentially suitable (see
https://osf.io/s8ery/).
We proceeded by selecting the easiest and the most difficult passages, using a computer algorithm that computes Flesch–Kincaid Grade Levels (FKGL). This algorithm calculates words-per-sentence ratio and syllables-per-words ratio to construct a score that reflects the US school grade level. After selecting very easy (level 1) and very difficult (level 5) texts, we selected texts with moderate difficulty levels (levels 2–4). We ensured that the steps between levels were approximately equal, based on the FKGLs. In total, we selected five texts with varying difficulty levels per topic (see Table
1).
Table 1
Characteristics of the stimulus materials
Animals | 742 | 1 | 5.8 | 13.6 | 83.9 | 52.8 | 31.9 | 52.8 | − 0.46 | 3.4 | 1.9 |
Animals | 1005 | 2 | 8.3 | 9.9 | 60.6 | 23.0 | 15.4 | 55.6 | − 0.08 | 3.4 | 2.3 |
Animals | 848 | 3 | 11.9 | 7.9 | 22.7 | 30.5 | 40.9 | 35.9 | 0.06 | 3.0 | 2.3 |
Animals | 1193 | 4 | 13.4 | 7.3 | 20.9 | 22.7 | 52.8 | 57.1 | − 0.28 | 3.0 | 2.3 |
Animals | 849 | 5 | 15.7 | 8.5 | 34.5 | 20.6 | 71.2 | 68.1 | 0.30 | 2.0 | 3.2 |
History | 939 | 1 | 7.1 | 16.7 | 92.9 | 39.4 | 16.9 | 78.5 | − 0.52 | 3.8 | 1.9 |
History | 1088 | 2 | 9.1 | 15.7 | 25.5 | 45.6 | 26.8 | 36.3 | 0.24 | 2.8 | 3.0 |
History | 889 | 3 | 11.1 | 14.4 | 63.3 | 33.0 | 16.6 | 83.9 | − 0.09 | 3.4 | 2.6 |
History | 1063 | 4 | 13.6 | 12.7 | 57.9 | 39.0 | 9.3 | 92.5 | 0.01 | 2.3 | 2.8 |
History | 1482 | 5 | 16.1 | 4.1 | 44.0 | 11.9 | 3.3 | 42.5 | 0.02 | 2.4 | 2.9 |
Before finalizing our selection, we used the Coh–Metrix online tool (Graesser et al.
2011; Graesser et al.
2014) to do a multi-dimensional text analysis, providing a deeper examination of difficulty levels for all texts. This was done to verify the difficulty rankings that we established using FKGL. The Coh–Metrix scores (Table
1, columns 6–11) generally confirmed that the selected texts could be characterized by five increasing difficulty levels with approximately equal steps between levels.
Procedure
Upon arrival, participants were seated in a cubicle. A computer script presented all stimuli and recorded all measurements. This script first gave participants a definition of mind wandering based on previous studies (Feng et al.
2013; Smallwood and Schooler
2006): “Mind wandering describes a state of mind that occurs when your attention shifts away from the task that you are doing at that moment”. Participants were instructed to read text passages at their own pace; they learned that they would spend 3 min on each passage (regardless of their pace).
Texts appeared in random order. Each text consisted of several pages, with roughly the same number of words on page one and two, and the variation in the length of text showing on page 3 (page 1: M = 399, SD = 22; page 2: M = 414, SD = 56; page 3: M = 199, SD = 141; average: M = 380, SD = 64). Participants could flip to the next page by pressing the spacebar; they could not go back to the previous page. A pilot test, in which participants (N = 8) freely read all texts, showed that the fastest readers finished the shortest texts in ± 3 min. Based on this, we set the time restriction mentioned earlier to 3 min, to ensure that participants would not mind wander because they had finished reading.
Within each reading period, participants were interrupted with one thought probe asking, “Were you just mind wandering?” with the answer options yes and no. The probes were presented at a random moment between 1.0 and 2.5 min after the onset of each reading interval. After the probe, participants continued reading until the 3-min period was over. After each reading period, participants answered two items about interest (How interesting did you find this text? 1 = not interesting at all; 5 = very interesting) and perceived text difficulty (How difficult did you find this text? 1 = very easy; 5 = very difficult). Finally, after reading all texts, participants completed a comprehension test. This test consisted of 30 multiple-choice questions, three per article, each question involving four answer options. The questions were designed to test the recollection of the explicit information from the respective text. All questions were about the content of the first page of each text, so that even slow readers would, in principle, still be able to correctly answer the questions.
Analyses
We used generalized mixed-effects models to analyze our data, using the
lme4 package in R (Bates et al.
2014). In all models, unless otherwise mentioned, mind wandering (
yes vs.
no, binary) was the dependent variable. All models included a fixed intercept, and two main effects of text difficulty, one linear and one quadratic. The main effects were both treated as continuous variables. To take into account that some people may generally mind wander more than others, we included per-participant adjustments to the intercept (i.e., a random intercept) in all models. Following well-established guidelines (Barr et al.
2013), we also included per-participant adjustments to the linear and quadratic effects of text difficulty (i.e., random slopes). These random slopes account for the possibility that some people are more responsive to the difficulty manipulation than others. Moreover, to take into account that either of the text topics may generally be more conductive to mind wandering, a per-topic adjustment to the intercept (i.e., another random intercept) was included. Finally, models included all correlations among the random effects. All continuous predictors were centered around the sample mean.
In all models, some of the variances were estimated to be zero or very close to zero (i.e., we encountered a ‘singularity warning’). Thus, as sensitivity analyses, we reproduced all our main analyses with models that had a simplified random-effects structure, i.e., models that included only per-participant adjustments to the intercept, and no other random effects. A report of these sensitivity analyses is included in the Supplementary Information (see Appendix, Table S1). These analyses yielded similar estimates as our main analyses, and are thus not discussed further.
To replicate the previous finding that mind wandering harms text comprehension (e.g. Feng et al.
2013; Mrazek et al.
2013), we used a generalized mixed-effects model, akin to the models we used to test our main hypotheses but with comprehension as the dependent variable.
Discussion
We found mind wandering increased linearly as people read more difficult texts. This finding extends previous work in two ways. First, our experiment suggests that a linear relationship between difficulty and mind wandering exists during natural, page-by-page reading. Second, our study suggests that this relationship holds across a large range of difficulty levels (i.e., 9 US grade levels). Although these findings replicate and extend previous work (Feng et al.
2013; Mills et al.
2013; Soemer et al.
2019; Soemer and Schiefele
2019), they do not support our novel hypothesis that mind wandering is lowest at moderate levels of difficulty, and highest at very easy and very difficult levels, even though our study was designed to be able to detect nonlinear effects.
Post hoc, our findings can still be reconciled with cost–benefit models of cognitive effort (Shenhav et al.
2017), when we only assume that (a) building a situational model requires cognitive effort, and (b) people may sometimes decide that the costs of such effort are not, or not anymore, worth expending. In our view, these two assumptions are plausible; at least, they are consistent with our findings
, with modern theories of mind wandering (that suggest that at least some mind wandering is voluntary; Seli et al.
2016a,
b), and with modern theories of cognitive control (that suggest that the allocation of control can be understood as resulting from a cost–benefit decision-making process; Shenhav et al.
2017). Thus, we think these assumptions can be used to generate further hypotheses regarding the interaction of task parameters (e.g., related to text difficulty) and motivational states (e.g., related to teacher expectations in educational settings) affecting mind wandering. However, we also reasoned a priori that when people allocate less cognitive effort to reading (i.e., when they are reading easy texts), they are more likely to mind wander. This idea seems incompatible with our findings.
Exploratory analyses suggested that the effect of difficulty was most pronounced for texts that people found very interesting; we found no evidence for an effect of difficulty for texts that people found uninteresting. At first sight, the direction of this interaction seems at odds with the one reported by Fulmer et al. (
2015), who found that the effect of difficulty was most pronounced for texts that people found uninteresting; for interesting texts, people’s mind wandered little regardless of difficulty. We should note, though, that the study by Fulmer et al. (
2015) was methodologically different from ours. In that study, the difficulty range of the texts was much smaller (i.e., 2 US grade levels), and the method used to manipulate interest was based on readers’ expectations before reading, not actual experiences. So, there may be several reasons for why the findings were not more similar. Still, both studies together suggest that, when trying to examine difficulty effects on mind wandering, it is wise to take reading motivation and interest into account. This conclusion also aligns with Soemer and Schiefele’s (
2019) finding that the effect of text interested fully mediated the relation between text difficulty and mind wandering.
The finding that difficulty affects mind wandering especially for texts that people perceive as interesting can be explained post-hoc by positing that people are categorically unwilling to invest substantial cognitive effort into building a situational model when they are not interested in the text; thus, the effect of difficulty becomes visible only when people are interested in the text. We note, though, that the test of this text difficulty * text interest interaction (a) was not pre-registered and (b) failed the sensitivity analysis in which we substituted objective (manipulated) difficulty with perceived difficulty (see Footnote 1). Even though this interaction is intriguing, it should be interpreted with great caution.
We replicated the ubiquitous finding that mind wandering impairs reading comprehension (e.g. Mrazek et al.
2013). Beyond this replication, exploratory analysis suggested the existence of an indirect route to failures of text comprehension: people fail to comprehend difficult texts not only because these texts are difficult per se, but also because these texts are more conductive to mind wandering. The existence of this indirect pathway suggests that the maximum level of text difficulty people can comprehend is not just constrained by people’s reading ability, but also by their mind’s tendency to drift off.
Our results show no support for the hypothesis that mind wandering is related to text difficulty in a U-shaped manner. Specifically, we found no evidence of people’s minds wandering more while they read (very) easy texts. Speculatively, we may have underestimated the cognitive effort it takes to build a situational model while reading, even for easy texts. That is, people may need to allocate cognitive effort mainly on reading to comprehend a text, even if that text is very easy. Further research—that employs behavioral, physiological, or subjective measures of effort (Bijleveld
2018; Scheiter et al.
2020)—may help to better understand whether and how the effort costs of building situational models can explain mind wandering during reading.
Another open question that pertains regarding the relation between text difficulty and mind wandering is whether such difficulty-triggered mind wandering is unintentional or intentional. Prior studies suggest both are possible (Seli et al.
2016a,
b; Soemer and Schiefele
2019), but the mechanisms through which intentional vs. unintentional mind wandering are triggered still need to be explored further. A promising avenue is to look at this issue through the lens of models of motivation–cognition interactions, like we did in the present study. Speculatively, both types of mind wandering may be underpinned by different cost–benefit weighting mechanisms (e.g., different in that they do vs. do not involve conscious awareness; Zedelius et al.
2014). Future research is needed to test this possibility.
In the present study, we used text interest as the main motivational factor when exploring the possible meaning of the models of motivation–cognition interactions for mind wandering. Time-on-task might be another relevant motivational factor to take into account, however. We controlled for this factor at the group level by randomly assigning the sequence of texts among the participants. However, it is important to note that previous studies have shown that time-on-task is an important motivational predictor of mind wandering on its own. Specifically, studies that measured mind wandering during cognitive tasks (e.g. working memory tasks) have shown that mind wandering increases with increased time-on-task (Brosowsky et al.
2020; Krimsky et al.
2017; Thomson et al.
2014). To explain this finding, Browosky et al. (
2020) suggested that at the start of a cognitive task—which is often new to participants—people allocate relatively high on-task focus, since they are then still lacking sufficient knowledge on the task’s costs and benefits (see also Kurzban et al.
2013). With time, they may sometimes learn that high performance has no benefits other than helping the researcher, after which they disengage However, when reading, participants may still see the personal benefit of learning something new. Future studies have yet to identify how the findings regarding other cognitive tasks, each of which has their unique costs and benefits, translate into the context of mind wandering while reading.
While the current study could support earlier findings and provide new perspectives, it also has some limitations. One limitation to the ecological validity is that we limited the time participants had to read each text and that they could not go back to earlier pages. We made this choice to ensure that participants can finish the task within the planned time, and to ensure that the effect of mind wandering on text comprehension cannot be biased by participants who choose to re-read passages.
Another limitation of our study is that we did not experimentally manipulate the text difficulties of the texts but chose texts based on their FKGL scores. Thus, it is possible that other text characteristics besides text difficulty influenced the current results. Previous studies did experimentally manipulate text difficulty by changing the words or sentence structure of a text without changing the content (Feng et al.
2013; Fulmer et al.
2015; Mills et al.
2013; Soemer et al.
2019; Soemer and Schiefele
2019). While this is a more controlled manipulation, we chose against it, as it would not allow the range in text difficulty that we strived for in the current study.
Related to the previous point, we primarily used FKGL to categorize our texts in text difficulty. FKGL is an often-used measure of text difficulty. However, it is also often criticized for its simplicity as its formula only relies on letters-per-word and words-per-sentence ratios (Dufty et al.
2006; Forrin et al.
2019; Fulmer et al.
2015; Graesser et al.
2011). Forrin et al.’ experiments (
2019) showed that the effects of text difficulty measured with the FKGL on mind wandering could largely be explained through an effect of section length rather than text difficulty itself. Coh–Metrix takes more facets of text difficulty into account by providing several measures of text difficulty. While we took the Coh–Metrix metrics (Graesser et al.
2011) into account while choosing our materials, it is hard, perhaps impossible, to find texts that present exactly the same pattern of text difficulty in all Coh–Metrix measures, or even on a composite Coh–Metrix measure (i.e., ‘Formality’; Graesser et al.,
2014). In other words, we acknowledge that text difficulty is not a unitary construct, even though we did treat it as such in the present study.
In sum, the current study showed that people’s minds tend to wander more with increasing text difficulty and decreased interest. Mind wandering in turn explains at least in part why more difficult texts lead to lower reading comprehension. From an applied perspective, our findings highlight the merits of the classic advice for writers to simplify their writing (“avoid fancy words”, “use the active voice”, “avoid the use of qualifiers”, “omit needless words”; Strunk and White
1959). Heeding such advice may well help readers mind wander less.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.