What matters in making demand-based decisions: Time alone or difficulty too?

Decisions between courses of actions are often assumed to be driven by attempts to minimize effort (Dunn & Risko, 2019; Dunn et al., 2016; Fournier et al., 2019; Gray et al., 2006; Hull, 1943; Kool et al., 2010; Zipf, 1949). What effort actually is, however, is unclear. Some researchers have suggested that effort amounts to time (Gray & Fu, 2004; Gray et al., 2006). Others have argued that effort amounts to demands on executive control (e.g., Kool et al., 2010; Taatgen, 2007), often operationalized with regard to task switching (Rogers & Monsell, 1995). Yet another hypothesis is that effort amounts to error avoidance or the associated demands (Dunn et al., 2019a, 2019b). A priori, any of these factors could hold in different circumstances, but it would be more satisfying to develop an understanding of the effort and task choice that appeals to some deeper, less ad hoc, understanding.

The main method for studying the effect of effort on choices is the Demand Selection Task (DST) used by Dunn and Risko (2019), Botvinick and Rosen (2009), and Kool et al. (2010), along with others who used the same method without explicitly referring to it by that name (Rosenbaum et al., 2013).¹ The DST has been used in investigations of the effects of the factors mentioned above, that is, the effect of task switching, time, and error likelihood, but has also been used in evaluations of physiological (Botvinick & Rosen, 2009) and neurological (McGuire & Botvinick, 2010) consequences of task engagement or anticipation. This body of work has shown that it is not necessarily the objective costs (of whatever kind) posed by a task, but rather “subjective costs and some degree of explicit awareness of said costs” (Dunn et al., 2016, p. 1373) that drive effort-based decision-making (see also Dunn et al., 2019a, 2019b; Gold et al., 2015). Accordingly, Dunn et al. (2016) have suggested that multiple cues can contribute to the evaluation of effort, provided those cues are sufficiently salient to be noticed (Dunn & Risko, 2019).

In typical DSTs, the choice options vary on cognitive dimensions. However, the same method can be used to investigate the comparison of physical and mental tasks (Feghhi & Rosenbaum, 2019, 2021; Feghhi et al., 2021; Potts et al., 2018; Rosenbaum & Bui, 2019; Rosenbaum et al., 2013). For example, if one chooses between doing ten challenging math problems or moving a wheelbarrow full of rocks back and forth between a couple of locations ten times, what would one choose? On what basis would effort be compared? The choice would surely depend on features of the tasks—how challenging the math problems were, how many of them there were, how far apart the rock locations were, how heavy the rocks were, how many trips were necessary, and so on. That these factors would affect the choice suggests that different kinds of effort can be compared. If so, what are the core elements for the decisions?

The tasks mentioned above may be said to differentially tax “brain” and “brawn.” One task, doing math problems, is “more cognitive.” The other, moving rocks, is “more physical.” Of course, these terms are intuitive at best, for “mental tasks” also require physical enactment, and “physical tasks” also require thought. If physical tasks only required brawn but no brain, robots would be more capable than they are of complex actions in unpredictable environments, and doing physical tasks would not affect cognitive performance or vice versa, though such interactions have been observed (e.g., Weigelt et al., 2009; Zhang et al., 2014).

As just intimated, the question posed here is, what aspect(s) of such tasks contribute(s) to a “common currency” that may be used to compare the effort associated with different kinds of tasks and, for that matter, tasks of a given kind? That there might be a common currency is suggested by the fact that when people compare the difficulty of physical and mental tasks—not math and rock-transport tasks, as in the just mentioned example, but digit memorization and walking through gaps of varying width—their choices are systematic (Feghhi & Rosenbaum, 2019, 2021; Feghhi et al., 2021). Even so, it is possible that the subjective difficulty of each task is measured with a different metric. For example, attentional demands might be used to determine mental difficulty, whereas caloric consumption might be used to determine physical difficulty. If mental difficulty values and physical difficulty values were mapped onto one another—say from smallest to largest in both cases—it might be possible to decide between the two kinds of tasks based on the relative positions of their values. No common currency would be needed.

Notwithstanding the latter possibility, several common-currency candidates have been considered in the literature. Energy has been suggested (Craig, 2013; Job et al., 2010), though, as far as we can tell, that term has been used metaphorically rather than literally as defined in physics (e.g., Navon & Miller, 2002; Tombu & Jolicoeur, 2003). Likelihood of error may also be considered (Dunn et al., 2019a, 2019b), but Feghhi and Rosenbaum (2019, 2021) showed that error reduction is not the sine qua non of the task choice. The latter point can be reached by recognizing that the cost of a car crash is much higher than the cost of a math-homework mistake even though the probabilities of the two events might be the same. Finally, time on task has been considered as well (Gray et al., 2006; Potts et al., 2018; Rosenbaum & Bui, 2019), and this candidate will be a focus of the current investigation.

On average, participants managed to perform 8.46 calculations of the addition task but only 7.47 calculations of the multiplication task, t(29) = 2.87, p = 0.008, d = 0.52. Error percentages were 10.92 and 11.17 in the addition and multiplication task, respectively. This difference was not significant, t(29) = 0.04, p = 0.966, d = 0.01 The probability, p(Bucket), of choosing the bucket as a function of the difficulty and duration of the cognitive task and bucket weight is shown in Fig. 2. The impression from the figure is that participants chose the bucket task more often when the bucket was empty than when it was weighted and more often as the duration of the other, cognitive, task increased. Difficulty of the cognitive task (addition versus multiplication) did not have a major impact on bucket choices, as confirmed in the ANOVA for these data (see Table 1 for the complete results). Only the two main effects of cognitive task duration and bucket weight were significant, Fs ≥ 20.44, ps < 0.001. All other effects were non-significant, all Fs ≤ 1.99, all ps ≥ 0.147.

Experiment 1 replicated and extended the results of Potts et al. (2018) by showing that when the cognitive task duration was controlled, the longer the cognitive task was to be performed the more often participants chose the alternative bucket task. We also observed a clear effect of bucket weight, that is, physical task difficulty. Participants chose the cognitive task more often when the bucket was loaded than when it was empty. This outcome accords with our expectation that with a within-participants manipulation, where the same participants got physical tasks with varying demands (buckets with different loads), they would show greater sensitivity to the demand levels than was the case in the study of Potts et al. where bucket weight was varied between- participants in their Experiment 1.

Unexpectedly, in the present experiment, the (presumed) difficulty of the cognitive task did not affect choices. Whether the cognitive task was addition or multiplication, it did not matter. Interestingly, the tasks differed in the rate at which problems were solved—more addition problems were solved per time unit than were multiplication problems—but error rates were comparable. Other studies have also reported a lack of differences in error rates between addition and multiplication of two digits, the number of digits per problem used here (e.g., Zhou et al., 2007). However, even though this dissociation may be interesting in itself, the absence of an error-rate difference also suggests that (a) the completion rate was not the determinant of the (perceived) task difficulty and (b) the intended manipulation of cognitive task difficulty was unsuccessful, or at least not strong enough to influence the choices made by our participants. To address this issue, we manipulated cognitive difficulty in a different way in Experiment 2.

Most aspects of this experiment were the same as in Experiment 1 with the major change relating to the cognitive task. Because it was unclear whether multiplication was substantially more difficult (objectively and subjectively) than addition, we turned to a different task.

In the exposure block, participants made fewer errors in the 2-digit condition than in the 4-digit condition, (5.68 vs. 11.61%), t(47) = 5.59, p < 0.001, d = 0.81, and responses were made more quickly in the 2-digit condition than in the 4-term condition (1851 vs. 5774 ms), t(47) = 20.05, p < 0.001, d = 2.89. These outcomes suggest, as expected, that the 2-digit condition would be less difficult than the 4-digit condition.

Figure 3 shows the probability of choosing the bucket, p(Bucket), as a function of the difficulty and duration of the cognitive task and bucket weight. The impression is that participants chose the bucket task more often as the cognitive task duration increased, when the bucket was empty compared to when the bucket was weighted, and when the cognitive task involved 4 digits compared to 2.

As shown in Table 2, this impression was corroborated by the respective three main effects, all of them being significant, assuming α = 0.05. Because the three-way interaction had a p-value that, in traditional hypothesis-testing terms would be “just significant,” we analyzed the two levels of cognitive task difficulty separately with ANOVAs that only had cognitive task duration and bucket weight as repeated measures.

For the 2-digit task, the main effect of duration was significant, F(2,94) = 7.36, p = 0.001, η_p² = 0.14, and this was also true for the main effect of bucket weight, F(1,47) = 11.41, p = 0.001, η_p² = 0.20. However, the interaction was not significant, F(2,94) = 2.03, p = 0.137, η_p² = 0.04. For the 4-digit task, the main effect of duration was significant, F(2,94) = 20.29, p < 0.001, η_p² = 0.30, whereas the main effect of weight was not, F(1,47) = 3.02, p = 0.089, η_p² = 0.06, and the p-value of the interaction fell just below the traditional α value for significance, F(2,94) = 3.21, p = 0.045, η_p² = 0.06.⁷

Experiment 2 was similar to Experiment 1 except that we varied the number of digits (2 or 4) in mixed addition and subtraction problems to manipulate the difficulty of the cognitive task. This manipulation differentially taxed participants in ways that the use of addition versus multiplication did not in Experiment 1. Given this outcome, the following conclusions could be drawn. First, there was an effect of the duration of the cognitive task on p(Bucket). The longer that duration, the higher the value of p(Bucket), replicating what was observed in Experiment 1. Second, loaded buckets were chosen less often than empty ones, also replicating what was observed in Experiment 1, and pointing to an effect of physical task difficulty. Third and finally, there was a clear effect of cognitive task difficulty on the likelihood of choosing the bucket in this experiment. When only two digits would be dealt with, p(Bucket) was lower than when four digits would be dealt with. These results show that all three variables—time, cognitive task difficulty, and physical task difficulty—affected choices. Perhaps most important given the main question of this study, the decisions about which task to carry out were not just based on time, as also suggested by Kool et al. (2010).

In this article, we have reported two experiments on choosing between a “more cognitive” task and a “more physical” task. Earlier studies (Potts et al., 2018; Rosenbaum & Bui, 2019) suggested that time may be the underlying metric for choosing a (subjectively) less difficult task. The present study built on those results and aimed to disentangle the contributions of duration and difficulty of a cognitive task—variables that were confounded in the earlier studies—and the difficulty of a physical task. To this end, we employed two cognitive tasks per experiment to manipulate (cognitive) difficulty. In Experiment 1, the cognitive tasks were addition versus multiplication of 2 digits. In Experiment 2, the cognitive tasks were mixed addition and subtraction of 2 versus 4 digits. We also varied how long the tasks were to be performed in both experiments. The durations were 9, 18, or 27 s. These values were based on 18 s as the approximate time of the bucket task, with 9 s and 27 s being 9 s shorter and longer, respectively, than that time. Further, we manipulated the difficulty of the physical task by varying the weight of the bucket, because this variable had unclear effects in earlier studies, where bucket weight was manipulated between-participants. Here we made bucket weight a within-participants factor and crossed this variable with the two levels of cognitive task difficulty and duration for each single participant.

Two results were clear in both experiments. First, the longer the duration of the cognitive task, the more often participants chose the bucket task. Second, when the bucket was loaded, the bucket task was chosen less often than when the bucket was empty. This latter result contrasts with Potts et al.’s (2018) report that bucket weight did not matter, though that study used a between-participants variation for bucket weight; for similar observations and a discussion, see Birnbaum (1999) and Schweitzer et al. (2013). The latter observation may also be discussed in reference to the General Evaluability Theory (GET; Hsee & Zhang, 2010). Varying bucket weight between- versus within-participants can be conceived as implementing a single versus a joint evaluation mode. However, in a single evaluation mode, some reference information is required to allow choices. Dunn et al. (2017) investigated the evaluability of stimulus rotation, set size, and weight. Stimulus rotation appeared as a not evaluable feature and the authors suggested that error or failure knowledge might be required to make a feature evaluable. In our case, bucket weight would probably matter in a between-participants design where bucket weights were increased enough to strain ethical guidelines regarding the loads that one group could be asked to carry. Thus, while there clearly is a point of failure in carrying a weighted bucket, the employed loads might have been too low to make this point salient to the participants. Notably, weight was an evaluable feature in the study by Dunn et al. This contrasts with the discussion just offered, and tentatively, we can only point to two differences. First, participants in the study by Dunn et al. were rating their perceived effort while in our study and the one of Potts et al. (2018) they needed to perform the task with the given weight. Second, the second level of weight (10 lbs) in the Dunn et al. study was higher than the heaviest buckets used here (3.2 kg or approximately 7 lbs).

With regard to the effect of cognitive task difficulty, choices did not depend on this factor in Experiment 1, but did so in Experiment 2. Thus, our expectation that multiplication would be more difficult than addition was not realized in Experiment 1. Still, an important conclusion could be reached from the fact that participants in Experiment 1 sometimes chose the cognitive task even though it had a higher error rate than the bucket task. The error rate for the bucket task was 0% but was close to 11% for the math tasks. Participants would have never chosen the math task if the sole criterion for doing so were the elimination of errors. Therefore, error elimination was not the sole basis for choosing tasks, a result observed as well by Kool et al. (2010) and Feghhi and Rosenbaum (2021). Where this leaves us is that neither error elimination nor time minimization was the sole basis for choice. Rather, all of the variables investigated here—duration of the cognitive task, its difficulty (at least in Exp. 2), and the difficulty of the physical task—influenced task choices.

How then can we account for our results? Can we do so with a single metric, as asked in the introduction of this article? Can we say there is a common currency for choosing tasks requiring less effort? We think we can.

We addressed these questions through modeling. More precisely, we asked whether the contributing factors (i.e., duration of the cognitive task, difficulty of the cognitive task, and difficulty of the physical task [i.e., bucket weight]) could be converted into a single variable. The idea was that if these factors are indeed convertible to a single subjective variable, then increasing the cognitive difficulty (e.g., from 2-digits to 4-digits in Experiment 2) should have an effect similar to increasing the duration of the 2-digit task. Similarly, increasing the duration of the cognitive task should have an effect similar to decreasing the difficulty of the physical task, that is, decreasing the bucket load.

The way we embarked on our modeling was to pursue transforms of objective duration and the other factors that would maximize the likelihood of the data. We imagined sliding three of the curves in Fig. 2 horizontally, such that all the points, plus the points along the unshifted curve, would hug a single curve. Similarly, we imagined sliding three of the curves in Fig. 3 horizontally, such that all the points, plus the points along the unshifted curve, would lie on a single curve. Finding the horizontal shifts that achieved the best fit of all the points per figure would amount to transforming the x-axis from objective time to a single variable referred to simply as Φ for now.

Beginning with Experiment 2 data, we defined two free parameters, denoted k (cognitive difficulty) and h (physical difficulty, i.e., bucket weight) in the following formula:

For given values of k and h, we fitted a logistic regression with four parameters (upper bound, lower bound, inflection point, and steepness) to maximize the coefficient of determination, that is, \({R}^{2}=1-\frac{S{S}_{residuals}}{S{S}_{total}}\). For Experiment 2, the largest value of R² was \({R}^{2}=.977,\) obtained with h = 8 and k = 24. The resulting logistic function is shown in the right panel of Fig. 4. Because we observed no effect of cognitive difficulty in Experiment 1, only one free parameter, h, was needed to reasonably explain choices in this experiment, and Φ would be

The largest value \({R}^{2}=0.976\) was obtained for h = 7 in this case and the resulting logistic function is shown in the left panel of Fig. 4. When setting h = 8, that is, to the same value as obtained when fitting the model in Experiment 2, the fit was negligibly worse with \({R}^{2}=0.970\). The resulting logistic function is shown as the red line in the left panel of Fig. 4.

The foregoing analysis shows that all of the choice data could be modeled by expressing costs of all kinds in terms of one variable Φ. This analysis and interpretation fit well with the metacognitive framework suggested by Dunn et al. (2016) where effort is a subjective inference from multiple cues, as long as they are sufficiently salient. This inference may, but need not, covary with objective costs (e.g., in terms of [response] time). For example, Dunn and Risko (2019) pitted two tasks against each other—one with a highly salient feature but less of a time difference (stimulus rotation) and one with a clear time difference but less salient variation (task switch)—and suggested that the highly salient variant also affected choices, even though the time costs were negligible. Thus, these authors went on to suggest “to conceptualize time and demands on executive control as potential cues” (p. 8). This proposal fits well with our results that all three factors contributed to choices. The single variable Φ might therefore simply be interpreted as the effort inferred from the available and salient factors (or cues). Arguably, all factors were salient to the participants, in particular in Experiment 2, where they were required to experience both levels of cognitive task difficulty in the exposure block. We can further speculate that this rather simple model might underlie a process model for making the choices: If the participants’ subjective estimate of Φ to do a task falls below a criterion value, choose that task; otherwise, choose the other task; in case the two estimates are the same, choose at random.

A drawback of our experimental variation of cognitive task difficulty is that we cannot disentangle cognitive task difficulty and error likelihood—in fact, we operationalized less versus more difficult tasks as those inducing less versus more errors, respectively. Error likelihood has, however, been associated with the effort in a study by Dunn et al., (2019a, 2019b; but see Feghhi & Rosenbaum, 2019, 2021). Yet, Dunn et al., (2019a, 2019b) also noted a role for time judgments in effort judgment in one their experiments (Experiment 4), but identification of situations where error likelihood might play a more prominent role seems a worthwhile future direction.

Reflecting on what we have achieved here, we note that while separate studies have compared the effects of each of the three factors that we examined—cognitive difficulty, physical difficulty, and time (i.e., duration of the cognitive task)—we have shown that these factors are lawfully convertible to a single variable Φ. This variable may be interpreted as effort, but it is tempting to ask whether Φ might correspond to some other nameable construct. In keeping with the results of Potts et al. (2018), this might be subjective time. Recall that Potts et al. (2018) concluded that subjective time, over and above objective time, provided a clear account of their choice data. This interpretation also receives support from a study by Rosenbaum and Bui (2019), where it was observed that subjective time did a better job of accounting for choices than another hypothesized measure of task difficulty, task sustainability. Task sustainability was not tested in the present experiments and was only estimated by the participants in the study Rosenbaum and Bui (2019). Similarly, subjective time was not tested directly here. More work is clearly needed to test the role of this hypothesized quantity. Collecting subjective time estimates would be an obvious next step.

Despite these limitations, we feel that the present work provides a useful advance for the study of task difficulty. By asking people to choose the easier of two tasks, we found that their choices were systematic and reliable, as in previous studies. Furthermore, through modeling, we were able to map costs of different kinds onto a single variable, Φ. The functional identity of Φ remains to be determined. It could be subjective time, as just suggested, or it might be some abstract, amodal quantity, perhaps corresponding to a common code for task difficulty. The fact that several variables contributed to it is consistent with the metacognitive framework of Dunn et al. (2016). Indeed, as Dunn et al. (2016) might argue, and we agree, Φ might be interpreted as the subjective effort inferred from the salient cues of the task alternatives—in our case, duration and difficulty of the cognitive task and difficulty of the physical task.

In a related way, subjective effort has recently been described as a “common code for task difficulty”. This idea, broached by Feghhi et al. (2021), was inspired by the common coding hypothesis (Prinz, 1984, 1992; see also Hommel et al., 2001) and is meant in terms of an analogy: the traditional concept of common codes refers to the representations underlying perception and action, here it refers to the effort representation underlying different kinds of tasks. Feghhi et al. (2021) used different tasks than the ones used in the present study, but like us, they observed that choice probabilities could be well fitted with a model in which costs of different kinds mapped onto a single quantity. Feghhi et al. (2021) suggested that the quantity might be abstract and amodal, but did not attempt to argue for or against subjective time per se.

What that “true value” might be is certainly a matter for future work. The idea that subjective effort is a multi-modal or amodal representation, regardless of whether it incorporates subjective time, sustainability (Rosenbaum & Bui, 2019), error likelihood (Dunn et al., 2019a, 2019b), executive control (Kool et al., 2010) or some other feature(s) is attractive to us, for it accommodates the flexibility of task choice which is needed in everyday life. Which task is easiest in any situation additionally might depend on a wide range of situational factors. A theory of task ease or difficulty must reckon with this variability.