Abstract
Humans and other animals live in dynamic environments. To reliably manipulate the environment and attain their goals they would benefit from a constant modification of motor-responding based on responses' current effect on the current environment. It is argued that this is exactly what is achieved by a mechanism that reinforces responses which have led to accurate sensorimotor predictions. We further show that evaluations of a response's effectiveness can occur simultaneously, driven by at least two different processes, each relying on different statistical properties of the feedback and affecting a different level of responding. Specifically, we show the continuous effect of (a) a sensorimotor process sensitive only to the conditional probability of effects given that the agent acted on the environment (i.e., action-effects) and of (b) a more abstract judgement or inference that is also sensitive to the conditional probabilities of occurrence of feedback given no action by the agent (i.e., inaction-effects). The latter process seems to guide action selection (e.g., should I act?) while the former the manner of the action's execution. This study is the first to show that different evaluation processes of a response’s effectiveness influence different levels of responding.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets analysed during the current study are available from the corresponding author on reasonable request.
Code availability
The code used in current study is available from the corresponding author on reasonable request.
Notes
A Bayes Factor greater than 3 or smaller than 0.33 is considered by convention to be the minimal conclusive ratio (See Jeffreys 1998; Kruschke and Liddell 2018); a Bayes factor of 3 means that the alternative hypothesis is three times more likely than the null hypothesis given the data (for support in the null, simply use the inverse of the Bayes factor ratio).
The experiment was pre-registered on OSF (Hemed et al. 2017) and we present it with all its conditions for the sake of completion (see Supplementary materials); yet in a recently published series of experiments (Karsh et al. 2020) we rather firmly established that adding monetary value to a neutral effect a response has on the environment does not facilitate response times more than merely adding such an effect.
We would like to thank Dr. Jan De-Houwer for raising this possibility.
It should be noted that we previously attempted to run this experiment dissociating between tones and the result of actions and inactions (using only the two conditions where action-effects are present, manipulated between-subjects). However, due to a design flaw the results of the experiment provided mixed results (we found the predicted difference in response frequency between the two conditions and a difference in RT between the conditions). The flaw was that we used the mean RT of previous valid trials as timepoint for deeming trial as No-Response (see Experiment 2a) and as a result the participants were gradually pushed to respond faster and faster as the time window shortened with their improved performance, resulting in truncated distribution of RTs. The current design bypasses this flaw and is more like the Experiments presented in the current study.
References
Aarts H, Bijleveld E, Custers R, Dogge M, Deelder M, Schutter D, van Haren NEM (2012) Positive priming and intentional binding: eye-blink rate predicts reward information effects on the sense of agency. Soc Neurosci 7(1):105–112
Bakbani-Elkayam S, Dolev-Amit T, Hemed E, Zilcha-Mano S, Eitam B (2019) Intact motivation in major depression: normative responsiveness to action-effectiveness demonstrated in a clinical sample. SSRN. https://doi.org/10.2139/ssrn.3472084
Bar-Hillel M, Wagenaar WA (1991) The perception of randomness. Adv Appl Math 12(4):428–454
Barlas Z, Kopp S (2018) Action choice and outcome congruency independently affect intentional binding and feeling of control judgments. Front Hum Neurosci 12:137
Barlas Z, Obhi SS (2014) Cultural background influences implicit but not explicit sense of agency for the production of musical tones. Conscious Cogn 28:94–103
Berberian B, Sarrazin J-C, Le Blaye P, Haggard P (2012) Automation technology and sense of control: a window on human agency. PLoS ONE 7(3):e34075. https://doi.org/10.1371/journal.pone.0034075
Blakemore SJ, Wolpert DM, Frith CD (1998) Central cancellation of self-produced tickle sensation. Nat Neurosci. https://doi.org/10.1038/2870
Blakemore SJ, Wolpert D, Frith C (2000) Why can’t you tickle yourself? NeuroReport. https://doi.org/10.1586/14737175.7.10.1337
Brass M, Haggard P (2008) The what, when, whether model of intentional action. Neuroscientist 14(4):319–325. https://doi.org/10.1177/1073858408317417
Buehner MJ (2015) Awareness of voluntary and involuntary causal actions and their outcomes. Psychol Conscious Theory Res Pract 2(3):237
Carruthers G (2012) The case for the comparator model as an explanation of the sense of agency and its breakdowns. Conscious Cogn. https://doi.org/10.1016/j.concog.2010.08.005
Chambon V, Haggard P (2012) Sense of control depends on fluency of action selection, not motor performance. Cognition 125(3):441–451
Chambon V, Haggard P (2013) Premotor or ldeomotor: how does the experience of action come about? In: Prinz W, Beisert M, Herwig A (eds) Action Science: Foundations of an Emerging Discipline Cambridge, MA: MIT Press, pp 359–380
Christensen JF, Di Costa S, Beck B, Haggard P (2019) I just lost it! Fear and anger reduce the sense of agency: a study using intentional binding. Exp Brain Res 237(5):1205–1212
Cohen J (1988) Statistical power analysis for the behavioral sciences. In: Statistical power analysis for the behavioral sciences
Desantis A, Roussel C, Waszak F (2011) On the influence of causal beliefs on the feeling of agency. Conscious Cogn 20(4):1211–1220. https://doi.org/10.1016/j.concog.2011.02.012
Desantis A, Weiss C, Schütz-Bosbach S, Waszak F (2012) Believing and perceiving: authorship belief modulates sensory attenuation. PLoS ONE. https://doi.org/10.1371/journal.pone.0037959
Dewey JA, Knoblich G (2014) Do implicit and explicit measures of the sense of agency measure the same thing? PLoS ONE 9(10):e110118
De Houwer J, Moors A (2015) Levels of analysis in social psychology. In: Gawronski B, Bodenhausen G (eds) Theory and explanation in social psychology, Guilford, pp 24–40
Dogge M, Schaap M, Custers R, Wegner DM, Aarts H (2012) When moving without volition: implied self-causation enhances binding strength between involuntary actions and effects. Conscious Cogn 21(1):501–506
Dogge M, Custers R, Aarts H (2019) Moving forward: on the limits of motor-based forward models. Trends Cogn Sci 23(9):743–753
Eder AB, Dignath D, Erle TM, Wiemer J (2017) Shocking action: facilitative effects of punishing electric shocks on action control. J Exp Psychol Gen 146(8):1204
Eitam B, Kennedy PM, Higgins ET (2013) Motivation from control. Exp Brain Res. https://doi.org/10.1007/s00221-012-3370-7
Elsner B, Hommel B (2004) Contiguity and contingency in action-effect learning. Psychol Res 68(2–3):138–154. https://doi.org/10.1007/s00426-003-0151-8
Gautier L (2021) rpy2 - r in python [Computer Software]. https://rpy2.github.io
Gentsch A, Weiss C, Spengler S, Synofzik M, Schütz-Bosbach S (2015) Doing good or bad: how interactions between action and emotion expectations shape the sense of agency. Soc Neurosci 10(4):418–430
Gibbons RD, Hedeker DR, Davis JM (1993) Estimation of effect size from a series of experiments involving paired comparisons. J Educ Stat 18(3):271–279
Gozli D (2019) Experimental psychology and human agency. Springer, New York
Gozli DG, Dolcini N (2018) Reaching into the unknown: actions, goal hierarchies, and explorative agency. Front Psychol 9:266
Gozli DG, Gao CJ (2019) Hope, exploration, and equilibrated action schemes. Behav Brain Sci 42:E41
Haering C, Kiesel A (2012) Mine is earlier than yours: causal beliefs influence the perceived time of action effects. Front Psychol 3:393
Haggard P (2005) Conscious intention and motor cognition. Trends Cogn Sci 9(6):290–295
Haggard P, Eitam B (2015) The sense of agency. Social cognition and social neuroscience. Oxford University Press, New York
Haggard P, Clark S, Kalogeras J (2002) Voluntary action and conscious awareness. Nat Neurosci 5(4):382–385. https://doi.org/10.1038/nn827
Hammond LJ (1980) The effect of contingency upon the appetitive conditioning of free-operant behavior. J Exp Anal Behav 34(3):297–304
Hauf P, Elsner B, Aschersleben G (2004) The role of action effects in infants’ action control. Psychol Res 68(2–3):115–125
Heald JB, Lengyel M, Wolpert DM (2021) Contextual inference underlies the learning of sensorimotor repertoires. Nature 600(7889):489–493
Hemed E, Karsh N, Mark-Tavger I, eitam, baruch (2017) The effectiveness of actions. https://doi.org/10.17605/OSF.IO/YZH5B
Hemed E, Bakbani-Elkayam S, Teodorescu AR, Yona L, Eitam B (2019) Evaluation of an action’s effectiveness by the motor system in a dynamic environment. Exp Psychol Gen 149(5):935–948
Hemed E, Bakbani-Elkayam S, Teodorescu AR, Yona L, Eitam B (2020) Evaluation of an action’s effectiveness by the motor system in a dynamic environment. J Exp Psychol Gen 149(5):935–948. https://doi.org/10.1037/xge0000692
Higgins ET (2011) Beyond pleasure and pain: how motivation works. Oxford University Press, Oxford. https://doi.org/10.1093/acprof:oso/9780199765829.001.0001
Higgins ET (2019) What reigns supreme: value, control, or truth? Motiv Sci 5(3):185–201. https://doi.org/10.1037/mot0000150
Hull CL (1943) Principles of behavior: an introduction to behavior theory. Appleton-Century, New York
Huster RJ, Enriquez-Geppert S, Lavallee CF, Falkenstein M, Herrmann CS (2013) Electroencephalography of response inhibition tasks: functional networks and cognitive contributions. Int J Psychophysiol 87(3):217–233
Jeffreys H (1998) Theory of probability. Clarendon Press, Oxford
Karsh N, Eitam B (2015a) I control therefore I do: judgments of agency influence action selection. Cognition. https://doi.org/10.1016/j.cognition.2015.02.002
Karsh N, Eitam B, Mark I, Higgins ET (2016) Bootstrapping agency: how control-relevant information affects motivation. J Exp Psychol Gen 145(10):1333–1350. https://doi.org/10.1037/xge0000212
Karsh N, Hemed E, Nafcha O, Elkayam SB, Custers R, Eitam B (2020) The differential impact of a response’s effectiveness and its monetary value on response-selection. Sci Rep 10(1):1–12. https://doi.org/10.1038/s41598-020-60385-9
Karsh N, Eitam B (2015b) Motivation from control: a response selection framework. In: The sense of agency. Oxford University Press, New York. https://doi.org/10.1093/acprof:oso/9780190267278.003
Karsh N, Haklay I, Raijman N, Ruud Custers AL (2021) Control alters risk-taking: the motivating impact of action-effectiveness in different risk contexts. Motiv Sci. https://psycnet.apa.org/record/2021-97718-001
Kilteni K, Andersson BJ, Houborg C, Ehrsson HH (2018) Motor imagery involves predicting the sensory consequences of the imagined movement. Nat Commun 9:1617
Kruschke JK, Liddell TM (2018) The Bayesian new statistics: hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychon Bull Rev 25(1):178–206
Li C-SR, Yan P, Sinha R, Lee T-W (2008) Subcortical processes of motor response inhibition during a stop signal task. Neuroimage 41(4):1352–1363
Liljeholm M, Tricomi E, O’Doherty JP, Balleine BW (2011) Neural correlates of instrumental contingency learning: differential effects of action–reward conjunction and disjunction. J Neurosci 31(7):2474–2480
Lynn MT, Muhle-Karbe PS, Aarts H, Brass M (2014) Priming determinist beliefs diminishes implicit (but not explicit) components of self-agency. Front Psychol 5:1483
Moore JW (2016) What is the sense of agency and why does it matter? Front Psychol 7:1272
Moore JW, Lagnado D, Deal DC, Haggard P (2009a) Feelings of control: contingency determines experience of action. Cognition 110(2):279–283. https://www.sciencedirect.com/science/article/pii/S0010027708002771?casa_token=PQqmKJMSD_8AAAAA:Lxh2F7APr3-6IKn_t6DB3QFp274UvI37BklVJ6ggrHMVlUh04onORztxuMJQRYHlkAf83BRjMe4
Moore JW, Wegner DM, Haggard P (2009b) Modulating the sense of agency with external cues. Conscious Cogn 18(4):1056–1064. https://doi.org/10.1016/j.concog.2009.05.004
Morey RD, Rouder JN, Jamil T, Morey MRD (2018) Bayesfactor. Computation of Bayes Factors for common designs. R package version 0.9. 12-4.2.
Nafcha O, Higgins ET, Eitam B (2016) Control feedback as the motivational force behind habitual behavior. Progress in brain research, vol 229. Elsevier, New York, pp 49–68
O’Callaghan C, Vaghi MM, Brummerloh B, Cardinal RN, Robbins TW (2019) Impaired awareness of action-outcome contingency and causality during healthy ageing and following ventromedial prefrontal cortex lesions. Neuropsychologia 128:282–289. https://doi.org/10.1016/J.NEUROPSYCHOLOGIA.2018.01.021
Peirce JW (2007) PsychoPy—psychophysics software in Python. J Neurosci Methods 162(1–2):8–13
Penton T, Wang X, Coll M-P, Catmur C, Bird G (2018) The influence of action–outcome contingency on motivation from control. Exp Brain Res. https://doi.org/10.1007/s00221-018-5374-4
Pfister R, Obhi SS, Rieger M, Wenke D (2014) Action and perception in social contexts: intentional binding for social action effects. Front Hum Neurosci 8:667
Press C, Kok P, Yon D (2020) The perceptual prediction paradox. Trends Cogn Sci 24(1):13–24
Preston C, Newport R (2010) Self-denial and the role of intentions in the attribution of agency. Conscious Cogn 19(4):986–998
Redgrave P, Gurney K (2006) The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci 7(12):967–975
Redgrave P, Gurney K, Reynolds J (2008) What is reinforced by phasic dopamine signals? Brain Res Rev 58(2):322–339
Rescorla RA (1967) Pavlovian conditioning and its proper control procedures. Psychol Rev 74(1):71
Saito N, Takahata K, Murai T, Takahashi H (2015) Discrepancy between explicit judgement of agency and implicit feeling of agency: implications for sense of agency and its disorders. Conscious Cogn 37:1–7
Shanks DR, Dickinson A (1991) Instrumental judgment and performance under variations in action-outcome contingency and contiguity. Mem Cognit. https://doi.org/10.3758/BF03197139
Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22(11):1359–1366
Singmann H, Bolker B, Westfall J, Aust F (2015) afex: analysis of factorial experiments. R package version 0.13–145
Skinner BF (1965) Science and human behavior (Issue 92904). Simon and Schuster, New York
Spellman BA (1996) Acting as intuitive scientists: contingency judgments are made while controlling for alternative potential causes. Psychol Sci 7(6):337–342
StataCorp (2015) Stata statistical software: release 14. https://doi.org/10.2307/2234838
Sun D, Custers R, Marian H, Liefooghe B, Aarts H (In press) Examining mechanistic explanations for ideomotor effects. J Exp Psychol Human Percept Perform
Suzuki K, Lush P, Seth A, Roseboom W (2019) Intentional binding without intentional action. Psychol Sci 30(6):842–853
Synofzik M, Vosgerau G, Newen A (2008) Beyond the comparator model: a multifactorial two-step account of agency. Conscious Cogn 17(1):219–239. https://doi.org/10.1016/j.concog.2007.03.010
Takahata K, Takahashi H, Maeda T, Umeda S, Suhara T, Mimura M, Kato M (2012) It’s not my fault: postdictive modulation of intentional binding by monetary gains and losses. PLoS ONE 7(12):e53421
Tanaka T, Watanabe K, Tanaka K (2021) Immediate action effects motivate actions based on the stimulus–response relationship. Exp Brain Res 239(1):67–78
Tapal A, Oren E, Dar R, Eitam B (2017) The sense of agency scale: a measure of consciously perceived control over one’s mind, body, and the immediate environment. Front Psychol 8:1552. https://doi.org/10.3389/fpsyg.2017.01552
Torchiano M, Torchiano MM (2020) Package ‘effsize.’ Package ‘Effsize.’
Vaghi MM, Cardinal RN, Apergis-Schoute AM, Fineberg NA, Sule A, Robbins TW (2019) Action-outcome knowledge dissociates from behavior in obsessive-compulsive disorder following contingency degradation. Biol Psychiatry Cogn Neurosci Neuroimaging 4(2):200–209
Vallée-Tourangeau F, Murphy RA, Baker AG (2005) Contiguity and the outcome density bias in action–outcome contingency judgements. Q J Exp Psychol Sect B 58(2b):177–192
Van Hamme LJ, Wasserman EA (1994) Cue competition in causality judgments: the role of nonpresentation of compound stimulus elements. Learn Motiv. https://doi.org/10.1006/LMOT.1994.1008
VandenBos GR (2007) APA dictionary of psychology. American Psychological Association
Waskom M (2018) Seaborn: statistical data visualization—Seaborn 0.9.0 Documentation. Sphinx 1.7.4
Wasserman EA, Chatlosh DL, Neunaber DJ (1983) Perception of causal relations in humans: factors affecting judgments of response-outcome contingencies under free-operant procedures. Learn Motiv 14(4):406–432
Watanabe H, Taga G (2006) General to specific development of movement patterns and memory for contingency between actions and events in young infants. Infant Behav Dev. https://doi.org/10.1016/j.infbeh.2006.02.001
Watanabe H, Taga G (2009) Flexibility in infant actions during arm- and leg-based learning in a mobile paradigm. Infant Behav Dev 32(1):79–90. https://doi.org/10.1016/j.infbeh.2008.10.003
Watanabe H, Taga G (2011) Initial-state dependency of learning in young infants. Hum Mov Sci 30(1):125–142
Wegner DM, Wheatley T (1999) Apparent mental causation. Sources of the experience of will. Am Psychol 54(7):480–492
Weller L, Schwarz KA, Kunde W, Pfister R (2020) Something from nothing: agency for deliberate nonactions. Cognition 196:104136
Wen W (2019) Does delay in feedback diminish sense of agency? A review. Conscious Cogn 73:102759. https://doi.org/10.1016/j.concog.2019.05.007
Wen W, Haggard P (2020) Prediction error and regularity detection underlie two dissociable mechanisms for computing the sense of agency. Cognition. https://doi.org/10.1016/j.cognition.2019.104074
Wen W, Yamashita A, Asama H (2015) The influence of action-outcome delay and arousal on sense of agency and the intentional binding effect. Conscious Cogn 36:87–95
White RW (1959) Motivation reconsidered: the concept of competence. Psychol Rev 66(5):297
Wolpert DM, Ghahramani Z, Jordan MI (1995) An internal model for sensorimotor integration. Sci AAAS Wkly Pap Ed 269(5232):1880–1882
Yon D, Bunce C, Press C (2020) Illusions of control without delusions of grandeur. Cognition 205:104429
Yon D, Frith CD (2021) Precision and the Bayesian brain. Curr Biol 31(17):R1026–R1032
Zaadnoordijk L, Otworowska M, Kwisthout J, Hunnius S (2018) Can infants’ sense of agency be found in their behavior? Insights from babybot simulations of the mobile-paradigm. Cognition. https://doi.org/10.1016/j.cognition.2018.07.006
Zaadnoordijk L, Otworowska M, Kwisthout J, Hunnius S, van Rooij I (2016) The mobile-paradigm as measure of infants’ sense of agency? Insights from babybot simulations. In: 2016 Joint IEEE international conference on development and learning and epigenetic robotics (ICDL-EpiRob), p 41–42
Acknowledgements
We would like to thank Prof. Jan De Houwer for his insights on previous versions of the manuscript. We would like to thank Rotem Ellenbogen and Shiran Sharabi (Tel-Hai College) for their help in running Experiment 1, Lilach Yona, Amier Kardosh and Rivka Aviv (University of Haifa, Israel) for their help in running experiments 2 through 4.
Funding
This research was supported by The Israel Science Foundation (ISF) grant number 339/16 and The Bi-national Science Foundation (BSF) grant number 2016/299 to B.E.
Author information
Authors and Affiliations
Contributions
EH, NK and BE developed the study concept. All authors contributed to the experiments’ design. EH, BE, NK, ITM, performed data analysis. EH and BE wrote the paper; the others commented and contributed to revisions. BE Supervised the study.
Corresponding author
Ethics declarations
Conflict of interest
The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.
Ethical approval
The study was approved by the Ethics Committee, Department of Psychology, University of Haifa (Approval No. 425/16).
Additional information
Communicated by Melvyn A. Goodale.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Hemed, E., Karsh, N., Mark-Tavger, I. et al. Motivation(s) from control: response-effect contingency and confirmation of sensorimotor predictions reinforce different levels of selection. Exp Brain Res 240, 1471–1497 (2022). https://doi.org/10.1007/s00221-022-06345-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00221-022-06345-3