Abstract
The ACT model consists of acceptance, cognitive defusion, contact with the present moment, self-as-context, values, and committed action, which together create psychological flexibility. Limited research has examined the unique contribution of values-focused work in acceptance-based therapies. To investigate this in a reliable and valid way, it is critical to ensure that the instruments used to measure values are empirically sound. This review aims to identify value-based psychometric tools currently in use, and examine their ability to reliably and validly measure the ACT-defined concept of values. The current study searched PsycINFO, Medline, and PubMed databases for psychometric validation papers of values-measurement instruments. Seventeen values-measures were evaluated by extracting data relating to their content, structural, construct, convergent, and discriminant validity, as well as internal consistency and test–retest reliability. The COSMIN manual for systematic reviews of Patient-Reported Outcome Measures (PROMs) was utilized as a guideline for assessing bias and examining the quality of psychometric tools identified. Outcomes suggest that the Valuing Questionnaire (Smout, Davies, Burns, & Christie, 2014), Engaged Living Scale (Trompetter et al., 2013), Valued Living Questionnaire (Wilson, Sandoz, Kitchens, & Roberts, 2010), Multidimensional Psychological Flexibility Inventory (Rolffs, Rogge, & Wilson, 2018), Acceptance and Action Questionnaire for Obsessions and Compulsions (Jacoby, Abramowitz, Buchholz, Reuman, & Blakey, 2018), and Bulls-Eye Values Survey (Lundgren, Luoma, Dahl, Strosahl, & Melin, 2012) have the best psychometric properties. A number of alternative values-based instruments demonstrate preliminary evidence for their utility, though further examination of these is necessary. This review also highlights a number of issues pertaining to the cohesiveness and psychometric comprehensiveness of current values-measurement research, with recommendations for improvement.
Similar content being viewed by others
References
References marked with an “†” indicate psychometric validation studies included
†Åkerblom, S., Perrin, S., Fischer, M. R., & McCracken, L. M. (2017). Further validation of the Chronic Pain Values Inventory in a Swedish chronic pain sample. Journal of Contextual Behavioral Science, 6(3), 261–267. https://doi.org/10.1016/j.jcbs.2017.06.001
Antonovsky, A. (1979). Health, stress, and coping. San Francisco, CA: Jossey-Bass.
Blackledge, J. T., Ciarrochi, J., & Bailey, A. (2006). Personal values questionnaire. Unpublished manuscript. Australia: University of Wollongong.
Blackledge, J. T., Spencer, R., & Ciarrochi, J. (2007). Initial validation of the personal values questionnaire. Paper presented at the 33rd Annual Convention of the Association for Behavior Analysis International, San Diego, CA.
Bond, F. W., & Bunce, D. (2003). The role of acceptance and job control in mental health, job satisfaction and work performance. Journal of Applied Psychology, 88, 1057–1067. https://doi.org/10.1037/0021-9010.88.6.1057.
Bond, F. W., Hayes, S. C., Baer, R. A., Carpenter, K. M., Guenole, N., Orcutt, H. K., et al. (2011). Preliminary psychometric properties of the Acceptance and Action Questionnaire–II: A revised measure of psychological inflexibility and experiential avoidance. Behavior Therapy, 42, 676–688. https://doi.org/10.1016/j.beth.2011.03.007.
Carey, K. B., Maisto, S. A., Carey, M. P., & Purnine, D. M. (2001). Measuring readiness-to-change substance misuse among psychiatric outpatients: I. Reliability and validity of self-report measures. Journal of Studies on Alcohol, 62(1), 79–88. https://doi.org/10.15288/jsa.2001.62.79.
†Carvalho, S. A., Palmeira, L., Pinto-Gouveia, J., Gillanders, D., & Castilho, P. (2018). The utility of the valuing questionnaire in chronic pain. Journal of Contextual Behavioral Science, 9, 21–29. https://doi.org/10.1016/j.jcbs.2018.06.002
†Cederberg, J. T., Strandskov, S. W., Dahl, J., & Ljungman, G. (2017). Parents’ relationship to pain during children’s cancer treatment: A preliminary validation of the Pain Flexibility Scale for parents. Journal of Pain Research, 10, 507–514. https://doi.org/10.2147/JPR.S127019
Ciarrochi, J., & Bailey, A. (2008). A CBT-practitioner's guide to ACT: How to bridge the gap between Cognitive Behavioral Therapy and Acceptance and Commitment Therapy. Oakland, CA: New Harbinger.
Cohen, L., Manion, L., & Morrison, K. (2002). Research methods in education. New York, NY: Routledge.
Demetriou, C., Ozer, B. U., & Essau, C. A. (2015). Self-report questionnaires. In The encyclopedia of clinical psychology (pp. 1–6). Wiley Online Library. https://doi.org/10.1002/9781118625392.wbecp507.
Doi, S., Yokomitsu, K., & Sakano, Y. (2014). Personal Values Questionnaire–II: Internal consistency and validity. Japanese Journal of Behavior Therapy, 40, 45–55 [in Japanese with English abstract].
†Drake, C. E., Masuda, A., Dalsky, D., Stevens, K. T., Kramer, S., Primeaux, S. J., . . . & Mitamura, T. (2019). Examining US and Japanese college students’ differences in psychological distress: The mediating roles of valued action and experiential avoidance. International Journal for the Advancement of Counselling, 41, 25–40. https://doi.org/10.1007/s10447-018-9342-2
Eisenbeck, N., Scheitz, K., & Szekeres, B. (2016). A brief Acceptance and Commitment Therapy-based intervention among violence-prone male inmates delivered by novice therapists. Psychology, Society & Education, 8(3), 187–199. https://doi.org/10.25115/psye.v8i3.173.
Emmons, R. A. (2003). Personal goals, life meaning, and virtue: Wellsprings of a positive life. In Flourishing: Positive psychology and the life well-lived (pp. 105–128). Washington, DC: American Psychological Association.
Emmons, R. A., & McAdams, D. (1991). Personal strivings and motive dispositions: Exploring the links. Personality & Social Psychology Bulletin, 17(6), 648–654. https://doi.org/10.1177/0146167291176007.
Fernandes, C., Castilho, P., & Pinto-Gouveia, J. (2012). Validação da versão Portuguesa do Questionário de Valores [Portuguese Validation of the Valued Living Questionnaire]. Unpublished manuscript.
Festinger, L. (1962). A theory of cognitive dissonance (Vol. 2). Stanford, CA: Stanford University Press.
Forman, E. M., Herbert, J. D., Moitra, E., Yeomans, P. D., & Geller, P. A. (2007). A randomized controlled effectiveness trial of acceptance and commitment therapy and cognitive therapy for anxiety and depression. Behavior Modification, 31(6), 772–799. https://doi.org/10.1177/0145445507302202.
†Francis, A. W., Dawson, D. L., & Golijani-Moghaddam, N. (2016). The development and validation of the Comprehensive assessment of Acceptance and Commitment Therapy processes (CompACT). Journal of Contextual Behavioral Science, 5(3), 134–145. https://doi.org/10.1016/j.jcbs.2016.05.003
Gaudiano, B. A., & Herbert, J. D. (2006). Acute treatment of inpatients with psychotic symptoms using Acceptance and Commitment Therapy: Pilot results. Behavior Research & Therapy, 44(3), 415–437. https://doi.org/10.1016/j.brat.2005.02.007.
Gifford, E. V., Kohlenberg, B. S., Hayes, S. C., Antonuccio, D. O., Piasecki, M. M., Rasmussen-Hall, M. L., & Palm, K. M. (2004). Acceptance-based treatment for smoking cessation. Behavior Therapy, 35(4), 689–705. https://doi.org/10.1016/S0005-7894(04)80015-7.
Glick, D M & Orsillo, S. M. (2009). Academic Values Questionnaire (AVQ). [5-item Likert-type scale assessing the extent to which students value academics]. Unpublished instrument.
Grégoire, S., Lachance, L., Bouffard, T., & Dionne, F. (2018). The use of acceptance and commitment therapy to promote mental health and school engagement in university students: A multisite randomized controlled trial. Behavior Therapy, 49(3), 360–372. https://doi.org/10.1016/j.beth.2017.10.003.
Hayes, S. C., Barnes-Holmes, D., & Roche, B. (2001). Relational frame theory: A post-Skinnerian approach to language and cognition. New York, NY: Plenum Press.
Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (2011). Acceptance and commitment therapy: The process and practice of mindful change. New York, NY: Guilford Press.
Henson, R. K., & Roberts, J. K. (2006). Use of exploratory factor analysis in published research: Common errors and some comment on improved practice. Educational & Psychological Measurement, 66, 393–416. https://doi.org/10.1177/0013164405282485.
Hickey, A. M., Bury, G., O’Boyle, C. A., Bradley, F., O’Kelly, F. D., & Shannon, W. (1996). A new short form individual quality of life measure (SEIQoL-DW): Application in a cohort of individuals with HIV/AIDS. BMJ, 313(7048), 29–33. https://doi.org/10.1136/bmj.313.7048.29.
†Jacoby, R. J., Abramowitz, J. S., Buchholz, J., Reuman, L., & Blakey, S. M. (2018). Experiential avoidance in the context of obsessions: Development and validation of the Acceptance and Action Questionnaire for Obsessions and Compulsions. Journal of Obsessive-Compulsive & Related Disorders, 19, 34–43. https://doi.org/10.1016/j.jocrd.2018.07.003
†Jensen, M. P., Vowles, K. E., Johnson, L. E., & Gertz, K. J. (2015). Living well with pain: Development and preliminary evaluation of the Valued Living Scale. Pain Medicine, 16(11), 2109–2120. https://doi.org/10.1111/pme.12802
Johns, R. (2010). Likert items and scales. Survey Question Bank: Methods Fact Sheet, 1, 1–11.
Jones, E. E. (1990). Interpersonal perception. WH Freeman/Times Books/Henry Holt & Co.
Kemani, M. K., Hesser, H., Olsson, G. L., Lekander, M., & Wicksell, R. K. (2016). Processes of change in Acceptance and Commitment Therapy and Applied Relaxation for long-standing pain. European Journal of Pain, 20(4), 521–531. https://doi.org/10.1002/ejp.754.
Levin, M. E., Hayes, S. C., & Vilardaga, R. (2012). Acceptance and commitment therapy: applying an iterative translational research strategy in behavior analysis. In G. J. Madden (Ed.), APA handbook of behavior analysis (Vol. 2, pp. 455–479). Washington DC: American Psychological Association.
Levitt, J. T., Brown, T. A., Orsillo, S. M., & Barlow, D. H. (2004). The effects of acceptance versus suppression of emotion on subjective and psychophysiological response to carbon dioxide challenge in patients with panic disorder. Behavior Therapy, 35(4), 747–766. https://doi.org/10.1016/S0005-7894(04)80018-2.
†Lundgren, T., Luoma, J. B., Dahl, J., Strosahl, K., & Melin, L. (2012). The bull's-eye values survey: A psychometric evaluation. Cognitive & Behavioral Practice, 9(4), 518–526. https://doi.org/10.1016/j.cbpra.2012.01.004
†Lundgren, T., Reinebo, G., Löf, P. O., Näslund, M., Svartvadet, P., & Parling, T. (2018). The Values, Acceptance, and Mindfulness Scale for ice hockey: A psychometric evaluation. Frontiers in Psychology, 9, 1794. https://doi.org/10.3389/fpsyg.2018.01794
†Luoma, J., Drake, C. E., Kohlenberg, B. S., & Hayes, S. C. (2011). Substance abuse and psychological flexibility: The development of a new measure. Addiction Research & Theory, 19(1), 3–13. https://doi.org/10.3109/16066359.2010.524956
†McCracken, L. M., & Yang, S. Y. (2006). The role of values in a contextual cognitive-behavioral approach to chronic pain. Pain, 123(1–2), 137–145. https://doi.org/10.1016/j.pain.2006.02.021
†Miller, M. B., Meier, E., Lombardi, N., Leavens, E. L., Grant, D. M., & Leffingwell, T. R. (2016). The Valued Living Questionnaire for Alcohol Use: Measuring value-behavior discrepancy in college student drinking. Psychological Assessment, 28(9), 1051–1060. https://doi.org/10.1037/pas0000235
Mokkink, L. B., De Vet, H. C., Prinsen, C. A., Patrick, D. L., Alonso, J., Bouter, L. M., & Terwee, C. B. (2018). COSMIN risk of Bias checklist for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1171–1179. https://doi.org/10.1007/s11136-017-1765-4.
†O’Connor, M., Tennyson, A., Timmons, M. & McHugh, L. (2019). The development and preliminary psychometric properties of the Values Wheel. Journal of Contextual Behavioral Science, 12, 39–46. https://doi.org/10.1016/j.jcbs.2019.01.005
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan: A web and mobile app for systematic reviews. Systematic Reviews, 5, 210. https://doi.org/10.1186/s13643-016-0384-4.
†Pielech, M., Bailey, R. W., McEntee, M. L., Ashworth, J., Levell, J., Sowden, G., & Vowles, K. E. (2016). Preliminary evaluation of the values tracker: A two-item measure of engagement in valued activities in those with chronic pain. Behavior Modification, 40(1–2), 239–256. https://doi.org/10.1177/0145445515616911
Pinto, R. A., Kienhuis, M., Slevison, M., Chester, A., Sloss, A., & Yap, K. (2017). The effectiveness of an outpatient Acceptance and Commitment Therapy Group programme for a transdiagnostic population. Clinical Psychologist, 21(1), 33–43. https://doi.org/10.1111/cp.12057.
Plumb, J. C., Stewart, I., Dahl, J., & Lundgren, T. (2009). In search of meaning: Values in modern clinical behavior analysis. The Behavior Analyst, 32, 85–104. https://doi.org/10.1007/BF03392177.
Prinsen, C. A. C., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., de Vet, H. C. W., & Terwee, C. B. (2018). COSMIN guideline for systematic reviews of patient-reported outcome measures. Quality of Life Research, 27(5), 1147–1157. https://doi.org/10.1007/s11136-018-1798-3.
Prinsen, C. A., Vohra, S., Rose, M. R., Boers, M., Tugwell, P., Clarke, M., et al. (2016). How to select outcome measurement instruments for outcomes included in a “Core Outcome Set”: A practical guideline. Trials, 17, 449. https://doi.org/10.1186/s13063-016-1555-2.
Reilly, E. D., Ritzert, T. R., Scoglio, A. A. J., Mote, J., Fukuda, S. D., Ahern, M. E., & Kelly, M. M. (2019). A systematic review of values measures in acceptance and commitment therapy research. Journal of Contextual Behavioral Science, 12, 290–304. https://doi.org/10.1016/j.jcbs.2018.10.004.
Rogers, C. R. (1964). Toward a modern approach to values: The valuing process in the mature person. Journal of Abnormal & Social Psychology, 68(2), 160–167. https://doi.org/10.1037/h0046419.
Rokeach, M. (1973). The nature of human values. New York, NY: Free Press.
†Rolffs, J. L., Rogge, R. D., & Wilson, K. G. (2018). Disentangling components of flexibility via the hexaflex model: Development and validation of the Multidimensional Psychological Flexibility Inventory (MPFI). Assessment, 25(4), 458–482. https://doi.org/10.1177/1073191116645905
†Romero-Moreno, R., Gallego-Alberto, L., Márquez-González, M., & Losada, A. (2017). Psychometric properties of the valued living questionnaire adapted to dementia caregiving. Aging & Mental Health, 21(9), 983–990. https://doi.org/10.1080/13607863.2016.1191055
Schünemann, R. (2013). In H. Schünemann, J. Brożek, G. Guyatt, & A. Oxman (Eds.)., The GRADE Working Group. GRADE Handbook for Grading Quality of Evidence and Strength of Recommendations. Available from gdt.guidelinedevelopment.org/app/handbook/handbook.html Updated October 2013.
Schwartz, S. H. (1992). Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in Experimental Social Psychology, 25, 1–65. https://doi.org/10.1016/S0065-2601(08)60281-6.
Schwartz, S. H. (1999). A theory of cultural values and some implications for work. Applied Psychology, 48(1), 23–47. https://doi.org/10.1111/j.1464-0597.1999.tb00047.x.
Schwartz, S. H., Melech, G., Lehmann, A., Burgess, S., Harris, M., & Owens, V. (2001). Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. Journal of Cross-Cultural Psychology, 32(5), 519–542. https://doi.org/10.1177/0022022101032005001.
Serowik, K. L., Khan, A. J., LoCurto, J., & Orsillo, S. M. (2018). The conceptualization and measurement of values: A review of the psychometric properties of measures developed to inform values work with adults. Journal of Psychopathology & Behavioral Assessment, 40(4), 615–635. https://doi.org/10.1007/s10862-018-9679-1.
†Smout, M., Davies, M., Burns, N., & Christie, A. (2014). Development of the valuing questionnaire (VQ). Journal of Contextual Behavioral Science, 3(3), 164–172. https://doi.org/10.1016/j.jcbs.2014.06.001.
Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. https://doi.org/10.1016/j.jclinepi.2006.03.012.
Terwee, C. B., Prinsen, C. A. C., Chiarotto, A., Westerman, M. J., Patrick, D. L., Alonso, J., et al. (2018). COSMIN methodology for evaluating the content validity of patient-reported outcome measures: A Delphi study. Quality of Life Research, 27(5), 1159–1170. https://doi.org/10.1007/s11136-018-1829-0.
†Trindade, I. A., Ferreira, C., Pinto-Gouveia, J., & Nooren, L. (2016). Clarity of personal values and committed action: Development of a shorter engaged living scale. Journal of Psychopathology & Behavioral Assessment, 38(2), 258–265. https://doi.org/10.1007/s10862-015-9509-7.
†Trompetter, H. R., ten Klooster, P. M., Schreurs, K. M., Fledderus, M., Westerhof, G. J., & Bohlmeijer, E. T. (2013). Measuring values and committed action with the Engaged Living Scale (ELS): Psychometric evaluation in a nonclinical sample and a chronic pain sample. Psychological Assessment, 25(4), 1235–1246. https://doi.org/10.1037/a0033813.
Twohig, M. P., Hayes, S. C., & Masuda, A. (2006). Increasing willingness to experience obsessions: Acceptance and commitment therapy as a treatment for obsessive-compulsive disorder. Behavior Therapy, 37(1), 3–13. https://doi.org/10.1016/j.beth.2005.02.001.
Tyrberg, M. J., Carlbring, P., & Lundgren, T. (2017). Brief acceptance and commitment therapy for psychotic inpatients: A randomized controlled feasibility trial in Sweden. Nordic Psychology, 69(2), 110–125. https://doi.org/10.1080/19012276.2016.119827.
†Van Buskirk, K., West, L., Malcarne, V., Afari, N., Liu, L., Petkus, A., & Wetherell, J. L. (2012). Confirmatory factor analysis of the valued living questionnaire in a black American sample: Implications for cognitive research and practice. Cognitive Therapy & Research, 36(6), 796–805. https://doi.org/10.1007/s10608-011-9405-8.
Van de Mortel, T. F. (2008). Faking it: Social desirability response bias in self-report research. Australian Journal of Advanced Nursing, 25(4), 40–48.
Varra, A. A., Hayes, S. C., Roget, N., & Fisher, G. (2008). A randomized control trial examining the effect of acceptance and commitment training on clinician willingness to use evidence-based pharmacotherapy. Journal of Consulting & Clinical Psychology, 76(3), 449–460. https://doi.org/10.1037/0022-006X.76.3.449.
Veehof, M. M., Oskam, M. J., Schreurs, K. M. G., & Bohlmeijer, E. T. (2011). Acceptance-based interventions for the treatment of chronic pain: A systematic review and meta-analysis. Pain, 152, 533–542. https://doi.org/10.1016/j.pain.2010.11.002.
Vowles, K. E., Fink, B., & Cohen, L. (2014). Acceptance and Commitment Therapy for chronic pain: A diary study of treatment process in relation to reliable change in disability. Journal of Contextual Behavioral Science, 3, 74–80. https://doi.org/10.1016/j.jcbs.2014.04.003.
Wilson, K. G., & Dufrene, T. (2009). Mindfulness for two: An acceptance and commitment therapy approach to mindfulness in psychotherapy. Oakland, CA: New Harbinger.
Wilson, K. G., & Murrell, A. R. (2004). Values work in Acceptance and Commitment Therapy: Setting a course for behavioral treatment. In S. C. Hayes, V. M. Follette, & M. M. Linehan (Eds.), Mindfulness and acceptance: Expanding the cognitive-behavioral tradition (pp. 120–151). New York, NY: Guilford Press.
†Wilson, K. G., Sandoz, E. K., Kitchens, J., & Roberts, M. (2010). The Valued Living Questionnaire: Defining and measuring valued action within a behavioral framework. The Psychological Record, 60(2), 249–272. https://doi.org/10.1007/BF03395706.
Zettle, R. D., Rains, J. C., & Hayes, S. C. (2011). Processes of change in acceptance and commitment therapy and cognitive therapy for depression: A mediation reanalysis of Zettle and Rains. Behavior Modification, 35(3), 265–283. https://doi.org/10.1177/0145445511398344.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Research Involving Human Participants and/or Animals
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed Consent
As this article is a review of previous psychometric validation studies, data was not collected from participants by the authors; therefore informed consent was not required.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Barrett, K., O’Connor, M. & McHugh, L. A Systematic Review of Values-Based Psychometric Tools Within Acceptance and Commitment Therapy (ACT). Psychol Rec 69, 457–485 (2019). https://doi.org/10.1007/s40732-019-00352-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40732-019-00352-7