Skip to main content
SpringerLink
Log in

Increasing the Sensitivity of Measures to Change

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

Little attention is paid in prevention research to the ability of measures to accurately assess change, termed “responsiveness” or “sensitivity to change.” This paper reviews definitions and measures of responsiveness, and suggests five strategies for increasing sensitivity to change, with central focus on prevention research with small samples: (a) improving understandability and cultural validity, (b) assuring that the measure covers the full range of the latent construct being measured, (c) eliminating redundant items, (d) maximizing sensitivity of the device used to collect responses; and (e) asking directly about change. Examples of the application of each strategy are provided. The discussion focuses on using the issues as a checklist for improving measures and the implications of sensitivity to change for prevention research with small samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Allen, J., Mohatt, G., Fok, C. C. T., Henry, D., & People Awakening Team. (2009). Suicide prevention as a community development process: Understanding circumpolar youth suicide prevention through community level outcomes. International Journal of Circumpolar Health, 68, 274–291. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2875412/.

    Article  PubMed Central  PubMed  Google Scholar 

  • Allen, J., Mohatt, G. V., Fok, C. C. T., Henry, D., Burket, R., & People Awakening Team. (2014). A protective factor model for alcohol abuse and suicide prevention among Alaska Native youth. American Journal of Community Psychology, 54, 125–139. doi:10.1007/s10464-014-9661-3.

    Article  PubMed Central  PubMed  Google Scholar 

  • Bartunek, J. M., & Moch, M. K. (1987). First-order, second-order, and third-order change and organization development interventions: A cognitive approach. The Journal of Applied Behavioral Science, 23, 483–500. doi:10.1177/002188638702300404.

    Article  Google Scholar 

  • Beneke, M., & Rasmus, W. (1992). ‘Clinical global impressions’ (ECDEU): Some critical comments. Pharmacopsychiatrie, 25, 171–176.

    Article  CAS  Google Scholar 

  • Campbell, D. T., Stanley, J. C., & Gage, N. L. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton, Mifflin and Company.

    Google Scholar 

  • Cox, E. P. (1980). The optimal number of response alternatives for a scale: A review. Journal of Marketing Research, 17, 407–422. doi:10.2307/3150495.

    Article  Google Scholar 

  • Dahlke, F., Lohaus, A., & Gutzmann, H. (1992). Reliability and clinical concepts underlying global judgments in dementia: Implications for clinical research. Psychopharmacology Bulletin, 28, 425–432.

    CAS  PubMed  Google Scholar 

  • Ebesutani, C., Drescher, C. F., Reise, S. P., Heiden, L., Hight, T. L., Damond, J. D., & Young, J. (2012). The Loneliness Questionnaire–Short version: An evaluation of reverse-worded and non-reverse-worded items via item response theory. Journal of Personality Assessment, 94, 427–437. doi:10.1080/00223891.2012.662188.

    Article  PubMed  Google Scholar 

  • Fok, C. C. T., Allen, J., Henry, D., Mohatt, G., & People Awakening Team. (2011a). Multicultural Mastery Scale for Youth: Multidimensional assessment of culturally mediated coping strategies. Psychological Assessment. doi:10.1037/a0025505.

    PubMed Central  PubMed  Google Scholar 

  • Fok, C.C.T., Allen, J.A., & Henry, D., The People Awakening Team. (2011b). The Brief Family Relationship Scale: A brief measure of the relationship dimension in family functioning. doi:10.1177/1073191111425856.

  • Fok, C.C.T., Henry, D., & Allen, J.A. (this issue). Maybe small is too small a term: Introduction to advancing small sample prevention science.

  • Frank, E., Prien, R. F., Jarrett, R. B., Keller, M. B., Kupfer, D. J., Lavori, P. W., Rush, A. J., & Weissman, M. M. (1991). Conceptualization and rationale for consensus definitions of terms in major depressive disorder. Archives of General Psychiatry, 48, 851–855.

    Article  CAS  PubMed  Google Scholar 

  • Grice, H. P. (1975). Logic and conversation. In P. Cole & J. L. Morgan (Eds.), Syntax and semantics (Vol. 3, pp. 41–58). New York: Academic.

    Google Scholar 

  • Guastello, S. J., Aruka, Y., Doyle, M., & Smerz, K. E. (2008). Cross-cultural generalizability of a cusp catastrophe model for binge drinking among college students. Nonlinear Dynamics, Psychology, and Life Sciences, 12, 397–407.

    PubMed  Google Scholar 

  • Guy, W. (1976). Clinical global impressions. In W. Guy (Ed.), ECDEU assessment manual for psychopharmacology, revised (pp. 218–222). Rockville: National Institute of Mental Health.

    Google Scholar 

  • Guyatt, G., Walter, S., & Norman, G. (1987). Measuring change over time: Assessing the usefulness of evaluative instruments. Journal of Chronic Diseases, 40, 171–178.

    Article  CAS  PubMed  Google Scholar 

  • Guyatt, G. H., Deyo, R. A., Charlson, M., Levine, M. N., & Mitchell, A. (1989). Responsiveness and validity in health status measurement: A clarification. Journal of Clinical Epidemiology, 42, 403–408.

    Article  CAS  PubMed  Google Scholar 

  • Hanley, J. A., & McNeil, B. J. (1983). A method of comparing the areas under receiver operating characteristics curves derived from the same cases. Radiology, 148, 839–843.

    Article  CAS  PubMed  Google Scholar 

  • Hays, R. D., & Hadorn, D. (1992). Responsiveness to change: An aspect of validity, not a separate dimension. Quality of Life Research, 1, 73–75.

    Article  CAS  PubMed  Google Scholar 

  • Henry, D., Pavuluri, M. N., Youngstrom, E., & Birmaher, B. (2008). Accuracy of brief and full forms of the child mania rating scale. Journal of Clinical Psychology, 64, 1–14. doi:10.1002/jclp.20464.

    Article  Google Scholar 

  • Himadi, W. G., Boice, R., & Barlow, D. H. (1986). Assessment of agoraphobia—II: Measurement of clinical change. Behaviour Research and Therapy, 24, 321–332.

    Article  CAS  PubMed  Google Scholar 

  • Jackson, T., McKenzie, J., & Hobfoll, S. E. (2000). Communal aspects of self-regulation. In M. Boekaerts, P. R. Pintrich, & M. Zeider (Eds.), Handbook of self-regulation. San Diego: Academic.

    Google Scholar 

  • Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12.

    Article  CAS  PubMed  Google Scholar 

  • Kristal, A. R., Beresford, S. A., & Lazovich, D. (1994). Assessing change in diet-intervention research. The American Journal of Clinical Nutrition, 59, 185S–189S.

    CAS  PubMed  Google Scholar 

  • Lewin, K. (1947). Frontiers in group dynamics. Human Relations, 1, 143–153.

    Article  Google Scholar 

  • Liang, M. H., Fossel, A. H., & Larson, M. G. (1990). Comparisons of five health status instruments for orthopaedic evaluation. Medical Care, 28, 632–642.

    Article  CAS  PubMed  Google Scholar 

  • Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85–106.

    PubMed  Google Scholar 

  • Metz, C. E. (1978). Basic principles of ROC analysis. Seminars in Nuclear Medicine, 8, 283–298.

    Article  CAS  PubMed  Google Scholar 

  • Mohatt, G. V., Fok, C. C. T., Henry, D., Allen, J., & People Awakening Team. (2014). Feasibility of a community intervention for the prevention of suicide and alcohol abuse with Yup’ik Alaska Native youth: The Elluam Tungiinun and Yupiucimta Asvairtuumallerkaa studies. American Journal of Community Psychology, 54, 153–169. doi:10.1007/s10464-014-9646-2.

    Article  PubMed Central  PubMed  Google Scholar 

  • Norman, G. R., Stratford, P., & Regehr, G. (1997). Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach alpha. Journal of Clinical Epidemiology, 50, 869–879.

    Article  CAS  PubMed  Google Scholar 

  • Pavuluri, M. N., Henry, D., Naylor, M., Carbray, J., Sampson, G., & Janicak, P. G. (2004). A pharmacotherapy algorithm for stabilization and maintenance of pediatric bipolar disorder. Journal of American Academy of Child and Adolescent Psychiatry, 43, 859–867.

    Article  Google Scholar 

  • Pavuluri, M. N., Henry, D. B., Devineni, B., Carbray, J. A., & Birmaher, B. (2006). Child Mania Rating Scale: Development, reliability, and validity. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 550–560. doi:10.1097/01.chi.0000205700.40700.50.

    Article  PubMed  Google Scholar 

  • Pearlin, L. I., Menaghan, E. G., Lieberman, M. A., & Mullan, J. T. (1981). The stress process. Journal of Health & Social Behavior, 22, 337–356.

    Article  CAS  Google Scholar 

  • Prochaska, J. O., & DiClemente, C. C. (1983). Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51, 390–395.

    Article  CAS  PubMed  Google Scholar 

  • Puhan, M. A., Bryant, D., Guyatt, G. H., Heels-Ansdell, D., & Schünemann, H. J. (2005). Internal consistency reliability is a poor predictor of responsiveness. Health and Quality of Life Outcomes, 3, 1–8. doi:10.1186/1477-7525-3-33.

    Article  Google Scholar 

  • Raudenbush, S. W., & Liu, X. (2001). Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychological Methods, 6, 387–401. doi:10.1037/1082-989X.6.4.387.

    Article  CAS  PubMed  Google Scholar 

  • Rizopoulos, D. (2006). ltm: An R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 17, 1–25.

    Article  Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34, 100–114.

    Google Scholar 

  • Sato, T. L., Turnbull, C. D., Davidson, J. R. T., & Madakasira, S. (1984). Depressive illness and placebo response. International Journal of Psychiatry Medicine, 14, 171–179.

    Article  Google Scholar 

  • Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.

    Article  Google Scholar 

  • Schwarz, N., Knäuper, B., Hippler, H. J., Noelle-Neumann, E., & Clar, F. (1991). Rating scales: Numeric values may change the meaning of scale labels. Public Opinion Quarterly, 55, 570–582.

    Article  Google Scholar 

  • Sharma, E., Beck, K. H., & Clar, P. I. (2013). Social context of smoking hookah among college students: Scale development and validation. Journal of American College Health, 61, 204–211. doi:10.1080/07448481.2013.787621.

    Article  PubMed  Google Scholar 

  • Spearing, M. K., Post, R. M., Levericha, G. S., Brandt, D., & Nolen, W. (1997). Modification of the Clinical Global Impressions (CGI) scale for use in bipolar illness (BP): The CGI-BP. Psychiatric Research, 73, 159–171.

    Article  CAS  Google Scholar 

  • Spybrook, J., Raudenbush, S. W., Congdon, R., & Martínez, A. (2009). Optimal design for longitudinal and multilevel research: Documentation for the “Optimal Design” software. New York: William T. Grant Foundation.

    Google Scholar 

  • Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use (3rd ed.). Oxford: Oxford University Press.

    Google Scholar 

  • Wakschlag, L.S., Briggs-Gowan, M.J., Choi, S.W., Nichols, S.R., Kestler, J., Burns, J.L., … & Henry, D. (2014). Advancing a multidimensional, developmental spectrum approach to preschool disruptive behavior. Journal of the American Academy of Child & Adolescent Psychiatry, 53(1), 82–96.

  • Wästberg, B. A., Haglund, L., & Eklund, M. (2012). The work environment impact scale–self-rating (WEIS-SR) evaluated in primary health care in Sweden. Work: A Journal of Prevention, Assessment and Rehabilitation, 42, 447–457.

    Google Scholar 

  • West, A. E., Celio, C. I., Henry, D. B., & Pavuluri, M. N. (2011). Child mania rating scale-parent version: A valid measure of symptom change due to pharmacotherapy. Journal of Affective Disorders, 128, 112–119.

    Article  PubMed Central  PubMed  Google Scholar 

  • Zimmerman, M., McGlinchey, J. B., Posternak, M. A., Friedman, M., Attiullah, N., & Boerescu, D. (2006). How should remission from depression be defined? The depressed patient’s perspective. American Journal of Psychiatry, 163, 148–150.

    Article  PubMed  Google Scholar 

Download references

Conflict of Interest

The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. University of Alaska Fairbanks, Fairbanks, AK, USA

    Carlotta Ching Ting Fok

  2. University of Illinois at Chicago, Chicago, IL, USA

    David Henry

Authors
  1. Carlotta Ching Ting Fok
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Henry
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlotta Ching Ting Fok.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fok, C.C.T., Henry, D. Increasing the Sensitivity of Measures to Change. Prev Sci 16, 978–986 (2015). https://doi.org/10.1007/s11121-015-0545-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-015-0545-z

Keywords

Search

Navigation