Abstract
Incremental validity, the ability of a measure to predict or explain variance over and above other measures, is an important psychometric characteristic of standardized measures, but has received little attention idiographically. Idiographic assessment may be an important part of developing a clinical case formulation, guiding treatment by developing an individualized understanding of the variables that trigger and maintain distress. This study examined whether the idiosyncratic cognitive schema hypothesized by a clinician in a cognitive case formulation explained distress incrementally over that of situational triggers. Using daily ratings of situational triggers, idiosyncratic cognitions, and distress, the incremental validity of cognitions in predicting each of six distress measures was tested in a case example using dynamic time series regression. The incremental variance explained by cognitions varied across the distress measures, suggesting that, in this case example, targeting thoughts and beliefs for treatment may be important for only certain types of distress.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Although “cognitions” is a broad term, in the present article they refer to the automatic thoughts and intermediate and core beliefs relevant to the ICS as defined here.
The term causal variable is used in the sense introduced by Haynes (1992; Haynes & O’Brien, 2000) as a variable that covaries with its effect, precedes the effect, has a logical mechanism for the causal relation on the effect, and cannot be wholly attributed to the common influence of a third variable. Temporal precedence is, at best, weakly demonstrated (via possible lagged effects) in the present study. In contrast, an approach frequently used to assess functional causal relationships in severely developmentally disabled individuals involves controlled observation after experimental manipulation of stimulus conditions (e.g., Iwata et al., 1994). However, this latter approach is not plausible in the present study given that the presence, absence, or degree of activation of idiosyncratic cognitions is difficult to control experimentally.
Watson et al. (1995) condensed the six MASQ subscales into three tripartite scales because the three dimensional factor structure fit their large samples well using R-technique factor analysis. However, so as to obtain a specific and fine-grained multidimensional assessment of symptoms and distress, each subscale was separately assessed. Four of these subscales were used in the present study: General Distress (GD): Depressed, GD: Mixed, Anhedonia (loss of interest), and Positive Affectivity. Within the tripartite model, the latter two are considered the most specific measures of depression. Permission to use selected items from the copyrighted MASQ obtained from L.E. Clark (9/29/1999).
Items measuring the ICSs and distress variables were first inspected for adequate variability, skewness, and kurtosis and then detrended with up to an 8th order polynomial regression to permit modeling of an overall linear and curvilinear trend as well as monthly cyclicity. Items which did not covary with other items intended to measure the same construct were eliminated based on the results of confirmatory dynamic factor analysis (for the ICSs) or an intraindividual item analysis (for distress measures). Of the 43 items proposed by the clinician to measure the ICSs in his CCF, 1 item was dropped due to high skewness and 6 were dropped due to inadequate convergent validity with other items assessing the ICS, resulting in retention of 84% of the clinician's items. Of the 39 items measuring the six distress scales for the present study, 7 were dropped.
Using loadings from the unidimensional model is preferable to the using loadings from the final multidimensional model for the ICSs because the latter model allows each ICS factor to covary. If ICS factor scores were created from the multidimensional model, they would include weighted contributions from all ICS items, thereby not limiting the factor score creation to only those items intended to measure that ICS. Also, the final multidimensional ICS model used item parcels instead of individual items.
Following Mumma (2004), the items on the symptom/distress scales were standardized intraindividually prior to creating the scale score because most of these scales were wholly or partly based on standardized measures constructed using equal weighting for (i.e., standardizing) each item.
For GD: Mixed entering all 4 ICSs resulted in a statistically significant increase in R 2 from .25 to .39. For the other 5 distress measures, putting all 4 ICSs in the model did not statistically significantly increase the proportion of daily variance in the distress measure explained by the ICSs that were included in the CCF.
Interestingly, the entire set of 4 ICSs explained a statistically significantly greater incremental proportion of variance over that explained by the ICSs included in the clinician's CCF for only one distress measure (GD: Mixed: z test for the difference in the R 2: z=3.67, p < .0004; Meng, Rosenthal, & Rubin, 1992 ). Thus, the clinician omitted incrementally important cognitive predictors from the CCF for only one of the six distress measures.
References
Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin, 3, 411–423.
Beck, A. T. (1996). Beyond belief: A theory of modes, personality, and psychopathology. In P. M. Salkovskis (Ed.), Frontiers of cognitive therapy. New York: Guilford.
Beck, A. T., Emery, G., & Greenberg, R. L. (1985). Anxiety disorders and phobias: A cognitive perspective. New York: Basic Books.
Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56, 893–897.
Beck, A. T., Rush, A. J., Shaw, B. F., & Emery, G. (1979). Cognitive therapy of depression. New York: Guilford.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). BDI-II: Beck depression inventory–-Second Edition: Manual. San Antonio: Psychological Corporation.
Beck, J. S. (1995). Cognitive therapy: Basics and beyond. New York: Guilford.
Bieling, P. J., & Kuyken, W. (2003). Is cognitive case formulation science or science fiction. Clinical Psychology: Science & Practice, 10, 52–69.
Clark, D. A., & Beck, A. T. (1999). Scientific foundations of cognitive theory and therapy of depression. New York: Wiley.
Craske, M. G., & Tsao, J. C. (1999). Self-monitoring with panic and anxiety disorders. Psychological Assessment, 11, 466–479.
Derogatis, L. R. (1983). The SCL-90-R: Administration, scoring, and procedures manual II. Baltimore: Clinical Psychometric research.
Durbin, J., & Koopman, S. J. (2001). Time series analysis by state space methods. New York: Oxford.
Eells, T. D. (Ed.) (1997). Handbook of psychotherapy case formulation. New York: Guilford.
Feldman, S. I., Downey, G., & Schaffer-Neitz, R. (1999). Pain, negative mood, and perceived support in chronic pain patients: A daily diary study of people with reflex sympathetic dystrophy syndrome. JCCP, 67, 776–785.
Ferrer, E., & Nesselroade, J. R. (2003). Modeling affective processes in dyadic relations via dynamic factor analysis. Emotion, 3, 344–360.
First, M. B., Gibbon, M., Spitzer, R. L., & Williams, J. B. (1996). Structured Clinical Interview for the Diagnostic and Statistical Manual IV: Axis I, Research Edition. New York: Biometrics.
Fresco, D. M., Mennin, D. S., Heimberg, R. G., & Turk, C. L. (2003). Using the Penn State Worry Questionnaire to identify individuals with generalized anxiety disorder: A receiver operating characteristic analysis. Journal of Behavior Therapy and Experimental Psychiatry, 34, 283–291.
Franklin, R. D., Gorman, B. S., Beasley, T. M., & Allison, D. B. (1996). Graphical display and visual analysis. In R. D. Franklin, D. B. Allison, & B. Gorman (Eds.), Design and analysis of single-case research. Mahwah, NJ: Erlbaum.
Gunthert, K. C., Cohen, L. H., Butler, A. C., & Beck, J. S. (2005). Predictive role of daily coping and affective reactivity in cognitive therapy outcome: Application of a daily process design to psychotherapy research. Behavior Therapy, 36, 77–88.
Hamaker, E. L., Dolan, C. V., & Molenaar, P. C. M. (2003). ARMA-based SEM when the number of time points T exceeds the number of cases N: Raw data likelihood. Structural Equation Modeling, 10, 352–379.
Hayes, S. C., Barlow, D. H., & Nelson-Gray, R. O. (1999). The scientist practitioner: Research and accountability in the age of managed care (2nd ed). Boston: Allyn and Bacon.
Haynes, S. N. (1992). Models of causality in psychopathology: toward dynamic, synthetic, and nonlinear models of behavior disorders. New York: Macmillan.
Haynes, S. N., Leisen, M. B., & Blaine, D. D. (1997). Functional analytic clinical case models and clinical decision making. Psychological Assessment, 9, 334–348.
Haynes, S. N., & Lench, H. C. (2003). Incremental validity of new clinical assessment measures. Psychological Assessment, 15, 456–466.
Haynes, S. N., Kaholokula, J. K., & Nelson, K. (1999). The idiographic application of nomothetic, empirically based treatments. Clinical Psychology: Science and Practice, 6, 456–461.
Haynes, S. N., & O’Brien, W. H. (1990). Functional analysis in behavior therapy. Clinical Psychology Review, 10, 649–668.
Haynes, S. N., & O’Brien, W. H. (2000). Principles and practice of behavioral assessment. New York: Plenum.
Hershberger, S. L. (1998). Dynamic factor analysis. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 217–249). Mahwah, NJ: Erlbaum.
Hokanson, J. E., Tate, R. L., Niu, X., Stader, S., & Flynn, H. (1994). Illustration of concomitant time series analyses in a case of somatoform disorder. Cognitive Therapy and Research, 18, 413–437.
Hox, J. (2002). Multilevel analysis: Techniques and applications. Erlbaum, NJ: Mahwah.
Kedem, B., & Fokianos, K. (2002). Regression models for time series analysis. Hoboken, NJ: Wiley.
Korchin, S. J. (1976). Modern clinical psychology. New York: Basic.
Latner, J. D., & Wilson, G. T. (2002). Self-monitoring and the assessment of binge eating. Behavior Therapy, 33, 465–477.
McCleary, R., & Hay, R. A. (1980). Applied time series analysis. Beverly Hills, CA: Sage.
McKnight, D. L., Nelson, R. O., Hayes, S. C., & Jarrett, R. B. (1984). Importance of treating individually assessed response classes in the amelioration of depression. Behavior Therapy, 15, 315–335.
Meng, X. L., Rosenthal, R., & Rubin, D. B. (1992). Comparing correlated correlation coefficients. Psychological Bulletin, 111, 172–175.
Meyer, T. J., Miller, M. L., Metzger, R. L., & Borkovec, T. D. (1990). Development and validation of the Penn State Worry Questionnaire. Behavior Research and Therapy, 28, 487–495.
Molenaar, P. C. M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement, 2, 201–218.
Molenaar, P. C. M., De Gooijer, J. C., & Schmitz, B. (1992). Dynamic factor analysis of nonstationary multivariate time series. Psychometrica, 57, 333–349.
Molenaar, P. C. M., & Valsiner, J. (2005). How generalization works through the single case: A simple idiographic process analysis of an individual psychotherapy. International Journal of Idiographic Science, 1. Retrieved April 20, 2005 from http://www.valsiner.com/articles/molenvals.htm
Mumma, G. H. (1998). Improving cognitive case formulations and treatment planning in clinical practice and research. Journal of Cognitive Psychotherapy, 12, 251–274.
Mumma, G. H. (2001a). Increasing accuracy in clinical decision making: Towards an integration of nomothetic-aggregate and intraindividual-idiographic approaches. The Behavior Therapist, 24, 77–94.
Mumma, G. H. (2001b). Manual for the Cognitive-Behavioral-Interpersonal Semi-Structured Assessment Interview. Unpublished manual.
Mumma, G. H. (2004). Validation of idiosyncratic cognitive schema in cognitive case formulations: An intraindividual idiographic approach. Psychological Assessment, 16, 211–230.
Mumma, G. H., & Mooney, S. R. (2005). Comparing Independently Generated Idiographic Cognitive Case Formulations: A Latent Variable, Multivariate Time Series Approach. Manuscript under review.
Mumma, G. H., & Smith, J. L. (2001). Cognitive-behavioral-interpersonal scenarios: Inter-formulator reliability and convergent validity. Journal of Psychopathology and Behavioral Assessment,23, 203–221.
Needleman, L. D. (1999). Cognitive case conceptualization. Mahwah, NJ: Erlbaum.
Nesselroade, J. R., & Molenaar, P. C. M. (1999). Pooling lagged covariance structures based on short, multivariate time series for dynamic factor analysis. In R. H. Hoyle (Ed.), Statistical strategies or small sample research (pp. 224–251). Newbury Park, CA: Sage.
Nezu, A. M., Nezu, C. M., Friedman, S. H., & Haynes, S. N. (1997). Case formulation in behavior therapy: Problem solving and functional analytic strategies. In T. D. Eells (Ed.), Handbook of psychotherapy case formulation (pp. 368–401). New York: Guilford.
Nezu, A. M., Nezu, C. M., & Lombardo, E. (2004). Cognitive-behavioral case formulation and treatment design: A problem-solving approach. New York: Springer.
Pankratz, A. (1991). Forecasting with dynamic regression models. New York: Wiley.
Persons, J. B. (1989). Cognitive therapy in practice: A case formulation approach. New York: Norton.
Persons, J. B., Davidson, J., & Tompkins, M. A. (2001). Essential components of cognitive-behavior therapy for depression. Washington, DC: American Psychological Association.
Persons, J. B., & Tompkins, M. A. (1997). Cognitive-behavioral case formulation. In T. D. Eells (Ed.), Handbook of psychotherapy case formulation (pp. 314–339). New York: Guilford.
Rodebaugh, T. L., Curran, P. J., & Chambless, D. L. (2002). Expectancy of panic in the maintenance of daily anxiety in panic disorder with agoraphobia: A longitudinal test of competing models. Behavior Therapy, 33, 315–336.
SAS Institute Inc. (2004). SAS/ETS User's Guide, Version 9.1. Cary, NC: SAS Institute Inc.
Steiger, H., Gauvin, L., Jabalpurwala, S., Séquin, J. R., & Stotland, S. (1999). Hypersensitivity to social interactions in bulimic syndromes: Relationship to binge eating. Journal of Consulting and Clinical Psychology, 67, 765–775.
Watson, D., Clark, L. A., Weber, K., Assenheimer, J. S., Strauss, M. E., & McCormick, R. A. (1995). Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. Journal of Abnormal Psychology, 104, 15–25.
Wood, P., & Brown, D. (1994). The study of intraindividual differences by means of dynamic factor models: Rationale, implementation, and interpretation. Psychological Bulletin, 116, 166–186.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mumma, G.H., Mooney, S.R. Incremental Validity of Cognitions in a Clinical Case Formulation: An Intraindividual Test in a Case Example. J Psychopathol Behav Assess 29, 17–28 (2007). https://doi.org/10.1007/s10862-006-9024-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-006-9024-y