Skip to main content

Advertisement

SpringerLink
Log in

The More You Say the Less It Means: Overreporting and Attenuated Criterion Validity in a Forensic Disability Sample

  • Published:
Psychological Injury and Law Aims and scope Submit manuscript

Abstract

A recent publication in Psychological Bulletin by McGrath et al. (Psychological Bulletin 136:450–470, 2010) challenged the necessity of response bias indicators in applied psychological assessment on the grounds that there is insufficient empirical support that shows that they are able to moderate the prediction of substantive measures. The current study challenges their conclusions by examining the effects of response bias in a sample of 2,275 disability litigants. We utilized the validity scales of the Minnesota Multiphasic Personality Inventory-2 Restructured Form (MMPI-2-RF; Ben-Porath and Tellegen, 2008) in order to establish a group of litigants who did not exhibit evidence of overreporting and compared them to a group of overreporting litigants. The overreporting group scored significantly higher on every Restructured Clinical Scale. Moreover, we compared the criterion validity between various substantive scales on the MMPI-2-RF and extratest measures [e.g., Beck Depression Inventory-II (Beck, Steer, & Brown, 1996)] and found evidence of significantly attenuated criterion validity in the overreporting group. Implications of the study in light of the review by McGrath and colleagues are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Several studies investigated both positive and negative response bias.

  2. As indicated by Rohling et al. (2011), the actual number of publications examined by McGrath et al. (2010) was 40, not 41 as indicated in the original paper.

  3. The practice from which the data were obtained uses a fixed battery approach, wherein all claimants complete the same selection of response bias, cognitive, and self-report measures, barring noncompliance or linguistic/cultural factors that preclude administration of the complete battery. No criterion measures were systematically excluded from the assessment battery.

  4. While exaggeration of severe psychopathology (e.g., psychosis) is rare in medico-legal settings (see Wygant et al., 2007), 15 % of the sample scored above the cutoff of 70 on Fp-r, reflective of exaggeration in this area. Examinees who scored above 70 on Fp-r had a mean RC8 of 73.5 (14.0 SD), whereas those scoring below 70 on Fp-r had a mean RC8 of 55.2 (11.7 SD). Consequently, it is likely that examinees producing elevations on Fp-r are responsible for the large overall effect size seen on RC8.

  5. The criterion measures were selected for comparison with conceptually relevant MMPI-2-RF scales prior to data analysis. The decision on which measures to examine was based on the assessment experience of three doctoral-level psychologists (DW, JH, and RG) and evaluation of the empirical correlates of the MMPI-2-RF based on the test's Interpretation (Ben-Porath & Tellegen, 2008) and Technical (Tellegen & Ben-Porath, 2008) manuals.

  6. We utilized an alpha of .001 to remain conservative in accepting significant findings. Given the number of correlations we examined (76), we felt that this was sufficient in reducing the possibility of type I error.

References

  • American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders, 4th edn, text revision. Washington: APA

  • Barthlow, D. L., Graham, J. R., Ben-Porath, Y. S., Tellegen, A. & McNulty, J. L. (2002). The appropriateness of the MMPI-2 K correction. Assessment, 9, 219–229.

    Article  PubMed  Google Scholar 

  • Beck, A. T. & Steer, R. A. (1993). Beck Anxiety Inventory. San Antonio: Psychological Corporation.

    Google Scholar 

  • Beck, A. T., Steer, R. A. & Brown, G. K. (1996). Beck Depression Inventory-II. San Antonio: Psychological Corporation.

    Book  Google Scholar 

  • Ben-Porath, Y. S. & Tellegen, A. (2008). Minnesota Multiphasic Personality Inventory-2 Restructured Form: Manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.

    Google Scholar 

  • Bianchini, K. J., Etherton, J. L., Greve, K. W., Heinly, M. T. & Meyers, J. E. (2008). Classification accuracy of the MMPI-2 in the detection of pain-related malingering. Assessment, 15, 435–449.

    Article  PubMed  Google Scholar 

  • Bianchini, K. J., Greve, K. W. & Glynn, G. (2005). Review article: On the diagnosis of malingered pain-related disability: Lessons from cognitive malingering research. The Spine Journal, 5, 404–417.

    Article  PubMed  Google Scholar 

  • Briere, J. (2001). DAPS: Detailed Assessment of Posttraumatic Stress. Odessa: Psychological Assessment Resources.

    Google Scholar 

  • Burchett, D. L. & Ben-Porath, Y. S. (2010). The Impact of overreporting on MMPI-2-RF substantive scale score validity. Assessment, 17, 497–516.

    Article  PubMed  Google Scholar 

  • Bush, S. S., Ruff, R. M., Troster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., … Silver, C. H. (2005). Symptom validity assessment: Practice issues and medical necessity NAN Policy & Planning Committee. Archives of Clinical Neuropsychology, 20, 419–426.

    Article  PubMed  Google Scholar 

  • Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G. & Kaemmer, B. (2001). MMPI-2 (Minnesota Multiphasic Personality Inventory-2):manual for administration, scoring, and interpretation, revised edition. Minneapolis: University of Minnesota Press.

    Google Scholar 

  • Camara, W. J., Nathan, J. S. & Puente, A. E. (2000). Psychological test usage: Implications in professional psychology. Professional Psychology: Research and Practice, 31, 141–154.

    Article  Google Scholar 

  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.

    Article  PubMed  Google Scholar 

  • Davidson, J. R. T., Book, S. W., Colket, J. T., Tupler, L.A., Roth, S., David, D., … Feldman, M.E., 1997. Assessment of a new self-rating scale for posttraumatic stress disorder: The Davidson Trauma Scale. Psychological Medicine 27, 153–160.

    Article  PubMed  Google Scholar 

  • Detrick, P., Chibnall, J. T. & Rosso, M. (2001). Minnesota Multiphasic Personality Inventory-2 in police officer selection: Normative data and relation to the Inwald Personality Inventory. Professional Psychology: Research and Practice, 32, 484–490.

    Article  Google Scholar 

  • Forbey, J. D. & Lee, T. T. C. (2011). An exploration of the impact of invalid MMPI-2 protocols on collateral self-report measure scores. Journal of Personality Assessment, 93, 556–565.

    Article  PubMed  Google Scholar 

  • Gervais, R. O., Ben-Porath, Y. S., Wygant, D. B. & Sellbom, M. (2010). Incremental validity of the MMPI-2-RF over-reporting scales and RBS in assessing the veracity of memory complaints. Archives of Clinical Neuropsychology, 25, 274–284.

    Article  PubMed  Google Scholar 

  • Gervais, R. O., Wygant, D. B., Sellbom, M. & Ben-Porath, Y. S. (2011). Associations between symptom validity test failure and scores on the MMPI-2-RF validity and substantive scales. Journal of Personality Assessment, 93, 508–517.

    Article  PubMed  Google Scholar 

  • Green, P. (2004). Memory Complaints Inventory. Edmonton: Green’s Publishing.

    Google Scholar 

  • Greve, K. W., Bianchini, K. J., Love, J. M., Brennan, A. & Heinly, M. T. (2006). Sensitivity and specificity of MMPI-2 validity scale and indicators to malingered neurocognitive dysfunction in traumatic brain injury. The Clinical Neuropsychologist, 20, 491–512.

    Article  PubMed  Google Scholar 

  • Groth-Marnat, G. (2009). Handbook of psychological assessment (5th ed.). Hoboken: Wiley.

    Google Scholar 

  • Hase, H. D. (1996). Manual for the Multifactor Health Inventory computer-supported and paper & pencil versions. Durham: CogniSyst.

    Google Scholar 

  • Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R. & Conference Participants (Nelson, N. W., et al.). (2009). American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23, 1093-1129.

    Article  PubMed  Google Scholar 

  • Kerns, R. D., Turk, D. C., et al (1985). The West Haven-Yale Multidimensional Pain Inventory (WHYMPI). Pain, 23, 345–356.

    Article  PubMed  Google Scholar 

  • McGrath, R. E., Kim, B. K. & Hough, L. (2011). Our main conclusion stands: Reply to Rohling et al. (2011). Psychological Bulletin, 137, 713–715.

    Article  Google Scholar 

  • McGrath, R. E., Mitchell, M., Kim, B. H. & Hough, L. (2010). Evidence for response bias as a source of error variance in applied assessment. Psychological Bulletin, 136, 450–470.

    Article  PubMed  Google Scholar 

  • Mittenberg, W., Patton, C., Canyock, E. M. & Condit, D. C. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 24, 1094–1102.

    Article  PubMed  Google Scholar 

  • Morey, L. C. (2007). The Personality Assessment Inventory: Professional Manual (2nd ed.). Odessa: Psychological Assessment Resources.

    Google Scholar 

  • Nelson, N. W., Hoelzle, J. B., Sweet, J. J., Arbisi, P. A. & Demakis, G. J. (2010). Updated meta-analysis of the MMPI-2 Symptom Validity Scale (FBS): Verified utility in forensic practice. The Clinical Neuropsychologist, 24, 701–724.

    Article  PubMed  Google Scholar 

  • Rogers, R., Sewell, K. W., Martin, M. A. & Vitacco, M. J. (2003). Detection of feigned mental disorders: A meta-analysis of the MMPI-2 and malingering. Assessment, 10, 160–177.

    Article  PubMed  Google Scholar 

  • Rohling, M. L., Larrabee, G. J., Greiffenstein, M. F., Ben-Porath, Y. S., Lees-Haley, P., Green, P. & Greve, K. W. (2011). A misleading review of response bias: Comment on McGrath, Mitchell, Kim, and Hough (2010). Psychological Bulletin, 137, 708–712.

    Article  PubMed  Google Scholar 

  • Schroeder, R. W., Baade, L. E., Peck, C. P., Von Dran, E. J., Brockman, C. J., Webster, B. K. & Heinrichs, R. J. (2012). Validation of the MMPI-2-RF validity scales in criterion group neuropsychological samples. The Clinical Neuropsychologist, 26, 129–146.

    Article  PubMed  Google Scholar 

  • Sellbom, M. & Bagby, R. M. (2008). Response styles on multi-scale inventories. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed.). New York: Guilford.

    Google Scholar 

  • Sellbom, M., Fischler, G. L. & Ben-Porath, Y. S. (2007). Identifying MMPI-2 predictors of police officer integrity and misconduct. Criminal Justice and Behavior, 34, 985–1004.

    Article  Google Scholar 

  • Sellbom, M., Toomey, J. A., Wygant, D. B., Kucharski, L. T. & Duncan, S. (2010). Utility of the MMPI-2-RF (Restructured Form) validity scales in detecting malingering in a criminal forensic setting: A known-groups design. Psychological Assessment, 22, 22–31.

    Article  PubMed  Google Scholar 

  • Slick, D. J., Sherman, E. M. S. & Iverson, G. L. (1999). Forum: Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13, 545–561.

    Article  PubMed  Google Scholar 

  • Tellegen, A. & Ben-Porath, Y. S. (2008). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): Technical manual. Minneapolis: University of Minnesota Press.

    Google Scholar 

  • Weiss, D. S. & Marmar, C. R. (1996). The impact of event scale—revised. In J. Wilson & T. M. Keane (Eds.), Assessing psychological trauma and PTSD (pp. 399–411). New York: Guilford.

    Google Scholar 

  • Wygant, D. B., Anderson, J. L., Sellbom, M., Rapier, J. L., Algeier, L. M. & Granacher, R. P. (2011). Association of MMPI-2 Restructured Form (MMPI-2-RF) validity scales with structured malingering criteria. Psychological Injury and Law, 4, 13–23.

    Article  Google Scholar 

  • Wygant, D. B., Ben-Porath, Y. S., Arbisi, P. A., Berry, D. T. R., Freeman, D. B. & Heilbronner, R. L. (2009). Examination of the MMPI-2 Restructured Form (MMPI-2-RF) validity scales in civil forensic settings: Findings from simulation and known group samples. Archives of Clinical Neuropsychology, 27, 671–680.

    Article  Google Scholar 

  • Wygant, D. B., Sellbom, M., Ben-Porath, Y. S., Stafford, K. P., Freeman, D. B. & Heilbronner, R. L. (2007). The relation between symptom validity testing and MMPI-2 scores as a function of forensic evaluation context. Archives of Clinical Neuropsychology, 22, 489–99.

    Article  PubMed  Google Scholar 

  • Wygant, D. B., Sellbom, M., Gervais, R. O., Ben-Porath, Y. S., Stafford, K. P., Freeman, D. B. & Heilbronner, R. L. (2010). Further validation of the MMPI-2 and MMPI-2-RF Response Bias Scale (RBS): Findings from disability and criminal forensic settings. Psychological Assessment, 22, 745–756.

    Article  PubMed  Google Scholar 

  • Zlotnick, C., Davidson, J., Shea, M. T. & Pearlstein, T. (1996). The validation of the Davidson Trauma Scale (DTS) in a sample of survivors of childhood sexual abuse. Journal of Nervous and Mental Disease, 184, 255–257.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Eastern Kentucky University, 521 Lancaster Ave, Richmond, KY, 40475, USA

    Christopher W. Wiggins & Dustin B. Wygant

  2. Department of Psychology, Marquette University, Milwaukee, WI, USA

    James B. Hoelzle

  3. Neurobehavioral Associates, University of Alberta, Edmonton, Alberta, Canada

    Roger O. Gervais

Authors
  1. Christopher W. Wiggins
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dustin B. Wygant
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James B. Hoelzle
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roger O. Gervais
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dustin B. Wygant.

Additional information

Portions of this project were presented at the 2012 Annual Conference of the American Psychology–Law Society in San Juan, Puerto Rico.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wiggins, C.W., Wygant, D.B., Hoelzle, J.B. et al. The More You Say the Less It Means: Overreporting and Attenuated Criterion Validity in a Forensic Disability Sample. Psychol. Inj. and Law 5, 162–173 (2012). https://doi.org/10.1007/s12207-012-9137-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12207-012-9137-4

Keywords

Search

Navigation