Skip to main content
SpringerLink
Log in

Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies

  • Methodological Paper
  • Published:
Journal of the Academy of Marketing Science Aims and scope Submit manuscript

Abstract

The results of this research suggest a new mandate for discriminant validity testing in marketing. Specifically, the authors demonstrate that the AVE-SV comparison (Fornell and Larcker 1981) and HTMT ratio (Henseler et al. 2015) with 0.85 cutoff provide the best assessment of discriminant validity and should be the standard for publication in marketing. These conclusions are based on a thorough assessment of the literature and the results of a Monte Carlo simulation. First, based on a content analysis of articles published in seven leading marketing journals from 1996 to 2012, the authors demonstrate that three tests—the constrained phi (Jöreskog 1971), AVE-SV (Fornell and Larcker 1981), and overlapping confidence intervals (Anderson and Gerbing 1988)—are by far most common. Further review reveals that (1) more than 20% of survey-based and over 80% of non-survey-based marketing studies fail to document tests for discriminant validity, (2) there is wide variance across journals and research streams in terms of whether discriminant validity tests are performed, (3) conclusions have already been drawn about the relative stringency of the three most common methods, and (4) the method that is generally perceived to be most generous is being consistently misapplied in a way that erodes its stringency. Second, a Monte Carlo simulation is conducted to assess the relative rigor of the three most common tests, as well as an emerging technique (HTMT). Results reveal that (1) on average, the four discriminant validity testing methods detect violations approximately 50% of the time, (2) the constrained phi and overlapping confidence interval approaches perform very poorly in detecting violations whereas the AVE-SV test and HTMT (with a ratio cutoff of 0.85) methods perform well, and (3) the HTMT.85 method offers the best balance between high detection and low arbitrary violation (i.e., false positive) rates.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Anderson, J. C., & Gerbing, D. W. (1988). Structural Equation Modeling in Practice: A Review and Recommended Two-Step Approach. Psychological Bulletin, 103(3), 411–423.

  • Bacharach, S. B. (1989). Organizational theories: Some criteria for evaluation. Academy of Management Review, 14(4), 496–515.

  • Baggozi, R. P., & Philips, L. W. (1982). Representing and Testing Organizational Theories: A Holistic Construal. Administrative Science Quarterly, 27(3), 459–489.

  • Bagozzi, R. P. (1981). Evaluating Structural equation models with unobservable variables and measurement error: a comment. Journal of Marketing Research, 18(3), 375–381.

  • Batra, R., & Sinha, I. (2000). Consumer-Level Factors Moderating the Success of Private Label Brands. Journal of Retailing, 76(2), 175–191.

  • Burton, S., Liechenstein, D. R., Netemeyer, R. G., & Garreston, J. A. (1998). A Scale for Measuring Attitude Toward Private Label Products and Examination of Its Psychological and Behavioral Correlates. Journal of the Academy of Marketing Science, 26(4), 293–306.

  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and Discriminant Validity by the Multitrait-Multimethod Matrix. Psychological Bulletin, 56(2), 81–105.

  • Cannon, J. P., & Homburg, C. (2001). Buyer–supplier Relationships and Firm Costs. Journal of Marketing, 65(1), 29–43.

  • Churchill, G. A. (1979). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, 16(1), 64–73.

    Article  Google Scholar 

  • Farrell, A. M. (2010). Insufficient discriminant validity: A comment on Bove, Pervan, Beatty, and Shiu (2009). Journal of Business Research, 63(3), 324–327.

  • Finn, J. D. (1974). A general model for multivariate analysis. New York: Holt, Rinehart, & Winston.

  • Folger, R. (1989). Significance Tests and the Duplicity of Binary Decisions. Psychological Bulletin, 106(1), 155–160.

  • Fornell, C., & Larcker, D. F. (1981). Evaluating Structural Equation Models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50.

  • Frambach, R. T., Prabhu, J., & Verhallen, T. M. M. (2003). The Influence of Business Strategy on New Product Activity: The Role of Market Orientation. International Journal of Research in Marketing, 20(4), 377–397.

  • Grewal, R., Cote, J. A., & Baumgartner, H. (2004). Multicollinearity and Measurement Error in Structural Equation Models: Implications for Theory Testing. Marketing Science, 23(4), 519–529.

  • Harris, L. C., & Goode, M. M. H. (2004). The four levels of loyalty and the pivotal role of trust: a study of online service dynamics. Journal of Retailing, 80, 139–158.

  • Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43, 115–135.

    Article  Google Scholar 

  • Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453.

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

  • Jap, S. D. (2001). “Pie Sharing” in Complex Collaboration Contexts. Journal of Marketing Research, 38(1), 86–99.

  • Joreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36(4), 409–426.

    Article  Google Scholar 

  • Kohli, A. K., Shervani, T. A., & Challagalla, G. N. (1998). Learning and Performance Orientation of Salespeople: The Role of Supervisors. Journal of Marketing Research, 35(2), 263–274.

    Article  Google Scholar 

  • Lord, F. M. (1957), A signif'icance test for the hypothesis that two variables measure the same trait except for errors of'measurement, Psychometrika, 207–220.

  • Low, G. S., & Mohr, J. J. (2001). Factors Affecting the Use of Information in the Evaluation of Marketing Communications Productivity. Journal of the Academy of Marketing Science, 29(1), 70–88.

    Article  Google Scholar 

  • Lytle, R. S., Hom, P. W., & Mokwa, M. P. (1998). SERV*OR: A Managerial Measure of Service Orientation. Journal of Retailing, 74(4), 455–589.

  • Mathwick, C., & Rigdon, E. (2004). Play, Flow, and the Online Search Experience. Journal of Consumer Research, 31(2), 324–332.

  • Mathwick, C., Malhotra, N., & Rigdon, E. (2001). Experiential Value: Conceptualization, Measurement, and Application in the Catalog and Internet Shopping Environment. Journal of Retailing, 77(1), 39–56.

  • Maxham, J. G., & Netemeyer, R. G. (2002). A Longitudinal Study of Complaining Customers’ Evaluations of Multiple Service Failures and Recovery Efforts. Journal of Marketing, 66(4), 57–71.

  • Peter, J. P. (1981). Construct Validity: A Review of Basic Issues and Marketing Practices. Journal of Marketing Research, 18(2), 133–145.

  • Pollard, P. (2014), How Significant is ‘Significance’? in A Handbook for Data Analysis in the Behaviorial Sciences, Volume 1: Methodological Issues Volume 2: Statistical Issues, 449

  • Rich, G. A. (1997). The Sales Manager as a Role Model: Effects on Trust, Job Satisfaction, and Performance of Salespeople. Journal of the Academy of Marketing Science, 25(4), 319–328.

  • Spreng, R. A., MacKenzie, S. B., & Olshavsky, R. W. (1996). A Reexamination of the Determinants of Consumer Satisfaction. Journal of Marketing, 60(3), 15–32.

  • Wang, G., & Netemeyer, R. G. (2002). The effects of job autonomy, customer demandingness, and trait competitiveness on salesperson learning, self-efficacy, and performance. Journal of the Academy of Marketing Science, 30(3), 217–228.

    Article  Google Scholar 

  • Yilmaz, C., & Hunt, S. D. (2001). Salesperson Cooperation: The Influence of Relational, Task, Organizational, and Personal Factors. Journal of the Academy of Marketing Science, 29(4), 335–357.

Download references

Acknowledgments

The authors would like to thank Peter Bentler for his comments on an earlier version of the simulations used in this paper.

Author information

Authors and Affiliations

  1. Department of Marketing, The Eli Broad Graduate School of Management, Michigan State University, N370 North Business Complex, East Lansing, MI, 48824-1122, USA

    Clay M. Voorhees & Roger Calantone

  2. Department of Marketing, Florida State University, 821 Academic Way, Tallahassee, FL, 32306-1110, USA

    Michael K. Brady

  3. Department of Marketing and Management, College of Business Administration, University of Texas at El Paso, 500 West University Avenue, El Paso, TX, 79968-0539, USA

    Edward Ramirez

Authors
  1. Clay M. Voorhees
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael K. Brady
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roger Calantone
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Edward Ramirez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Clay M. Voorhees.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Voorhees, C.M., Brady, M.K., Calantone, R. et al. Discriminant validity testing in marketing: an analysis, causes for concern, and proposed remedies. J. of the Acad. Mark. Sci. 44, 119–134 (2016). https://doi.org/10.1007/s11747-015-0455-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11747-015-0455-4

Keywords

Search

Navigation