Skip to main content
Log in

Invariance of Measures of Prevention Program Effectiveness: A Replication

  • Published:
Journal of Quantitative Criminology Aims and scope Submit manuscript

Abstract

Recent literature has suggested that measures of risk and protective factorsfor delinquency and substance use are not equally reliable or valid acrossgender and ethnic groups and has recommended differentiated programming andculturally specific evaluation methods. Three data sets containing up tofive ethnic groups were used to determine the degree to which risk andprotective factors are equally reliable and valid predictors of drug use anddelinquency across gender and ethnic groups. Congeneric measurement modelsand structural equation models were evaluated to determine if the factorstructures for these measures and their covariances with measures of druguse and delinquency were equivalent across gender and ethnic groups. Half ofthe risk and protective factors included in this analysis were found to beequally reliable across gender and ethnic groups. When controlling forreliability differences, all of the risk and protective factors were foundto predict both drug use and delinquncy for all gender and ethnic groups. Interms of the magnitude of these associations, no substantive differenceswere found in the validity of risk and protective factors for drug use anddelinquency. Differences in the validity of risk and protective factors weremore prevalent for delinquency than for drug use. However, all differenceswere substantively trivial. We conclude that measures of prevention programeffectiveness are invariant across gender and ethnic groups.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

REFERENCES

  • Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automatic Control 19: 716–723.

    Google Scholar 

  • Akaike, H. (1987). Factor analysis and AIC. Psychometrika 52: 317–332.

    Google Scholar 

  • Akers, R. L., Krohn, M. D., Lanza-Kaduce, L., and Radosevich, N. (1979). Social learning and deviant behavior: A specific test of a general theory. Am. Sociol. Rev. 44: 636–655.

    Google Scholar 

  • Alwin, D. F., and Jackson, D. J. (1979). Measurement models for response errors in surveys: Issues and applications. In Schuessler, K. F. (ed.), Sociological Methodology: 1980, Jossey-Bass, San Francisco, pp. 68–119.

    Google Scholar 

  • Anderson, J. C., and Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychol. Bull. 103: 411–423.

    Google Scholar 

  • Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., and Cullen, F. T. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology 28: 369–404.

    Google Scholar 

  • Bandura, A. (1977). Social Learning Theory, Prentice Hall, Englewood Cliffs, NJ.

    Google Scholar 

  • Benson, J. (1987). Detecting item bias in affective scales. Educ. Psychol. Measure. 47: 55–67.

    Google Scholar 

  • Bentler, P. M., and Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 88: 588–606.

    Google Scholar 

  • Bollen, K. A. (1986). Sample size and Bentler and Bonett's nonormed fit index. Psychometrika, 51: 375–377.

    Google Scholar 

  • Bollen, K. A. (1989). Structural Equations with Latent Variables, Wiley, New York.

    Google Scholar 

  • Boomsma, A. (1982). The robustness of LISREL against small sample size in factor analysis models. In Jöreskog, K., and Wold, H. (eds.), Systems Under Indirect Observation, Part I, North-Holland, Amsterdam, pp. 149–173.

    Google Scholar 

  • Bozdogan, H. C. (1987). Model selection and Akaike's information criterion (AIC): The general theory and its analytical extensions. Psychometrika 52: 345–370.

    Google Scholar 

  • Byram, O. W., and Fly, J. W. (1984). Family structure, race and adolescents' alcohol use: A research note. Am. J. Drug Alcohol Abuse 10: 467–478.

    Google Scholar 

  • Byrne, B. M. (1988). The Self Description Questionnaire III: Testing for equivalent factorial validity across ability. Educ. Psychol. Measure. 48: 397–406.

    Google Scholar 

  • Carmines, E. G., and McIver, J. P. (1981). Analyzing models with unobserved variables: Analysis of covariance structures. In Bohrnstedt, G. W., and Borgatta, E. F. (eds.), Social Measurement: Current Issues, Sage, Beverly Hills, CA, pp. 66–115.

    Google Scholar 

  • Carmines, E. G., and Zeller, R. A. (1979). Reliability and Validity Assessment, Sage, Newbury Park, CA.

    Google Scholar 

  • Castro, F. G., Maddahian, E., Newcomb, M. D., and Bentler, P. M. (1987). A multivariate model of the determinants of cigarette smoking among adolescents. J. Health Soc. Behav. 28: 273–289.

    Google Scholar 

  • Catalano, R. F., Morrison, D. M., Wells, E. A., Gillmore, M. R., Iritani, B., and Hawkins, J. D. (1992). Ethnic differences in family factors related to early drug initiation. J. Stud. Alcohol. 53: 208–217.

    Google Scholar 

  • Cole, D. A., and Maxwell, S. E. (1985). Multitrait-multimethod comparisons across populations: A confirmatory factor analytic approach. Multivar. Behav. Res. 20: 389–417.

    Google Scholar 

  • Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika 16: 297–334.

    Google Scholar 

  • Gerbing, D. W., and Hunter, J. E. (1982). The metric of the latent variables in the LISRELIV analysis. Educ. Psychol. Measure. 42: 423–427.

    Google Scholar 

  • Gillmore, M. R., Catalano, R. F., Morrison, D. M., Wells, E. A., Iritani, B., and Hawkins, J. D. (1990). Racial differences in acceptability and availability of drugs and early initiation of substance use. Am. J. Drug Alcohol Abuse 16: 185–206.

    Google Scholar 

  • Gottfredson, D. C. (1997). School-based crime prevention. In Preventing Crime: What Works, What Doesn't, What's Promising, Office of Justice Programs, Washington, DC.

    Google Scholar 

  • Gottfredson, D. C., and Koper, C. S. (1996). Race and sex differences in the prediction of drug use. J. Consult. Clin. Psychol. 64: 305–313.

    Google Scholar 

  • Gottfredson, D. C., and Koper, C. S. (1997). Race and sex differences in the measurement of risk for drug use. J. Quant. Criminol. 13: 325–347.

    Google Scholar 

  • Gottfredson, D. C., Sealock M. D., and Koper, C. S. (1996). Delinquency. In DiClemente, R. J., Hansen, W. B., and Ponton, L. E. (eds.), Handbook of Adolescent Health Risk Behavior, Plenum, New York, pp. 259–288.

    Google Scholar 

  • Gottfredson, G. D., and Gottfredson, D. C. (1992). Development and Applications of Theoretical Measures for Evaluating Drug and Delinquency Prevention Programs, Gottfredson Associates, Elliott City, MD.

    Google Scholar 

  • Gottfredson, M. R., and Hirschi, T. (1990). A General Theory of Crime, Stanford University, Stanord, CA.

    Google Scholar 

  • Hawkins, J. D., Catalano, R. F., and Miller, J. L. (1992). Risk and protective factors for alcohol and other drug problems in early adulthood: Implications for substance abuse prevention. Psychol. Bull. 112: 64–105.

    Google Scholar 

  • Hayduk, L. A. (1987). Structural Equation Modeling with LISREL, Johns Hopkins University, Baltimore.

    Google Scholar 

  • Hayduk, L. A. (1996). Structural Equation Modeling with LISREL: Essentials and Advances, Johns Hopkins Univesity, Baltimore.

    Google Scholar 

  • Hirschi, T. (1969). Causes of Delinquency, University of California, Berkeley.

    Google Scholar 

  • Hoelter, J. W. (1983). The analysis of covariance structures. Goodness-of-fit indices. Sociol. Methods Res. 11: 325–344.

    Google Scholar 

  • Horn, J. L., and McArdle, J. J. (1980). Perspectives on mathematicalystatistical model building (MASMOB) in research on aging. In Poon, L. W. (ed.), Aging in the 1980s: Selected Contemporary Issues in the Psychology of Aging, American Psychological Association, Washington, DC, pp. 503–541.

    Google Scholar 

  • Howell, J. C., Krisberg, B., Wilson, J. J., and Hawkins, J. D. (1995). A Sourcebook on Serious Violent, and Chronic Juvenile Offenders, Sage, Newbury Park, CA.

    Google Scholar 

  • James, L. R., Mulaik, S. A., and Brett, J. M. (1982). Causal Analysis: Assumptions, Models and Data, Sage, Beverly Hills.

    Google Scholar 

  • Jessor, R., and Jessor, S. (1977). Problem Behavior and Psychosocial Development: A Longitudinal Study of Youth, Academic Press, New York.

    Google Scholar 

  • Jöreskog, K. G., and Sörbom, D. (1981). LISREL5: Analysis of Linear Structural Relations by the Method of Maximum Likelihod, International Educational Services, Chicago.

    Google Scholar 

  • Jöreskog, K. G., and Sörbom, D. (1989). LISREL7: A Guide to the Program and Applictions, 2d ed., SPSS, Chicago.

    Google Scholar 

  • Kleinman, P., and Lukoff, I. (1978). Ethnic differences in factors related to drug use. J. Health Soc. Behav. 19: 190–199.

    Google Scholar 

  • Long, J. S. (1983). Covariance Structure Models: An Introduction to LISREL, Sage, Beverly Hills, CA.

    Google Scholar 

  • MacCallum, R. C., and Tucker, L. R. (1991). Representing sources of error in the commonfactor model: Implications for theory and practice. Psychol. Bull. 109: 502–511.

    Google Scholar 

  • Marsh, H. W., Balla, J. R., and McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychol. Bull. 103: 391–410.

    Google Scholar 

  • McArdle, J. J. (1986). Latent variable growth within behavior genetic models. Behav. Genet. 16: 163–200.

    Google Scholar 

  • McDonald, R. P., and Marsh, H. W. (1990). Choosing a multivariate model: Noncentrality and goodness of fit. Psychol. Bull. 107: 247–255.

    Google Scholar 

  • Newcomb, M. D., and Bentler, P. M. (1986). Substance use and ethnicity: Differential impact of peer and adult models. J. Psychol. 120: 83–95.

    Google Scholar 

  • Newcomb, M. D., Maddahian, E., Skager, R., and Bentler, P. M. (1987). Substance abuse and psychological risk factors among teenagers: Associations with sex, age, ethnicity, and type of school. Am. J. Drug Alcohol Abuse 13: 413–433.

    Google Scholar 

  • Nunnally, J. C. (1978). Psychometric Theory, McGraw-Hill, New York.

    Google Scholar 

  • Orlandi, M. A. (ed.) (1992). Cultural Competence for Evaluators: A Guide for Alcohol and Other Drug Abuse Prevention Practitioners Working with Ethnic/Racial Communities, Office of Substance Abuse Prevention, U.S. Department of Health and Human Services, Rockville, MD.

    Google Scholar 

  • Piquero, A. R., and Rosay, A. B. (1998).The reliability and validity of Grasmick, et al.'s Self-Control Scale: A comment on Longshore, et al., Criminology 36: 157–173.

    Google Scholar 

  • Reise, S. P., Widaman, K. F., and Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. Psychol. Bull. 114: 552–566.

    Google Scholar 

  • Rowe, D. C., Vazsonyi, A. T., and Flannery, D. J. (1994). No more than skin deep: ethnic and racial similarity in developmental process. Psychol. Rev. 101: 396–413.

    Google Scholar 

  • Sampson, R. J. (1985). Sex differences in self-reported delinquency and official records: A multiple-group structural modeling approach. J. Quant. Criminol. 1: 345–367.

    Google Scholar 

  • Schwarz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6: 461–464.

    Google Scholar 

  • Tanaka, J. S. (1987). “How big is enough?” Sample size and goodness-of-fit structural equation models with latent variables. Child Dev. 58: 135–146.

    Google Scholar 

  • Tanaka, J. S., and Huba, G. J. (1985). A fit index for covariance structure models under arbitrary GLS estimation. Br. J. Math. Stat. Psychol. 38: 197–201.

    Google Scholar 

  • Tucker, L. R., and Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrika 38: 1–10.

    Google Scholar 

  • Wells, E. A., Morrison, D. M., Gillmore, M. R., Catalano, R. F., Iritani, B., and Hawkins, J. D. (1992). Race differences in antisocial behaviors and attitudes and early initiation of substance use. J. Drug Educ. 22: 115–130.

    Google Scholar 

  • Wheaton, B. (1988). Assessment of fit in overidentified models with latent variables. In Long, J. S. (ed.), Common Problems/Proper Solutions: Avoiding Error in Quantitative Research, Sage, Newbury Park, CA, pp. 193–225.

    Google Scholar 

  • Wheaton, B., Muthen, B., Alwin, D., and Summers, G. (1977). Assessing reliability and stability in panel models. In Heise, D. (ed.), Sociological Methodology, Jossey-Bass, San Francisco, pp. 84–136.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rosay, A.B., Gottfredson, D.C., Armstrong, T.A. et al. Invariance of Measures of Prevention Program Effectiveness: A Replication. Journal of Quantitative Criminology 16, 341–367 (2000). https://doi.org/10.1023/A:1007527022941

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1007527022941

Navigation