Abstract
Recent literature has suggested that measures of risk and protective factorsfor delinquency and substance use are not equally reliable or valid acrossgender and ethnic groups and has recommended differentiated programming andculturally specific evaluation methods. Three data sets containing up tofive ethnic groups were used to determine the degree to which risk andprotective factors are equally reliable and valid predictors of drug use anddelinquency across gender and ethnic groups. Congeneric measurement modelsand structural equation models were evaluated to determine if the factorstructures for these measures and their covariances with measures of druguse and delinquency were equivalent across gender and ethnic groups. Half ofthe risk and protective factors included in this analysis were found to beequally reliable across gender and ethnic groups. When controlling forreliability differences, all of the risk and protective factors were foundto predict both drug use and delinquncy for all gender and ethnic groups. Interms of the magnitude of these associations, no substantive differenceswere found in the validity of risk and protective factors for drug use anddelinquency. Differences in the validity of risk and protective factors weremore prevalent for delinquency than for drug use. However, all differenceswere substantively trivial. We conclude that measures of prevention programeffectiveness are invariant across gender and ethnic groups.
Similar content being viewed by others
REFERENCES
Akaike, H. (1974). A new look at the statistical model identification. IEEE Trans. Automatic Control 19: 716–723.
Akaike, H. (1987). Factor analysis and AIC. Psychometrika 52: 317–332.
Akers, R. L., Krohn, M. D., Lanza-Kaduce, L., and Radosevich, N. (1979). Social learning and deviant behavior: A specific test of a general theory. Am. Sociol. Rev. 44: 636–655.
Alwin, D. F., and Jackson, D. J. (1979). Measurement models for response errors in surveys: Issues and applications. In Schuessler, K. F. (ed.), Sociological Methodology: 1980, Jossey-Bass, San Francisco, pp. 68–119.
Anderson, J. C., and Gerbing, D. W. (1988). Structural equation modeling in practice: A review and recommended two-step approach. Psychol. Bull. 103: 411–423.
Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., and Cullen, F. T. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology 28: 369–404.
Bandura, A. (1977). Social Learning Theory, Prentice Hall, Englewood Cliffs, NJ.
Benson, J. (1987). Detecting item bias in affective scales. Educ. Psychol. Measure. 47: 55–67.
Bentler, P. M., and Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychol. Bull. 88: 588–606.
Bollen, K. A. (1986). Sample size and Bentler and Bonett's nonormed fit index. Psychometrika, 51: 375–377.
Bollen, K. A. (1989). Structural Equations with Latent Variables, Wiley, New York.
Boomsma, A. (1982). The robustness of LISREL against small sample size in factor analysis models. In Jöreskog, K., and Wold, H. (eds.), Systems Under Indirect Observation, Part I, North-Holland, Amsterdam, pp. 149–173.
Bozdogan, H. C. (1987). Model selection and Akaike's information criterion (AIC): The general theory and its analytical extensions. Psychometrika 52: 345–370.
Byram, O. W., and Fly, J. W. (1984). Family structure, race and adolescents' alcohol use: A research note. Am. J. Drug Alcohol Abuse 10: 467–478.
Byrne, B. M. (1988). The Self Description Questionnaire III: Testing for equivalent factorial validity across ability. Educ. Psychol. Measure. 48: 397–406.
Carmines, E. G., and McIver, J. P. (1981). Analyzing models with unobserved variables: Analysis of covariance structures. In Bohrnstedt, G. W., and Borgatta, E. F. (eds.), Social Measurement: Current Issues, Sage, Beverly Hills, CA, pp. 66–115.
Carmines, E. G., and Zeller, R. A. (1979). Reliability and Validity Assessment, Sage, Newbury Park, CA.
Castro, F. G., Maddahian, E., Newcomb, M. D., and Bentler, P. M. (1987). A multivariate model of the determinants of cigarette smoking among adolescents. J. Health Soc. Behav. 28: 273–289.
Catalano, R. F., Morrison, D. M., Wells, E. A., Gillmore, M. R., Iritani, B., and Hawkins, J. D. (1992). Ethnic differences in family factors related to early drug initiation. J. Stud. Alcohol. 53: 208–217.
Cole, D. A., and Maxwell, S. E. (1985). Multitrait-multimethod comparisons across populations: A confirmatory factor analytic approach. Multivar. Behav. Res. 20: 389–417.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika 16: 297–334.
Gerbing, D. W., and Hunter, J. E. (1982). The metric of the latent variables in the LISRELIV analysis. Educ. Psychol. Measure. 42: 423–427.
Gillmore, M. R., Catalano, R. F., Morrison, D. M., Wells, E. A., Iritani, B., and Hawkins, J. D. (1990). Racial differences in acceptability and availability of drugs and early initiation of substance use. Am. J. Drug Alcohol Abuse 16: 185–206.
Gottfredson, D. C. (1997). School-based crime prevention. In Preventing Crime: What Works, What Doesn't, What's Promising, Office of Justice Programs, Washington, DC.
Gottfredson, D. C., and Koper, C. S. (1996). Race and sex differences in the prediction of drug use. J. Consult. Clin. Psychol. 64: 305–313.
Gottfredson, D. C., and Koper, C. S. (1997). Race and sex differences in the measurement of risk for drug use. J. Quant. Criminol. 13: 325–347.
Gottfredson, D. C., Sealock M. D., and Koper, C. S. (1996). Delinquency. In DiClemente, R. J., Hansen, W. B., and Ponton, L. E. (eds.), Handbook of Adolescent Health Risk Behavior, Plenum, New York, pp. 259–288.
Gottfredson, G. D., and Gottfredson, D. C. (1992). Development and Applications of Theoretical Measures for Evaluating Drug and Delinquency Prevention Programs, Gottfredson Associates, Elliott City, MD.
Gottfredson, M. R., and Hirschi, T. (1990). A General Theory of Crime, Stanford University, Stanord, CA.
Hawkins, J. D., Catalano, R. F., and Miller, J. L. (1992). Risk and protective factors for alcohol and other drug problems in early adulthood: Implications for substance abuse prevention. Psychol. Bull. 112: 64–105.
Hayduk, L. A. (1987). Structural Equation Modeling with LISREL, Johns Hopkins University, Baltimore.
Hayduk, L. A. (1996). Structural Equation Modeling with LISREL: Essentials and Advances, Johns Hopkins Univesity, Baltimore.
Hirschi, T. (1969). Causes of Delinquency, University of California, Berkeley.
Hoelter, J. W. (1983). The analysis of covariance structures. Goodness-of-fit indices. Sociol. Methods Res. 11: 325–344.
Horn, J. L., and McArdle, J. J. (1980). Perspectives on mathematicalystatistical model building (MASMOB) in research on aging. In Poon, L. W. (ed.), Aging in the 1980s: Selected Contemporary Issues in the Psychology of Aging, American Psychological Association, Washington, DC, pp. 503–541.
Howell, J. C., Krisberg, B., Wilson, J. J., and Hawkins, J. D. (1995). A Sourcebook on Serious Violent, and Chronic Juvenile Offenders, Sage, Newbury Park, CA.
James, L. R., Mulaik, S. A., and Brett, J. M. (1982). Causal Analysis: Assumptions, Models and Data, Sage, Beverly Hills.
Jessor, R., and Jessor, S. (1977). Problem Behavior and Psychosocial Development: A Longitudinal Study of Youth, Academic Press, New York.
Jöreskog, K. G., and Sörbom, D. (1981). LISREL5: Analysis of Linear Structural Relations by the Method of Maximum Likelihod, International Educational Services, Chicago.
Jöreskog, K. G., and Sörbom, D. (1989). LISREL7: A Guide to the Program and Applictions, 2d ed., SPSS, Chicago.
Kleinman, P., and Lukoff, I. (1978). Ethnic differences in factors related to drug use. J. Health Soc. Behav. 19: 190–199.
Long, J. S. (1983). Covariance Structure Models: An Introduction to LISREL, Sage, Beverly Hills, CA.
MacCallum, R. C., and Tucker, L. R. (1991). Representing sources of error in the commonfactor model: Implications for theory and practice. Psychol. Bull. 109: 502–511.
Marsh, H. W., Balla, J. R., and McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychol. Bull. 103: 391–410.
McArdle, J. J. (1986). Latent variable growth within behavior genetic models. Behav. Genet. 16: 163–200.
McDonald, R. P., and Marsh, H. W. (1990). Choosing a multivariate model: Noncentrality and goodness of fit. Psychol. Bull. 107: 247–255.
Newcomb, M. D., and Bentler, P. M. (1986). Substance use and ethnicity: Differential impact of peer and adult models. J. Psychol. 120: 83–95.
Newcomb, M. D., Maddahian, E., Skager, R., and Bentler, P. M. (1987). Substance abuse and psychological risk factors among teenagers: Associations with sex, age, ethnicity, and type of school. Am. J. Drug Alcohol Abuse 13: 413–433.
Nunnally, J. C. (1978). Psychometric Theory, McGraw-Hill, New York.
Orlandi, M. A. (ed.) (1992). Cultural Competence for Evaluators: A Guide for Alcohol and Other Drug Abuse Prevention Practitioners Working with Ethnic/Racial Communities, Office of Substance Abuse Prevention, U.S. Department of Health and Human Services, Rockville, MD.
Piquero, A. R., and Rosay, A. B. (1998).The reliability and validity of Grasmick, et al.'s Self-Control Scale: A comment on Longshore, et al., Criminology 36: 157–173.
Reise, S. P., Widaman, K. F., and Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. Psychol. Bull. 114: 552–566.
Rowe, D. C., Vazsonyi, A. T., and Flannery, D. J. (1994). No more than skin deep: ethnic and racial similarity in developmental process. Psychol. Rev. 101: 396–413.
Sampson, R. J. (1985). Sex differences in self-reported delinquency and official records: A multiple-group structural modeling approach. J. Quant. Criminol. 1: 345–367.
Schwarz, G. (1978). Estimating the dimension of a model. Ann. Stat. 6: 461–464.
Tanaka, J. S. (1987). “How big is enough?” Sample size and goodness-of-fit structural equation models with latent variables. Child Dev. 58: 135–146.
Tanaka, J. S., and Huba, G. J. (1985). A fit index for covariance structure models under arbitrary GLS estimation. Br. J. Math. Stat. Psychol. 38: 197–201.
Tucker, L. R., and Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrika 38: 1–10.
Wells, E. A., Morrison, D. M., Gillmore, M. R., Catalano, R. F., Iritani, B., and Hawkins, J. D. (1992). Race differences in antisocial behaviors and attitudes and early initiation of substance use. J. Drug Educ. 22: 115–130.
Wheaton, B. (1988). Assessment of fit in overidentified models with latent variables. In Long, J. S. (ed.), Common Problems/Proper Solutions: Avoiding Error in Quantitative Research, Sage, Newbury Park, CA, pp. 193–225.
Wheaton, B., Muthen, B., Alwin, D., and Summers, G. (1977). Assessing reliability and stability in panel models. In Heise, D. (ed.), Sociological Methodology, Jossey-Bass, San Francisco, pp. 84–136.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Rosay, A.B., Gottfredson, D.C., Armstrong, T.A. et al. Invariance of Measures of Prevention Program Effectiveness: A Replication. Journal of Quantitative Criminology 16, 341–367 (2000). https://doi.org/10.1023/A:1007527022941
Issue Date:
DOI: https://doi.org/10.1023/A:1007527022941