The role of behavior observation in theory-driven prevention intervention trials is examined. A model is presented to guide choice of strategies for the measurement of five core elements in theoretically informed, randomized prevention trials: (1) training intervention agents, (2) delivery of key intervention conditions by intervention agents, (3) responses of clients to intervention conditions, (4) short-term risk reduction in targeted client behaviors, and (5) long-term change in client adjustment. It is argued that the social processes typically thought to mediate interventionist training (Element 1) and the efficacy of psychosocial interventions (Elements 2 and 3) may be powerfully captured by behavior observation. It is also argued that behavior observation has advantages in the measurement of short-term change (Element 4) engendered by intervention, including sensitivity to behavior change and blinding to intervention status.
Similar content being viewed by others
Notes
Some theories may not distinguish between Core Elements 3 and 4. For example, intervention with young adolescents may entail using instruction and role playing to shape skills to resist peer influence or to refuse offers to use drugs. This same set of skills may be the desired short-term outcome. Even if this is the case, distinction between Elements 3 and 4 may be useful. Core Element 4 in contrast to Element 3 refers to the display of the target response outside of the intervention session—its generalization to natural environmental settings and its persistence over time after its acquisition and practice. Heuristically, measurement of the same response in two different contexts may be optimized by different methods.
There is another step in the training process—training PMT therapists who provide training for parents. For sake of simplicity of exposition, this additional element is not described here although observational methods and sequential analyses are clearly relevant to measurement of the training of PMT therapists as well (Forgatch et al., 2005).
This model of social exchange and influence can also be applied to therapists' training of parents in PMT skills. Parents react to actions of the PMT therapist, and these reactions may be cooperative or resistant. As such, reciprocal influence is embedded in this element. Efforts to assess this process have been made (Stoolmiller et al., 1993) and strategies by which PMT therapists can effectively deal with resistance are explicitly addressed in intervention manuals and in training of PMT therapists (Forgatch et al., 2005). These therapist–parent actions and reactions reflect sources of variation due to social Interchange (\(S _{\rm I}^2\)). This suggests that behavior observation methods are also relevant to the measurement of Core Element 1.
REFERENCES
Bakeman, R., & Gottman, J. M. (1997). Observing interaction (2nd ed.). New York: Cambridge University Press.
Botvin, G. J. (2000). Life skills training: Promoting health and personal development. Princeton, NJ: Princeton Health Press.
Brown, C. H. (1993). Statistical methods for prevention trials in mental health. Statistics in Medicine, 12, 289–300.
Cairns, R. B., & Green, J. A. (1979). How to assess personality and social patterns: Observations or ratings? In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 209–226). Hillsdale, NJ: Erlbaum.
Carroll, K. M., Rounsaville, B. J., & Nich, C. (1994). Blind man's bluff: Effectiveness and significance of psychotherapy and pharmacotherapy blinding procedures in a clinical trial. Journal of Consulting and Clinical Psychology, 62, 276–280.
Collins, L. M. (1991). Measurement in longitudinal research. In L. M. Collins & J. L. Horn (Eds.), Best methods for analysis of change (pp. 137–148). Washington, DC: American Psychological Association.
Crosby, L., Stubbs, J., Forgatch, M., & Capaldi, D. (1998). Family and peer process code training manual. Eugene: Oregon Social Learning Center.
Dagne, G. A., Howe, G. W., Brown, C. H., & Muthen, B. O. (2002). Hierarchical modeling of sequential behavioral data: An empirical Bayesian approach. Psychological Methods, 7, 262–280.
DeGarmo, D. S., Patterson, G. R., & Forgatch, M. S. (2004). How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science, 5, 73–90.
Dishion, T. J., & Kavanagh, K. (2004). Adolescent problem behavior: An intervention and assessment sourcebook for working with families in schools. New York: Guilford Press.
Dishion, T. J., Rivera, E. K., Jones, L., Verberkmoes, S., & Patras, J. (2002). Relationship process code. Unpublished Coding Manual, Child and Family Research Center, University of Oregon, Eugene.
Eddy, J. M., Dishion, T. J., & Stoolmiller, M. (1998). The analysis of intervention change in children and families: Methodological and conceptual issues embedded in intervention studies. Journal of Abnormal Child Psychology, 26, 53–69.
Follette, W. C. (1995). Correcting methodological weaknesses in the knowledge base used to derive practice standards. In S. C. Hayes, W. C. Follette, R. M. Dawes, & K. E. Grady (Eds.), Scientific standards of psychological practice: Issues and recommendations (pp. 229–247). Reno, NV: Context Press.
Forgatch, M. S. (1994). Parenting through change: A training manual. Eugene: Oregon Social Learning Center.
Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon Model of Parent Management Training (PMTO). Behavior Therapy, 36, 3–14.
Gardner, W., & Griffin, W. A. (1989). Methods for the analysis of parallel streams of continuously recorded behavior. Psychological Bulletin, 105, 446–455.
Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy, 33, 332–345.
Ialongo, N., Poduska, J., Werthamer, L., & Kellam, S. (2001). The distal impact of two first grade preventive interventions on conduct problems and disorder in early adolescence. Journal of Emotional and Behavioral Disorders, 9, 146–160.
Jones, R. R., Reid, J. B., & Patterson, G. R. (1975). Naturalistic observation in clinical assessment. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 3, pp. 42–95). San Francisco, CA: Jossey-Bass.
Kellam, S. G. (1990). Developmental epidemiologic framework for family research on depression and aggression. In G. R. Patterson (Ed.), Depression and aggression in family interaction (pp. 1–48). Hillsdale, NJ: Erlbaum.
Kent, R. N., O'Leary, K. D., Diament, C., & Dietz, A. (1974). Expectation biases in observational evaluation of therapeutic change. Journal of Consulting and Clinical Psychology, 42, 774–780.
Kraemer, H. C. (1991). To increase power in randomized clinical trials without increasing sample size. Psychopharmacology Bulletin, 27, 217–224.
MacKinon, D. P., & Lockwood, C. M. (2003). Advances in statistical methods for substance use prevention research. Prevention Science, 4, 155–171.
Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people to change addictive behavior. New York: Guilford Press.
Muthén, L. K., & Muthén, B. O. (1998–2004). MPLUS user's guide (3rd ed.). Los Angeles, CA: Muthén &Muthén.
Olds, D. L. (2002). Prenatal and infancy home-visiting by nurses: From randomized trials to community replication. Prevention Science, 3, 153–172.
Patterson, G. R. (1982). Coercive family process. Eugene, OR: Castalia.
Patterson, G. R., & Reid, J. B. (1973). Interventions for aggressive boys: A replication study. Behavior Research and Therapy, 11, 383–394.
Radke-Yarrow, M., & Zahn-Waxler, C. (1979). Observing interaction: A confrontation with methodology. In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 37–66). Hillsdale, NJ: Erlbaum.
Reid, J. (2003). Development of measurement systems for randomized trials. Paper presented to the American Institutes for Research, Center for Integrating Education and Prevention Research in Schools, Washington, DC.
Reid, J. B., Patterson, G. R., & Snyder, J. (2002). Antisocial behavior in children and adolescents: A developmental analysis and model for intervention. Washington, DC: American Psychological Association.
Rotheram-Borus, M. J., Song, J., Gwadz, M., Lee, M., Van Rossem, R., & Koopman, C. (2003). Reductions in HIV risk among runaway youth. Prevention Science, 4, 173–188.
Schelle, J. (1974). A brief report on the invalidity of parent evaluations of behavior change. Journal of Applied Behavior Analysis, 7, 341–343.
Schrepferman, L., & Snyder, J. (2002). Coercion: The link between treatment mechanisms in behavioral parent training and risk reduction in child antisocial behavior. Behavior Therapy, 33, 339–359.
Schrepferman, L., Snyder, J., Prichard, J., & Suarez, M. (2004). An observational system for children's social interaction with peers: Reliability and validity. Manuscript submitted for publication, Wichita State University, Wichita, KS.
Skindrud, K. D. (1973). Field observation of observer bias under overt and covert monitoring. In L. Handy & E. Mash (Eds.), Behavior change: Methodology, concepts, and practice (pp. 97–118). Champaign, IL: Research Press.
Snyder, J., & Stoolmiller, M. (2002). Reinforcement and coercion mechanisms in the development of antisocial behavior: Family processes. In J. R. Reid, J. Snyder, & G. R. Patterson (Eds.), Antisocial behavior: Prevention, intervention and basic research (pp. 65–100). Washington, DC: American Psychological Association.
Stoolmiller, M., Duncan, T. E., & Patterson, G. R. (1995). Some problems and solutions in the study of change: Significant patterns of client resistance. Journal of Consulting and Clinical Psychology, 61, 920–928.
Stoolmiller, M., Eddy, J. M., & Reid, J. B. (2000). Detecting and describing preventive intervention effects in a universal school-based randomized trial targeting delinquent and violent behavior. Journal of Consulting and Clinical psychology, 68, 296–306.
Walter, H. I., & Gilmore, S. K. (1973). Placebo versus social learning effects in parent training procedures designed to alter the behaviors of aggressive boys. Behavior Therapy, 4, 361–377.
White, L., Tursky, B., & Schwartz, G. E. (1985). Placebo: Theory, research and mechanisms. New York: Guilford Press.
Willett, J. B. (1989). Some results on the reliability for longitudinal measurement of change: Implications for the design of studies of individual growth. Educational and Psychological Measurement, 49, 587–602.
ACKNOWLEDGMENTS
This report is a result of a collaborative effort by members of the Workgroup for the Analysis of Observational Data (WODA), supported in part by grants 3P30 MH46690-13S1, R01 MH 57342, R01 MH40859, MH59855, R01 DA015409, and T32 MH18911.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Snyder, J., Reid, J., Stoolmiller, M. et al. The Role of Behavior Observation in Measurement Systems for Randomized Prevention Trials. Prev Sci 7, 43–56 (2006). https://doi.org/10.1007/s11121-005-0020-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-005-0020-3