Skip to main content
Log in

The Role of Behavior Observation in Measurement Systems for Randomized Prevention Trials

  • Original Article
  • Published:
Prevention Science Aims and scope Submit manuscript

The role of behavior observation in theory-driven prevention intervention trials is examined. A model is presented to guide choice of strategies for the measurement of five core elements in theoretically informed, randomized prevention trials: (1) training intervention agents, (2) delivery of key intervention conditions by intervention agents, (3) responses of clients to intervention conditions, (4) short-term risk reduction in targeted client behaviors, and (5) long-term change in client adjustment. It is argued that the social processes typically thought to mediate interventionist training (Element 1) and the efficacy of psychosocial interventions (Elements 2 and 3) may be powerfully captured by behavior observation. It is also argued that behavior observation has advantages in the measurement of short-term change (Element 4) engendered by intervention, including sensitivity to behavior change and blinding to intervention status.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.

Similar content being viewed by others

Notes

  1. Some theories may not distinguish between Core Elements 3 and 4. For example, intervention with young adolescents may entail using instruction and role playing to shape skills to resist peer influence or to refuse offers to use drugs. This same set of skills may be the desired short-term outcome. Even if this is the case, distinction between Elements 3 and 4 may be useful. Core Element 4 in contrast to Element 3 refers to the display of the target response outside of the intervention session—its generalization to natural environmental settings and its persistence over time after its acquisition and practice. Heuristically, measurement of the same response in two different contexts may be optimized by different methods.

  2. There is another step in the training process—training PMT therapists who provide training for parents. For sake of simplicity of exposition, this additional element is not described here although observational methods and sequential analyses are clearly relevant to measurement of the training of PMT therapists as well (Forgatch et al., 2005).

  3. This model of social exchange and influence can also be applied to therapists' training of parents in PMT skills. Parents react to actions of the PMT therapist, and these reactions may be cooperative or resistant. As such, reciprocal influence is embedded in this element. Efforts to assess this process have been made (Stoolmiller et al., 1993) and strategies by which PMT therapists can effectively deal with resistance are explicitly addressed in intervention manuals and in training of PMT therapists (Forgatch et al., 2005). These therapist–parent actions and reactions reflect sources of variation due to social Interchange (\(S _{\rm I}^2\)). This suggests that behavior observation methods are also relevant to the measurement of Core Element 1.

REFERENCES

  • Bakeman, R., & Gottman, J. M. (1997). Observing interaction (2nd ed.). New York: Cambridge University Press.

  • Botvin, G. J. (2000). Life skills training: Promoting health and personal development. Princeton, NJ: Princeton Health Press.

  • Brown, C. H. (1993). Statistical methods for prevention trials in mental health. Statistics in Medicine, 12, 289–300.

    Article  PubMed  CAS  Google Scholar 

  • Cairns, R. B., & Green, J. A. (1979). How to assess personality and social patterns: Observations or ratings? In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 209–226). Hillsdale, NJ: Erlbaum.

  • Carroll, K. M., Rounsaville, B. J., & Nich, C. (1994). Blind man's bluff: Effectiveness and significance of psychotherapy and pharmacotherapy blinding procedures in a clinical trial. Journal of Consulting and Clinical Psychology, 62, 276–280.

    Article  PubMed  CAS  Google Scholar 

  • Collins, L. M. (1991). Measurement in longitudinal research. In L. M. Collins & J. L. Horn (Eds.), Best methods for analysis of change (pp. 137–148). Washington, DC: American Psychological Association.

  • Crosby, L., Stubbs, J., Forgatch, M., & Capaldi, D. (1998). Family and peer process code training manual. Eugene: Oregon Social Learning Center.

  • Dagne, G. A., Howe, G. W., Brown, C. H., & Muthen, B. O. (2002). Hierarchical modeling of sequential behavioral data: An empirical Bayesian approach. Psychological Methods, 7, 262–280.

    Article  PubMed  Google Scholar 

  • DeGarmo, D. S., Patterson, G. R., & Forgatch, M. S. (2004). How do outcomes in a specified parent training intervention maintain or wane over time? Prevention Science, 5, 73–90.

    Article  PubMed  Google Scholar 

  • Dishion, T. J., & Kavanagh, K. (2004). Adolescent problem behavior: An intervention and assessment sourcebook for working with families in schools. New York: Guilford Press.

  • Dishion, T. J., Rivera, E. K., Jones, L., Verberkmoes, S., & Patras, J. (2002). Relationship process code. Unpublished Coding Manual, Child and Family Research Center, University of Oregon, Eugene.

  • Eddy, J. M., Dishion, T. J., & Stoolmiller, M. (1998). The analysis of intervention change in children and families: Methodological and conceptual issues embedded in intervention studies. Journal of Abnormal Child Psychology, 26, 53–69.

    Article  PubMed  CAS  Google Scholar 

  • Follette, W. C. (1995). Correcting methodological weaknesses in the knowledge base used to derive practice standards. In S. C. Hayes, W. C. Follette, R. M. Dawes, & K. E. Grady (Eds.), Scientific standards of psychological practice: Issues and recommendations (pp. 229–247). Reno, NV: Context Press.

  • Forgatch, M. S. (1994). Parenting through change: A training manual. Eugene: Oregon Social Learning Center.

  • Forgatch, M. S., Patterson, G. R., & DeGarmo, D. S. (2005). Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon Model of Parent Management Training (PMTO). Behavior Therapy, 36, 3–14.

    Article  PubMed  Google Scholar 

  • Gardner, W., & Griffin, W. A. (1989). Methods for the analysis of parallel streams of continuously recorded behavior. Psychological Bulletin, 105, 446–455.

    Article  Google Scholar 

  • Hogue, A., Liddle, H. A., & Rowe, C. (1996). Treatment adherence process research in family therapy: A rationale and some practical guidelines. Psychotherapy, 33, 332–345.

    Google Scholar 

  • Ialongo, N., Poduska, J., Werthamer, L., & Kellam, S. (2001). The distal impact of two first grade preventive interventions on conduct problems and disorder in early adolescence. Journal of Emotional and Behavioral Disorders, 9, 146–160.

    Article  Google Scholar 

  • Jones, R. R., Reid, J. B., & Patterson, G. R. (1975). Naturalistic observation in clinical assessment. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 3, pp. 42–95). San Francisco, CA: Jossey-Bass.

  • Kellam, S. G. (1990). Developmental epidemiologic framework for family research on depression and aggression. In G. R. Patterson (Ed.), Depression and aggression in family interaction (pp. 1–48). Hillsdale, NJ: Erlbaum.

  • Kent, R. N., O'Leary, K. D., Diament, C., & Dietz, A. (1974). Expectation biases in observational evaluation of therapeutic change. Journal of Consulting and Clinical Psychology, 42, 774–780.

    Article  PubMed  CAS  Google Scholar 

  • Kraemer, H. C. (1991). To increase power in randomized clinical trials without increasing sample size. Psychopharmacology Bulletin, 27, 217–224.

    PubMed  CAS  Google Scholar 

  • MacKinon, D. P., & Lockwood, C. M. (2003). Advances in statistical methods for substance use prevention research. Prevention Science, 4, 155–171.

    Article  Google Scholar 

  • Miller, W. R., & Rollnick, S. (2002). Motivational interviewing: Preparing people to change addictive behavior. New York: Guilford Press.

  • Muthén, L. K., & Muthén, B. O. (1998–2004). MPLUS user's guide (3rd ed.). Los Angeles, CA: Muthén &Muthén.

  • Olds, D. L. (2002). Prenatal and infancy home-visiting by nurses: From randomized trials to community replication. Prevention Science, 3, 153–172.

    Article  PubMed  Google Scholar 

  • Patterson, G. R. (1982). Coercive family process. Eugene, OR: Castalia.

  • Patterson, G. R., & Reid, J. B. (1973). Interventions for aggressive boys: A replication study. Behavior Research and Therapy, 11, 383–394.

    Article  CAS  Google Scholar 

  • Radke-Yarrow, M., & Zahn-Waxler, C. (1979). Observing interaction: A confrontation with methodology. In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 37–66). Hillsdale, NJ: Erlbaum.

  • Reid, J. (2003). Development of measurement systems for randomized trials. Paper presented to the American Institutes for Research, Center for Integrating Education and Prevention Research in Schools, Washington, DC.

  • Reid, J. B., Patterson, G. R., & Snyder, J. (2002). Antisocial behavior in children and adolescents: A developmental analysis and model for intervention. Washington, DC: American Psychological Association.

  • Rotheram-Borus, M. J., Song, J., Gwadz, M., Lee, M., Van Rossem, R., & Koopman, C. (2003). Reductions in HIV risk among runaway youth. Prevention Science, 4, 173–188.

    Article  PubMed  Google Scholar 

  • Schelle, J. (1974). A brief report on the invalidity of parent evaluations of behavior change. Journal of Applied Behavior Analysis, 7, 341–343.

    Article  Google Scholar 

  • Schrepferman, L., & Snyder, J. (2002). Coercion: The link between treatment mechanisms in behavioral parent training and risk reduction in child antisocial behavior. Behavior Therapy, 33, 339–359.

    Article  Google Scholar 

  • Schrepferman, L., Snyder, J., Prichard, J., & Suarez, M. (2004). An observational system for children's social interaction with peers: Reliability and validity. Manuscript submitted for publication, Wichita State University, Wichita, KS.

  • Skindrud, K. D. (1973). Field observation of observer bias under overt and covert monitoring. In L. Handy & E. Mash (Eds.), Behavior change: Methodology, concepts, and practice (pp. 97–118). Champaign, IL: Research Press.

  • Snyder, J., & Stoolmiller, M. (2002). Reinforcement and coercion mechanisms in the development of antisocial behavior: Family processes. In J. R. Reid, J. Snyder, & G. R. Patterson (Eds.), Antisocial behavior: Prevention, intervention and basic research (pp. 65–100). Washington, DC: American Psychological Association.

  • Stoolmiller, M., Duncan, T. E., & Patterson, G. R. (1995). Some problems and solutions in the study of change: Significant patterns of client resistance. Journal of Consulting and Clinical Psychology, 61, 920–928.

    Article  Google Scholar 

  • Stoolmiller, M., Eddy, J. M., & Reid, J. B. (2000). Detecting and describing preventive intervention effects in a universal school-based randomized trial targeting delinquent and violent behavior. Journal of Consulting and Clinical psychology, 68, 296–306.

    Article  PubMed  CAS  Google Scholar 

  • Walter, H. I., & Gilmore, S. K. (1973). Placebo versus social learning effects in parent training procedures designed to alter the behaviors of aggressive boys. Behavior Therapy, 4, 361–377.

    Article  Google Scholar 

  • White, L., Tursky, B., & Schwartz, G. E. (1985). Placebo: Theory, research and mechanisms. New York: Guilford Press.

  • Willett, J. B. (1989). Some results on the reliability for longitudinal measurement of change: Implications for the design of studies of individual growth. Educational and Psychological Measurement, 49, 587–602.

    Article  Google Scholar 

Download references

ACKNOWLEDGMENTS

This report is a result of a collaborative effort by members of the Workgroup for the Analysis of Observational Data (WODA), supported in part by grants 3P30 MH46690-13S1, R01 MH 57342, R01 MH40859, MH59855, R01 DA015409, and T32 MH18911.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to James Snyder.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Snyder, J., Reid, J., Stoolmiller, M. et al. The Role of Behavior Observation in Measurement Systems for Randomized Prevention Trials. Prev Sci 7, 43–56 (2006). https://doi.org/10.1007/s11121-005-0020-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-005-0020-3

KEY WORDS:

Navigation