Skip to main content
main-content
Top

Tip

Swipe om te navigeren naar een ander artikel

Gepubliceerd in: Journal of Child and Family Studies 9/2016

08-06-2016 | Original Paper

The Use of Prospective Versus Retrospective Pretests with Child-Welfare Involved Families

Auteurs: Jody Brook, Becci A. Akin, Margaret Lloyd, Jackie Bhattarai, Thomas P. McDonald

Gepubliceerd in: Journal of Child and Family Studies | Uitgave 9/2016

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

The purpose of the current study is to examine the use of both a prospective and a retrospective pretest in evaluating the impact of the Strengthening Families Program on foster care involved families affected by parental substance abuse. Debate has existed in the literature for over 60 years regarding the use of retrospective pretests in assessing self-reported program impact, with key stakeholders often arguing strongly that, due to response-shift bias, sensitization, and the nature of many human service settings, retrospective pretesting may be more appropriate. However, program evaluators must also incorporate into the evaluation design funding mandates to collect data at specific points in time. In order to mitigate potential pretest-only biases and enhance the evaluation’s rigor, this study sought to address the pretest debate through the use of both a traditional, prospective pretest and a retrospective pretest. Using data provided by 411 caregivers, program effectiveness was measured in the areas of family, child and parent functioning. Statistical significance tests and effect sizes were analyzed to compare traditional prospective pretest to posttest scores and retrospective pretests to posttest scores. Findings indicate that overall, the Strengthening Families Program positively impacted family, child and parent functioning and that there were few differences between testing approaches when testing for statistical significance; however, relative differences between prospective pretests and retrospective pretests appeared more prominent in effect size computations. This research informs the longstanding debate, and suggests that program evaluators consider the advantages and disadvantages of using a retrospective pretest in design planning.
Literatuur
go back to reference Allen, J. M., & Nimon, K. (2007). Retrospective pretest: A practical technique for professional development evaluation. Journal of Industrial Teacher Education, 44(3), 27–42. Allen, J. M., & Nimon, K. (2007). Retrospective pretest: A practical technique for professional development evaluation. Journal of Industrial Teacher Education, 44(3), 27–42.
go back to reference Barth, R. P., Landsverk, J., Chamberlain, P., Reid, J. B., Rolls, J. A., Hurlburt, M. S., & Kohl, P. L. (2005). Parent-training programs in child welfare services: Planning for a more evidence-based approach to serving biological parents. Research on Social Work Practice, 15(5), 353–371. CrossRef Barth, R. P., Landsverk, J., Chamberlain, P., Reid, J. B., Rolls, J. A., Hurlburt, M. S., & Kohl, P. L. (2005). Parent-training programs in child welfare services: Planning for a more evidence-based approach to serving biological parents. Research on Social Work Practice, 15(5), 353–371. CrossRef
go back to reference Betz, D. L., & Hill, L. G. (2006). Real world evaluation. Journal of Extension, 44(2), 2RIB9. Betz, D. L., & Hill, L. G. (2006). Real world evaluation. Journal of Extension, 44(2), 2RIB9.
go back to reference Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
go back to reference Deutsch, M., & Collins, M. E. (1951). Interracial housing: A psychological evaluation of a social experiment. Minneapolis: University of Minnesota Press. Deutsch, M., & Collins, M. E. (1951). Interracial housing: A psychological evaluation of a social experiment. Minneapolis: University of Minnesota Press.
go back to reference Falconer, M. K., Haskett, M. E., McDaniels, L., Dirkes, T., & Siegel, E. C. (2008). Evaluation of support groups for child abuse prevention: Outcomes of four state evaluations. Social Work with Groups, 31(2), 165–182. CrossRef Falconer, M. K., Haskett, M. E., McDaniels, L., Dirkes, T., & Siegel, E. C. (2008). Evaluation of support groups for child abuse prevention: Outcomes of four state evaluations. Social Work with Groups, 31(2), 165–182. CrossRef
go back to reference Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501–517. CrossRef Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501–517. CrossRef
go back to reference Kellam, S., Werthamer-Larsson, L., Dolan, L., Brown, C. H., Mayer, L., Rebok, G., & Wheeler, L. (1991). Developmental epidemiologically based preventive trials: Baseline modeling of early target behaviors and depressive symptoms. American Journal of Community Psychology, 19(4), 563–584. doi: 10.​1007/​BF00937992. CrossRefPubMed Kellam, S., Werthamer-Larsson, L., Dolan, L., Brown, C. H., Mayer, L., Rebok, G., & Wheeler, L. (1991). Developmental epidemiologically based preventive trials: Baseline modeling of early target behaviors and depressive symptoms. American Journal of Community Psychology, 19(4), 563–584. doi: 10.​1007/​BF00937992. CrossRefPubMed
go back to reference Kreulen, G. J., Stommel, M., Gutek, B. A., Burns, L. R., & Braden, C. J. (2002). Utility of retrospective pretest ratings of patient satisfaction with health status§. Research in Nursing and Health, 25(3), 233–241. doi: 10.​1002/​nur.​10031. CrossRefPubMed Kreulen, G. J., Stommel, M., Gutek, B. A., Burns, L. R., & Braden, C. J. (2002). Utility of retrospective pretest ratings of patient satisfaction with health status§. Research in Nursing and Health, 25(3), 233–241. doi: 10.​1002/​nur.​10031. CrossRefPubMed
go back to reference Kumpfer, K., & Alvarado, R. (1998). Effective family strengthening interventions. US Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention Washington, DC. Kumpfer, K., & Alvarado, R. (1998). Effective family strengthening interventions. US Department of Justice, Office of Justice Programs, Office of Juvenile Justice and Delinquency Prevention Washington, DC.
go back to reference Kumpfer, K., DeMarsh, J., & Child, W. (1989). Strengthening Families Program: Children’s skills training curriculum manual, parent training manual, children’s skills training manual, and family skills training manual Prevention Services to Children of Substance-Abusing Parents. Salt Lake City: Social Research Institute, Graduate School of Social Work, University of Utah. Kumpfer, K., DeMarsh, J., & Child, W. (1989). Strengthening Families Program: Children’s skills training curriculum manual, parent training manual, children’s skills training manual, and family skills training manual Prevention Services to Children of Substance-Abusing Parents. Salt Lake City: Social Research Institute, Graduate School of Social Work, University of Utah.
go back to reference Kumpfer, K., Whiteside, H., Greene, J. A., & Allen, K. C. (2010). Effectiveness outcomes of four age versions of the Strengthening Families Program in statewide field sites. Group Dynamics: Theory, Research, and Practice, 14(3), 211. CrossRef Kumpfer, K., Whiteside, H., Greene, J. A., & Allen, K. C. (2010). Effectiveness outcomes of four age versions of the Strengthening Families Program in statewide field sites. Group Dynamics: Theory, Research, and Practice, 14(3), 211. CrossRef
go back to reference Lamb, T. (2005). The retrospective pretest: An imperfect but useful tool. The Evaluation Exchange, 11, 181. Lamb, T. (2005). The retrospective pretest: An imperfect but useful tool. The Evaluation Exchange, 11, 181.
go back to reference Miller, M., & Hinshaw, R. E. (2012). The retrospective pretest as a gauge of change. Journal of Instructional Psychology, 39(3/4), 251–258. Miller, M., & Hinshaw, R. E. (2012). The retrospective pretest as a gauge of change. Journal of Instructional Psychology, 39(3/4), 251–258.
go back to reference Nimon, K., Zigarmi, D., & Allen, J. (2011). Measures of program effectiveness based on retrospective pretest data: Are all created equal? American Journal of Evaluation,. doi: 10.​1177/​1098214010378354​. Nimon, K., Zigarmi, D., & Allen, J. (2011). Measures of program effectiveness based on retrospective pretest data: Are all created equal? American Journal of Evaluation,. doi: 10.​1177/​1098214010378354​.
go back to reference Piwowar, V., & Thiel, F. (2014). Evaluating response shift in training evaluation: Comparing the retrospective pretest with an adapted measurement invariance approach in a classroom management training program. Evaluation review, 38(5), 420–448. doi: 10.​1177/​0193841x14546932​. CrossRefPubMed Piwowar, V., & Thiel, F. (2014). Evaluating response shift in training evaluation: Comparing the retrospective pretest with an adapted measurement invariance approach in a classroom management training program. Evaluation review, 38(5), 420–448. doi: 10.​1177/​0193841x14546932​. CrossRefPubMed
go back to reference Radloff, L. S. (1977). The CES-D scale A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401. CrossRef Radloff, L. S. (1977). The CES-D scale A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401. CrossRef
go back to reference Taylor, P. J., Russ-Eft, D. F., & Taylor, H. (2008). Gilding the outcome by tarnishing the past: Inflationary biases in retrospective pretests. American Journal of Evaluation,. doi: 10.​1177/​1098214008328517​. Taylor, P. J., Russ-Eft, D. F., & Taylor, H. (2008). Gilding the outcome by tarnishing the past: Inflationary biases in retrospective pretests. American Journal of Evaluation,. doi: 10.​1177/​1098214008328517​.
Metagegevens
Titel
The Use of Prospective Versus Retrospective Pretests with Child-Welfare Involved Families
Auteurs
Jody Brook
Becci A. Akin
Margaret Lloyd
Jackie Bhattarai
Thomas P. McDonald
Publicatiedatum
08-06-2016
Uitgeverij
Springer US
Gepubliceerd in
Journal of Child and Family Studies / Uitgave 9/2016
Print ISSN: 1062-1024
Elektronisch ISSN: 1573-2843
DOI
https://doi.org/10.1007/s10826-016-0446-1