Top

07-01-2025 | Original Paper

Exploring Machine Learning to Support Decision-Making for Placement Stabilization and Preservation in Child Welfare

Auteurs: Ka Ho Brian Chor, Zhidi Luo, Kit T. Rodolfa, Rayid Ghani

Gepubliceerd in: Journal of Child and Family Studies

Abstract

The Family First Prevention Services Act requires youth’s placement in residential care to be clinically appropriate, time-limited, and only when youth’s needs cannot be met in family-like settings in foster care. State child welfare agencies can benefit from upstream, empirical decision support to preempt youth’s placement disruption, coordinate proactive placement stabilization services, prevent unnecessary step-up to residential care, and improve outcomes for the youth. This statewide case study explores the potential benefit to child welfare decision support for placement stabilization and diversion from residential care, by comparing predictive machine learning (ML) models with conventional regression models. We analyzed child welfare spells of 12,621 youth in one large Midwestern state between January 2017 and January 2020. Caseworkers could refer youth to a placement stabilization and preservation program. To predict youth’s monthly program need in the next 6 months, we developed and validated a wide grid of ML models—random forest, regularized logistic regression, decision tree, dummy classifier—and a conventional unregularized logistic regression model, using literature-informed predictors from child welfare administrative data. We retrained, retested, and compared all models over time using temporal hold-out sets. Based on anticipated program capacity, model evaluation focused on accuracy in identifying the 100 highest-need youth, fairness, and equity of resource allocation. Random forest models produced the best performance with a precision (positive predictive value) 10 times greater than baseline precision. Common important predictors across models included youth’s age, history of placement changes, and emotional/behavioral needs. We discuss potential applications of ML to support preventive child welfare decisions, adapt to policy changes, and allocate limited resources.

Alleen toegankelijk voor geautoriseerde gebruikers

Ackermann, K., Walsh, J., Unánue, A. D., Naveed, H., Rivera, A. N., Lee, S.-J., Bennett, J., Defoe, M., Cody, C., Haynes, L., & Ghani, R. (2018). Deploying machine learning models for public policy: A framework Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, United Kingdom. https://doi.org/10.1145/3219819.3219911.

Ahn, E., Gil, Y., & Putnam-Hornstein, E. (2021). Predicting youth at high risk of aging out of foster care using machine learning methods. Child Abuse & Neglect, 117, 105059. https://doi.org/10.1016/j.chiabu.2021.105059.CrossRef

Allegheny County Department of Human Services. (2017). Developing predictive risk models to support child maltreatment hotline screening decisions. Allegheny County Department of Human Services.

American Academy of Pediatrics, & Chapin Hall at the University of Chicago. (2023). Family First implementation: A one-year review of state progress in reforming congregate care. American Academy of Pediatrics & Chapin Hall at the University of Chicago.

Annie, E. Casey Foundation. (2002). Team decision-making involving the family and community in child welfare decisions part two: Building community partnership in child welfare. Annie E. Casey Foundation.

Archer, K. J., & Kimes, R. V. (2008). Empirical characterization of random forest variable importance measures. Computational Statistics & Data Analysis, 52(4), 2249–2260. https://doi.org/10.1016/j.csda.2007.08.015.CrossRef

Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324.CrossRef

Centre for Social Data Analytics. (2019). Implementing a child welfare decision aide in Douglas County: Methodology report. Centre for Social Data Analytics.

Chambers, R. M., Crutchfield, R. M., Willis, T. Y., Cuza, H. A., Otero, A., Goddu Harper, S. G., & Carmichael, H. (2018). “It’s just not right to move a kid that many times:” A qualitative study of how foster care alumni perceive placement moves. Children and Youth Services Review, 86, 76–83. https://doi.org/10.1016/j.childyouth.2018.01.028.CrossRef

Child Welfare Information Gateway. (2020). Determining the best interests of the child. U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau.

Chor, K. H. B., Epstein, R. A., & Luo, Z. (2023). Developing and validating a predictive risk model for youth placement in residential care to support decision-making under the Family First Prevention Services Act. Residential Treatment for Children & Youth, 40(3), 324–347. https://doi.org/10.1080/0886571X.2022.2111018.CrossRef

Chor, K. H. B., Luo, Z., Dworsky, A., Raman, R., Courtney, M. E., & Epstein, R. A. (2022). Development and validation of a predictive risk model for runaway among youth in child welfare. Children and Youth Services Review, 143, 106689. https://doi.org/10.1016/j.childyouth.2022.106689.CrossRef

Chor, K. H. B., McClelland, G. M., Weiner, D. A., Jordan, N., & Lyons, J. S. (2015). Out-of-home placement decision-making and outcomes in child welfare: A longitudinal study. Administration and Policy in Mental Health and Mental Health Services Research, 42(1), 70–86. https://doi.org/10.1007/s10488-014-0545-5.CrossRefPubMed

Chor, K. H. B., & Oltmans, C. (2023). Cost–benefit of Treatment Foster Care Oregon (TFCO) versus residential care in Illinois. Research on Social Work Practice, 10497315231206752. https://doi.org/10.1177/10497315231206752.

Chor, K. H. B., Rodolfa, K. T., & Ghani, R. (2022). A conceptual framework for using machine learning to support child welfare decisions. arXiv:2207.05855. Retrieved February 1, 2024, from https://doi.org/10.48550/arXiv.2207.05855.

Chouldechova, A., Putnam-Hornstein, E., Benavides-Prado, D., Fialko, O., & Vaithianathan, R. (2018). A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions Proceedings of the 1st Conference on Fairness, Accountability and Transparency, Proceedings of Machine Learning Research. http://proceedings.mlr.press.

Church, C. E., & Fairchild, A. J. (2017). In search of a silver bullet: Child welfare’s embrace of predictive analytics. Juvenile and Family Court Journal, 68(1), 67–81. https://doi.org/10.1111/jfcj.12086.CrossRef

Courtney, M. E., & Zinn, A. (2009). Predictors of running away from out-of-home care. Children and Youth Services Review, 31(12), 1298–1306. https://doi.org/10.1016/j.childyouth.2009.06.003.CrossRef

Cuccaro-Alamin, S., Foust, R., Vaithianathan, R., & Putnam-Hornstein, E. (2017). Risk assessment and decision making in child protective services: Predictive risk modeling in context. Children and Youth Services Review, 79, 291–298. https://doi.org/10.1016/j.childyouth.2017.06.027.CrossRef

Drake, B., Jonson-Reid, M., Ocampo, M. G., Morrison, M., & Dvalishvili, D. (2020). A practical framework for considering the use of predictive risk modeling in child welfare. The ANNALS of the American Academy of Political and Social Science, 692(1), 162–181. https://doi.org/10.1177/0002716220978200.CrossRefPubMed

Epstein, Jr, R. A. (2004). Inpatient and residential treatment effects for children and adolescents: a review and critique. Child and Adolescent Psychiatric Clinics, 13(2), 411–428. https://doi.org/10.1016/S1056-4993(03)00126-3.CrossRef

Epstein, R. A., Schlueter, D., Gracey, K. A., Chandrasekhar, R., & Cull, M. J. (2015). Examining placement disruption in child welfare. Residential Treatment for Children & Youth, 32(3), 224–232. https://doi.org/10.1080/0886571X.2015.1102484.CrossRef

Ghani, R., & Schierholz, M. (2020). Machine learning. In I. Foster, R. Ghani, R. S. Jarmin, F. Kreuter, & J. Lane (Eds.), Big data and social science: Data science methods and tools for research and practice (2nd ed., pp. 143–191). CRC Press.

Glaberson, S. K. (2019). Coding over the cracks: Predictive analytics and child protection. Fordham Urban Law Journal, 46(2), 307–363.

Hall, S. F., Sage, M., Scott, C. F., & Joseph, K. (2023). A systematic review of sophisticated predictive and prescriptive analytics in child welfare: Accuracy, equity, and bias. Child and Adolescent Social Work Journal. https://doi.org/10.1007/s10560-023-00931-2.

Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems (pp. 3315–3323). https://ui.adsabs.harvard.edu/abs/2016arXiv161002413H.

Havlicek, J. (2011). Lives in motion: A review of former foster youth in the context of their experiences in the child welfare system. Children and Youth Services Review, 33(7), 1090–1100. https://doi.org/10.1016/j.childyouth.2011.02.007.CrossRefPubMedPubMedCentral

Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: Principles and Practice (2nd ed.). OTexts.

Illinois Department of Children and Family Services. (2013). Policy Guide 2013.03: Clinical Intervention for Placement Preservation (CIPP). Illinois Department of Children and Family Services.

Jackson, D., & Marx, G. (2017). Data mining program designed to predict child abuse proves unreliable, DCFS says. Chicago Tribune. https://www.chicagotribune.com/investigations/ct-dcfs-eckerd-met-20171206-story.html.

James, S. (2004). Why do foster care placements disrupt? An investigation of reasons for placement change in foster care. Social Service Review, 78(4), 601–627. https://doi.org/10.1086/424546.CrossRef

Janczewski, C. E., & Nitkowski, J. (2023). Predicting mental and behavioral health service utilization among child welfare-involved caregivers: A machine learning approach. Children and Youth Services Review, 155, 107150. https://doi.org/10.1016/j.childyouth.2023.107150.CrossRef

Jenkins, B. Q. (2021). Measuring the equity of risk assessment instruments used in child protection. Children and Youth Services Review, 131, 106266. https://doi.org/10.1016/j.childyouth.2021.106266.CrossRef

Keddell, E. (2019). Algorithmic justice in child protection: Statistical fairness, social justice and the implications for practice. Social Sciences, 8(10), 281. https://doi.org/10.3390/socsci8100281.CrossRef

Kiani, S., Barton, J., Sushinsky, J., Heimbach, L., & Luo, B. (2023). Counterfactual prediction under selective confounding. Frontiers in Artificial Intelligence and Applications, 372, 1256–1263. https://doi.org/10.3233/FAIA230403.CrossRef

Kim, H., & Kao, D. (2014). A meta-analysis of turnover intention predictors among U.S. child welfare workers. Children and Youth Services Review, 47, 214–223. https://doi.org/10.1016/j.childyouth.2014.09.015.CrossRef

Lee, S. I., Lee, H., Abbeel, P., & Ng, A. (2006). Efficient L1 regularized logistic regression. American Association for Artificial Intelligence (AAAI), 6, 401–408.

Liaw, A., & Wiener, M. (2002). Classification and regression by randomForest. R News, 2(3), 18–22.

Lyons, J. S. (2009). Communimetrics: A communication theory of measurement in human service settings. Springer.

McCurdy, B. L., & McIntyre, E. K. (2004). ‘And what about residential…?’ Re-conceptualizing residential treatment as a stop-gap service for youth with emotional and behavioral disorders. Behavioral Interventions, 19(3), 137–158. https://doi.org/10.1002/bin.151.CrossRef

Myles, A. J., Feudale, R. N., Liu, Y., Woody, N. A., & Brown, S. D. (2004). An introduction to decision tree modeling. Journal of Chemometrics, 18(6), 275–285. https://doi.org/10.1002/cem.873.CrossRef

Negriff, S., Dilkina, B., Matai, L., & Rice, E. (2022). Using machine learning to determine the shared and unique risk factors for marijuana use among child-welfare versus community adolescents. PLOS ONE, 17(9), e0274998. https://doi.org/10.1371/journal.pone.0274998.CrossRefPubMedPubMedCentral

Pecora, P. J., Whittaker, J. K., Barth, R. P., Borja, S., & Vesneski, W. (2018). The child welfare challenge: Policy, practice, and research (4th, Ed.). Routledge.

Python Software Foundation. (2020). Python 3.7 release. Python Software Foundation.

Rodolfa, K. T., Saleiro, P., & Ghani, R. (2020). Machine learning. In I. Foster, R. Ghani, R. S. Jarmin, F. Kreuter, & J. Lane (Eds.), Big data and social science: Data science methods and tools for research and practice (2nd ed.). CRC Press.

Rodolfa, K. T., Salomon, E., Haynes, L., Mendieta, I. H., Larson, J., & Ghani, R. (2020). Case study: Predictive fairness to reduce misdemeanor recidivism through social service interventions Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, Barcelona, Spain. https://doi.org/10.1145/3351095.3372863.

Russell, J. (2015). Predictive analytics and child protection: Constraints and opportunities. Child Abuse & Neglect, 46, 182–189. https://doi.org/10.1016/j.chiabu.2015.05.022.CrossRef

Saxena, D., Badillo-Urquiola, K. A., Wisniewski, P., & Guha, S. (2020). A human-centered review of algorithms used within the U.S. child welfare system. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems.

Schwartz, I. M., York, P., Nowakowski-Sims, E., & Ramos-Hernandez, A. (2017). Predictive and prescriptive analytics, machine learning and child welfare risk assessment: The Broward County experience. Children and Youth Services Review, 81, 309–320. https://doi.org/10.1016/j.childyouth.2017.08.020.CrossRef

State of Illinois Office of the Auditor General. (2022). Performance audit of the Department of Children and Family Services child safety and well-being. State of Illinois Office of the Auditor General.

Stott, T., & Gustavsson, N. (2010). Balancing permanency and stability for youth in foster care. Children and Youth Services Review, 32(4), 619–625. https://doi.org/10.1016/j.childyouth.2009.12.009.CrossRef

The R Foundation for Statistical Computing. (2020). R version 4.0.3. The R Foundation for Statistical Computing.

Wisconsin Department of Children and Families. (2014). The P.S. program: Using predictive analytics in program implementation. Wisconsin Department of Children and Families.

Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12(6), 1100–1122. https://doi.org/10.1177/1745691617693393.CrossRefPubMedPubMedCentral

Ye, T., Johnson, R., Fu, S., Copeny, J., Donnelly, B., Freeman, A., Lima, M., Walsh, J., & Ghani, R. (2019). Using machine learning to help vulnerable tenants in New York City Proceedings of the 2nd ACM SIGCAS Conference on Computing and Sustainable Societies, Accra, Ghana. https://doi.org/10.1145/3314344.3332484.

Titel: Exploring Machine Learning to Support Decision-Making for Placement Stabilization and Preservation in Child Welfare
Auteurs: Ka Ho Brian Chor
Zhidi Luo
Kit T. Rodolfa
Rayid Ghani
Publicatiedatum: 07-01-2025
Uitgeverij: Springer US
Gepubliceerd in: Journal of Child and Family Studies
Print ISSN: 1062-1024
Elektronisch ISSN: 1573-2843
DOI: https://doi.org/10.1007/s10826-024-02993-x

Bohn Stafleu van Loghum

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Log in om toegang te krijgen