Skip to main content
SpringerLink
Log in

Is It Safe? Reliability and Validity of Structured Versus Unstructured Child Safety Judgments

  • Original Paper
  • Published:
Child & Youth Care Forum Aims and scope Submit manuscript

Abstract

Background

The LIRIK, an instrument for the assessment of child safety and risk, is designed to improve assessments by guiding professionals through a structured evaluation of relevant signs, risk factors, and protective factors.

Objective

We aimed to assess the interrater agreement and the predictive validity of professionals’ judgments made with the LIRIK in comparison to unstructured judgments.

Method

In study 1, professionals made safety and risk judgments for 12 vignettes with the LIRIK (group 1, n = 36) or without an instrument (group 2, n = 43). In study 2, we compared professionals’ safety and risk judgments for 370 children made with the LIRIK (group 1, n = 278) or with no instrument (group 2, n = 92), with outcomes indicating actual unsafety in files 6 months later.

Results

In study 1, agreement about safety and risks was poor to moderate in both groups. Differences between groups were small and inconsistent. In study 2, the predictive validity of judgments was weak to moderate in both groups. In neither group had unsafe outcomes increased consistently when unsafety or risks were assessed as higher.

Conclusions

Judgments made with the LIRIK were not more reliable or valid than unstructured professional judgments. These findings raise important questions about the value of risk assessment instruments and about how professional safety and risk judgments can be improved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The fifth category, lack of information, can be considered to be an uninformative category with respect to participants’ safety judgments. Uninformative categories in a coding system are best replaced by informative values, or otherwise excluded from analyses of agreement (Krippendorff 2011).

  2. To control for biases that might result from our recoding, we performed two additional transformations, that excluded no judgments. When two or three other categories than 5 were chosen, for one transformation we took the highest option, for the other transformation the lowest option. In both groups, calculation of agreement with the three different transformations did not result in significantly different alphas (see “Results” section, and Bartelink et al. 2015).

  3. We had 181 judgments in the LIRIK group and 183 judgments in the non-LIRIK group. These numbers are sufficient for reliably deciding (p ≤ .05) whether Krippendorff’s alpha would exceed a minimum of .80 [see Krippendorff (2011): Table 1, p 105]. For reliably deciding on lower minimum levels fewer judgments are needed.

  4. The use of the LIRIK was a standard procedure within the agencies. To collect data for the control group, they adjusted their procedure temporarily.

  5. We found no difference between the LIRIK group and the control group, except for the safety judgments and number of out-of-home placements. The safety judgments of the control group were statistically significantly higher than those of the LIRIK group, meaning that professionals in the control group judged cases to be more unsafe (χ2 (3) = 10.52, p = .02). In the control group children were significantly more often placed out-of-home than in the LIRIK group (χ2 (1) = 14.84, p = .00).

  6. The LIRIK was revised just before the studies started, in response to youth care professionals’ feedback. Differences between the earlier and the latest version consist of textual changes and changes in item ordering. The risk judgments changed from a four point to a five point scale. For the validity study, professionals’ assessments of 6 months earlier were collected, therefore the risk judgments were on a four point scale here.

  7. The relations between judgments and outcomes were the same when the cases were included in which the professional concluded that there was a life-threatening situation.

References

  • Arad-Davidson, B., & Benbenishty, R. (2008). The role of workers’ attitudes and parent and child wishes in child protection workers’ assessments and recommendation regarding removal and reunification. Children and Youth Services Review, 30, 107–121.

    Article  Google Scholar 

  • Baird, C., & Wagner, D. (2000). The relative validity of actuarial- and consensus-based risk assessment systems. Children and Youth Services Review, 22, 839–871.

    Article  Google Scholar 

  • Baird, C., Wagner, D., Healy, T., & Johnson, K. (1999). Risk assessment in child protection services: Consensus and actuarial model reliability. Child Welfare, 78, 723–748.

    PubMed  Google Scholar 

  • Barber, J., Trocmé, N., Goodman, D., Shlonsky, A., Black, T., & Leslie, B. (2007). The reliability and predictive validity of consensus-based risk assessment. Toronto: Centre of Excellence for Child Welfare.

    Google Scholar 

  • Bartelink, C., De Kwaadsteniet, L., ten Berge, I., Witteman, C., & Van Gastel, W. (2015). Betrouwbaarheid en validiteit van de LIRIK: Eindrapport LIRIK valideringsonderzoek. [Reliability and validity of the LIRIK: Final report LIRIK validation study.]. Utrecht: Nederlands Jeugdinstituut.

    Google Scholar 

  • Bartelink, C., & Kooijman, K. (2013). Inschatten van veiligheid en kans op kindermishandeling: Noodzaak, instrumenten en ontwikkelingen. [Estimating the safety and risk of child maltreatment: Necessity, instruments and developments.]. Tijdschrift voor Sociale Geneeskunde, 91, 391–393.

    Google Scholar 

  • Bartelink, C., Van Yperen, T. A., ten Berge, I. J., De Kwaadsteniet, L., & Witteman, C. L. M. (2014). Agreement on child maltreatment decisions: A nonrandomized study on the effects of structured decision-making. Child & Youth Care Forum, 43, 639–654.

    Article  Google Scholar 

  • Baumann, D. J., Dalgleish, L., Fluke, J., & Kern, H. (2011). The decision-making ecology. Washington: American Humane Association.

    Google Scholar 

  • Baumann, D. J., Law, J. R., Sheets, J., Reid, G., & Graham, J. C. (2005). Evaluating the effectiveness of actuarial risk assessment models. Children and Youth Services Review, 27, 465–490.

    Article  Google Scholar 

  • Begle, A. M., Dumas, J. E., & Hanson, R. F. (2010). Predicting child abuse potential: An empirical investigation of two theoretical frameworks. Journal of Clinical Child & Adolescent Psychology, 39, 208–219.

    Article  Google Scholar 

  • Berben, E. G. M. J. (2000). Als iedereen hetzelfde was… indicatiestelling in de jeugdzorg. [If everybody would be the same… assessment of youth care.]. Maastricht: Shaker Publishing B.V.

    Google Scholar 

  • Camasso, M. J., & Jagannathan, R. (2000). Modeling the reliability and predictive validity of risk assessment in child protective services. Children and Youth Services Review, 22, 873–896.

    Article  Google Scholar 

  • Camasso, M. J., & Jagannathan, R. (2013). Decision making in child protective services: A risky business? Risk Analysis, 33, 1636–1649.

    Article  PubMed  Google Scholar 

  • Cichetti, D. V. (2001). The precision of reliability and validity estimates re-visited: Distinguishing between clinical and statistical significance of sample size requirements. Journal of Clinical and Experimental Neuropsychology, 23, 695–700.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New Jersey: Lawrence Erlbaum Associates.

    Google Scholar 

  • D’Andrade, A., Benton, A., & Austin, M. J. (2005). Risk and safety assessment in child welfare: Instrument comparisons. Berkeley: Bay Area Social Services Consortium.

    Google Scholar 

  • Dalgleish, L. I. (1988). Decision-making in child abuse cases: Applications of social judgment theory and signal detection theory. In B. Brehmer & C. R. B. Joyce (Eds.), Human Judgment: The SJT view (pp. 317–360). North Holland: Elsevier.

    Chapter  Google Scholar 

  • Dalgleish, L. I. (2000). Assessing the Situation and Deciding to do Something: Risk, Needs and Consequences. Paper presented at the 13th International congress on child abuse and neglect, Durban, September 2000.

  • Dalgleish, L. I. (2003). Risk, needs and consequences. In M. C. Calder (Ed.), Assessments in child care: A comprehensive guide to frameworks and their use (pp. 86–99). Dorset: Russell House Publishing.

    Google Scholar 

  • Dana, J., Dawes, R., & Peterson, N. (2013). Belief in the unstructured interview: The persistence of an illusion. Judgment and Decision Making, 8, 512–520.

    Google Scholar 

  • Danktert, E. W., & Johnson, K. (2013). Risk assessment validation: A prospective study. Los Angeles: California Department of Social Services, Children and Family Services Division.

    Google Scholar 

  • Davidson-Arad, B. (2005). Fifteen month follow-up of children at risk: Comparison of the quality of life of children removed from home and children remaining at home. Child and Youth Services Review, 27, 1–20.

    Article  Google Scholar 

  • Davidson-Arad, B. (2010). Four perspectives on the quality of life of children at risk kept at home and removed from home in Israel. British Journal of Social Work, 40, 1719–1735.

    Article  Google Scholar 

  • Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1674.

    Article  PubMed  Google Scholar 

  • De Kwaadsteniet, L., Bartelink, C., Witteman, C. L. M., ten Berge, I. J., & Van Yperen, T. A. (2013). Improved decision making about suspected child maltreatment: Results of structuring the decision process. Children and Youth Services Review, 35, 347–352.

    Article  Google Scholar 

  • De Ruiter, C., Hildebrand, M., & Van der Hoorn, S. (2012). Risicotaxatie bij kindermishandeling: De Child Abuse Risk Evaluation–Nederlandse versie (CARE-NL). [Risk assessment in child maltreatment cases: The Child Abuse Risk Evaluation–Dutch version (CARE = NL.). In H. P. B. Lodewijks & L. Van Domburg (Eds.), Instrumenten voor risicotaxatie: Kinderen en jeugdigen [Instruments for risk assessment: Children and youth] (pp. 169–190). Amsterdam: Pearson.

    Google Scholar 

  • Doyle, J. (2007). Child protection and child outcomes: Measuring the effects of foster care. The American Economic Review, 97, 1583–1608.

    Article  Google Scholar 

  • Dumbrill, G. C. (2005). Child welfare in Ontario: Developing a collaborative intervention model. Toronto: Ontario Association of Children’s Aid Societies.

    Google Scholar 

  • Edwards, V. J., Anda, R. F., Dube, S. R., Dong, M., Chapman, D. F., & Felitti, V. J. (2005). The wide-ranging health consequences of adverse childhood experiences. In K. Kendall-Tackett & S. Giacomoni (Eds.), Victimization of children and youth: Patterns of abuse, response strategies. Kingston, NJ: Civic Research Institute.

    Google Scholar 

  • Enosh, G., & Bayer-Topilsky, T. (2014). Reasoning and bias: Heuristics in safety assessment and placement decisions for children at risk. British Journal of Social Work, 45, 1–17.

    Google Scholar 

  • Evers, A., Lucassen, W., Meijer, R., & Sijtsma, K. (2010). COTAN Beoordelingssysteem voor de Kwaliteit van Tests [COTAN review system for the quality of tests]. Amsterdam: NIP, COTAN.

    Google Scholar 

  • Felitti, V. J., Anda, R. F., Nordenberg, D., Williamson, D. F., Spitz, A. M., Edwards, V., et al. (1998). Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults: The adverse childhood experiences (ACE) study. American Journal of Preventive Medicine, 14, 245–258.

    Article  PubMed  Google Scholar 

  • Finnila, K., Santtila, P., Mattila, J., & Niemi, P. (2012). The effects of experience, outcome feedback, and cognitive feedback on decision-making in child sexual abuse cases: A simulation study. Nordic Psychology, 64, 242–257.

    Article  Google Scholar 

  • Gambrill, E., & Shlonsky, A. (2000). Risk assessment in context. Children and Youth Services Review, 22, 813–837.

    Article  Google Scholar 

  • Gambrill, E., & Shlonsky, A. (2001). The need for comprehensive risk management programs in child protective services. Children and Youth Services Review, 23, 79–107.

    Article  Google Scholar 

  • Gold, N., Benbenishty, R., & Osmo, R. (2001). A comparative study of risk assessment and recommended interventions in Canada and Israel. Child Abuse and Neglect, 25, 607–622.

    Article  PubMed  Google Scholar 

  • Golnik, A., Maccabee-Ryaboy, N., Scal, P., Wey, A., & Gaillard, P. (2012). Shared decision making: Improving care for children with autism. Intellectual and Developmental Disabilities, 50, 322–331.

    Article  PubMed  Google Scholar 

  • Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19–30.

    Article  PubMed  Google Scholar 

  • Hacking, I. (1992). World-making by kind-making: Child abuse for example. In M. Douglas & D. Hull (Eds.), How classification works (pp. 180–238). Edinburgh: Edinburgh University Press.

    Google Scholar 

  • Hardman, D. (2009). Judgment and decision making: Psychological perspectives. West Sussex: BPS Blackwell.

    Google Scholar 

  • Hart, S. D., Mitchie, C., & Cooke, (2007). Precision of actuarial risk assessment instruments: Evaluating the ‘margins of error’ of group v. individual predictions of violence. British Journal of Psychiatry, 190(49), s60–s65.

    Article  Google Scholar 

  • Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77–79.

    Article  Google Scholar 

  • Hinkle, D. E., Wiersma, W., & Jurs, S. G. (2003). Applied statistics for the behavioral sciences (5th ed.). Boston: Houghton Mifflin.

    Google Scholar 

  • Hollander, M., Van Klaveren, S., Faun, H., & Spijkerman, M. (2013). Arbeidsmarkteffectrapportage transitie jeugdzorg (labour market outcomes report transition youth care). Zoetermeer: Panteia.

    Google Scholar 

  • Johnson, W. (2006). The risk assessment wars: A commentary response to “Evaluating the effectiveness of actuarial risk assessment models” by Donald Baumann, J. Randolph Law, Janess Sheets, Grant Reid, and J. Christopher Graham, Children and Youth Services Review, 27, 465–490. Children and Youth Services Review, 28, 704–714.

  • Johnson, W. L. (2011). The validity and utility of the California Family Risk Assessment under practice conditions in the field: A prospective study. Child Abuse and Neglect, 35, 18–28.

    Article  PubMed  Google Scholar 

  • Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64, 515–526.

    Article  PubMed  Google Scholar 

  • Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendation. Human Communication Research, 30, 411–433.

    Google Scholar 

  • Krippendorff, K. (2011). Agreement and information in the reliability of coding. Communication Methods and Measures, 5, 93–112.

    Article  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    Article  PubMed  Google Scholar 

  • Lekkerkerker, L., Bartelink, C., & Eijgenraam, K. (2011). De indicatiestelling bij de Brabantse Bureaus Jeugdzorg nader bekeken. Een onderzoek naar de kwaliteit van het indicatieproces en de betrouwbaarheid van het indicatiebesluit. [A closer look at the assessment of Youth Care Agency Brabant. A study on the quality of the assessment process and reliability of the care decision.]. Utrecht: Nederlands Jeugdinstituu.

  • Lindsey, D. (1992). Reliability of the foster care placement decision: A review. Research on Social Work Practice, 2, 65–80.

    Article  Google Scholar 

  • MacKenzie, M. J., Kotch, J. B., & Lee, L. (2011). Toward a cumulative ecological risk model for the etiology of child maltreatment. Children and Youth Services Review, 33, 1638–1647.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mandel, D. R., Lehman, D. R., & Yuille, J. C. (1994). Should this child be removed from home? Hypothesis generation and information seeking as predictors of case decisions. Child Abuse and Neglect, 18, 1051–1062.

    Article  PubMed  Google Scholar 

  • Munro, E. (1999). Common errors of reasoning in child protection work. Child Abuse and Neglect, 23, 745–758.

    Article  PubMed  Google Scholar 

  • Munro, E. (2005). Improving practice: Child protection as a systems problem. Children and Youth Services Review, 27, 375–391.

    Article  Google Scholar 

  • Munro, E. (2008). Effective child protection. London: Sage.

    Google Scholar 

  • Munro, E. (2014). Understanding the causal pathways to child maltreatment: Implications for health and social care policy and practice. Child Abuse Review, 23, 61–74.

    Article  Google Scholar 

  • Nanni, V., Uher, R., & Danese, A. (2012). Childhood maltreatment predicts unfavorable course of illness and treatment outcome in depression: A meta-analysis. The American Journal of Psychiatry, 169, 141–151.

    Article  PubMed  Google Scholar 

  • Orsi, R., Drury, I. J., & Mackert, M. J. (2014). Reliable and valid: A procedure for establishing inter-item level interrater reliability for child maltreatment risk and safety assessments. Children and Youth Services Review, 43, 58–66.

    Article  Google Scholar 

  • Perry, B. D. (2009). Examining child maltreatment through a neurodevelopmental lens: Clinical applications of the neurosequential model of therapeutics. Journal of Loss and Trauma, 14, 240–255.

    Article  Google Scholar 

  • Pinto, R. J., & Maia, A. C. (2013). Psychopathology, physical complaints and health risk behaviours among youths who were victims of childhood maltreatment: A comparison between home and institutional interventions. Children and Youth Services Review, 35, 603–610.

    Article  Google Scholar 

  • Prins, D. (2011). Een onderzoek naar de ORBA-werkwijze: Onderzoek, Risicotaxatie en Besluitvorming van de Advies- en Meldpunten Kindermishandeling. [A study on the ORBA method: Investigation, risk assessment and decision-making in the advice and reporting centres on child abuse and neglect.]. Utrecht: University of Utrecht (masterthesis).

  • Regehr, C., Bogo, M., Shlonsky, A., & LeBlanc, V. (2010). Confidence and professional judgment in assessing children’s risk of abuse. Research on Social Work Practice, 20, 621–628.

    Article  Google Scholar 

  • Rycus, J. S., & Hughes, R. C. (2003). Issues in risk assessment: Policy white paper. Columbus, Ohio: North American Resource Center for Child Welfare.

    Google Scholar 

  • Schuerman, J., Rossi, P. H., & Budde, S. (1999). Decisions on placement and family preservation: Agreement and targeting. Evaluation Review, 23, 599–618.

    Article  PubMed  Google Scholar 

  • Shlonsky, A., & Wagner, D. (2005). The next step: Integrating actuarial risk assessment and clinical judgment into an evidence-based practice framework in CPS case management. Children and Youth Services Review, 27, 409–427.

    Article  Google Scholar 

  • Smithgall, C., Jarpe-Ratner, E., Gnedko-Berry, N., & Mason, S. (2015). Developing and testing a framework for evaluating the quality of comprehensive family assessment in child welfare. Child Abuse and Neglect, 44, 194–206.

    Article  PubMed  Google Scholar 

  • Ten Berge, I. J. (1998). Besluitvorming in de kinderbescherming. De ontwikkeling en evaluatie van een checklist voor de beoordeling van meldingen bij de raad voor de kinderbescherming. [Decision-making in Child Protective Services. The development and evaluation of a checklist for decision-making at Child Protective Services intake.]. Dissertation, Eburon, Delft.

  • Ten Berge, I. J., Eijgenraam, K., & Bartelink, C. (2014a). Licht instrument risicotaxatie kindveiligheid: Herziene versie juni 2014 [Light instrument risk assessment child safety: Revised version June 2014]. Utrecht: Nederlands Jeugdinstituut.

  • Ten Berge, I. J., Eijgenraam, K., & Bartelink, C. (2014b). Licht instrument risicotaxatie kindveiligheid: Toelichting en instructie [Light instrument risk assessment child safety: Explanation and instruction]. Utrecht: Nederlands Jeugdinstituut.

  • Ten Berge, I., & Meuwissen, I. (2013). Bruikbaarheid en mogelijke aanpassingen van de LIRIK voor de toepassing in de (L)VB-sector: Bevindingen van de pilot augustus 2012–oktober 2013 [Utility and possible adaptations tot he LIRIK for the use in the (mild) mental disabilities field: Findings from the pilot August 2012–October 2013]. Utrecht: Nederlands Jeugdinstituut.

  • Ten Berge, I., & Van Rossum, J. (2009). Evaluatie en bijstelling GCT en LIRIK. Samenvatting resultaten en aanpassingen [Evaluation and adaptation GCT and LIRIK. Summary results and adaptations]. Utrecht: Nederlands Jeugdinstituut.

  • Ten Berge, I., & Vinke, A. (2006a). Beslissen over vermoedens van kindermishandeling: Eindrapport project Onderzoek, Risicotaxatie en Besluitvorming Advies- en Meldpunten Kindermishandeling (ORBA) [Deciding on suspicions of child maltreatment: Final report on ORBA project]. Utrecht, Woerden: Nederlands Jeugdinstituut, Adviesbureau Van Montfoort.

  • Ten Berge, I., & Vinke, A. (2006b). Beslissen over vermoedens van kindermishandeling: Handreiking en hulpmiddelen voor het Advies- en Meldpunt Kindermishandeling [Deciding on suspicions of child maltreatment: Practice manual and tools for the Advice and Reporting Centres of Child Abuse and Neglect]. Utrecht, Woerden: Nederlands Jeugdinstituut Adviesbureau Van Montfoort.

  • Turnell, A., & Edwards, S. (1999). Signs of safety: A solution and safety oriented approach to child protection casework. New York, London: Norton.

    Google Scholar 

  • Van der Elst, M., Sondeijker, F., Vogel, I., Jansen, W., & Hermanns, J. (2012). Veiligheidsrisicotaxatie bij Opvoedhulp en Opgroeihulp aan Gezinnen met Kinderen van 0-12 jaar: Validering van de California Family Risk Assessment. [Safety risk assessment in child and youth care to families with children aged 0–12 year: Validation of the California Family Risk Assessment]. Woerden: GGD Rotterdam-Rijnmond Van Montfoort Collegio.

  • Van der Put, C. E., Assink, M., & Stams, G. J. J. M. (2016). Predicting relapse of problematic child-rearing situations. Children and Youth Services Review, 61, 288–295.

    Article  Google Scholar 

  • Vis, S. A., Strandbu, A., Holtan, A., & Thomas, N. (2011). Participation and health: A research review of child participation in planning and decision making. Child and Family Social Work, 16, 325–335.

    Article  Google Scholar 

  • White, A., & Walsh, P. (2006). Risk assessment in child welfare: An issues paper. Ashfield: Centre for Parenting & Research.

    Google Scholar 

Download references

Acknowledgements

Stichting Kinderpostzegels Nederland sponsored this study. We thank all participants for their cooperation. We specifically thank Carolien Konijn (Spirit), Mariëtte Blik and Mariëtte Brandenburg (Juzt), Mariëlle Hornstra (GGD-Zuid-Gelderland) for their collaboration and for their insightful contributions. We thank Floor Minkhorst for designing the online questionnaires and analyzing participants’ comments about the LIRIK, Raoul Hakkenberg for coding the case files, Pieter Groenestijn for support with Perseus and Pierre Souren for support with the statistical analyses, and all members of the research committee for their valuable feedback during the whole research project.

Funding

This study was funded by Stichting Kinderpostzegels Nederland (Grant No. 8329641).

Author information

Author notes

  1. Cora Bartelink and Leontien de Kwaadsteniet contributed equally to the paper.

Authors and Affiliations

  1. The Netherlands Youth Institute, Nederlands Jeugdinstituut, PO Box 19221, 3501 DE, Utrecht, The Netherlands

    Cora Bartelink & Ingrid J. ten Berge

  2. Behavioural Science Institute, Radboud University, Nijmegen, The Netherlands

    Leontien de Kwaadsteniet & Cilia L. M. Witteman

Authors
  1. Leontien de Kwaadsteniet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ingrid J. ten Berge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cilia L. M. Witteman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cora Bartelink.

Ethics declarations

Conflict of interest

Cora Bartelink was involved in the development of the LIRIK. Leontien de Kwaadsteniet declares that she has no conflict of interest. Ingrid ten Berge was involved in the development of the LIRIK. Cilia Witteman declares that she has no conflict of interest.

Ethical Approval

All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Appendix

Appendix

See Table 6.

Table 6 Proportions of how often participants in the LIRIK group believed a specific factor was present, measured over cases and raters, and Krippendorff’s alphas for interrater agreement (with bootstrap 95%-confidence intervals) for these items in Study 1, and Spearman’s rank correlations between the items and any unsafe outcome in Study 2
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bartelink, C., de Kwaadsteniet, L., ten Berge, I.J. et al. Is It Safe? Reliability and Validity of Structured Versus Unstructured Child Safety Judgments. Child Youth Care Forum 46, 745–768 (2017). https://doi.org/10.1007/s10566-017-9405-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10566-017-9405-2

Keywords

Search

Navigation