Elsevier

Evaluation and Program Planning

Volume 53, December 2015, Pages 99-106
Evaluation and Program Planning

Using concept mapping as a planning tool: Child welfare citizen review panels

https://doi.org/10.1016/j.evalprogplan.2015.08.001Get rights and content

Highlights

  • Concept mapping (CM) as a valuable tool for planning and evaluation procedures.

  • CM inherently allayed challenges explicitly identified in the literature.

  • Methodology was easily understood by study participants.

  • More research needs to be conducted related to the evaluation of CRPs.

Abstract

Citizen Review Panels (CRPs) are groups of citizen volunteers authorized by U.S. federal law to examine state child welfare agencies. These groups inspect policies and practices related to child protection responsibilities and are tasked with making recommendations for systemic improvement. Despite the federal mandate for each state to develop a CRP and the potential of these groups to positively impact child welfare practices, there is a dearth in the literature related to CRPs. Consequently, planning and evaluation processes of these groups vary widely. This study reports on the use of concept mapping (CM) to outline a framework for planning and subsequently evaluating the CRP in one southeastern state. CM is a mixed-method research approach that uses multi-dimensional scaling and hierarchical cluster analyses to explore an area of study. Through these analyses, the method creates visual depictions of conceptual relationships between ideas. Data yielded a seven cluster concept map that CRP members (N = 36) utilized for planning processes, and subsequently for developing an internal evaluation tool. Results from this study offer a unifying framework by which CRPs, and similar groups in other areas can utilize for planning and evaluation purposes. After a review of pertinent literature on CRPs, this article explicates CM processes utilized in this study, describes results, discusses lessons learned, and outlines apposite areas for future CRP research.

Introduction

Participation of an engaged citizenry has long been recognized as a pertinent part of planning governmental programs and interventions (e.g., Kinney, 2008, Roberts, 2004, Stivers, 1990), specifically in the area of child welfare (Bryan et al., 2007, Blome and Steib, 2007, Schorr, 2000). One way this participation has manifested in child welfare is through the formation of Citizen Review Panels (CRPs). CRPs are groups of citizen volunteers authorized by U.S. federal law to examine state child welfare agencies. These groups inspect policies and practices related to child protection responsibilities and are tasked with making recommendations for systemic improvement (U.S. Department for Health & Human Services, 2014).

Despite the federal mandates for CRPs, the millions of dollars of resources allotted to these panels, and the potential that these panels have in improving the child welfare system, little research has explored planning processes and evaluation related to CRPs (Bryan, Jones, & Lawson, 2010). Divergent CRP planning and implementation processes between states and a lack of evaluation models and tools have undoubtedly contributed to this absence of published literature. This paper seeks to uniquely contribute to addressing limitations in the current literature.

This study reports on the use of concept mapping (CM) to outline a framework for planning and subsequently evaluating the CRP in one southeastern state. CM is a mixed-method research approach that uses multi-dimensional scaling and hierarchical cluster analyses to explore an area of study (Kane & Trochim, 2007). Through these analyses, the method creates visual depictions of conceptual relationships between ideas (Anderson et al., 2006). After a review of pertinent literature on CRPs, this article explicates CM processes utilized in this study, describes results, discusses lessons learned, and outlines apposite areas for future CRP research.

Section snippets

History of Citizen Review Panels

The origins of CRP can be traced to the Child Abuse Prevention and Treatment Act (P.L. 93-247), or CAPTA. This seminal piece of child welfare legislation, passed by Congress and then signed into law by President Nixon in 1974, was the first substantive federal response to improve responses to child maltreatment in the United States (National Child Abuse and Neglect Training and Publications Project [NCANTPP], 2014). CAPTA mandated the creation of the National Center on Child Abuse and Neglect,

Method

To delineate a planning framework for a CRP in a southeastern state this study employed concept mapping (CM). CM is a participatory, mixed-method research approach that utilizes quantitative processes to analyze qualitative data (Brown, 2008). This approach pairs multidimensional scaling with hierarchical cluster analysis to create clusters in a two-dimensional space (along x and y axes). Through these analyses, pictorial representations of the data are created. These visual representations

Results

The final cluster map yielded seven distinct clusters. Fig. 1 illustrates this map, which includes the following clusters: Internal Group Communication, Leadership, Meeting Guidelines, Membership, External Resource Needs, Statement Government, and Research. Each cluster was named based on the labels ascribed to each “pile” by participants during the statement structuring exercise denoted above. The cluster names are intended to represent the overall conceptual meaning of the statements that

Discussion

The overarching aim of this study was to use CM to identify a framework germane to programmatic planning for a state CRP group. This aim in mind, this study had a very positive impact on constructing and organizing data in a way that proved useful for the panel. The following paragraphs outline salient discussion points derived from this study.

First, much of the data collected in consistent with existing, albeit very limited, data on CRPs. For example, findings in the literature point to the

Lessons learned

There are many lessons to be taken from this study, particularly related to process. The following paragraphs outline a few examples.

First, CM is a viable methodology for unifying development/planning and evaluation processes. While this study, and associated data, are restricted to the state in which the study was conducted, this documented process may offer a framework for future planning and evaluation of CRPs and similar groups in other locations. CM gives a common language that utilizes

Conclusion

For this study, CM provided a participatory and structured mechanism for planning, and subsequently evaluating, a federally mandated citizen group tasked with examining child welfare practices in one south-eastern state. Results from this study coincide with existing literature, as well as shed light on other pertinent aspects for CRP planning procedures. Additionally, data from this study provides the foundation for the development of an internal evaluation tool. While this study does

Dr. Justin Jay Miller is an Assistant Professor in the College of Social Work at the University of Kentucky. Jay has previously worked as a social worker at the Cabinet for Health and Family Services, Louisville's Crimes Against Children Unit, and the Ireland Army Hospital at Fort Knox. Jay is dedicated to social issues and community outreach, a passion which he brings to his work as an educator and scholar. His research and academic interests focus on child welfare, particularly outcomes

References (46)

  • C. Baldwin et al.

    Complementary and conventional medicine: A concept map

    BioMed Central

    (2004)
  • R. Bedi

    Concept mapping the client's perspective on counseling alliance formation

    Journal of Counseling Psychology

    (2006)
  • D. Biegel et al.

    Overcoming barriers faced by African-American families with a family member with mental illness

    Family Relations

    (1997)
  • W. Blome et al.

    An examination of oversight and review in the child welfare system: The many watch the few serve many

    Journal of Public Child Welfare

    (2007)
  • J. Brown

    Foster parents’ perceptions of factors needed for successful foster placements

    Journal of Child and Family Studies

    (2008)
  • J. Brown et al.

    Children with fetal alcohol spectrum disorders: A concept map of parenting children with FASD

    Journal of Family Social Work

    (2004)
  • N. Buckwalter

    The potential for public empowerment through government-organized participation

    Public Administration Review

    (2014)
  • W. Collins

    Citizen review panels: An opportunity to improve child protection?

    (1998)
  • J. Donnelly et al.

    The needs of children with life-limiting conditions: A healthcare-provider-based model

    American Journal of Hospice and Palliative Medicine

    (2005)
  • M. Gonzalez-Block et al.

    Mapping of health system functions to strengthen priority programs: The case of maternal health in Mexico

    BioMed Public Health

    (2011)
  • L. Hurt et al.

    Gender differences in evaluating social-sexual conduct in the workplace

    Behavioral Sciences and the Law

    (1999)
  • M. Iris et al.

    The development of a conceptual model for understanding elder self-neglect

    The Gerentologist

    (2010)
  • K. Jackson et al.

    Concept mapping as an alternative approach for the analysis of open-ended survey responses

    Organizational Research Methods

    (2002)
  • Cited by (15)

    • Concept mapping the needs of Flemish nonkinship foster parents who care for unaccompanied refugee minors

      2019, Children and Youth Services Review
      Citation Excerpt :

      It is a participatory, mixed-method approach using a quantitative fixed statistical algorithm to analyze qualitative data (Brown, 2008; Nabitz et al., 2017). Through these analyses, visual representations of the data are constructed (Miller & Jones, 2015). Concept mapping has been used several times to examine the needs of (specific groups of) foster parents (e.g., Brown, 2008; Brown et al., 2013; Brown & Calder, 2000; Van Holen, Van Loock, et al., 2017).

    • Training child welfare citizen review panel members: A promising approach?

      2018, Children and Youth Services Review
      Citation Excerpt :

      Once collected, data were analyzed for the explicit purpose of answering the previously proposed evaluation queries. Based on a literature review on CRPs (Bryan et al., 2010; Jones et al., 2003; Jones & Royse, 2008; Miller & Jones, 2015), the workgroup developed a questionnaire intended to collect primary data related to evaluating the training. The survey entailed three distinct sections and was piloted with a small group of former CRP members (n = 5) for item clarity and readability.

    View all citing articles on Scopus

    Dr. Justin Jay Miller is an Assistant Professor in the College of Social Work at the University of Kentucky. Jay has previously worked as a social worker at the Cabinet for Health and Family Services, Louisville's Crimes Against Children Unit, and the Ireland Army Hospital at Fort Knox. Jay is dedicated to social issues and community outreach, a passion which he brings to his work as an educator and scholar. His research and academic interests focus on child welfare, particularly outcomes related to foster care and adoption. Jay is actively involved in several community endeavors and has served as the co-founder/past president of the Louisville Association of Social Workers, and the founder of the Jefferson County Foster Care Peer Support Program and the Kentucky Chapter of the Foster Care Alumni of America. Last but not least, Jay is a proud foster care alum!

    Blake L. Jones, MSW, LCSW, Ph.D. is a leading national authority on Citizen Review Panels (CRP) for child protective services. Based at the University of Kentucky, Dr. Jones has published numerous scholarly articles on CRPs and has provided technical assistance to over thirty states in the area of citizen participation and child welfare. He has hosted five national Citizen Review Panel conferences and maintained a national “virtual community” for CRPs. Dr. Jones is the Program Coordinator for Kentucky's Citizen Review Panels, an adjunct professor at the University of Kentucky's College of Social Work, and a husband and father of two sons.

    View full text