Results from a randomized trial of the Healthy Families Oregon accredited statewide program: Early program impacts on parenting
Introduction
Home visiting has become increasingly accepted as an effective strategy for supporting healthy development of infants and toddlers, improving parenting practices, and reducing family and child risk factors associated with child maltreatment, juvenile delinquency, and other negative outcomes (Doggett, 2013, Peacock et al., 2013). Federal support for home visiting has greatly expanded the reach of these services through the Maternal, Infant, and Early Childhood Home Visiting initiative (MIECHV, U.S. Department of Health and Human Services, Health Resources and Services Administration, 2013), which has provided over $1.5 billion in funding for home visiting programs nationally. At the same time, however, scholars have repeatedly noted that the outcomes of home visiting programs are modest in magnitude, as well as inconsistent in demonstrating positive outcomes (Daro, 2006, Gomby et al., 1999, Howard and Brooks-Gunn, 2009).
The Healthy Families America (HFA) program, although it is widely implemented nationally and one of 13 home visiting models identified as meeting federal criteria for “evidence based” home visitation services, has a history of inconsistent evaluation results, and poses particular challenges in terms of cross-study synthesis of findings. The model, by design, allows considerable local variability in terms of such key program components as target population and curriculum. This local variability is both a strength of the model, in that specific aspects of the program can be tailored to best meet individual community needs, as well as a challenge — in particular, that this local variability makes the synthesis and generalizability of outcomes from studies of tHFA more difficult, and that outcome studies have had more inconsistent outcomes than those of more prescriptive models (Azzi-Lessing, 2011, LeCroy and Krysik, 2011). More research on this widely disseminated and popular model that can better identify and specify how model variations may influence outcomes is needed.
More generally, several recent articles have identified the need for more research that can identify program and family characteristics that may contribute to the variability in program outcomes in the home visiting literature (Azzi-Lessing, 2011, Howard and Brooks-Gunn, 2009, Kahn and Moore, 2010, Peacock et al., 2013). Characteristics that have been highlighted as particularly important in this regard include the quality of service delivery (Azzi-Lessing, 2011, Daro, 2006, Howard and Brooks-Gunn, 2009, Kahn and Moore, 2010, Peacock et al., 2013); the timing of initiation of services (specifically, prenatal vs. postnatal enrollment and enrollment of first-time vs. multiparous mothers; Huntington and Galano, 2013, Kahn and Moore, 2010); community and cultural context (Azzi-Lessing, 2011, Del Grosso et al., 2012); and effectiveness for families with specific risk factors (e.g., teen parents, depressed and/or psychologically vulnerable mothers; Kahn and Moore, 2010, Rodriguez et al., 2010) or different levels of cumulative risk (Azzi-Lessing, 2013, Peterson et al., 2013). Below we briefly review features of programs and families that may influence outcomes for HFA and other home visiting models, with a particular emphasis on findings related to HFA programs. The current study will add to our understanding of the effectiveness of home visiting programs, and specifically of the Healthy Families America (HFA) model, by rigorously evaluating early program impacts and by systematically assessing outcomes for key subgroups defined by characteristics that have been hypothesized to influence program effectiveness.
The quality of program implementation, and in particular the dosage, frequency, and content of home visits is a near-universal challenge for home visiting programs and associated research. Two early randomized studies of the HFA model in Alaska (Caldera et al., 2007, Duggan et al., 2007) and Hawaii (Duggan, Fuddy, et al., 2004, Duggan, McFarlane, et al., 2004, Duggan et al., 2000) found few positive impacts for the program, while at the same time describing significant implementation issues in terms of programs' ability to engage and retain families, and to deliver the expected level of home visits. For example, in Healthy Families Alaska, fewer than 4% of families received 75% of expected home visits during the first two years of the program (Caldera et al., 2007). Further, neither of these programs went through the rigorous quality assurance process now available through HFA accreditation (HFA, 2014). Accreditation involves documentation by external reviewers and site visitors of the extent to which programs meet over 100 research-based standards of practice related to training, supervision, staff characteristics, curriculum implementation, service delivery and retention, and ongoing evaluation (HFA, 2014). While accreditation does not guarantee high quality implementation, it provides a clear structure and process for ongoing quality assurance; programs are re-accredited every five years and those which do not meet the standards risk losing their accredited status (HFA, 2014).
Three more recent randomized studies, of Healthy Families programs in New York (DuMont et al., 2008, DuMont et al., 2010), Massachusetts (Easterbrooks et al., 2012, Easterbrooks et al., 2013), and Arizona (LeCroy & Krysik, 2011) have examined the effectiveness of accredited HFA programs using rigorous randomized designs. Perhaps not surprisingly, the outcomes of these accredited programs have shown more positive results across a number of domains, including the frequency of positive discipline strategies (DuMont et al., 2008); reduction in harsh and severe parenting (DuMont et al., 2008, LeCroy and Krysik, 2011) and maternal parenting stress (Easterbrooks et al., 2012); and lower maternal alcohol use (LeCroy & Krysik, 2011). These studies suggest that evaluations of HFA programs must clearly identify the quality of program implementation, and call into questions results from programs that do not meet HFA accreditation standards. The current study, while it does not directly assess program implementation, involves a long-standing statewide accredited HFA program, Healthy Families Oregon, with a strong history of data-driven quality improvement (e.g., Green, Tarte, Aborn, & Talkington, 2014).
Another key program characteristic that has varied across studies of HFA is the point of entry into services, specifically, whether mothers are enrolled prenatally or postnatally, and whether enrollment is restricted to first-time parents. While some have argued that services to first-time mothers may be more effective, and have restricted enrollment to this subgroup (e.g., Olds, 2007) most evaluations of HFA have included both primiparous and multiparous mothers enrolled both prenatally and postnatally (DuMont et al., 2008, DuMont et al., 2010, LeCroy and Krysik, 2011) or postnatally only (Duggan, Fuddy, et al., 2004, Duggan, McFarlane, et al., 2004, Duggan et al., 2000). One exception is Healthy Families Massachusetts (HFMA), which enrolls only first-time mothers under age 20; early results for HFMA were promising in terms of reducing parenting stress (Easterbrooks et al., 2012) for this target population. DuMont et al. (2010), in their study of the HFNY program, had sufficient sample size to examine effects specifically for young (< 20), first-time, prenatally enrolled mothers, and found some evidence that they may, indeed, show more positive outcomes in terms of reported harsh/severe parenting, compared to similar controls. Rodriguez et al. (2010) similarly found improvements in positive parenting behavior for all mothers who received HFNY services, but found a reduction in negative/harsh parenting only for the young, first-time, prenatally enrolled mothers. Further, comparing prenatally vs. postnatally enrolled mothers, Lee et al. (2009) found a significant effect of HFNY participation on the likelihood of having low birth weight infants (the sample size was not sufficient to also compare first-time vs. subsequent births).
While results of HFA evaluations involving first time mothers have been promising, Huntington and Galano (2013), using a quasi-experimental longitudinal data, directly compared outcomes within the Healthy Families Virginia program for first-time vs. other mothers, and found no evidence of differential program effects. This study did not explore whether there were outcome differences for prenatally vs. postnatally enrolled mothers. Green et al. (2014), compared outcomes within the HFO sample of first-time parents, and found that mothers enrolled prenatally (compared to postnatally) were more likely to report breastfeeding at the child's 6-month birth date and had somewhat lower rates of premature birth, but did not find differences in parenting-related stress. Thus, while there is some evidence that Healthy Families may have particular benefits for young, prenatally enrolled first-time mothers, the findings are mixed at best and lack a clear pattern of either testing for differences within studies or clearly specifying differences in target populations that may help to better synthesize results across studies. The Healthy Families Oregon study will compare outcomes for first-time mothers who are enrolled prenatally vs. postnatally as well as for teenage, prenatally enrolled mothers vs. older, postnatally enrolled mothers (because all mothers are first time, differential effectiveness for primiparous vs. multiparous mothers cannot be assessed).
Azzi-Lessing (2013) notes that the findings of home visiting programs may be substantially impacted by cultural and community norms, citing differences in the racial/ethnic populations served as well as the communities in which studies have been conducted. Several studies of Early Head Start services have compared differences for White, Hispanic, and African American families (Love et al., 2002, Peterson et al., 2013, Raikes et al., 2013), finding consistent evidence that African American families may benefit most, at least in selected domains. However, few HFA studies have directly examined differential impacts for various racial/ethnic groups; nor have most studies addressed or discussed the substantial culturally differences that may characterize program communities. For example, the Hawaii HFA study involved almost two-thirds Native Hawaiian and Asian/Pacific Islander families (Duggan et al., 2000), while the Alaska study was characterized by a high proportion of Native Alaskan mothers (Duggan et al., 2007). Both HFNY and HFMA enrolled a large proportion of minority participants; however, subgroup effects for race/ethnicity were not reported. Instead, outcome analyses controlled for race, a common statistical approach but one that might serve to mask positive outcomes that occur only within a particular subgroup. The current study will take advantage of the relatively large proportion of Hispanic mothers served by Healthy Families Oregon to systematically compare outcomes for Hispanic vs. White/Caucasian (non-Hispanic) mothers.
While there are numerous family, parental, and social risk factors that may influence the effectiveness of home visiting services, several have received particular attention in the research to date and will be examined in the current research: (1) maternal depression (Ammerman et al., 2010, Duggan et al., 2009, Duggan, Fuddy, et al., 2004, Duggan, McFarlane, et al., 2004, Easterbrooks et al., 2013, Peterson et al., 2013); (2) teen parent status (DuMont et al., 2008; Olds et al., 2002, Olds et al., 2004); and (3) overall level of family risk (Duggan, Fuddy, et al., 2004, Duggan, McFarlane, et al., 2004, Duggan et al., 2007, Olds, 2002, Peterson et al., 2013).
Several studies have examined whether maternal depression moderates the impact of home visiting effectiveness. Generally, if differences have been found, mothers higher in depressive symptomatology appear somewhat less likely to benefit from home visiting (Ammerman et al., 2010). For example, Easterbrooks et al. (2013) found that non-depressed home visited mothers had fewer substantiated maltreatment reports than did non-depressed mothers in the control group; no differences were found for depressed mothers. Mitchell-Herzfeld, Izzo, Greene, Lee, and Lowenfels (2005) also found that at children's 1-year birthday, non-depressed mothers served by Healthy Families New York reported less punitive parenting compared to non-depressed controls. Duggan et al. (2009) found a complex relationship of depression to outcomes, wherein the impact of depression on program outcomes was moderated by mothers' attachment style. In their analysis of the Healthy Families Alaska sample, depressed mothers who also had insecure/avoidant attachment styles were less likely, compared to depressed mothers with healthier attachment styles, to show program benefits. Thus, several studies have found somewhat greater program impacts on parenting-related outcomes for non-depressed mothers. This is in contrast, however, to studies of the Nurse–Family Partnership program that suggest that mothers with ‘low psychological resources’ (defined, in part, by the presence of mental health symptoms) may benefit more from home visitation (Olds, 2002). The current study will assess differential outcomes for mothers with and without depressive symptomatology.
Effectiveness of home visiting services for teen parents vs. older mothers has also been examined in several studies, with mixed results (Kahn & Moore, 2010). Some programs (e.g., Healthy Families Massachusetts) have restricted program enrollment to teenage mothers; thus, results of these studies are generalizable only to this target population. As noted previously, research conducted in programs with only prenatally enrolled mothers (e.g., Easterbrooks et al., 2012, Easterbrooks et al., 2013; Olds et al., 2002, 2010) have not been able to separate the effects of teenage parent status and prenatal enrollment. Duggan and her colleagues also directly examined the potential effect of maternal age on outcomes (Duggan, Fuddy, et al., 2004, Duggan, McFarlane, et al., 2004, Duggan et al., 2007), and failed to find differences; however, as noted previously these studies were of non-accredited programs with documented implementation challenges. Kahn and Moore (2010) specifically call for additional research that can address whether home visiting programs are most effective when they target adolescent mothers, which will be directly addressed in the current study.
Finally, the current study will examine whether families with different levels of overall (cumulative) risk may benefit differentially from home visitation. This issue has been identified as critical for helping guide decisions about how to prioritize program enrollment and target limited program resources; however, research studies as well as programs themselves have varied widely in terms of how levels of risk have been operationalized (Azzi-Lessing, 2011). While some have argued that higher risk families may benefit more (Howard and Brooks-Gunn, 2009, Olds, 2002) other studies have found that those families with more moderate risk levels may be most appropriate for voluntary prevention services such as home visiting (Peterson et al., 2013). Clearly, more research to understand home visiting effectiveness for families with different levels of risk is needed.
The current study uses a randomized, intent-to-treat design to evaluate short-term outcomes of the Healthy Families Oregon (HFO) program. HFO is a comprehensive home visiting program serving first-time parents who have two or more identified risk factors. Families are screened for risk and enrolled either prenatally or within 90 days of the infant's birth, and are provided with up to 3 years of home visiting services by a trained home visitor. Visits are weekly for the first 6 months, and then may be reduced based on family progress and needs. Oregon was fully accredited as a multi-site state HFA system in June 2007, and in 2012 was successfully re-accredited following external review of the quality implementation of services.
As shown in Fig. 1, the program logic model for HFO centers on three sets of program activities designed to impact the long term outcomes of reduced child maltreatment and improved school readiness. First, parenting education, coaching, and modeling are provided to increase parents' knowledge of child development to improve parenting knowledge and skills. Programs use the Parents as Teachers and/or the Growing Great Kids curriculum for the parent education component. Second, home visitors identify individualized family issues that may interfere with child development and parenting, such as depression, substance abuse, and parental unemployment. Third, home visitors support healthy child development by promoting breastfeeding, conducting regular developmental screening and by connecting families to preventative health services. Together, these services are also designed to reduce parenting stress and to reduce levels of maternal depression, both identified risk factors for maltreatment (Abidin, 1990, Cummings and Davies, 1994).
The current study will address the following research questions based on the program logic model and on the existing research identifying key subgroups that may benefit differentially from home visiting:
- (1)
What short-term program effects can be detected at children's 1-year birthday? In particular, compared to control families: (a) Do parents in the HFO group report more positive parenting behaviors and skills compared to families in the control group? (b) Do parents in the HFO group report lower parenting stress, less depressive symptomatology, and more positive family functioning compared to families in the control group? and (c) Do children in the HFO treatment group experience more supports for healthy development, specifically increased breastfeeding and increased rates of developmental screening?
- (2)
Are there outcome differences for key subgroups of families? In particular, do outcomes differ for: (a) prenatally vs. postnatally enrolled mothers; (b) Hispanic vs. White/Caucasian mothers; (c) teenage vs. older mothers; (d) mothers with depressive symptomatology vs. non-depressed mothers; and (e) families with more vs. fewer total risk factors.
Section snippets
Study sites
Of the 35 HFO programs in Oregon, seven (7) met our criteria for inclusion in this study, specifically that (1) the program met state-established performance standards for the quality of program implementation during the last two fiscal years; and (2) estimates indicated that the program would have a minimum of 25 unserved eligible applicants per year (range was 25–272). Of the seven programs meeting these criteria, four are medium-sized programs (300–1000 first births per county) and three
Sample characteristics
The final telephone survey sample included 803 first-time mothers. Table 1 shows the demographic and risk characteristics for the interview study subsample, as well as for the full sample of randomized study participants. Study participants had an average of three out of 10 risk factors. Slightly over half were White/Caucasian; about one-fourth were Hispanic/Latina; the remainder were African American, American Indian/Alaska Native, or Asian/Pacific Islander. The average age of participants was
Discussion
Results from this study represent a very early look at the impacts of the Healthy Families Oregon program. As such, some promising findings emerged that are consistent with the program's emphasis on supporting positive parent–child relationships and early precursors of school readiness in children, as well as providing general support that may reduce parent stress. These HFO mothers were reading to their infants more frequently than were control parents, with 62.4% of HFO mothers reported
Acknowledgments
The authors would like to thank the participating Healthy Families Oregon program staff and families, without whom this research would not have been possible, especially Christi Peeples and Lisa Sutter, the statewide program managers. We would also like to acknowledge the work of other members of our research team, namely Adam Talkington, Jason Wheeler, Maria Rhodes, Andrea Doyle Hugmeyer, and Carrie Williamson.
This research is supported by grant # 90CA1782 from the Children's Bureau, U.S.
References (66)
- et al.
Maternal depression in home visitation: A systematic review
Aggression and Violent Behavior
(2010) Home visitation programs: Critical issues and future directions
Early Childhood Research Quarterly
(2011)- et al.
Impact of a statewide home visitation program on parenting and on child health and development
Child Abuse and Neglect
(2007) - et al.
Measuring the potential for child maltreatment: The reliability and validity of the Adult Adolescent Parenting Inventory — 2
Child Abuse & Neglect
(2006) - et al.
The development and validation of the protective factors survey: A self-report measure of protective factors against child maltreatment
Child Abuse & Neglect
(2010) - et al.
Impact of a statewide home visiting program to prevent child abuse
Child Abuse & Neglect
(2007) - et al.
Randomized trial of a statewide home visiting program to prevent child abuse: Impact in reducing parental risk factors
Child Abuse and Neglect
(2004) - et al.
Randomized trial of a statewide home visiting program: Impact in preventing child abuse and neglect
Child Abuse & Neglect
(2004) - et al.
Healthy families New York (HFNY) randomized trial: Effects on early child abuse and neglect
Child Abuse & Neglect
(2008) - et al.
Randomized trial of the Healthy Families Arizona home visiting program
Children and Youth Services Review
(2011)