Rationale

The presence and quality of mandated reporter education is inconsistent across professional groups and jurisdictions. Numerous studies have found that members of various professions feel they have not had sufficient training to discharge their duty [16] [19] [20] [22] [24]. Research has identified low levels of knowledge in some professions about the reporting duty and attitudes precluding sound reporting [25] [26].

To date, educational interventions differ widely in delivery modality (paper materials only, in-person, or online), target audience (single or multiple professional group; pre-service or in-service), duration, approach (didactic or interactive; single discipline or multidisciplinary), content (one or multiple forms of maltreatment); and outcome measures [27]. Most educational programs focus only on cognitive skills, such as knowledge of indicators of child abuse, or of the content of the reporting duty, with less attention paid to the affective dimension of attitudinal development [27] [28]. Many have focused only on sexual abuse [29] [30].

Web-based training of professionals about child maltreatment has been adopted since the mid-2000s [31]. Delivery online offers multiple advantages: it is practicable; is cost-effective; enables access to large numbers of participants over broad geographic areas; enables flexible access in time and place; allows use of multi-media and multiple learning and delivery modes; and ensures program fidelity. Online education has been conducted and evaluated for knowledge gains with education and counselling students [31], nurses [32], and mixed audiences [33].

Educational programs on child abuse are rarely rigorously tested for effects, whether delivered online or traditionally. A systematic review of studies from 1994–2005 found only 15 programs met inclusion criteria against outcome measures of one or more of learning achievement, attitudinal change and clinical behaviour [34]. Only three used a control group, and these relied only on self-reported change in knowledge and confidence rather than objective measures [34]. Controlled trials outside 1994–2005 are rare [28] [32] [33]. Most programs have been directed towards professions such as medicine, nursing, dentistry, and school-based education [34]. Educational efforts for CCPs are rare, with isolated exceptions [33] [35]. Therefore, there is a pressing need to conduct true experimental studies of CCPs to identify baseline levels of knowledge and attitudes, and to explore changes produced by educational interventions.

Objectives

CCPs are mandated reporters of child abuse and neglect under Pennsylvania law, and are in a unique position to identify and respond to these children. The evidence base is underdeveloped regarding CCPs’ knowledge of the law and attitudes towards the reporting duty, and there is also a lack of evidence about how to improve their knowledge and attitudes. This study was primarily designed to explore:

1. the baseline knowledge of CCPs regarding the legal reporting duty;
2. the baseline attitudes of CCPs towards the legal reporting duty;
3. whether knowledge of the duty is increased by a customized online educational program, and whether this is maintained over time;
4. whether attitudes towards the duty are enhanced by a customized online educational program, and whether this is maintained over time.

The overall aim of the experiment was to explore whether a multidisciplinary online educational program could improve these professionals’ knowledge of and attitudes towards reporting child maltreatment. This aim was achieved and the results demonstrated significant improvements in both domains.

Design

In a sample of 762 childcare providers (CCPs), we randomized participants in a single-blind controlled trial. Using a test-retest design, we measured baseline knowledge of the legal reporting duty and attitudes towards reporting, and compared increase in knowledge and change to attitudes for those assigned to the intervention group (n = 388 completed the iLookOut) with the control group (n = 374). The study was approved by the Institutional Review Board of Penn State University on May 5, 2014 (#0376) and registered at the US National Institutes of Health #NCT02225301 https://clinicaltrials.gov/ct2/show/NCT02225301?term=NCT02225301&rank=1) (S1 File). The study was informed by the CONSORT approach [36] [37] and reports results accordingly (S1 Fig). The authors confirm that all ongoing and related trials for this intervention are registered. The short delay in registration of the trial was due to initial indications that since no clinical patients or health outcomes were being evaluated, registration may not be required; IRB advice was then provided to register the trial in any event and this advice was followed.

Participants

Participants were CCPs from the state of Pennsylvania, which is the sixth most populous U.S. state (12.8 million), predominantly Caucasian (82%) [38], has a sizeable rural population (27%) and has a high median per capita income ($US46,000). Eligibility criteria included being at least 18 years of age, English-speaking, and working as paid or volunteer staff at a licensed childcare facility (commercial, non-commercial, home-based, or other) taking care of children under 5 years of age. Recruitment letters were sent to directors of all licensed childcare facilities in Pennsylvania via the mailing lists of Penn State Better Kid Care, which provides online professional development to 1,900 childcare facilities, the Pennsylvania Child Care Association, and Pennsylvania’s Office of Child Development and Early Learning. Childcare facilities were chosen by sampling respondents to obtain a representative distribution of participants, based on region of the state, rurality, and type of facility. Directors of childcare facilities were then provided with web links enabling their staff to access iLookOut. Participants provided informed written consent at the outset of iLookOut, and indicated willingness to be recontacted several months later for follow-up. No financial compensation was offered, but completion of iLookOut conferred two hours of professional development credit.

CCP directors were contacted about the study on May 22, 2014 and the date range for participant recruitment was June 23, 2014–July 18, 2014. Participants therefore accessed and completed iLookOut during a single four week period in June–July 2014. The iLookOut intervention was available 24 hours a day and could be paused and resumed as desired, across multiple sessions, in individuals’ residences or workplaces. Program software was designed to capture data about the participant’s responses. All data were de-identified and collected by the host website. The date range for the four month follow-up was December 8, 2014–February 20, 2015.

Study intervention

The iLookOut for Child Abuse—Online Learning Module for Early Childcare Providers intervention was created in 2013–2014 by a multi-disciplinary team at the Center for the Protection of Children at the Penn State Children’s Hospital, involving experts in child abuse, instructional design, pediatrics, early childhood education, online learning, mandated reporter training, law, ethics, and child advocacy. Two co-investigators provided psychometric and statistical expertise. iLookOut was programmed using responsive web design to allow for mobile access, as well as features to accommodate individuals with sensory disabilities. iLookOut is hosted on a secure server by the Center for the Application of Information Technologies, which provides technological support for users, and stores all user data.

Content of iLookOut was informed by prior studies of professionals’ reporting [18] [20] [21] [22] [28] [31], and the professional expertise of the interdisciplinary team. The first four sections captured participant information about demographics; work information; knowledge about reporting child abuse and neglect, and attitudes towards reporting child abuse and neglect.

The fifth section of iLookOut comprised the online education program. This was designed to address standard cognitive aspects of mandated reporter training (e.g., definitions of abuse, signs of abuse, legal requirements for reporting), and to have a strong additional focus on affective elements thought to contribute to deeper understanding as well as effective practice. To do this, iLookOut employed an interactive, video-based storyline with films shot in point-of-view (i.e., the camera functioning as the learner’s eyes), with the learner taking the role of a teacher of 4 year olds at a childcare center. As key events unfold through interactions involving children, parents, and co-workers (all played by actors), the learner had to decide how to best respond. After some episodes, learners were posed questions, and based on their response were provided with didactic material to educate them about aspects of child abuse. Other videos were followed by questions where the learner had to choose how to respond to events. At different junctures in the story, learners could access resource files (e.g., Facts about Abuse, Red Flags for Abuse), and more information in text or video about the children. As in real life, the more information sought, the better informed would be the participant’s choices. The interactive storyline was designed to increase CCPs’ awareness about child abuse, and to help CCPs feel both empowered and responsible to contact CPS when there was a reasonable suspicion of child abuse. By immersing CCPs into real-life scenarios and requiring them to practice using their knowledge and skills, iLookOut was designed to help learners operationalize new information, and develop knowledge and attitudinal dispositions to help protect real children from harm.

Sample size and statistical analysis

It was anticipated that small gains in knowledge may be present even post-intervention, due to child abuse reporting duties having existed in Pennsylvania for many years, high profile cases in the area having drawn broad public attention to child maltreatment and its reporting [39], and general educational efforts in this sector about child abuse. Accordingly, to detect a moderate effect size of Cohen’s d = 0.25 using two-sided two-sample t-test with 80% statistical power at type I error = 0.05 level, and assuming 15% drop-off, it was calculated that 300 participants for each study arm were needed. By comparison, sample sizes for the few controlled trials involving mandated reporter training have been much smaller [32] [33]. Another study involved a control group of 94 and an experimental group of 37 [28]. Our sample (n = 762) experienced low non-completion and provided a strong basis for statistical analysis.

All variables were summarized before analyses using descriptive statistics, means, medians, and standard deviations for continuous variables and frequencies and percentages for categorical variables. Distributions of continuous variables were assessed using histograms, box plots, normal probability plots, and the Shapiro-Wilk test for normality. Demographic variables were compared by intervention groups using Chi-square tests, and exact tests were used as needed. Statistical significance was set at p < 0.05. Baseline knowledge scores were calculated using descriptive statistics for each knowledge item and were aggregated to produce a mean score for each item and a mean total knowledge score. Baseline attitudes were calculated using descriptive statistics for each attitude item. Attitude items a, b, e, i, j, l, and m, which were negatively-worded, were reverse-coded so that for all attitude items a higher number represented a more positive response.

For the knowledge scores (individual items and total knowledge score), and for the attitude items, the change from baseline (pre-test) to re-test was calculated in the control group, while the change from baseline (pre-test) to post-test was calculated in the experimental group. A paired t-test was used to make comparisons between pre and post responses within each group, while a two-sample t-test was then applied to compare the mean change between the control and experimental groups. For the individual knowledge items, a generalized estimating equations (GEE) model was used to compare the change in the proportion of correct answers from pre-test or re-test to post-test between the control and experimental groups. A Cohen’s d statistic for means and a Cohen’s h statistic for proportions was calculated to further quantify the effect size of the change from pre-test to post-test, using 0.2, 0.5, and 0.8 as cut-off values for small, medium, and large effect sizes for both d and h. To rule out any bias introduced by pre-test differences between groups in knowledge or attitude items, Chi-square tests or two-sample t-tests were used to make group comparisons at pre-test, and no significant differences were found. In the follow-up cohort after 4 months, paired t-tests measured change in knowledge and attitudes. All analyses were performed using SAS version 9.4, and a significance level of 0.05 was used for all comparisons.

Outcomes

Follow-up after 4 months post-intervention.

On re-contact for follow-up testing after four months, there were 201 participants (43.69%), all of whom had received the intervention and whose knowledge and attitude had been measured after it during the initial 4-week study. The follow-up of these participants demonstrated a sustained significant increase in mean knowledge from baseline of 13.54 to 15.16 (Table 4). However, the mean follow-up knowledge score for these 201 CCPs declined to 15.16 (95% CI = 14.87, 15.45) from its post-test peak of 16.86 (p < 0.001, 95% CI = 16.60, 17.12). Three knowledge items showed notable declines of > 15% (items 1g, 6, 8).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Post-intervention to 4 month follow-up–knowledge.

https://doi.org/10.1371/journal.pone.0177777.t004

For attitudes, several items remained stable from a very high post-intervention baseline (Table 5). There were some significant negative changes with five of 13 items changing with medium effect sizes over 0.5 and also decreasing by over 0.5 of a point on the Likert scale (items c, g, h, i, k). However, these declines were from a very high base so that even with the downward trend, the attitudes remained strong.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Post-intervention to 4 month follow-up–attitudes.

https://doi.org/10.1371/journal.pone.0177777.t005

Knowledge

This study found that iLookOut produced significant improvements in knowledge regarding all aspects of the reporting duty, ranging from the types of injuries required to be reported, protections for reporters, the consequences for failing to report, and the mechanism for reporting. Improved total knowledge was evident, and improvement on 15 out of 21 knowledge items was shown with either large or medium effect size, despite the relatively high baseline knowledge possessed by participants. However, despite these impressive improvements, some knowledge items remained less accurately answered, with participants demonstrating mistaken beliefs that they were required to report any kind of injury caused by a parent, any witnessing of domestic violence, and any bruising in a child aged under five, and a further persistent misapprehension about to whom they should report.

Attitudes

This study also found that iLookOut produced significant improvements in the attitude towards the reporting duty. The intervention group showed improvement on all 13 attitude items, with the most significant changes on three items regarding the role of reporting in promoting children’s long-term interests; the safety of reporters from legal liability if the report is unsubstantiated; and the duty to report applying even if one’s supervisor disagrees. Some of the positive changes in attitude were less expected, as iLookOut only indirectly addressed key issues underlying these attitudes. For example, after completing iLookOut participants were more likely to hold that child protection services (CPS) would respond effectively to reports, and that children and their families could receive helpful services. Because iLookOut did not present detailed data about the efficacy of CPS, this suggests that when CCPs better understand the nature and consequences of child abuse, as well as how to engage CPS, they are more likely to view the system as responsive to the needs of at-risk children and their families.

Four month follow-up: Sustained gains

The study found that initial positive gains in both knowledge and attitudes were sustained over a four month follow-up period. Knowledge regression occurred significantly on three items, although notably these were poorly answered initially (domestic violence, how soon to report, to whom to report). Regression in knowledge was expected given the normal transience of single dose interventions and the need for longitudinal reinforcement to sustain gains in learning, but the sustained knowledge gains overall indicate the efficacy of iLookOut in producing sustained cognitive gain. To address the identified regression, forthcoming versions of iLookOut will add further content to clarify areas of confusion, and leverage electronic communication and spaced content retrieval techniques to enhance cognitive mapping. Similarly, attitudes at four month follow-up demonstrated persistent overall gain, with attitudes towards some items being further strengthened, suggesting a deepened attitudinal response over time, fostered by the unusually deep engagement of the intervention with the affective dimension of this context. However, attitudes towards some items regressed, perhaps indicating a growing understanding of the complexity of some aspects of this context. For example, subsequent decreased desire to fulfil professional responsibility by reporting may be explicated by a growing understanding that some meritorious reports do not yield positive outcomes for families.

To address this, forthcoming versions of the intervention will explore such possibilities. However, other attitudinal regression may be due to contamination of attitudes by incorrect knowledge. For example, participants were less committed than they had previously been about whether they should be required by law to report child maltreatment. We would hypothesise this regression is linked with the identified knowledge gaps; this less positive attitude could be explained by an incorrect belief that the law requires all cases of exposure to domestic violence to be reported. To address this, future versions of iLookOut will add further content to clarify areas of confusion in knowledge, and create further connections between the development of knowledge and attitudes to create a coherent cognitive and affective framework.

Contributions to knowledge

To our knowledge, relative to other studies, the iLookOut intervention is the first randomized controlled trial with CCPs to explore changes in knowledge of the duty to report all kinds of child maltreatment, and attitudes towards the duty. Other types of study have shown increased knowledge, including one study which showed a significant difference between pretest and post-test scores (15.3 to 17.8 out of 20) [31]. However, there are few if any comparable studies with a large sample, control and intervention groups, carefully designed scales, application to multiple forms of maltreatment, and with a focus on attitudinal development. The systematic review of training and procedural interventions to improve child protection [34] identified only three studies using a control group and outcome measures of one or more of learning achievement, attitudinal change, and clinical behaviour [29] [40] [41]. Even these three studies were limited, either assessing knowledge only of sexual abuse [29] [41], or relying only on self-reported change in knowledge rather than objective measures [29] [40]. Furthermore, none of these studies were conducted with CCPs, which are a group of professionals who have rarely received educational efforts that have been rigorously designed and evaluated, with two exceptions [33] [35]. Since the systematic review [34], only two controlled trials have been conducted and each had a different focus to iLookOut. One was a small study of nurses’ clinical capacity to detect cases [32], and the other was devoted to exploring knowledge of child sexual abuse and attitudes towards it [33], with this program being further explored for effects on reporting of child sexual abuse [42].

Overall, the research implications are that a customized, multimethod educational intervention delivered online to a large group of CCPs can significantly increase knowledge and change attitudes about the duty to report suspected child maltreatment, with these increases able to be largely sustained over time. While the precise mechanism of change in developing knowledge and attitudes has not been isolated in this context [43], these findings can be plausibly explained as an exemplar of adult learning [44]. Child protection training for professionals is an educational intervention through which professionals develop cognitive capacity, building knowledge and skills, as well as attitudes. While previous studies have focused on the cognitive domain, the iLookOut intervention was designed to include measures to enhance empathy towards maltreated children as a means of fostering positive attitudes towards the duty to report suspected cases. Empathy is known to encourage prosocial altruistic behaviour, and can be fostered through education [45] [46]. Combined with greater cognitive understanding of the nature and function of child protection systems, this additional attention to the affective domain may have acted as a mechanism to foster more positive attitudes towards the duty to report. Further research into the empathic domain would be beneficial, especially to investigate areas known to be more resistant to empathic development and response [45]. As done to an extent in some other studies [40] [42], research to explore the effect of increased knowledge and attitudes on actual reporting practice would be beneficial.

Clinically and practically, the significance of these findings derives from the low rate of reports made by CCPs, and the need for tools to overcome barriers to preparing and motivating CCPs to identify and report child maltreatment. There is a need for robust education of CCPs, and other professionals, using proven methodologies. Studies of other professionals indicate reports of maltreatment are more likely when the reporter has higher knowledge of indicators of maltreatment [47], and higher knowledge of the reporting law and procedures [22] [25] [48]. In addition, as suggested elsewhere [49], this study confirmed the feasibility of administering an online, interactive educational intervention without financial compensation for a large cohort of professionals with heavy workloads and competing time demands.