Comparison of parent report and direct assessment of child skills in toddlers

Highlights

• Parent report of child receptive and expressive language and fine motor skills is generally consistent with direct assessment.

• Child diagnosis does not influence consistency between parent report and direct testing.

• When disagreement between sources exists, parents are more likely to report a skill as present than seen on direct testing.

Abstract

Background

There are unique challenges associated with measuring development in early childhood. Two primary sources of information are used: parent report and direct assessment. Each approach has strengths and weaknesses, particularly when used to identify and diagnose developmental delays. The present study aimed to evaluate consistency between parent report and direct assessment of child skills in toddlers with and without Autism Spectrum Disorder (ASD) across receptive language, expressive language, and fine motor domains.

Method

109 children were evaluated at an average age of two years; data on child skills were collected via parent report and direct assessment. Children were classified into three groups (i.e., ASD, Other Developmental Disorder, or Typical Development) based on DSM-IV-TR diagnosis. Mixed design ANOVAs, with data source as a within subjects factor and diagnostic group as a between subjects factor, were used to assess agreement. Chi square tests of agreement were then used to examine correspondence at the item level.

Results

Results suggested that parent report of language and fine motor skills did not significantly differ from direct assessment, and this finding held across diagnostic groups. Item level analyses revealed that, in most cases of significant disagreement, parents reported a skill as present, but it was not seen on direct testing.

Conclusions

Results indicate that parents are generally reliable reporters of child language and fine motor abilities in toddlerhood, even when their children have developmental disorders such as ASD. However, the fullest picture may be obtained by using both parent report and direct assessment.

Introduction

Early detection of developmental delays, including autism spectrum disorder (ASD), has been shown to facilitate earlier intervention and better outcomes (Orinstein et al., 2014, Rogers and Vismara, 2008). These findings have precipitated greater focus on improving routine developmental screening and evaluation rates for at risk young children and the general population. There are, however, unique challenges associated with obtaining accurate developmental data in early childhood, as toddlers tend to respond differently across contexts (Sachse & Von Suchodoletz, 2008). Currently, two primary methods are used to evaluate child development: parent report and direct assessment (Luyster, Kadlec, Carter, & Tager-Flusberg, 2008; Nordahl-Hansen, Kaale, & Ulvund, 2014). There is limited consensus regarding which method offers the best picture of child ability, as each approach has strengths and limitations.

Parents are an important source of information regarding child skills deficits and atypical behaviors, as they are uniquely positioned to observe and interact with children across various situations (Sachse & Von Suchodoletz, 2008). Thus, they may provide data regarding child development that could otherwise not be measured in a clinical setting. Parent report, too, is not subject to issues with child motivation and cooperation that frequently occur in testing situations (Nordahl-Hansen et al., 2014). Furthermore, parent report measures are an increasingly attractive option for detecting developmental delays, as they are quick, easy to use, and cost effective relative to formal evaluation (Nordahl-Hansen et al., 2014, Sachse and Von Suchodoletz, 2008). Based on these strengths, the pediatric healthcare system is moving toward greater involvement of parents in the process of identifying early developmental delays (Feldman et al., 2005, Nordahl-Hansen et al., 2013). Specifically, routine developmental screening via parent report is increasingly being used to identify at risk children, in accordance with American Academy of Pediatrics recommendations (Emerson, Morrell, & Neece, 2016; Johnson & Myers, 2007).

However, concerns have been raised about the accuracy of parent report (Ozonoff et al., 2011, Tomasello and Mervis, 1994). Although they are often keen observers of their child’s early development, parents generally lack expertise in evaluating developmental milestones, sometimes making it difficult to report reliably (Nordahl-Hansen et al., 2014). Alternatively, parents may attend more to challenging or unusual behaviors, thus introducing bias into their reporting (Zapolski & Smith, 2013). In addition, parents may overestimate child abilities because of a reluctance to acknowledge that their child has a delay (Ozonoff et al., 2011). As parent report necessarily reflects a parent’s perception of child functioning, it is considered subjective rather than objective (Sachse & Von Suchodoletz, 2008).

In contrast, standardized tests administered by a trained tester are, by definition, objective. Direct testing should thus offer an unbiased picture of child development, as each child is evaluated in a highly similar (i.e., standard) manner, and data is based on observations made by a professional with expertise and experience in assessing early development (Nordahl-Hansen et al., 2014, Sachse and Von Suchodoletz, 2008). However, in an unfamiliar clinical setting, children tend to behave differently than they may in familiar settings. Issues of behavioral noncompliance and poor attention and motivation commonly seen in young children may influence test results, potentially limiting validity (Nordahl-Hansen et al., 2014). Particularly when evaluating language skills, children may produce fewer utterances when outside of their everyday communicative activities, thus limiting the validity of formal assessment. In addition, formal developmental and diagnostic testing can be time intensive and cost ineffective (Sachse & Von Suchodoletz, 2008). Furthermore, some critics suggest that standardized tests are inappropriate for children with ASD and global delays, as they often measure skills that are too advanced for the child (Luyster et al., 2008). Thus, it is unclear whether direct testing, long regarded as the gold standard, best estimates child abilities in very young children, particularly those with developmental deficits.

To address concerns with both parent report and direct assessment as outlined above, efforts to determine the ideal approach to assessing child development have focused on evaluating agreement between these two primary sources of information. However, the existing literature relies on a variety of methodological approaches and definitions of ‘parent report’ and ‘direct assessment.’ Reliability of parent recall of early developmental milestones and health events, as compared to medical records, indicates that parents are generally good reporters of gross motor milestones (e.g., age at first steps) and medical outcomes (e.g., birth weight, illnesses during early infancy), but that their ability to recall early language milestones is lower (Majnemer & Rosenblatt, 1994; Pless & Pless, 1995). Most often, though, parent report has been used to refer to parent completed screening tools or developmental checklists, which have then been compared to standardized developmental evaluation measures (Bodnarchuk and Eaton, 2004, Luyster et al., 2008, Nordahl-Hansen et al., 2014, Sachse and Von Suchodoletz, 2008, Voigt et al., 2007) or, in the case of ASD symptoms, home videos (Ozonoff et al., 2011). Findings suggest that parents are adequate reporters of child language ability (Nordahl-Hansen et al., 2014), although agreement between parent report and direct assessment is stronger for speech production than for comprehension (Luyster et al., 2008, Sachse and Von Suchodoletz, 2008, Tomasello and Mervis, 1994). This discrepancy may be due to the generally low reliability of measures of early receptive language (Sachse & Von Suchodoletz, 2008). Alternatively, for children with ASD, discrepancies between parent report and direct testing may be an artifact of their difficulty generalizing language across contexts (Nordahl-Hansen et al., 2014). Parents demonstrate good concordance with trained assessors when evaluating gross motor milestones (Bodnarchuk & Eaton, 2004), yet limited research is available on agreement for fine motor skills in early childhood.

The present study aims to evaluate consistency between two sources of information, parent report and direct assessment, when measuring child development in three domains: receptive language, expressive language, and fine motor skills. Here, we define ‘parent report’ as information obtained from a parent during a structured interview, the Vineland Adaptive Behavior Scales, Second Edition (Vineland), which we then compare to results of direct testing using the Mullen Scales of Early Learning (Mullen). Although the basis of information about a child’s development in a parent report is different from that in direct assessment, in that the Vineland aims to assess what skills a child uses in his or her daily life and the Mullen aims to measure a child’s competence, both approaches yield similar information at a content level. A similar method of comparing parent report and direct testing of child language skills using the Vineland and Mullen was employed by Luyster et al. (2008) to study development in toddlers with ASD. We expand on this approach by investigating agreement between parent report and direct testing of early developmental outcomes in children with ASD, other developmental disorders, and typical development. The aims of the current study are threefold:

• We examine consistency between Vineland and Mullen scores in the domains of receptive language, expressive language, and fine motor skills. Based on prior research showing good agreement between parent report and direct testing of language production, but somewhat weaker agreement when assessing language comprehension (Luyster et al., 2008, Nordahl-Hansen et al., 2014, Sachse and Von Suchodoletz, 2008, Tomasello and Mervis, 1994), we expected to find a similar pattern in our data. We also hypothesized that parent report of fine motor skills would be generally consistent with standardized assessment, as similar findings exist for gross motor functioning (Bodnarchuk & Eaton, 2004).

• We assess the impact of child diagnosis (i.e., ASD, other developmental disorders, or typical development) on consistency between Vineland and Mullen scores. As children with ASD often have difficulty generalizing skills and behavior across contexts (Nordahl-Hansen et al., 2014), we hypothesized that agreement between parent report and direct testing would be weaker for children with ASD as compared to those toddlers with other developmental delays or typical development.

• We explore agreement between parent report and direct assessment at the individual item level in order to evaluate parent reporting of particular developmental skills. Although we consider these analyses exploratory, we expected to see the greatest discrepancies for items measuring receptive language skills, based on prior research findings (see Aim 1).