Investigating the Validity of the Australian Early Development Census

The period of early childhood has been identified as foundational for future health, socioeconomic and emotional success [1, 2]. Developmental level at school-entry particularly has been found to be predictive of future capacity for learning, and socioeconomic attainment [3]. A range of indicators have been identified for early childhood which allow for assessment of current and future levels of development. The Australian Early Development Census (AEDC) collects data using the Australian version of the Early Development Instrument (AvEDI).¹ Australia is the only high income country to collect developmental data for children nationally at the time of entry to full-time school [4]. It is a revision of the Canadian Early Development Instrument (EDI), a teacher-completed measure of school readiness assessed through five domains: Physical health and wellbeing, Social competence, Emotional maturity, Language and cognitive development, and Communication skills and general knowledge [5, 6]. The AEDC enables a population level assessment of the key aspects of child development within a single instrument and yields data with the potential for benchmarking on universal child development approaches [7, 8]. Testing of the AvEDI using the Rasch model demonstrated high reliability and validity [9, 10], validity has been demonstrated in both Canada and Australia [11], and it has been shown to be predictive of later educational outcomes [12]. Validity of the AvEDI has been further explored against independently reported datasets. A 2007 study by Brinkman et al. [9] examined AvEDI validity using a nested study comparing AvEDI results from 642 children against contemporaneous data from the Longitudinal Study of Australian Children (LSAC), a nationally representative cross-sectional sample of Australian children [13]. The study reviewed the consistency of correlations of early child development measures between the AvEDI subconstructs and independently reported multimethod measures collected contemporaneously by the LSAC. Correlations between AvEDI domains and subconstructs were found to be moderate to high for LSAC measures based on teacher-report, but lower when compared to parent-completed or interview-directly assessed measures. Several factors were proposed to account for the lack of stronger correlations between the datasets. These include variation in the ages of the children and their relative exposure to formal primary education at the time of data collection within the two datasets. AvEDI discriminant validity was suggested by the consistency of lower correlations between constructs which were conceptually different.

In the current study, we proposed further validity testing of the AvEDI against a larger and more age appropriate² LSAC sample than has previously been available. We hypothesise that this sample will provide increased strength or correlations for validity measures. We use linked AEDC/LSAC data to provide empirical evidence on the validity of the AEDC.

The LSAC is a biennial nationally representative clustered cross-sequential sample of two cohorts of Australian children: the first group from all children born between March 2003 and February 2004 (Birth or B-Cohort), and the second from all children born between March 1999 and February 2000 (Kindergarten or K-Cohort) [14]. The initial sample of B-Cohort consisted of 5107 infants aged 0–1 in 2004, and K-Cohort 4983 children aged 4–5 years in 2004. The LSAC was initiated in 2004 and the latest available release was surveyed in 2020 when B (K) cohort children were 16–17 (20–21) years old. LSAC explored 3 domains, Health and physical development, Social and emotional functioning, and Learning and cognitive ability [15]. Health and physical development explored general health and medical conditions, nutrition and motor skills. Social and emotional functioning measures mental health, relationships, and social and emotional well-being. Learning and cognitive development looks at academic skills and outcomes, numeracy and literacy. The survey collected data on early development, health, education and living situation, from a variety of sources: teacher/caregiver completed questionnaires, parent self-completed questionnaires, face-to-face parent interviews, interviewer observation, and direct child observation. Assessments were based on well-established tools including Parent’s Evaluation of Developmental Status (PEDS), Pediatric Quality of life Inventory (PedsQL) [16], Short Temperament Scale for Children (STSC), Strengths and Difficulties Questionnaire (SDQ) [17], Peabody Picture Vocabulary Test (PPVT), Who am I? (WAI), and additional measures such as teacher rating of literacy and numeracy skills and rating of peer relationship quality [9]. Nationwide, 1976 postcodes were stratified by state, territory and capital city statistical division to ensure the sample was representative of the target population. Postcodes with less than 20 children were excluded from the survey. The current study used data from the B-Cohort collected during Wave 3, which overall consisted of 4386 4–5 year old children (88.2% response rate) transitioning to school [18]. Wave 3 data was collected from March 2008 to February 2009.

The Australian Early Development Census has collected data every 3 years since 2009, and measures developmental outcomes for children commencing their first year of full-time school through five key developmental domains [19].The Physical health and well-being domain measures children’s readiness for the school day, fine and gross motor skills, and physical independence. Social competence is measured overall, and also includes responsibility and respect, plus readiness to learn. The Emotional maturity domain measures prosocial behaviour and willingness to help, plus behaviours concerning anxiety, fear, aggression, hyperactivity, and inattention. The domain of Language and cognitive skills measures school-based basic and advanced literacy, basic numeracy, and interest in literacy, numeracy and memory tasks. The final domain, Communication skills and general knowledge, assesses these features in terms of broad developmental skills. The domains are assessed through 96 core questions, assessed using either a yes/no or Likert scale response. Children receive a score between 0 and 10 for each domain, with higher scores indicating a higher level of development. Context for the child is provided through collection of details such as the child’s care arrangements, participation in early education programs prior to school, and demographic and geographical information. The instrument is completed by the primary classroom teacher based on at least one month’s knowledge of the child. AvEDI is not an individual diagnostic instrument but a tool to assess development of children according to a range of developmental domains. Individual scores are de-identified and reported as data aggregated by group (class, school, neighbourhood, province/state, country).

The current study is based on a subset of the 2009 AEDC collection cycle, the latter of which involved 261,147 children, representing 97.5% of the eligible child population. AEDC data was only available for that subset of B-cohort children (n = 2459) who took the AEDC and then consented to have their AEDC data linked to LSAC. This subset of the LSAC B-Cohort were 5–6 years of age at the time of the AEDC. AEDC data collection took place between 1st May and 31st July 2009. This linked LSAC-AEDC dataset forms the basis of our analysis. We used continuous scores for AEDC as per Brinkman et al. [9]. As AEDC is designed to capture “school readiness”, we use developmental outcomes recorded at LSAC wave 3 (or year 2008) when the B cohort children were 4–5 years of age, and before they started their formal school years.

As the constructs that the AEDC measures that relate to early child development and school readiness are latent, that is, not directly observable, their validity as a general measure of child development can only be assessed indirectly. As such, assessment of validity is based on establishing consistent correlations between the overall, domain and subdomain scores, and other measures of early child development.

Our key research question was whether a larger LSAC sample than was previously available allows for an improved calibration of the AEDC. Following Brinkman [9] we analysed the strength of association between each AEDC domain and the comparable LSAC measure using Spearman correlation coefficients. Correlational analyses were used to analyse nomological network of associations between the AEDC and the LSAC. Also, following Brinkman [9], we adopt the popular thresholds suggested by Hopkins [20] for pairwise correlation strength in which thresholds of 0.90, 0.70, 0.50, 0.30 and 0.0 are used to indicate a “very large”, “large”, “moderate”, “small” and “little or none” correlation, respectively.

The Participant Flow Diagram (Fig. 1) shows the linkage protocol and associated participant numbers for the linked LSAC/AEDC sample used in the current study, which resulted in a final matched sample of 2459 children. This follows the linkage process outlined in detail in Bandara et al. [21] which uses the child’s first name, surname, gender, date of birth, school name and school address as identifying variables. Some children were classified as being out of scope, or ineligible to receive a consent form. Out of scope exclusions included home schooling, special care needs/intellectual disability, or distance education. Using the matched sample, we make a further restriction that matched children have valid scores for all subjects. This restriction led to a final sample of 2216 children.

The comparative demographic detail of included and excluded children in provided in Appendix Table 8. As the number of linked children excluded from our study due to incomplete AEDC scores was small (243), they were included with unlinked children in this table. Table 8 suggests that included children were statistically significant more likely to come from families with higher maternal education (mother with a postgraduate degree), be living with both parents and/or living in own home, or live in an area where a higher percentage of the community are working. This suggests that linked children were of overall higher socioeconomic status than unlinked children. With excluded children being more likely to come from a non-English-speaking background and have younger mothers, the patterns were similar to children from non-responding families in the subsequent rounds of LSAC [18].

The demographics of the sample are provided in Appendix 2. The average age of the AEDC sample (5.17 years) was slightly older than the LSAC sample (4.75). The sample was evenly divided by gender, with 52% males. The majority of children did not identify as being Aboriginal or Torres Strait Islander (97%). Some 10% of children had mothers who were from a non-English speaking background (NESB) while 8% had mothers born overseas in an English-speaking background (ESB) country. Approximately three-quarters of children had been breastfed at early childhood (74%) and lived in non-rental accommodation (77%). The majority of children (89%) lived with both parents and had on average one-two siblings. Approximately two thirds of children had mothers with either a diploma or certificate (39%) or postgraduate degree (38%).

Following Brinkman et al. [9], we employ a range of child development outcomes as recorded in LSAC which map to the five AEDC domains. Moreover, to make the scaling consistent across comparable measures between LSAC and AEDC, where necessary, we rescale some LSAC measures in such a way that a higher score indicates a more desirable outcome. Table 1 shows the number of children with valid data, and the mean, standard error, median, and minimum and maximum values for each variable. The difference between N for each measure and the total number of participants indicates any missing data for each LSAC outcome for this matched sample.

Table 2 shows the correlation between the AvEDI instruments. All correlations were statistically significant at 0.1%.

In 2007, Brinkman et al. [9] investigated the construct and concurrent validity of the Australian Early Development Index, an earlier version of AEDC. In Brinkman et al.’s study, the AvEDI data from 641 to 4–5 year old children were analysed in relation to other measures of early child development from the first wave of LSAC data for K-cohort in 2004, assessed by parent interview, direct assessment and teacher rating. Moderate to strong correlations were apparent between the AvEDI domains and subconstructs and the comparable LSAC measures based on teacher report. Less correlation was apparent when AvEDI domains were compared with parent reported LSAC items. Through the linkage of AEDC data to B-Cohort children from Wave 3, the current study afforded an opportunity to further assess validity based on a larger sample than was available in Brinkman et al. original study.

Overall, there was poor correlation between LSAC and AEDC scores in the current paper. The highest correlation, although only moderately strong, was between LSAC interviewer administered PPVT and WAI and AEDC Communication skills and general knowledge scales. The strongest association in Brinkman et al.’s analysis was similarly found in the areas of literacy and communication, although between LSAC teacher rated reading, writing and numeracy and AvEDI Language and cognition [9]. In the current study, moderate correlations were also found between the LSAC Who Am I? (WAI) interviewer administered questionnaire and AEDC Basic literacy score, Advanced literacy score and Basic numeracy score, again reflecting the strength of correlation in the literacy and numeracy domains. The WAI is a narrow construct measuring fine motor skills of drawing and copying [22], with suggestions that it assesses concepts of pre-literacy [23]. That the correlation was not stronger may be due to the simpler literacy concepts measured in the WAI, with copying and recognition of symbols, compared with the more sophisticated literacy and numeracy concepts tested in the AEDC, such as identify letters and sounds of the alphabet, awareness of rhyming words and counting to 20. Weaker correlations were observed across many measures such as LSAC Prosocial and AEDC Social competence scales, and LSAC Child separation behaviour and AEDC Overall social competence.

Little or no correlation was observed across various domains, such as the LSAC parent-rated Emotional maturity measures and the AEDC Emotional maturity subdomain scores. For instance, there was no or little correlation between LSAC parent reported PedsQL Social functioning or STSC Sociability, and AEDC Social competence measures. Similarly, no correlation was found between language competencies such as the LSAC parent reported PEDS Expressive language concern, Receptive and Expressive language scales and the AEDC Language domain. Positive moderate correlations were noted between LSAC and AEDC measures of hyperactivity and LSAC Conflict/anger and AEDC Aggressive behaviour scales. These scales use very similar items. For example, LSAC asks the teacher if the child “Often has temper tantrums or hot tempers” compared with the AEDC “Has temper tantrums”.

There was overall poor or no correlation between LSAC parent-report and AEDC measures. A similar relationship was noted both between parent- and teacher-reported scores within LSAC and in Brinkman et al.’s 2007 paper. Possible reasons for this will be discussed below.

This article continues evaluation of the construct validity of the Australian Early Development Census (AEDC) through comparison with linked data from a sample of 2216 4–5 year old children collected as part of the Longitudinal Study of Australian Children (LSAC). LSAC data on early child development was independently derived from multimethod measures derived approximately contemporaneous with the AEDC data collection. This builds on the construct validity assessments of Brinkman et al. [9] which was based on a smaller sample of 642 linked the Australian Early Development Instrument (AvEDI) and LSAC children in which moderate to large correlations were apparent between teacher-rated AvEDI domains and subconstructs and LSAC measures, with lower levels apparent for parent reported LSAC measures. In the current study, the data showed moderate to low correlations between the domains and subdomains from the AEDC and teacher reported LSAC data. We considered that factors such as the gap in the timing when LSAC and AEDC outcomes are measured, comparing data collected from parent versus teacher report, the measures utilised differing in the characteristics of child development being captured, and different levels of exposure to formal schooling at the time of testing were all likely to have impacted on the lower levels of correlation between the two measures. Overall, this paper finds a lack of validity between the scales across LSAC and AEDC. Given the previously recognised validity of the AEDC, further research is required to explore the apparent differences between the AEDC and LSAC data sets.