Computerized social-emotional assessment measures for early childhood settings☆
Introduction
Ensuring that children are on positive school developmental trajectories at school entry is essential for later academic and social success. Previous research has highlighted social-emotional competence (SEC) as being especially important in establishing such success (Denham, Brown, & Domitrovich, 2010; Murray & Harrison, 2011; Romano, Babchishin, Pagani, & Kohen, 2010). In particular, the extant literature has identified educators as key promoters of Social Emotional Learning (SEL) across five important areas of competence: (a) self-management, the ability to regulate thoughts, emotions, and behaviors, (b) self-awareness, including the ability to recognize one’s emotions, (c) social awareness of culture, beliefs, and feelings of others, (d) relationship skills, the abilities to effectively communicate with and work well with peers, and build meaningful relationships, and (e) responsible decision-making, including the abilities to make plans for the future, and solving social problems (Oberle, Domitrovich, Meyers, & Weissberg, 2016; Payton et al., 2000). A preschooler who has attained such age-appropriate SEC skills is more likely than one who has not to become the kindergartner who can better plan and pay attention to academic tasks and devote more resources to learning. S/he can benefit more from teachers’ instructions, sharing academic information and resources with peers and modeling peers’ learning skills (Denham et al., 2010, Romano et al., 2010). In this study we focus on two SEC skills that are important during early childhood: emotion knowledge and social problem solving.
There is increasing stakeholder acknowledgement that SEC is key in the transition from preschool to elementary school. Both parents and early childhood educators point to an urgent need for SEC programming, especially for children who live in low income families and/or are members of minority groups who historically have faced challenges to academic achievement (Bridgeland, Bruce, & Hariharan, 2013; Piotrkowski, Botsko, & Matthews, 2000). Further, all state education systems and Head Start programs now incorporate SEC skills, including emotion knowledge and social problem solving, in prekindergarten learning standards (albeit with fewer indicators and implemented less systematically in comparison to cognitive skills; see Dusenbury et al., 2015). National legislation has also been introduced in the US authorizing funds for technical assistance, training, and programming on SEC (O’Connor, DeFeyter, Carr, Luo, & Romm, 2017). As a result, SEC is receiving increasing attention and incorporation in early childhood instruction.
Therefore, because SEC skills are so important and viewed as such not only by academicians, but vehemently by early childhood educators, parents, and even legislators, it would behoove us to assess them well. However, educators lack the requisite psychometrically sound tools to identify, track and assess SEC skills (McKown, 2017). Without sound measurement tools to facilitate early assessment of SEC, educators cannot best tailor instruction to each child, and stakeholders cannot ascertain progress or whether a given SEC curriculum is more or less fruitful (Denham, Bassett, & Zinsser, 2012; Denham, 2015). Hence, our goal in this study is to promote useful SEC assessments measuring preschoolers’ emotion knowledge and social problem solving.
There are, of course, definite requirements for such assessment tools (Denham, 2015; Kendziora, Weissberg, Ji, & Dusenbury, 2011). They should be developmentally appropriate, integrated with curricula, beneficial to all parties, primarily reliant on the child’s everyday activities in realistic situations, culturally and linguistically responsive, and reflect children’s actual performance (Copple & Bredekamp, 2009). Data emanating from such assessment should, moreover, not be used for high stakes decisions, such as retention in kindergarten. Instead, formative and summative functions of assessment should be undertaken to effectively identify children needing intervention or higher-level services, highlight specific needs of children and classrooms in terms of instruction, and show overall effects of programming (Denham, 2015; Denham, Ferrier, Howarth, Herndon, & Bassett, 2019).
With these stipulations in mind, our research team has developed several well-validated assessments for aspects of SEC, including direct in-person assessments of emotion knowledge with the Affect Knowledge Test-Shortened (AKT-S) and social problem solving with the Challenging Situations Task (CST). The AKT-S and CST measures have been widely used by researchers around the world (e.g., Leyva, Berrocal, & Nolivos, 2014; Kılıç & Aytar, 2016; Rebelo, Verissimo, Machado, & Silva, 2013; Sette, Bassett, Baumgartner, & Denham, 2015; Sheard, Ross, & Cheung, 2013; Upshur, Heyman, & Wenz-Gross, 2017). In fact, one recent meta-analysis found that over a third of all effect sizes in studies of children’s emotion socialization were measured with the AKT (Zinsser, Gordon, & Jiang, 2019).
As theoretically predicted, research has consistently shown that the AKT and CST predict social competence, classroom adjustment, and academic readiness (e.g., Denham, Bassett, Way, et al., 2012; Denham et al., 2013; Denham, Way, Kalb, Warren-Khot, & Bassett, 2013; Denham, Bassett, Zinsser, & Wyatt, 2014). The theoretical proposition underlying the AKT’s relations with these attributes is that emotion knowledge “frees up” personal resources, allowing young children to interact positively with others in the preschool environment, and focus more specifically on learning. Regarding social problem solving as assessed by the CST, being calmly “ok” or sad instead of angry during a peer provocation may allow the child to take time to think about a prosocial solution, whereas the angry child may merely lash out aggressively, such that sad or “just ok” self-emotion choices on the CST are related to their teacher-rated social competence and classroom adjustment (Denham et al., 2013; Izard, Stark, Trentacosta, & Schultz, 2008; Schultz et al., 2010). In contrast, choosing happy emotion responses connotes lack of understanding or denial that a provocation has occurred (or perhaps social desirability in the context of assessment). Choosing angry emotion responses would more directly relate to deficits of social competence and classroom adjustment (Orobio de Castro, Merk, Koops, Veerman, & Bosch, 2005). Regarding behavior response choices on the CST, children who choose socially competent behaviors as part of response decision-making, as opposed to aggressive or passive behaviors, are likely to demonstrate greater social competence and overall classroom adjustment.
Despite these theoretical foundations, the popularity of these tools among researchers, and the evidence base supporting their use, their administration and scoring by trained examiners makes them prohibitive for classroom teachers’ use. To provide educators with the most psychometrically rigorous tools, we have adapted these tools to maximize their potential utility and feasibility in preschool, Head Start, and childcare classrooms. Our goals are to (a) allow teachers to track students’ progress and inform instruction (formative assessment), (b) measure child outcomes (summative assessment), and (c) evaluate program outcomes. To accomplish these goals, we adapt these in-person measures for electronic administration, using tablet devices.
In sum, given the growing importance of SEC and its concomitant assessment in the early childhood classroom, the focus of the current study is to evaluate the psychometric efficacy of two direct assessment tools measuring emotion knowledge and social problem solving, adapted for computer administration. The goal is, with continued improvement based on these evaluations, to use these tools to inform early childhood instruction. In accessing what the child knows and thinks, direct assessment can complement and add to information emanating from the observable and ratable behavioral aspects of SEC (McKown, 2017).
Why computerize measures of early childhood SEC? In the busy early childhood classroom, teachers must have easy-to-administer assessment tools. Tablets have become very common in preschool classrooms in the last two decades (e.g., Fletcher, Whitaker, Marino, & Anderson, 2014; NAEYC, 2012), and meet this need. Other principal advantages of computer-based systems over conventional assessment methods are that: (a) stimuli are standardized, making assessment more precise; (b) significant savings can be made in both time and labor; (c) scoring can be immediately available, without error-prone optical scoring; and (d) training for assessment administrators (i.e., teachers or educational resource personnel) can be dramatically reduced and streamlined. In attempts to computerize our preschool SEC measures, we strove to meet these criteria, to help teachers move toward use of electronic portfolios of both formative and summative SEC assessment.
Such assessment also can take advantage of the capabilities of technology for animation, speech, and sound. In fact, young children engage with educational software as soon as they can manipulate a touchscreen (NAEYC, 2012; Vandewater et al., 2007). They show intense interest and pleasure, and surprising stamina, in interacting with computers (Ellis and Blashki, 2004, McCarrick and Li, 2007). Thus, computer use, whether for instruction or assessment, can be a highly motivating, positive experience for young children.
In summary, to facilitate teacher use of these SEC assessments in effective, efficient, and valid ways, it is crucial to develop appropriate means of standardizing and streamlining direct assessments emotion knowledge and social problem solving, via computer usage. Although our original research-based measures are valuable in predicting early school success, the training, coding, and administration requirements that attend them can be “deal breakers” for applied use in the early childhood classroom. If we are to make these means of assessment useful, to move toward both formative and summative assessment, we feel that they must be computerized, with much thought given to user support (for, e.g., preschool, Head Start, and childcare teachers, Head Start mental health consultants, and others). We also consider making the assessment attractive and fun to children to be an important goal, to maintain their engagement and enjoyment of the learning process.
In this study, we present findings from our direct assessments of emotion knowledge and social problem solving. A first step in creating usable assessment tools is to examine psychometric properties of the measures. To that end, data from in-person and computerized modes of each measure were collected. We used with two versions of each measure within each mode, to yield answers for four research questions. Research Question 1 (RQ1) asks whether internal consistency and between-version temporal stability for each version of the computerized mode are adequate. The adequacy of concurrent validity, via mean equivalence of in-person and computerized modes, is addressed in Research Question 2 (RQ2). For Research Question 3 (RQ3), we further examine the adequacy of the computerized versions’ concurrent validity via associations between corresponding versions across in-person and computerized modes. Finally, in Research Question Four (RQ4) we determine the adequacy of the computerized versions’ predictive validity via associations between the computerized mode and both teacher-rated SEC and learning behavior/attitudes (given the importance of emotion knowledge and social problem solving to early school success, and in accordance with the theoretical propositions related to these associations).
Section snippets
Participants
Participants included 450 preschoolers from urban, suburban, and more rural preschools (nsuburban = 337; nurban = 40; nrural = 73). Because we collected data in three locations, we examined regional differences. Participants’ average age across all regions was 51 months, but age differed across regions (F (2, 442) = 11.25, partial 2 = 0.05, p < 0.001); mean age in the suburban group was 52 months, but in both the urban and rural samples, the average age was 48 months (ranges 35–71 months,
Research question 1: reliability
First, all associations between versions of both measures were generally significant, exemplifying short-term test-retest reliability (see Table 1). Over one- to three-week intervals, correlations for the AKT ranged from weak in magnitude for negative recognition to moderate for other subscales. For CST emotion choices, all correlations were moderate in magnitude, and for CST behavior choices these ranged from weak for avoidant/passive choices to moderate for the other three choices. Thus, over
Discussion
For early childhood educators to make informed decisions about how best to support children’s social-emotional development, they must have access to rigorous formative assessments of children’s competencies that can easily be administered in classroom settings. This study set out, therefore, to develop and validate computer-based parallel versions of two highly regarded measures of children’s emotion knowledge (the AKT-S) and social problem solving (the CST). These measures are psychometrically
References (66)
- et al.
Factors influencing digital technology use in early childhood education
Computers & Education
(2014) - et al.
Social-emotional learning profiles of preschoolers' early school success: A person-centered approach
Learning and Individual Differences
(2012) - et al.
Parents’ and teachers’ beliefs about children’s school readiness in a high-need community
Early Childhood Research Quarterly
(2000) - et al.
Efficacy trial of the Second Step Early Learning (SSEL) curriculum: Preliminary outcomes
Journal of Applied Developmental Psychology
(2017) Computer-based testing and young children
- et al.
Effects of Head Start REDI on children’s outcomes 1 year later in different kindergarten contexts
Child Development
(2014) - et al.
The missing piece: A national teacher survey on how social and emotional learning can empower children and transform schools. A report for CASEL
(2013) - et al.
The role of factor analysis in the development and evaluation of personality scales
Journal of Personality
(1986) Reliability and predictive validity of preschool web-based phonological awareness assessment for identifying school-aged reading difficulty
Communication Disorders Quarterly
(2017)- et al.
Predicting reading outcomes in the classroom using a computer-based phonological awareness screening and monitoring assessment (Com-PASMA)
International Journal of Speech-Language Pathology
(2014)
Computer-administrated versus paper-based assessment of school-entry phonological awareness ability
Asia Pacific Journal of Speech, Language and Hearing
Constructing validity: Basic issues in objective scale development
Psychological Assessment
Young pupils’, their teacher’s and classroom assistants’ experiences of iPads in a Northern Ireland school: “Four and five years old, who would have thought they could do that?”
British Journal of Educational Technology
Developmentally appropriate practice in early childhood programs serving children from birth through age 8
Computer-based assessment of school readiness and early reasoning
Journal of Educational Psychology
“Plays nice with others”: Social-emotional learning and academic success
Early Education and Development
Computerizing social-emotional assessment for school readiness: First steps toward an assessment battery for early childhood settings
Journal of Applied Research on Children: Informing Policy for Children at Risk
Preschoolers’ emotion knowledge: Self-regulatory foundations, and predictions of early school success
Cognition and Emotion
Social and emotional information processing in preschoolers: Indicator of early school success?
Early Child Development and Care
Preschoolers' social information processing and early school success: The Challenging Situations Task
British Journal of Developmental Psychology
“How would you feel? What would you do?” Development and underpinnings of preschoolers’ social information processing
Journal of Research in Childhood Education
How preschoolers’ social-emotional learning (SEL) predicts their early school success: Developing theory-promoting, competency-based assessments
Infant and Child Development
Assessment of SEL in educational contexts
“I know how you feel”: Preschoolers’ emotion knowledge contributes to early school success
Journal of Early Childhood Research
Key considerations in assessing young children’s emotional competence
The case for preschool through high school state learning standards for SEL
Handbook of social and emotional learning: Research and practice
Toddler techies: A study of young children’s interaction with computers
Information Technology in Childhood Education Annual
Preschool approaches to learning and their relationship to other relevant classroom competencies for low-income children
School Psychology Quarterly
Screen time at home and school among low-income children attending head start
Child Indicators Research
Assessment of children’s digital courseware in light of developmentally appropriate courseware criteria
British Journal of Educational Technology
Beyond emotion regulation: Emotion utilization and adaptive functioning
Child Development Perspectives
A step in the right direction: Development of the computerized assessment for preschool social emotional learning (CAPSEL)
Journal of Applied Research on Children: Informing Policy for Children at Risk
Using multiple child assessments to inform practice in early childhood programs: Lessons from Milpitas Unified School District
Cited by (5)
Intervention response among preschoolers with ADHD: The role of emotion understanding
2021, Journal of School PsychologyCitation Excerpt :Total scores on the human and cartoon faces were combined, with higher scores indicative of better ERU (α's = 0.72–0.78 across cartoon and human faces, pre- and post-intervention). This task has shown good internal consistency and test-retest reliability, along with concurrent and predictive validity (Denham, 2006; Denham et al., 2012, 2020). Previous research using this task has shown ERU be associated with school readiness (Denham et al., 2012; Shields et al., 2001), EF (Denham et al., 2012), teacher and peer rating of prosocial behavior (Cassidy, Werner, Rourke, Zubernis, & Balaraman, 2003; Dunsmore & Karn, 2004), academic achievement (Leerkes et al., 2008), attention (Rhoades, Warren, Domitrovich, & Greenberg, 2011), and has demonstrated sensitivity to intervention effects (Graziano & Hart, 2016).
Multimodal Assessment of Emotion Dysregulation in Children with and without ADHD and Disruptive Behavior Disorders
2024, Journal of Clinical Child and Adolescent PsychologyThe PETaL approach to bilingual and intercultural education in early childhood education
2022, Research Anthology on Early Childhood Development and School Transition in the Digital EraDevelopment and Psychometric Properties of a Computer-Based Standardized Emotional Competence Inventory (MeKKi) for Preschoolers and School-Aged Children
2022, Child Psychiatry and Human Development
- ☆
The present study was funded by NICHD grant #R21HD068744 and IES grant #IES R305A110730. We are grateful to the many children, families, and teachers who participated in this study, and the directors of the facilities who so cooperatively worked with us. Special thanks go to Andrew Steed, Adrian Grajdeanu, and Dustin Rhodes for programming our computerized measures. We also thank Claire Christensen, Candace DeVito, Charlotte Edwards, Nicole Fettig, Grace Howarth, Sarah Lim, Sara O’Malley, Tanvai Rawat, Caitlin Tarr, Ruvi Tsokodayi, and Naomi Watanabe for their major contributions to this work, as well as the numerous other graduate and undergraduate research assistants who also administered the measures.