NEW RESEARCH
Observational Assessment of Preschool Disruptive Behavior, Part I: Reliability of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS)

https://doi.org/10.1097/CHI.0b013e31816c5bdbGet rights and content

ABSTRACT

Objective:

To examine the reliability of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS), a new observational method for assessing preschool disruptive behavior.

Method:

The DB-DOS is a structured clinic-based assessment designed to elicit clinically salient behaviors relevant to the diagnosis of disruptive behavior in preschoolers. Child behavior is assessed in three interactional contexts that vary by partner (parent versus examiner) and level of support provided. Twenty-one disruptive behaviors are coded within two domains: problems in Behavioral Regulation and problems in Anger Modulation. A total of 364 referred and nonreferred preschoolers participated: interrater reliability and internal consistency were assessed on a primary sample (n = 335) and test-retest reliability was assessed in a separate sample (n = 29).

Results:

The DB-DOS demonstrated good interrater and test-retest reliability. Confirmatory factor analysis demonstrated an excellent fit of the DB-DOS multidomain model of disruptive behavior.

Conclusions:

The DB-DOS is a reliable observational tool for clinic-based assessment of preschool disruptive behavior. This standardized assessment method holds promise for advancing developmentally sensitive characterization of preschool psychopathology.

Section snippets

Participants

Two samples were recruited for the reliability study (primary reliability and test-retest). Samples were recruited from clinics affiliated with two Midwestern U.S. universities serving predominantly urban, disadvantaged populations. Inclusion criteria included age between 3 and 5 years, residence with biological mother, attendance in out-of-home day care or school, and low income. Children were excluded if they had a serious developmental disability or medical condition. Because a primary goal

Item-Level Interrater Reliability

Diagnostic observation measures may include clinically salient items whose occurrence is too low to be evaluated statistically.38 Five items (all in the Behavioral Regulation domain) were rated as present fewer than six times. These items were verbal aggression (two items), directed aggression, spiteful behavior, and sneaky behavior. Because estimates of reliability can be unduly biased by differences between a single pair of raters for items with very low base rates, these items were excluded

DISCUSSION

We have shown that a relatively brief, laboratory-based diagnostic observation yields reliable information about patterns of disruptive behavior in young children. Establishing the reliability of the DB-DOS is the first step in the validation process (see Part II for a report on the validity of the DB-DOS). The promise of the DB-DOS for advancing the science and practice of preschool mental health is substantial. Disruptive behavior problems are the most common reasons for mental health

Behavioral Regulation Domain

  • 1.

    Defiance

  • 2.

    Passive noncompliance

  • 3.

    Predominance of noncompliance

  • 4-5.

    Rule breaking (coded in all contexts; in examiner contexts, rule-breaking that occurred with and without supervision was coded separately)

  • 6-7.

    Lack of admission of rule-breaking (coded only in examiner contexts with separate codes for rule-breaking that occurred with and without supervision)

  • 8.

    Provocative behavior

  • 9.

    Behavioral inflexibility

  • 10.

    Destructiveness

  • 11.

    Directed aggression*

  • 12-13.

    Verbal aggression* (coded separately for threats and cursing)

  • 14.

    Spiteful

REFERENCES (44)

  • M Rutter

    Child psychiatric disorders: measures, causal mechanisms, and interventions

    Arch Gen Psychiatry.

    (1997)
  • D Pine et al.

    Advances in developmental science and DSM-IV

  • S Campbell

    Behavior Problems in Preschool Children: Clinical and Developmental Issues

    (2002)
  • C Lord et al.

    The Autism Diagnostic Observation Schedule-Generic: a standard measure of social and communication deficits associated with the spectrum of autism

    J Autism Dev Disord.

    (2000)
  • American Academy of Child and Adolescent Psychiatry

    Practice parameters for the psychiatric assessment of infants and toddlers

    J Am Acad Child Adolesc Psychiatry.

    (1997)
  • A Benham

    The observation and assessment of young children including use of the Infant-Toddler Mental Status Exam

  • C Zeanah et al.

    Relationship assessment in infant mental health

    Infant Ment Health J.

    (1997)
  • S Campbell et al.

    Parent-identified problem preschoolers: mother-child interaction during play at intake and 1 year follow-up

    J Abnorm Child Psychol.

    (1986)
  • EA Robinson et al.

    The dyadic parent-child interaction coding system: standardization and validation

    J Consult Clinic Psychol.

    (1981)
  • C. Webster-Stratton

    Mother perceptions and mother-child interactions; comparison of clinic-referred and a nonclinic group

    J Clinic Child Psychol.

    (1985)
  • E Mash et al.

    Exporting analogue behavioral observation from research to clinical practice: useful or cost-defective?

    Psychol Assess.

    (2001)
  • M Roberts

    Clinic observation of structured parent-child interactions designed to evaluate externalizing disorders

    Psychol Assess.

    (2001)
  • Cited by (84)

    • Differential associations of maternal behavior to preschool boys' and girls' executive function

      2022, Journal of Applied Developmental Psychology
      Citation Excerpt :

      A trained research assistant tested children individually over three sessions, separated by approximately one week. In the first session, children completed the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS), a standardized clinical observation designed to differentiate normative misbehavior in early childhood from clinically salient patterns of behavioral dysregulation (Wakschlag et al., 2008). The DB-DOS examines children's behavior during parent and examiner social interaction contexts with parallel demands.

    • Adversity is Linked with Decreased Parent-Child Behavioral and Neural Synchrony

      2021, Developmental Cognitive Neuroscience
      Citation Excerpt :

      Factor scores were extracted based on the two-factor solution using regression-based weights and used in further analysis. Neural synchrony was assessed using the DB-DOS: BioSync task (Quiñones-Camacho et al., 2019), a modification of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS) a developmentally sensitive, observational index of child disruptive behavior (Wakschlag et al., 2008). The DB-DOS: BioSync contained two experimental conditions, one in which frustration was induced for both members of the dyad, and one in which the dyad was given time to recover from the frustration-induction condition.

    • Dimension- and context-specific expression of preschoolers' disruptive behaviors associated with prenatal tobacco exposure

      2020, Neurotoxicology and Teratology
      Citation Excerpt :

      To address this need, we conceptualized a developmental framework and devised a suite of assessment tools for assessing core dimensions of DB in young children. This includes an observational assessment paradigm, the Disruptive Behavior - Diagnostic Observation System (DB-DOS) to capture typical and atypical behavior in response to frustration across varied interactional contexts (Wakschlag et al., 2008a; Wakschlag et al., 2008b) and the Multidimensional Assessment of DB (MAP-DB), to capture parental reports of the frequency, quality and intransigence of children's behavior across a variety of everyday settings (Wakschlag et al., 2014). The combination of these multiple-informant, multi-dimension, cross-contextual measures creates a developmentally sensitive, empirically derived framework for capturing patterns of DB that deviate from the normative misbehavior of early childhood.

    View all citing articles on Scopus

    This project was supported by National of Institute of Mental Health grants R01 MH68455 and MH62437, National 0-3, and the Walden and Jean Young Shaw and Children's Brain Research Foundations.

    This work has been importantly shaped by ongoing critical discussions with our colleagues Patrick Tolan, Daniel Pine, Edwin Cook, Jr., Catherine Lord, Kimberly Espy, David Henry, Nathan Fox, and Chaya Roth, and our students Anil Chacko, Nicole Bush, and Melanie Dirks. We thank Drs. Janis Mendolsohn and Saul Weiner for facilitation of pediatric recruitment. The DB-DOS is dedicated to the memory of beloved student and colleague Kathleen Kennedy Martin.

    This article is the subject of an editorial by Dr. Charles Zeanah in this issue.

    View full text