Childhood and adolescent adversity have been shown to predict later mental and physical health outcomes. Understanding which aspects and developmental timings of adversity are important, and the mechanisms by which they have their impact may help guide intervention approaches. A large subset of adolescents (N = 457; Female 68.9 %) from the 10-year longitudinal Youth Emotion Project was examined to better understand the associations among childhood/adolescent adversity, substance use disorder, and later health quality. Adolescent (but not childhood) adversities were associated with poorer health in late adolescence/early adulthood, adolescent adversities were associated with subsequent onset of substance use disorder, and adolescent adversities continued to be associated with poorer health in late adolescence/early adulthood after accounting for the variance explained by substance use disorder onset. These associations were observed after statistically accounting for emotional disorders and socioeconomic status. Specific domains of adversity uniquely predicted substance use disorder and poorer health outcomes. In contrast with current recent research, our findings suggest the association between childhood/adolescent adversity and poorer health outcomes in late adolescence and emerging adulthood are not entirely accounted for by substance use disorder, suggesting efforts to curtail family-based adolescent adversity may have downstream health benefits.