Previous research has examined the factor structure of ADHD symptoms in adults using confirmatory factor analysis and there is mixed support for two-factor, three-factor, or more complex bifactor models. The current study re-examines these factor structures on the basis of not only overall model fit, but also criteria such as chi-square difference tests, invariance, reliability, and validity. Three datasets were pooled to form a total sample of 430 adults (50% female). Participants completed questionnaires assessing their ADHD symptoms, education, psychopathology, and parenting (the latter as validity variables). Although the bifactor models demonstrated the best fit, the reliability and construct replicability of specific factors in these models was poor. The three-factor model had good fit and demonstrated good reliability, validity, construct replicability, and some invariance across gender and dataset. Our results replicate previous studies that find superior fit for bifactor models. However, the superior statistical fit may reflect only the increased complexity of these models. The more parsimonious three-factor model may demonstrate stronger validity and clinical utility.