Abstract
Randomized controlled trials (RCTs) are criticized for the questionable generalizability of their patient samples to real world populations. This study compared the demographic and clinical characteristics of children from the Multimodal Treatment Study of Children with ADHD (MTA) to those from the NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) Study and the National Health and Nutrition Examination Survey (NHANES). On the whole, we found no compelling evidence against the representativeness of the MTA sample, but the limited power from MECA and NHANES may have lead to noteworthy differences between datasets being statistically nonsignificant.
Similar content being viewed by others
References
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.
Arnold, L. E., Abikoff, H. B., Cantwell, D. P., Conners, C. K., Elliott, G., Greenhill, L. L., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., March, J. S., Newcorn, J. H., Pelham, W. E., Richters, J. E., Schiller, E., Severe, J. B., Swanson, J. M., Vereen, D., & Wells, K. C. (1997). National Institute of Mental Health Collaborative Multimodal Treatment Study of Children with ADHD (the MTA). Design challenges and choices. Archives of General Psychiatry, 54, 865–870.
Arnold, L. E., Elliott, M., Sachs, L., Bird, H., Kraemer, H. C., Wells, K. C., Abikoff, H. B., Comarda, A., Conners, C. K., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. J., March, J. S., Newcorn, J., Pelham, W. E., Severe, J. B., Swanson, J. M., Vitiello, B., & Wigal, T. (2003). Effects of ethnicity on treatment attendance, stimulant response/dose, and 14-month outcome in ADHD. Journal of Consulting and Clinical Psychology, 71, 713–727.
Barkley, R. A. (2000). Commentary on the multimodal treatment study of children with ADHD. Journal of Abnormal Child Psychology, 28, 595–599.
Beck, A. T., Ward, C. H., Mendelson, M., Mock, J. E., & Erbaugh, J. K. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571.
Bird, H. R., Gould, M. S., & Staghezza, B. M. (1993). Patterns of diagnostic comorbidity in a community sample of children aged 9 through 16 years. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 361–368.
Conners, C. K., Epstein, J. N., March, J. S., Angold, A., Wells, K. C., Klaric, J., Swanson, J. M., Arnold, L. E., Abikoff, H. B., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., Newcorn, J. H., Pelham, W. E., Severe, J. B., Vitiello, B., & Wigal, T. (2001). Multimodal treatment of ADHD in the MTA: An alternative outcome analysis. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 159–167.
Dunn, L. M., Dunn, L. M. (1981). Peabody picture vocabulary test-revised. Circle Pines, MN: American Guidance Service.
Farmer, E. M., Stangl, D. K., Burns, B. J., Costello, E. J., & Angold, A. (1999). Use, persistence, and intensity: Patterns of care for children’s mental health across one year. Community Mental Health Journal, 35, 31–46.
Fisher, P., Wicks, J., Shaffer, D., Piacentini, J., & Lapkin, J. (1992). NIMH diagnostic interview schedule for children, version 2.3, user’s manual. New York: New York State Psychiatric Institute.
General Accounting Office. (1992). Cross design synthesis: A new strategy for medical effectiveness research. Washington, DC: GAO.
Greenhill, L. L., Swanson, J. M., Vitiello, B., Davies, M., Clevenger, W., Wu, M., Arnold, L. E., Abikoff, H. B., Bukstein, O. G., Conners, C. K., Elliott, G. R., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., March, J. S., Newcorn, J. H., Severe, J. B., Wells, K., & Wigal, T. (2001). Impairment and deportment responses to different methylphenidate doses in children with ADHD: The MTA titration trial. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 180–187.
Jensen, P. S., Hinshaw, S. P., Kraemer, H. C., Lenora, N., Newcorn, J. H., Abikoff, H. B., March, J. S., Arnold, L. E., Cantwell, D. P., Conners, C. K., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hoza, B., Pelham, W. E., Severe, J. B., Swanson, J. M., Wells, K. C., Wigal, T., & Vitiello, B. (2001). ADHD comorbidity findings from the MTA study: Comparing comorbid subgroups. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 147–158.
Korn, E. L., & Graubard, B. I. (1999). Analysis of health surveys. New York: Wiley Press.
Lahey, B. B., Flagg, E. W., Bird, H. R., Schwab-Stone, M. E., Canino, G., Dulcan, M. K., Leaf, P. J., Davies, M., Brogan, D., Bourdon, K., Horwitz, S. M., Rubio-Stipec, M., Freeman, D. H., Lichtman, J. H., Shaffer, D., Goodman, S. H., Narrow, W. E., Weissman, M. M., Kandel, D. B., Jensen, P. S., Richters, J. E., & Regier, D. A. (1996). The NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) study: Background and methodology. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 855–864.
Leaf, P., Alegria, M., Cohen, P., Goodman, S., Horwitz, S. M., Hoven, C. W., Narrow, W. E., Vaden-Kiernan, M., & Regier, D. A. (1996). Mental health service use in the community and schools: Results from the four-community MECA study. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 889–897.
MTA Cooperative Group. (1999a). A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Multimodal Treatment Study of Children with ADHD. Archives of General Psychiatry, 56, 1073–1086.
MTA Cooperative Group. (1999b). Moderators and mediators of treatment response for children with attention-deficit/hyperactivity disorder: The Multimodal Treatment Study of children with Attention-deficit/hyperactivity disorder. Archives of General Psychiatry, 56, 1088–1096.
Owens, E. B., Hinshaw, S. P., Kraemer, H. C., Arnold, L. E., Abikoff, H. B., Cantwell, D. P., Conners, C. K., Elliott, G., Greenhill, L. L., Hechtman, L., Hoza, B., Jensen, P. S., March, J. S., Newcorn, J. H., Pelham, W. E., Severe, J. B., Swanson, J. M., Vitiello, B., Wells, K. C., & Wigal, T. (2003). Which treatment for whom for ADHD? Moderators of treatment response in the MTA. Journal of Consulting and Clinical Psychiatry, 71, 540–552.
Rieppi, R., Greenhill, L. L., Ford, R. E., Chuang, S., Wu, M., Davies, M., Abikoff, H. B., Arnold, L. E., Conners, C. K., Elliott, G. R., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., March, J. S., Newcorn, J. H., Pelham, W. E., Severe, J. B., Swanson, J. M., Vitiello, B., Wells, K. C., & Wigal, T. (2002). Socioeconomic status as a moderator of ADHD treatment outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 269–277.
Rothwell, P. (2005). External validity of randomised controlled trials: To whom do the results of this trial apply? Lancet, 365, 82–93.
Shaffer, D., Fisher, P., Dulcan, M. K., Davies, M., Piacentini, J., Schwab-Stone, M. E., Lahey, B. B., Bourdon, K., Jensen, P. S., Bird, H. R., Canino, G., & Regier, D. A. (1996). The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA Study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 865–877.
Persons, J. B., & Silberschatz, G. (1998). Are results of randomized controlled trials useful to psychotherapists? Journal of Consulting and Clinical Psychology, 66, 126–135.
SPSS Inc. (2003). SPSS base 12.0 application guide. Chicago, IL: Author.
Stevens, J., Harman, J. S., & Kelleher, K. J. (2005). Race/ethnicity and insurance status as factors associated with ADHD treatment patterns. Journal of Child and Adolescent Psychopharmacology, 15, 88–96.
Stirman, S. W., DeRubeis, R. J., Crits-Christoph, P., & Brody, P. E. (2003). Are samples in randomized controlled trials of psychotherapy representative of community outpatients? A new methodology and initial findings. Journal of Consulting and Clinical Psychology, 71, 963–972.
Swanson, J. M., Kraemer, H. C., Hinshaw, S. P., Arnold, L. E., Conners, C. K., Abikoff, H. B., Clevenger, W., Davies, M., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hoza, B., Jensen, P. S., March, J. S., Newcorn, J. H., Owens, E. B., Pelham, W. E., Schiller, E., Severe, J. B., Simpson, S., Vitiello, B., Wells, K., Wigal, T., & Wu, M. (2001). Clinical relevance of the primary findings of the MTA: Success rates based on severity of ADHD and ODD symptoms at the end of treatment. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 168–179.
Wechsler, D. (1991). Wechsler intelligence Scale for Children—Third Edition San Antonio. TX: Psychological Corporation.
Westen, D. I. (2006). Patients and treatments in clinical trials are not adequately representative of clinical practice. In J. C. Norcross, L. E. Beutler, & R. F. Levant (Eds.), Evidence-based practices in mental health: Debate and dialogue on the fundamental questions. Washington, DC: American Psychological Association.
Zimmerman, M., Chelminski, I., & Posternak, M. A. (2005). Generalizability of antidepressant efficacy trials: Differences between depressed psychiatric outpatients who would versus would not qualify for an efficacy trial. American Journal of Psychiatry, 162, 1370–1372.
Zimmerman, M., Mattia, J. I., & Posternak, M. A. (2002). Are subjects in pharmacological treatment trials of depression representative of patients in routine clinical practice? American Journal of Psychiatry, 159, 469–473.
Author information
Authors and Affiliations
Corresponding author
Additional information
This study was supported in part by NIMH grants MH66960, MH65430, and MH30915. The Multimodal Treatment Study of Children with Attention Deficit/Hyperactivity Disorder (MTA) was conducted and supported by the National Institute of Mental Health (NIMH) in collaboration with the MTA Investigators. The NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) Study was conducted and supported by the National Institute of Mental Health (NIMH) in collaboration with the MECA investigators. This manuscript reflects the views of the authors and may not reflect the opinions or views of all the MTA investigators, the MECA investigators, or NIMH.
Rights and permissions
About this article
Cite this article
Stevens, J., Kelleher, K., Greenhouse, J. et al. Empirical Evaluation of the Generalizability of the Sample from the Multimodal Treatment Study for ADHD. Adm Policy Ment Health 34, 221–232 (2007). https://doi.org/10.1007/s10488-006-0097-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-006-0097-4