Skip to main content

Advertisement

Log in

Empirical Evaluation of the Generalizability of the Sample from the Multimodal Treatment Study for ADHD

  • Original Paper
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Randomized controlled trials (RCTs) are criticized for the questionable generalizability of their patient samples to real world populations. This study compared the demographic and clinical characteristics of children from the Multimodal Treatment Study of Children with ADHD (MTA) to those from the NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) Study and the National Health and Nutrition Examination Survey (NHANES). On the whole, we found no compelling evidence against the representativeness of the MTA sample, but the limited power from MECA and NHANES may have lead to noteworthy differences between datasets being statistically nonsignificant.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.

    Google Scholar 

  • Arnold, L. E., Abikoff, H. B., Cantwell, D. P., Conners, C. K., Elliott, G., Greenhill, L. L., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., March, J. S., Newcorn, J. H., Pelham, W. E., Richters, J. E., Schiller, E., Severe, J. B., Swanson, J. M., Vereen, D., & Wells, K. C. (1997). National Institute of Mental Health Collaborative Multimodal Treatment Study of Children with ADHD (the MTA). Design challenges and choices. Archives of General Psychiatry, 54, 865–870.

    PubMed  CAS  Google Scholar 

  • Arnold, L. E., Elliott, M., Sachs, L., Bird, H., Kraemer, H. C., Wells, K. C., Abikoff, H. B., Comarda, A., Conners, C. K., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. J., March, J. S., Newcorn, J., Pelham, W. E., Severe, J. B., Swanson, J. M., Vitiello, B., & Wigal, T. (2003). Effects of ethnicity on treatment attendance, stimulant response/dose, and 14-month outcome in ADHD. Journal of Consulting and Clinical Psychology, 71, 713–727.

    Article  PubMed  Google Scholar 

  • Barkley, R. A. (2000). Commentary on the multimodal treatment study of children with ADHD. Journal of Abnormal Child Psychology, 28, 595–599.

    Article  PubMed  CAS  Google Scholar 

  • Beck, A. T., Ward, C. H., Mendelson, M., Mock, J. E., & Erbaugh, J. K. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561–571.

    PubMed  CAS  Google Scholar 

  • Bird, H. R., Gould, M. S., & Staghezza, B. M. (1993). Patterns of diagnostic comorbidity in a community sample of children aged 9 through 16 years. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 361–368.

    Article  PubMed  CAS  Google Scholar 

  • Conners, C. K., Epstein, J. N., March, J. S., Angold, A., Wells, K. C., Klaric, J., Swanson, J. M., Arnold, L. E., Abikoff, H. B., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., Newcorn, J. H., Pelham, W. E., Severe, J. B., Vitiello, B., & Wigal, T. (2001). Multimodal treatment of ADHD in the MTA: An alternative outcome analysis. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 159–167.

    Article  PubMed  CAS  Google Scholar 

  • Dunn, L. M., Dunn, L. M. (1981). Peabody picture vocabulary test-revised. Circle Pines, MN: American Guidance Service.

    Google Scholar 

  • Farmer, E. M., Stangl, D. K., Burns, B. J., Costello, E. J., & Angold, A. (1999). Use, persistence, and intensity: Patterns of care for children’s mental health across one year. Community Mental Health Journal, 35, 31–46.

    Article  PubMed  CAS  Google Scholar 

  • Fisher, P., Wicks, J., Shaffer, D., Piacentini, J., & Lapkin, J. (1992). NIMH diagnostic interview schedule for children, version 2.3, user’s manual. New York: New York State Psychiatric Institute.

    Google Scholar 

  • General Accounting Office. (1992). Cross design synthesis: A new strategy for medical effectiveness research. Washington, DC: GAO.

    Google Scholar 

  • Greenhill, L. L., Swanson, J. M., Vitiello, B., Davies, M., Clevenger, W., Wu, M., Arnold, L. E., Abikoff, H. B., Bukstein, O. G., Conners, C. K., Elliott, G. R., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., March, J. S., Newcorn, J. H., Severe, J. B., Wells, K., & Wigal, T. (2001). Impairment and deportment responses to different methylphenidate doses in children with ADHD: The MTA titration trial. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 180–187.

    Article  PubMed  CAS  Google Scholar 

  • Jensen, P. S., Hinshaw, S. P., Kraemer, H. C., Lenora, N., Newcorn, J. H., Abikoff, H. B., March, J. S., Arnold, L. E., Cantwell, D. P., Conners, C. K., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hoza, B., Pelham, W. E., Severe, J. B., Swanson, J. M., Wells, K. C., Wigal, T., & Vitiello, B. (2001). ADHD comorbidity findings from the MTA study: Comparing comorbid subgroups. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 147–158.

    Article  PubMed  CAS  Google Scholar 

  • Korn, E. L., & Graubard, B. I. (1999). Analysis of health surveys. New York: Wiley Press.

    Google Scholar 

  • Lahey, B. B., Flagg, E. W., Bird, H. R., Schwab-Stone, M. E., Canino, G., Dulcan, M. K., Leaf, P. J., Davies, M., Brogan, D., Bourdon, K., Horwitz, S. M., Rubio-Stipec, M., Freeman, D. H., Lichtman, J. H., Shaffer, D., Goodman, S. H., Narrow, W. E., Weissman, M. M., Kandel, D. B., Jensen, P. S., Richters, J. E., & Regier, D. A. (1996). The NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) study: Background and methodology. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 855–864.

    Article  PubMed  CAS  Google Scholar 

  • Leaf, P., Alegria, M., Cohen, P., Goodman, S., Horwitz, S. M., Hoven, C. W., Narrow, W. E., Vaden-Kiernan, M., & Regier, D. A. (1996). Mental health service use in the community and schools: Results from the four-community MECA study. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 889–897.

    Article  PubMed  CAS  Google Scholar 

  • MTA Cooperative Group. (1999a). A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Multimodal Treatment Study of Children with ADHD. Archives of General Psychiatry, 56, 1073–1086.

    Article  Google Scholar 

  • MTA Cooperative Group. (1999b). Moderators and mediators of treatment response for children with attention-deficit/hyperactivity disorder: The Multimodal Treatment Study of children with Attention-deficit/hyperactivity disorder. Archives of General Psychiatry, 56, 1088–1096.

    Article  Google Scholar 

  • Owens, E. B., Hinshaw, S. P., Kraemer, H. C., Arnold, L. E., Abikoff, H. B., Cantwell, D. P., Conners, C. K., Elliott, G., Greenhill, L. L., Hechtman, L., Hoza, B., Jensen, P. S., March, J. S., Newcorn, J. H., Pelham, W. E., Severe, J. B., Swanson, J. M., Vitiello, B., Wells, K. C., & Wigal, T. (2003). Which treatment for whom for ADHD? Moderators of treatment response in the MTA. Journal of Consulting and Clinical Psychiatry, 71, 540–552.

    Article  Google Scholar 

  • Rieppi, R., Greenhill, L. L., Ford, R. E., Chuang, S., Wu, M., Davies, M., Abikoff, H. B., Arnold, L. E., Conners, C. K., Elliott, G. R., Hechtman, L., Hinshaw, S. P., Hoza, B., Jensen, P. S., Kraemer, H. C., March, J. S., Newcorn, J. H., Pelham, W. E., Severe, J. B., Swanson, J. M., Vitiello, B., Wells, K. C., & Wigal, T. (2002). Socioeconomic status as a moderator of ADHD treatment outcomes. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 269–277.

    Article  PubMed  Google Scholar 

  • Rothwell, P. (2005). External validity of randomised controlled trials: To whom do the results of this trial apply? Lancet, 365, 82–93.

    Article  PubMed  Google Scholar 

  • Shaffer, D., Fisher, P., Dulcan, M. K., Davies, M., Piacentini, J., Schwab-Stone, M. E., Lahey, B. B., Bourdon, K., Jensen, P. S., Bird, H. R., Canino, G., & Regier, D. A. (1996). The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA Study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 865–877.

    Article  PubMed  CAS  Google Scholar 

  • Persons, J. B., & Silberschatz, G. (1998). Are results of randomized controlled trials useful to psychotherapists? Journal of Consulting and Clinical Psychology, 66, 126–135.

    Article  PubMed  CAS  Google Scholar 

  • SPSS Inc. (2003). SPSS base 12.0 application guide. Chicago, IL: Author.

    Google Scholar 

  • Stevens, J., Harman, J. S., & Kelleher, K. J. (2005). Race/ethnicity and insurance status as factors associated with ADHD treatment patterns. Journal of Child and Adolescent Psychopharmacology, 15, 88–96.

    Article  PubMed  Google Scholar 

  • Stirman, S. W., DeRubeis, R. J., Crits-Christoph, P., & Brody, P. E. (2003). Are samples in randomized controlled trials of psychotherapy representative of community outpatients? A new methodology and initial findings. Journal of Consulting and Clinical Psychology, 71, 963–972.

    Article  PubMed  Google Scholar 

  • Swanson, J. M., Kraemer, H. C., Hinshaw, S. P., Arnold, L. E., Conners, C. K., Abikoff, H. B., Clevenger, W., Davies, M., Elliott, G. R., Greenhill, L. L., Hechtman, L., Hoza, B., Jensen, P. S., March, J. S., Newcorn, J. H., Owens, E. B., Pelham, W. E., Schiller, E., Severe, J. B., Simpson, S., Vitiello, B., Wells, K., Wigal, T., & Wu, M. (2001). Clinical relevance of the primary findings of the MTA: Success rates based on severity of ADHD and ODD symptoms at the end of treatment. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 168–179.

    Article  PubMed  CAS  Google Scholar 

  • Wechsler, D. (1991). Wechsler intelligence Scale for Children—Third Edition San Antonio. TX: Psychological Corporation.

    Google Scholar 

  • Westen, D. I. (2006). Patients and treatments in clinical trials are not adequately representative of clinical practice. In J. C. Norcross, L. E. Beutler, & R. F. Levant (Eds.), Evidence-based practices in mental health: Debate and dialogue on the fundamental questions. Washington, DC: American Psychological Association.

    Google Scholar 

  • Zimmerman, M., Chelminski, I., & Posternak, M. A. (2005). Generalizability of antidepressant efficacy trials: Differences between depressed psychiatric outpatients who would versus would not qualify for an efficacy trial. American Journal of Psychiatry, 162, 1370–1372.

    Article  PubMed  Google Scholar 

  • Zimmerman, M., Mattia, J. I., & Posternak, M. A. (2002). Are subjects in pharmacological treatment trials of depression representative of patients in routine clinical practice? American Journal of Psychiatry, 159, 469–473.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jack Stevens.

Additional information

This study was supported in part by NIMH grants MH66960, MH65430, and MH30915. The Multimodal Treatment Study of Children with Attention Deficit/Hyperactivity Disorder (MTA) was conducted and supported by the National Institute of Mental Health (NIMH) in collaboration with the MTA Investigators. The NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) Study was conducted and supported by the National Institute of Mental Health (NIMH) in collaboration with the MECA investigators. This manuscript reflects the views of the authors and may not reflect the opinions or views of all the MTA investigators, the MECA investigators, or NIMH.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Stevens, J., Kelleher, K., Greenhouse, J. et al. Empirical Evaluation of the Generalizability of the Sample from the Multimodal Treatment Study for ADHD. Adm Policy Ment Health 34, 221–232 (2007). https://doi.org/10.1007/s10488-006-0097-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-006-0097-4

Keywords

Navigation