Skip to main content
Top
Gepubliceerd in: Journal of Autism and Developmental Disorders 2/2011

01-02-2011 | Original Paper

From Bayes Through Marginal Utility to Effect Sizes: A Guide to Understanding the Clinical and Statistical Significance of the Results of Autism Research Findings

Auteurs: Domenic V. Cicchetti, Kathy Koenig, Ami Klin, Fred R. Volkmar, Rhea Paul, Sara Sparrow

Gepubliceerd in: Journal of Autism and Developmental Disorders | Uitgave 2/2011

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

The objectives of this report are: (a) to trace the theoretical roots of the concept clinical significance that derives from Bayesian thinking, Marginal Utility/Diminishing Returns in Economics, and the “just noticeable difference”, in Psychophysics. These concepts then translated into: Effect Size (ES), strength of agreement, clinical significance, and related concepts, and made possible the development of Power Analysis; (b) to differentiate clinical significance from statistical significance; and (c) to demonstrate the utility of measures of ES and related concepts for enhancing the meaning of Autism research findings. These objectives are accomplished by applying criteria for estimating clinical significance, and related concepts, to a number of areas of autism research.
Literatuur
go back to reference Bartko, J. J. (1966). The intraclass correlation coefficient as a measure of reliability. Psychological Reports, 19, 3–11.PubMed Bartko, J. J. (1966). The intraclass correlation coefficient as a measure of reliability. Psychological Reports, 19, 3–11.PubMed
go back to reference Bartko, J. J. (1974). Corrective note to “the intraclass correlation coefficient as a measure of reliability”. Psychological Reports, 34, 418. Bartko, J. J. (1974). Corrective note to “the intraclass correlation coefficient as a measure of reliability”. Psychological Reports, 34, 418.
go back to reference Bayes, T. (1763). “An essay, by the late Reverend Mr. Bayes, F.R.S. communicated by Mr. Price, in a letter to John Canton, A.M.F.R.S. Philosophical Transactions, giving some account of the present undertakings, studies and labours of the ingenious in many considerable parts of the world, vol 53, 370–418. Bayes, T. (1763). “An essay, by the late Reverend Mr. Bayes, F.R.S. communicated by Mr. Price, in a letter to John Canton, A.M.F.R.S. Philosophical Transactions, giving some account of the present undertakings, studies and labours of the ingenious in many considerable parts of the world, vol 53, 370–418.
go back to reference Bolanowski, S. J., Jr., & Gescheider, G. A. (Eds.). (1991). Ratio scaling of psychological magnitude: In honor of the memory of S.S. Stevens. Hillsdale, NJ: Lawrence Erlbaum Associates. Bolanowski, S. J., Jr., & Gescheider, G. A. (Eds.). (1991). Ratio scaling of psychological magnitude: In honor of the memory of S.S. Stevens. Hillsdale, NJ: Lawrence Erlbaum Associates.
go back to reference Borenstein, M. (1998). The shift from significance testing to effect size estimation. In A. S. Bellak & M. Hershen (Series Eds.) & N. Schooler (Vol. Ed.), Research and methods: Comprehensive clinical psychology (Vol. 3, pp. 313–349). New York, NY: Pergamon. Borenstein, M. (1998). The shift from significance testing to effect size estimation. In A. S. Bellak & M. Hershen (Series Eds.) & N. Schooler (Vol. Ed.), Research and methods: Comprehensive clinical psychology (Vol. 3, pp. 313–349). New York, NY: Pergamon.
go back to reference Borenstein, M., Rothstein, H., & Cohen, J. (2001). Power and precision: A computer program for statistical power analysis and confidence intervals. Englewood, NJ: Biostat, Inc. Borenstein, M., Rothstein, H., & Cohen, J. (2001). Power and precision: A computer program for statistical power analysis and confidence intervals. Englewood, NJ: Biostat, Inc.
go back to reference Cicchetti, D. V. (1988). When diagnostic agreement is high, but reliability is low: Some paradoxes occurring in joint independent neuropsychology assessments. Journal of Clinical and Experimental Neuropsychology, 10, 605–622.CrossRefPubMed Cicchetti, D. V. (1988). When diagnostic agreement is high, but reliability is low: Some paradoxes occurring in joint independent neuropsychology assessments. Journal of Clinical and Experimental Neuropsychology, 10, 605–622.CrossRefPubMed
go back to reference Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.CrossRef Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.CrossRef
go back to reference Cicchetti, D. V. (2001). The precision of reliability and validity estimates re-visited: istinguishing between clinical and statistical significance of sample size requirements. Journal of Clinical and Experimental Neuropsychology, 23, 695–700.CrossRefPubMed Cicchetti, D. V. (2001). The precision of reliability and validity estimates re-visited: istinguishing between clinical and statistical significance of sample size requirements. Journal of Clinical and Experimental Neuropsychology, 23, 695–700.CrossRefPubMed
go back to reference Cicchetti, D. V. (2008). From Bayes to the just noticeable difference to effect sizes: A note to understanding the clinical and statistical significance of oenologic research findings. Journal of Wine Economics, 3, 185–193.CrossRef Cicchetti, D. V. (2008). From Bayes to the just noticeable difference to effect sizes: A note to understanding the clinical and statistical significance of oenologic research findings. Journal of Wine Economics, 3, 185–193.CrossRef
go back to reference Cicchetti, D. V., Bronen, R., Spencer, S., Haut, S., Berg, A., Oliver, P., et al. (2006). Rating scales, scales of measurement, issues of reliability: Resolving some critical issues for clinicians and researchers. Journal of Nervous and Mental Disease, 194, 557–564.CrossRefPubMed Cicchetti, D. V., Bronen, R., Spencer, S., Haut, S., Berg, A., Oliver, P., et al. (2006). Rating scales, scales of measurement, issues of reliability: Resolving some critical issues for clinicians and researchers. Journal of Nervous and Mental Disease, 194, 557–564.CrossRefPubMed
go back to reference Cicchetti, D. V., Lord, C., Koenig, K., Klin, A., & Volkmar, F. (2008). Reliability of the ADI-R: Multiple examiners evaluate a single case. Journal of Autism and Developmental Disorders, 38, 764–770.CrossRefPubMed Cicchetti, D. V., Lord, C., Koenig, K., Klin, A., & Volkmar, F. (2008). Reliability of the ADI-R: Multiple examiners evaluate a single case. Journal of Autism and Developmental Disorders, 38, 764–770.CrossRefPubMed
go back to reference Cicchetti, D. V., & Sparrow, S. S. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86, 127–137.PubMed Cicchetti, D. V., & Sparrow, S. S. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86, 127–137.PubMed
go back to reference Cicchetti, D. V., & Sparrow, S. S. (1990). Assessment of adaptive behavior in young children. In J. J. Johnson & J. Goldman (Eds.), Developmental assessment in clinical child psychology: A handbook (chap. 8 (pp. 173–196). New York: Pergamon. Cicchetti, D. V., & Sparrow, S. S. (1990). Assessment of adaptive behavior in young children. In J. J. Johnson & J. Goldman (Eds.), Developmental assessment in clinical child psychology: A handbook (chap. 8 (pp. 173–196). New York: Pergamon.
go back to reference Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 23, 37–46.CrossRef Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 23, 37–46.CrossRef
go back to reference Cohen, J. (1965). Some statistical issues in psychological research. In B.B. Wolman (Ed.). Handbook of clinical psychology. Cohen, J. (1965). Some statistical issues in psychological research. In B.B. Wolman (Ed.). Handbook of clinical psychology.
go back to reference Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for partial credit. Psychological Bulletin, 70, 213–220.CrossRefPubMed Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for partial credit. Psychological Bulletin, 70, 213–220.CrossRefPubMed
go back to reference Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press. Cohen, J. (1977). Statistical power analysis for the behavioral sciences. New York, NY: Academic Press.
go back to reference Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Glendale, NJ: Lawrence Erlbaum, Associates. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Glendale, NJ: Lawrence Erlbaum, Associates.
go back to reference Cronbach, L. J. (1950). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.CrossRef Cronbach, L. J. (1950). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.CrossRef
go back to reference Durlak, J. A. (2009). How to select, calculate and interpret Effect Sizes. Journal of Pediatric Psychology, 34, 917–928.CrossRefPubMed Durlak, J. A. (2009). How to select, calculate and interpret Effect Sizes. Journal of Pediatric Psychology, 34, 917–928.CrossRefPubMed
go back to reference Fechner, G. (1907). Elemente der Psychophysik I u. II Leipsig. Germany: Breitkopf & Hartel. Fechner, G. (1907). Elemente der Psychophysik I u. II Leipsig. Germany: Breitkopf & Hartel.
go back to reference Finch, S., & Cumming, G. (2009). Putting research in context: Understanding, confidence intervals from one or more studies. Journal of Pediatric Psychlogy, 34, 903–916.CrossRef Finch, S., & Cumming, G. (2009). Putting research in context: Understanding, confidence intervals from one or more studies. Journal of Pediatric Psychlogy, 34, 903–916.CrossRef
go back to reference Fleiss, J. L. (1975). Measuring agreement between two judges on the presence or absence of a trait. Biometrics, 31, 651–659.CrossRefPubMed Fleiss, J. L. (1975). Measuring agreement between two judges on the presence or absence of a trait. Biometrics, 31, 651–659.CrossRefPubMed
go back to reference Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613–619.CrossRef Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33, 613–619.CrossRef
go back to reference Klin, A., Lang, J., Cicchetti, D. V., & Volkmar, F. (2000). Inter-rater reliability of clinical diagnosis and DSM-IV criteria for autistic disorder: Results of the DSM-IV autism field trial. Journal of Autism and Developmental Disorders, 30, 163–167.CrossRefPubMed Klin, A., Lang, J., Cicchetti, D. V., & Volkmar, F. (2000). Inter-rater reliability of clinical diagnosis and DSM-IV criteria for autistic disorder: Results of the DSM-IV autism field trial. Journal of Autism and Developmental Disorders, 30, 163–167.CrossRefPubMed
go back to reference Kraemer, H. C., Morgan, G. H., Leech, N. L., Gliner, J. A., Vaske, J. J., & Harmon, R. J. (2003). Measures of clinical significance. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 1524–1529.CrossRefPubMed Kraemer, H. C., Morgan, G. H., Leech, N. L., Gliner, J. A., Vaske, J. J., & Harmon, R. J. (2003). Measures of clinical significance. Journal of the American Academy of Child and Adolescent Psychiatry, 42, 1524–1529.CrossRefPubMed
go back to reference Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefPubMed Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefPubMed
go back to reference Laupacis, A., Sackett, D. L., & Roberts, R. S. (1988). An assessment of clinically useful measures of the consequences of treatment. New England Journal of Medicine, 318, 1728–1733.CrossRefPubMed Laupacis, A., Sackett, D. L., & Roberts, R. S. (1988). An assessment of clinically useful measures of the consequences of treatment. New England Journal of Medicine, 318, 1728–1733.CrossRefPubMed
go back to reference Neyman, J., & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference. Biometrika, 20A, 175–240. and 263–294. Neyman, J., & Pearson, E. S. (1928). On the use and interpretation of certain test criteria for purposes of statistical inference. Biometrika, 20A, 175–240. and 263–294.
go back to reference Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Transactions of the Royal Society of London Series A, 231, 289–337.CrossRef Neyman, J., & Pearson, E. S. (1933). On the problem of the most efficient tests of statistical hypotheses. Transactions of the Royal Society of London Series A, 231, 289–337.CrossRef
go back to reference Nunnally, J. C. (1978). Psychometric theory. New York, NY: McGraw-Hill. Nunnally, J. C. (1978). Psychometric theory. New York, NY: McGraw-Hill.
go back to reference Paul, R., Chawarska, K., Cicchetti, D., & Volkmar, F. (2008). Language outcomes of toddlers with autism spectrum disorders: A two year follow-up. Autism Research, 1(2), 97–107.CrossRefPubMed Paul, R., Chawarska, K., Cicchetti, D., & Volkmar, F. (2008). Language outcomes of toddlers with autism spectrum disorders: A two year follow-up. Autism Research, 1(2), 97–107.CrossRefPubMed
go back to reference Paul, R., Miles-Orlovsky, S., Marcinko, H. C., & Volkmar, F. (2010). Conversational behaviors in youth with high-functioning ASD and Asperger Syndrome. Journal of Autism and Developmental Disorders, 39, 115–125.CrossRef Paul, R., Miles-Orlovsky, S., Marcinko, H. C., & Volkmar, F. (2010). Conversational behaviors in youth with high-functioning ASD and Asperger Syndrome. Journal of Autism and Developmental Disorders, 39, 115–125.CrossRef
go back to reference Rosenthal, R. (1991). Meta-analytic procedures for social research. Applied Social Research Methods Series, 6, 1–155. Rosenthal, R. (1991). Meta-analytic procedures for social research. Applied Social Research Methods Series, 6, 1–155.
go back to reference Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland II: A revision of the vineland adaptive behavior scales: I. Survey/caregiver form (2nd edn). Circle Pines, Minnesota: American Guidance Service. Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland II: A revision of the vineland adaptive behavior scales: I. Survey/caregiver form (2nd edn). Circle Pines, Minnesota: American Guidance Service.
go back to reference Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2008). Vineland II: A revision of the vineland adaptive behavior scales: II. Expanded form (2nd edn). Circle Pines, Minnesota: American Guidance Service. Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2008). Vineland II: A revision of the vineland adaptive behavior scales: II. Expanded form (2nd edn). Circle Pines, Minnesota: American Guidance Service.
go back to reference Stevens, S. S. (1946). On the theory of scales of measurement. Science, 10, 677–680.CrossRef Stevens, S. S. (1946). On the theory of scales of measurement. Science, 10, 677–680.CrossRef
go back to reference Stevens, S. S. (1951). Mathematics, measurement, and psychophysics. In Stevens, S. S. (Ed.). Handbook of experimental psychology, chap. 1 (pp. 1–49). New York, NY: Wiley. Stevens, S. S. (1951). Mathematics, measurement, and psychophysics. In Stevens, S. S. (Ed.). Handbook of experimental psychology, chap. 1 (pp. 1–49). New York, NY: Wiley.
go back to reference Stone, H., & Sidel, J.l. (Eds.). (1993). Sensory evaluation practices (2nd ed.). New York, NY: Academic Press. Stone, H., & Sidel, J.l. (Eds.). (1993). Sensory evaluation practices (2nd ed.). New York, NY: Academic Press.
go back to reference Von Wieser, F. (1893). Natural value (English ed ed.). New York, NY: MacMillan. Von Wieser, F. (1893). Natural value (English ed ed.). New York, NY: MacMillan.
Metagegevens
Titel
From Bayes Through Marginal Utility to Effect Sizes: A Guide to Understanding the Clinical and Statistical Significance of the Results of Autism Research Findings
Auteurs
Domenic V. Cicchetti
Kathy Koenig
Ami Klin
Fred R. Volkmar
Rhea Paul
Sara Sparrow
Publicatiedatum
01-02-2011
Uitgeverij
Springer US
Gepubliceerd in
Journal of Autism and Developmental Disorders / Uitgave 2/2011
Print ISSN: 0162-3257
Elektronisch ISSN: 1573-3432
DOI
https://doi.org/10.1007/s10803-010-1035-6

Andere artikelen Uitgave 2/2011

Journal of Autism and Developmental Disorders 2/2011 Naar de uitgave