Skip to main content
main-content
Top

Tip

Swipe om te navigeren naar een ander artikel

Gepubliceerd in: Quality of Life Research 10/2022

29-10-2021 | Special Section: Reducing Research Waste in (Health-Related) Quality of Life Research

Power(ful) myths: misconceptions regarding sample size in quality of life research

Auteur: Samantha F. Anderson

Gepubliceerd in: Quality of Life Research | Uitgave 10/2022

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Purpose

Carefully selecting the sample size for a research study is one of the most fundamental ways to utilize resources in an ethical manner, maximize impact and replicability, and minimize research waste when investigating questions relevant to health-related quality of life (HRQOL). Despite an increased focus on sample size in the methodological literature, the topic has received limited attention in the HRQOL field, and there are still misconceptions that can weaken even well-intentioned sample size planning. This article aims to highlight common misconceptions, provide accessible and non-technical corrections to these misconceptions, and show how HRQOL researchers can benefit from a more nuanced understanding of sample size planning.

Method

Misconceptions were identified broadly through examples within the health, psychology, and HRQOL literatures. In examining these misconceptions, study-level (e.g., missing data, multilevel designs, multiple reported outcomes) and field-level (e.g., publication bias, replicability) issues relevant to HRQOL research were considered.

Results

Misconceptions include: (a) researchers should use rules of thumb or the largest sample size possible, (b) sample size planning should always focus on power, (c) planned power = actual power, (d) there is only one level of power per study, and (e) power is only relevant for the individual researcher. Throughout the article, major themes linked to these misconceptions are mapped onto recent HRQOL studies to make the connections more tangible.

Conclusion

By clarifying several challenges and misconceptions regarding sample size planning and statistical power, HRQOL researchers will have the tools needed to augment the research literature in effective and meaningful ways.
Bijlagen
Alleen toegankelijk voor geautoriseerde gebruikers
Literatuur
1.
go back to reference Collins, F. S., & Tabak, L. A. (2014). NIH plans to enhance reproducibility. Nature, 505(7485), 612–613. CrossRef Collins, F. S., & Tabak, L. A. (2014). NIH plans to enhance reproducibility. Nature, 505(7485), 612–613. CrossRef
7.
go back to reference Ioannidis, J. P. A. (2008). Some main problems eroding the credibility and relevance of randomized trials. Bulletin of the NYU hospital for joint diseases, 66(2), 135–139. PubMed Ioannidis, J. P. A. (2008). Some main problems eroding the credibility and relevance of randomized trials. Bulletin of the NYU hospital for joint diseases, 66(2), 135–139. PubMed
18.
go back to reference Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17(2), 137–152. CrossRef Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17(2), 137–152. CrossRef
46.
go back to reference Cook, K. F., Taylor, P. W., Dodd, B. G., Teal, C. R., & McHorney, C. A. (2007). Evidence-based practice for equating health status items: Sample size and IRT model. Journal of Applied Measurement, 8(2), 175–189. PubMed Cook, K. F., Taylor, P. W., Dodd, B. G., Teal, C. R., & McHorney, C. A. (2007). Evidence-based practice for equating health status items: Sample size and IRT model. Journal of Applied Measurement, 8(2), 175–189. PubMed
48.
go back to reference Bakker, M., Hartgerink, C. H. J., Wicherts, J. M., & van der Maas, H. L. J. (2016). Researchers’ intuitions about power in psychological research. Psychological Science, 27(8), 1069–1077. CrossRef Bakker, M., Hartgerink, C. H. J., Wicherts, J. M., & van der Maas, H. L. J. (2016). Researchers’ intuitions about power in psychological research. Psychological Science, 27(8), 1069–1077. CrossRef
51.
go back to reference Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company.
53.
go back to reference Hunter, J. E., & Schmidt, F. L. (2004). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings (2nd ed.). SAGE Publications Inc. CrossRef Hunter, J. E., & Schmidt, F. L. (2004). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings (2nd ed.). SAGE Publications Inc. CrossRef
54.
go back to reference Maxwell, S. E., & Kelley, K. (2011). Ethics and sample size planning. In A. T. Panter & S. K. Sterba (Eds.), Handbook of Ethics in Quantitative Methodology (pp. 159–184). Taylor & Francis. Maxwell, S. E., & Kelley, K. (2011). Ethics and sample size planning. In A. T. Panter & S. K. Sterba (Eds.), Handbook of Ethics in Quantitative Methodology (pp. 159–184). Taylor & Francis.
62.
go back to reference Abelson, R. P. (1997). On the surprising longevity of flogged hotses: Why there is a case for the significance test. Psychological Science, 8(1), 12–15. CrossRef Abelson, R. P. (1997). On the surprising longevity of flogged hotses: Why there is a case for the significance test. Psychological Science, 8(1), 12–15. CrossRef
64.
go back to reference Schwartz, C. E., Stark, R. B., Stucky, B. D., Li, Y., & Rapkin, B. D. (2021). Response-shift effects in neuromyelitis optica spectrum disorder: estimating response-shift-adjusted scores using equating. Quality of Life Research An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02727-8 CrossRef Schwartz, C. E., Stark, R. B., Stucky, B. D., Li, Y., & Rapkin, B. D. (2021). Response-shift effects in neuromyelitis optica spectrum disorder: estimating response-shift-adjusted scores using equating. Quality of Life Research An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02727-8 CrossRef
65.
go back to reference Cohen, J. (1988). Statistical Power Analysis for the Behavioral Science (2nd ed.). Lawrence Erlbaum. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Science (2nd ed.). Lawrence Erlbaum.
66.
go back to reference Amit Aharon, A., Dubovi, I., & Ruban, A. (2021). Differences in mental health and health-related quality of life between the Israeli and Italian population during a COVID-19 quarantine. Quality of Life Research An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02746-5 CrossRef Amit Aharon, A., Dubovi, I., & Ruban, A. (2021). Differences in mental health and health-related quality of life between the Israeli and Italian population during a COVID-19 quarantine. Quality of Life Research An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02746-5 CrossRef
67.
go back to reference Prichard, R. A., Zhao, F.-L., Mcdonagh, J., Goodall, S., Davidson, P. M., Newton, P. J., & Hayward, C. S. (2021). Discrepancies between proxy estimates and patient reported, health related, quality of life: Minding the gap between patient and clinician perceptions in heart failure. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 30(4), 1049–1059. https://​doi.​org/​10.​1007/​s11136-020-02722-z CrossRef Prichard, R. A., Zhao, F.-L., Mcdonagh, J., Goodall, S., Davidson, P. M., Newton, P. J., & Hayward, C. S. (2021). Discrepancies between proxy estimates and patient reported, health related, quality of life: Minding the gap between patient and clinician perceptions in heart failure. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 30(4), 1049–1059. https://​doi.​org/​10.​1007/​s11136-020-02722-z CrossRef
69.
go back to reference Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Sage Publications Inc. Lipsey, M. W. (1990). Design sensitivity: Statistical power for experimental research. Sage Publications Inc.
71.
go back to reference Gibbs, N. M., & Weightman, W. M. (2016). The minimum clinically important difference is fundamentalto all clinical trials. European Journal of Anaesthesiology EJA, 33(1), 60. CrossRef Gibbs, N. M., & Weightman, W. M. (2016). The minimum clinically important difference is fundamentalto all clinical trials. European Journal of Anaesthesiology EJA, 33(1), 60. CrossRef
72.
go back to reference Kelley, K., & Maxwell, S. E. (2012). Sample size. In H. Cooper (Ed.), APA Handbook of Research Methods in Psychology (Vol. 1, pp. 181–202). American Psychological Association. Kelley, K., & Maxwell, S. E. (2012). Sample size. In H. Cooper (Ed.), APA Handbook of Research Methods in Psychology (Vol. 1, pp. 181–202). American Psychological Association.
78.
go back to reference O’Brien, R. G., & Muller, K. E. (1993). 8 Unified Power Analysis for t-Tests through Multivariate Hypotheses. Applied Analysis of Variance in the Behavioral Sciences (pp. 297–344). Marcel Dekker. O’Brien, R. G., & Muller, K. E. (1993). 8 Unified Power Analysis for t-Tests through Multivariate Hypotheses. Applied Analysis of Variance in the Behavioral Sciences (pp. 297–344). Marcel Dekker.
80.
go back to reference Anderson, S. F., & Kelley, K. (2020). Bias-Uncertainty Corrected Sample Size. R Package. Anderson, S. F., & Kelley, K. (2020). Bias-Uncertainty Corrected Sample Size. R Package.
82.
go back to reference Prentice, D. A., & Miller, D. T. (1992). When small effects are impressive. Psychological Bulletin, 112(1), 160–164. CrossRef Prentice, D. A., & Miller, D. T. (1992). When small effects are impressive. Psychological Bulletin, 112(1), 160–164. CrossRef
85.
go back to reference Siembida, E. J., Smith, A. W., Potosky, A. L., Graves, K. D., & Jensen, R. E. (2021). Examination of individual and multiple comorbid conditions and health-related quality of life in older cancer survivors. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 30(4), 1119–1129. https://​doi.​org/​10.​1007/​s11136-020-02713-0 CrossRef Siembida, E. J., Smith, A. W., Potosky, A. L., Graves, K. D., & Jensen, R. E. (2021). Examination of individual and multiple comorbid conditions and health-related quality of life in older cancer survivors. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 30(4), 1119–1129. https://​doi.​org/​10.​1007/​s11136-020-02713-0 CrossRef
86.
go back to reference Timal, R. J., de Gucht, V., Rotmans, J. I., Hensen, L. C. R., Buiten, M. S., de Bie, M. K., & Jukema, J. W. (2021). The impact of transvenous cardioverter-defibrillator implantation on quality of life, depression and optimism in dialysis patients report on the secondary outcome of QOL in the randomized controlled ICD2 trial. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02744-7 CrossRef Timal, R. J., de Gucht, V., Rotmans, J. I., Hensen, L. C. R., Buiten, M. S., de Bie, M. K., & Jukema, J. W. (2021). The impact of transvenous cardioverter-defibrillator implantation on quality of life, depression and optimism in dialysis patients report on the secondary outcome of QOL in the randomized controlled ICD2 trial. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02744-7 CrossRef
88.
go back to reference Rebora, P., Spedale, V., Occhino, G., Luciani, M., Alvaro, R., Vellone, E., & Ausili, D. (2021). Effectiveness of motivational interviewing on anxiety, depression, sleep quality and quality of life in heart failure patients: secondary analysis of the MOTIVATE-HF randomized controlled trial. Quality of Life Research An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-021-02788-3 CrossRef Rebora, P., Spedale, V., Occhino, G., Luciani, M., Alvaro, R., Vellone, E., & Ausili, D. (2021). Effectiveness of motivational interviewing on anxiety, depression, sleep quality and quality of life in heart failure patients: secondary analysis of the MOTIVATE-HF randomized controlled trial. Quality of Life Research An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-021-02788-3 CrossRef
90.
go back to reference Jarab, A. S., Alefishat, E. A., Al-Qerem, W., Mukattash, T. L., & Abu-Zaytoun, L. (2021). Variables associated with poor health-related quality of life among patients with dyslipidemia in Jordan. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02726-9 CrossRef Jarab, A. S., Alefishat, E. A., Al-Qerem, W., Mukattash, T. L., & Abu-Zaytoun, L. (2021). Variables associated with poor health-related quality of life among patients with dyslipidemia in Jordan. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-020-02726-9 CrossRef
91.
go back to reference Maxwell, S. E., Delaney, H., & Kelley, K. (2018). Designing Experiments and Analyzing Data: A Model Comparison Approach (3rd ed.). Routledge. Maxwell, S. E., Delaney, H., & Kelley, K. (2018). Designing Experiments and Analyzing Data: A Model Comparison Approach (3rd ed.). Routledge.
92.
go back to reference Pan, Z., & Kupper, L. L. (1999). Sample size determination for multiple comparison studies treating confidence interval width as random. Statistics in Medicine, 18(12), 1475–1488. CrossRef Pan, Z., & Kupper, L. L. (1999). Sample size determination for multiple comparison studies treating confidence interval width as random. Statistics in Medicine, 18(12), 1475–1488. CrossRef
97.
go back to reference Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. CrossRef Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. CrossRef
99.
go back to reference Enders, C. K. (2010). Applied Missing Data Analysis. Guilford Press. Enders, C. K. (2010). Applied Missing Data Analysis. Guilford Press.
101.
go back to reference Ahmed, A., Saqlain, M., Bashir, N., Dujaili, J., Hashmi, F., Mazhar, F., & Awaisu, A. (2021). Health-related quality of life and its predictors among adults living with HIV/AIDS and receiving antiretroviral therapy in Pakistan. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-021-02771-y CrossRef Ahmed, A., Saqlain, M., Bashir, N., Dujaili, J., Hashmi, F., Mazhar, F., & Awaisu, A. (2021). Health-related quality of life and its predictors among adults living with HIV/AIDS and receiving antiretroviral therapy in Pakistan. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation. https://​doi.​org/​10.​1007/​s11136-021-02771-y CrossRef
106.
go back to reference Julious, S. A., Campbell, M. J., Walker, S. J., George, S. L., & Machin, D. (2000). Sample sizes for cancer trials where Health Related Quality of Life is the primary outcome. The British Journal of Cancer, 83(7), 959–963. CrossRef Julious, S. A., Campbell, M. J., Walker, S. J., George, S. L., & Machin, D. (2000). Sample sizes for cancer trials where Health Related Quality of Life is the primary outcome. The British Journal of Cancer, 83(7), 959–963. CrossRef
107.
go back to reference Julious, S. A., George, S., Machin, D., & Stephens, R. J. (1997). Sample sizes for randomized trials measuring quality of life in cancer patients. Quality of Life Research, 6(2), 109–117. CrossRef Julious, S. A., George, S., Machin, D., & Stephens, R. J. (1997). Sample sizes for randomized trials measuring quality of life in cancer patients. Quality of Life Research, 6(2), 109–117. CrossRef
108.
go back to reference Zimmerman, D., & Zumbo, B. (2015). Resolving the issue of how reliability is related to statistical power adhering to mathematical definitions. Journal of Modern Applied Statistical Methods, 14(2), 9–26. CrossRef Zimmerman, D., & Zumbo, B. (2015). Resolving the issue of how reliability is related to statistical power adhering to mathematical definitions. Journal of Modern Applied Statistical Methods, 14(2), 9–26. CrossRef
110.
go back to reference Kanyongo, G., Brook, G., Kyei-Blankson, L., & Gocmen, G. (2007). Reliability and statistical power how measurement fallibility affects power and required sample sizes for several parametric and nonparametric statistics. Journal of Modern Applied Statistical Methods, 6(1), 81–90. CrossRef Kanyongo, G., Brook, G., Kyei-Blankson, L., & Gocmen, G. (2007). Reliability and statistical power how measurement fallibility affects power and required sample sizes for several parametric and nonparametric statistics. Journal of Modern Applied Statistical Methods, 6(1), 81–90. CrossRef
111.
go back to reference Kelley, K. (2007). Confidence intervals for standardized effect sizes theory, application, and implementation. Journal of Statistical Software, 20(1), 1–24. Kelley, K. (2007). Confidence intervals for standardized effect sizes theory, application, and implementation. Journal of Statistical Software, 20(1), 1–24.
112.
go back to reference Schmidt, F., & Hunter, J. E. (1995). The impact of data-analysis methods on cumulative research knowledge: Statistical significance testing, confidence intervals, and meta-analysis. Evaluation & the Health Professions, 18(4), 408–427. CrossRef Schmidt, F., & Hunter, J. E. (1995). The impact of data-analysis methods on cumulative research knowledge: Statistical significance testing, confidence intervals, and meta-analysis. Evaluation & the Health Professions, 18(4), 408–427. CrossRef
Metagegevens
Titel
Power(ful) myths: misconceptions regarding sample size in quality of life research
Auteur
Samantha F. Anderson
Publicatiedatum
29-10-2021
Uitgeverij
Springer International Publishing
Gepubliceerd in
Quality of Life Research / Uitgave 10/2022
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-021-03020-y