Skip to main content
Top
Gepubliceerd in:

01-01-2015 | Brief Communication

Sample size for pre-tests of questionnaires

Auteurs: Thomas V. Perneger, Delphine S. Courvoisier, Patricia M. Hudelson, Angèle Gayet-Ageron

Gepubliceerd in: Quality of Life Research | Uitgave 1/2015

Log in om toegang te krijgen
share
DELEN

Deel dit onderdeel of sectie (kopieer de link)

  • Optie A:
    Klik op de rechtermuisknop op de link en selecteer de optie “linkadres kopiëren”
  • Optie B:
    Deel de link per e-mail

Abstract

Purpose

To provide guidance regarding the desirable size of pre-tests of psychometric questionnaires, when the purpose of the pre-test is to detect misunderstandings, ambiguities, or other difficulties participants may encounter with instrument items (called «problems»).

Methods

We computed (a) the power to detect a problem for various levels of prevalence and various sample sizes, (b) the required sample size to detect problems for various levels of prevalence, and (c) upper confidence limits for problem prevalence in situations where no problems were detected.

Results

As expected, power increased with problem prevalence and with sample size. If problem prevalence was 0.05, a sample of 10 participants had only a power of 40 % to detect the problem, and a sample of 20 achieved a power of 64 %. To achieve a power of 80 %, 32 participants were necessary if the prevalence of the problem was 0.05, 16 participants if prevalence was 0.10, and 8 if prevalence was 0.20. If no problems were observed in a given sample, the upper limit of a two-sided 90 % confidence interval reached 0.26 for a sample size of 10, 0.14 for a sample size of 20, and 0.10 for a sample of 30 participants.

Conclusions

Small samples (5–15 participants) that are common in pre-tests of questionaires may fail to uncover even common problems. A default sample size of 30 participants is recommended.
Literatuur
1.
go back to reference Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., et al. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68, 109–130.CrossRef Presser, S., Couper, M. P., Lessler, J. T., Martin, E., Martin, J., Rothgeb, J. M., et al. (2004). Methods for testing and evaluating survey questions. Public Opinion Quarterly, 68, 109–130.CrossRef
2.
go back to reference Converse, J. M., & Presser, S. (1986). Survey questions. Handcrafting the standardized questionnaire. Newbury Park: Sage Publications Inc.CrossRef Converse, J. M., & Presser, S. (1986). Survey questions. Handcrafting the standardized questionnaire. Newbury Park: Sage Publications Inc.CrossRef
3.
go back to reference Backstrom, C. H., & Hursch-César, G. (1981). Survey Research (2nd ed.). New York: Macmillan Publishing Company. Backstrom, C. H., & Hursch-César, G. (1981). Survey Research (2nd ed.). New York: Macmillan Publishing Company.
4.
go back to reference Streiner, D. L., & Norman, G. R. (2003). Health Measurement Scales. A practical guide to their development and use (3rd ed.). Oxford: Oxford University Press. Streiner, D. L., & Norman, G. R. (2003). Health Measurement Scales. A practical guide to their development and use (3rd ed.). Oxford: Oxford University Press.
5.
go back to reference Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken NJ: Wiley. Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken NJ: Wiley.
6.
go back to reference Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71, 287–311.CrossRef Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71, 287–311.CrossRef
7.
go back to reference DeVellis, R. F. (2012). Scale development. Theory and applications (3rd ed.). Los Angeles, Newbury Park: Sage Publications Inc. DeVellis, R. F. (2012). Scale development. Theory and applications (3rd ed.). Los Angeles, Newbury Park: Sage Publications Inc.
8.
go back to reference Wild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S., Verjee-Lorenz, A., et al. (2005). Principles of good pratice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: Report of the ISPOR task force for translation and cultural adaptation. Value Health, 8, 94–104.PubMedCrossRef Wild, D., Grove, A., Martin, M., Eremenco, S., McElroy, S., Verjee-Lorenz, A., et al. (2005). Principles of good pratice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: Report of the ISPOR task force for translation and cultural adaptation. Value Health, 8, 94–104.PubMedCrossRef
9.
go back to reference Patrick, D. L., Burke, L. B., Gwaltney, C. J., Kline Leidy, N., Martin, L., Molsen, E., et al. (2011). Content validity—Establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2—Assessing respondent understanding. Value Health, 14, 978–988.PubMedCrossRef Patrick, D. L., Burke, L. B., Gwaltney, C. J., Kline Leidy, N., Martin, L., Molsen, E., et al. (2011). Content validity—Establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2—Assessing respondent understanding. Value Health, 14, 978–988.PubMedCrossRef
10.
go back to reference Sprangers, M. A., Cull, A., Groenvold, M., Bjordal, K., Blazeby, J., & Aaronson, N. K. (1998). The European Organization for Research and Treatment of Cancer approach to developing questionnaire modules: An update and overview. Quality of Life Research, 7, 291–300.PubMedCrossRef Sprangers, M. A., Cull, A., Groenvold, M., Bjordal, K., Blazeby, J., & Aaronson, N. K. (1998). The European Organization for Research and Treatment of Cancer approach to developing questionnaire modules: An update and overview. Quality of Life Research, 7, 291–300.PubMedCrossRef
11.
go back to reference Fayers, P. M., & Machin, D. (2000). Quality of life. Assessment, analysis and interpretation. New York: Wiley. Fayers, P. M., & Machin, D. (2000). Quality of life. Assessment, analysis and interpretation. New York: Wiley.
12.
go back to reference Beaton, D. E., Bombardier, C., Guillemin, F., & Ferrsaz, M. C. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25, 3186–3191.PubMedCrossRef Beaton, D. E., Bombardier, C., Guillemin, F., & Ferrsaz, M. C. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25, 3186–3191.PubMedCrossRef
13.
go back to reference Blair, J., & Conrad, F. G. (2011). Sample size for cognitive interview pretesting. Public Opinion Quarterly, 75, 636–658.CrossRef Blair, J., & Conrad, F. G. (2011). Sample size for cognitive interview pretesting. Public Opinion Quarterly, 75, 636–658.CrossRef
14.
15.
go back to reference Clopper, C., & Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26, 404–413.CrossRef Clopper, C., & Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26, 404–413.CrossRef
16.
go back to reference StatXact version 4. Cytel software, Cambridge, MA. StatXact version 4. Cytel software, Cambridge, MA.
17.
go back to reference Perneger, T. V., Kossovsky, M. P., Cathieni, F., di Florio, V., & Burnand, B. (2003). A randomized trial of four patient satisfaction questionnaires. Medical Care, 41, 1343–1352.PubMedCrossRef Perneger, T. V., Kossovsky, M. P., Cathieni, F., di Florio, V., & Burnand, B. (2003). A randomized trial of four patient satisfaction questionnaires. Medical Care, 41, 1343–1352.PubMedCrossRef
18.
go back to reference Cleopas, A., Kolly, V., & Perneger, T. V. (2006). Longer response scales improved the acceptability and performance of the Nottingham Health Profile. Journal of Clinical Epidemiology, 59, 1183–1190.PubMedCrossRef Cleopas, A., Kolly, V., & Perneger, T. V. (2006). Longer response scales improved the acceptability and performance of the Nottingham Health Profile. Journal of Clinical Epidemiology, 59, 1183–1190.PubMedCrossRef
Metagegevens
Titel
Sample size for pre-tests of questionnaires
Auteurs
Thomas V. Perneger
Delphine S. Courvoisier
Patricia M. Hudelson
Angèle Gayet-Ageron
Publicatiedatum
01-01-2015
Uitgeverij
Springer International Publishing
Gepubliceerd in
Quality of Life Research / Uitgave 1/2015
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI
https://doi.org/10.1007/s11136-014-0752-2