Skip to main content
SpringerLink
Log in

An Introduction to Modern Statistical Methods in HCI

  • Chapter
  • First Online:
Modern Statistical Methods for HCI

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

This chapter explains why we think statistical methodology matters so much to the HCI community and why we should attempt to improve it. It introduces some flaws in the well-accepted methodology of Null Hypothesis Significance Testing and briefly introduces some alternatives. Throughout the book we aim to critically evaluate current practices in HCI and support a less rigid, procedural view of statistics in favour of “fair statistical communication”. Each chapter provides scholars and practitioners with the methods and tools to do so.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    While technically the often applied methods are an—arguably erroneous—hybrid between methods introduced by Neymann-Pearson and Fisher, we focus here on the common practice.

References

  • American Psychological Association (2009) Publication Manual of the American Psychological Association, APA, 6th edn, p. 272

    Google Scholar 

  • Baguley T (2012) Serious stats: a guide to advanced statistics for the behavioral sciences. Palgrave Macmillan, Basingstoke

    Google Scholar 

  • Barnard GA, Bayes T (1958) Studies in the history of probability and statistics: IX. In: Thomas Bayes’s essay towards solving a problem in the doctrine of chances. Biometrika 45(3/4):293–315

    Google Scholar 

  • Box JF (1987) Guinness, gosset, fisher, and small samples. Stat Sci 2(1):45–52

    Google Scholar 

  • Cairns P (2007) HCI... not as it should be: inferential statistics in HCI research. In: Proceedings of HCI 2007. Lancaster, UK, pp 195–201

    Google Scholar 

  • Chan ISF, Zhang Z (1999) Test-based exact confidence intervals for the difference of two binomial proportions. Biometrics 55(4):1202–1209

    Google Scholar 

  • Cohen J (1992) Statistical power analysis. In: Current directions in psychological science, pp 98–101

    Google Scholar 

  • Conley T, Hansen C, Mcculloch R, Rossi P (2008) A semi-parametric Bayesian approach to the instrumental variable problem. J Econom 144(1):276–305

    Google Scholar 

  • Dienes Z (2008) Understanding psychology as a science: an introduction to scientific and statistical inference, 1st edn. Palgrave Macmillan, Basingstoke, p 150

    Google Scholar 

  • Dunlop MD, Baillie M(2009) Paper rejected (p> 0.05): an introduction to the debate on appropriateness of null-hypothesis testing. Int J Mob Hum Comput Interact 1(3):86

    Google Scholar 

  • Fisher RA (1934) Statistical methods for research workers, 5th edn. Oliver and Boyd, Edinburgh

    MATH  Google Scholar 

  • Fox RJ, Dimmic MW (2006) A two-sample Bayesian t-test for microarray data. BMC Bioinform 7:126

    Google Scholar 

  • Gelman A (2005) Analysis of variance: why it is more important than ever. Ann Stat 33(1):1–31

    Google Scholar 

  • Gelman A, David W (2009) Of beauty, sex, and power: statistical challenges in estimating small effects. Am Sci 97:310–316

    Article  Google Scholar 

  • Gray WD, Salzman MC (1998) Damaged merchandise? a review of experiments that compare usability evaluation methods. Hum Comput Interact 13:203–261

    Google Scholar 

  • Ioannidis’s JPA (2005) Why most published research findings are false. In: Jantsch W, Schaffler F (eds.) PLoS Med 2(8), e124

    Google Scholar 

  • Kampenes V, Dyba T, Hannay J, Sjoberg D (2007) A systematic review of effect size in software engineering experiments. Inform Softw Technol 49(11–12):1073–1086

    Google Scholar 

  • Kaptein MC, Robertson J (2012) Rethinking statistical methods for HCI. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, CHI 212. ACM Press, New York, New York, USA, pp 1105–1114

    Google Scholar 

  • Kruschke JK (2010) What to believe: Bayesian methods for data analysis. Trends Cogn Sci 14(7):293–300

    Google Scholar 

  • Kruschke JK (2011a) Bayesian assessment of null values via parameter estimation and model comparison. Perspect Psychol Sci 6(3):299–312

    Google Scholar 

  • Kruschke JK (2011b) Doing bayesian data analysis: a tutorial with R and BUGS. Elsevier, Burlington

    MATH  Google Scholar 

  • Maxwell SE (2004) The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychol Methods 9(2):147

    Google Scholar 

  • Oakes M (1986) Statistical inference: a commentary for the social and behavioural sciences, p. 196. Wiley, New York

    Google Scholar 

  • Paternoster R, Brame R, Mazerolle P, Piquero A (1998) Using the correct statistical test for the equality of regression coefficients. Criminology 36(4):859–866

    Google Scholar 

  • Pearson K, Fisher RA, Inman HF (1994) Karl Pearson and R.A. Fisher on statistical tests: a 1935 exchange from nature. Am Stat 48(1):2–11

    Google Scholar 

  • Rosenthal R (1979) The file drawer problem and tolerance for null results. Psychol Bull 86(3):638

    Google Scholar 

  • Rouder JN, Speckman PL, Sun D, Morey RD, Iverson G (2009) Bayesian t tests for accepting and rejecting the null hypothesis. Psychon Bull Rev 16(2):225–237

    Google Scholar 

  • Seaman, C. B. (1999). Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572

    Google Scholar 

  • Simmons JP, Nelson LD, Simonsohn U (2011) False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 22(11):1359–1366

    Google Scholar 

  • Student (1908) The probable error of a mean. Biometrika 6:1–25

    Google Scholar 

  • Wagenmakers E, Wetzels R, Borsboom D, Van der Maas HLJ (2011) Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem. J Pers Soc Psychol 100:426–443

    Google Scholar 

  • Ziliak S, McCloskey D (2008) The cult of statistical significance: how the standard error costs us jobs, justice and lives. University of Michigan Press, Ann Arbor

    MATH  Google Scholar 

  • Ziliak ST (2008) The Cult of statistical significance: how the standard error costs us jobs, justice, and lives. J Econ Lit 47(2):499–503

    Google Scholar 

  • Ziliak ST, McCloskey DN (2004) Size matters: the standard error. J Socio-Econ 33(5):527–546

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Moray House School of Education, Edinburgh University, Edinburgh, UK

    Judy Robertson

  2. Donders Centre for Cognition, Radboud University Nijmegen, Nijmegen, The Netherlands

    Maurits Kaptein

Authors
  1. Judy Robertson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurits Kaptein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Judy Robertson .

Editor information

Editors and Affiliations

  1. Moray School of Education, Edinburgh University, Edinburgh, United Kingdom

    Judy Robertson

  2. Donders Centre for Cognition, Radboud University Nijmegen, Tilburg, The Netherlands

    Maurits Kaptein

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Robertson, J., Kaptein, M. (2016). An Introduction to Modern Statistical Methods in HCI. In: Robertson, J., Kaptein, M. (eds) Modern Statistical Methods for HCI. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-26633-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26633-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26631-2

  • Online ISBN: 978-3-319-26633-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation