Skip to main content
SpringerLink
Log in

Validity of Consumer Activity Wristbands and Wearable EEG for Measuring Overall Sleep Parameters and Sleep Structure in Free-Living Conditions

  • Research Article
  • Published:
Journal of Healthcare Informatics Research Aims and scope Submit manuscript

Abstract

Consumer sleep tracking technologies offer an unobtrusive and cost-efficient way to monitor sleep in free-living conditions. Technological advances in hardware and software have significantly improved the functionality of the new gadgets that recently appeared in the market. However, whether the latest gadgets can provide valid measurements on overall sleep parameters and sleep structure such as deep and REM sleep has not been examined. In this study, we aimed to investigate the validity of the latest consumer sleep tracking devices including an activity wristband Fitbit Charge 2 and a wearable EEG-based eye mask Neuroon in comparison to a medical sleep monitor. First, we confirmed that Fitbit Charge 2 can automatically detect the onset and offset of sleep with reasonable accuracy. Second, analysis found that both consumer devices produced comparable results in measuring total sleep duration and sleep efficiency compared to the medical device. In addition, Fitbit accurately measured the number of awakenings, while Neuroon with good signal quality had satisfactory performance on total awake time and sleep onset latency. However, measuring sleep structure including light, deep, and REM sleep remains to be challenging for both consumer devices. Third, greater discrepancies were observed between Neuroon and the medical device in nights with more disrupted sleep and when the signal quality was poor, but no trend was observed in Fitbit Charge 2. This study suggests that current consumer sleep tracking technologies may be immature for diagnosing sleep disorders, but they are reasonably satisfactory for general purpose and non-clinical use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Liang Z, Chapa-Martell MA (2015) Framing self-quantification for individual-level preventive health care. In: Proceedings of the International Conference on Health Informatics, pp 336–343

  2. Dittmar A, Axisa F, Delhomme G, Gehin C (2004) New concepts and technologies in home care and ambulatory monitoring. Stud Health Technol Inform 108:9–35

    Google Scholar 

  3. Mantua J, Gravel N, Spencer R (2016) Reliability of sleep measures from four personal health monitoring device compared to research-based actigraphy and polysomnography. Sensors 16:646

    Article  Google Scholar 

  4. Meltzer LJ, Hiruma LS, Avis K, Montgomery-Downs H, Valentin J (2015) Comparison of a commercial accelerometer with polysomnography and actigraphy in children and adolescents. Sleep 38(8):1323–1330. https://doi.org/10.5665/sleep.4918

    Article  Google Scholar 

  5. De Zambotti M, Baker FC, Colrain IM (2015) Validation of sleep-tracking technology compared with polysomnography in adolescents. Sleep 38(9):1461–1468. https://doi.org/10.5665/sleep.4990

    Article  Google Scholar 

  6. De Zambotti M, Godino JG, Baker FC et al (2016) The boom in wearable technology: cause for alarm or just what is needed to better understand sleep? Sleep 39(9):1761–1762. https://doi.org/10.5665/sleep.6108

    Article  Google Scholar 

  7. Buysse DJ (2014) Sleep health: can we define it? Does it matter? Sleep 37(1):9–17. https://doi.org/10.5665/sleep.3298

    Article  Google Scholar 

  8. Coates TJ, Killen JD, George J, Marchini E, Silverman S, Thoresen C (1982) Estimating sleep parameters: a multitrait-multimethod analysis. J Consult Clin Psychol 50(3):345–352. https://doi.org/10.1037/0022-006X.50.3.345

    Article  Google Scholar 

  9. Carskadon MA, Dement WC (2015) Normal human sleep: an overview. In: Kryger MH, Roth T, Dement WC (eds) Principle and practice of sleep medicine, 4th edn. Elsevier Saunders, Philadelphia, pp 13–23

    Google Scholar 

  10. Hall M (2010) Behavioral medicine and sleep: concept, measures and methods. In: Steptoe A (ed) Handbook of behavioral medicine: methods and applications. Springer, New York, pp 749–765. https://doi.org/10.1007/978-0-387-09488-5_49

    Chapter  Google Scholar 

  11. Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ (1989) The Pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res 28(2):193–213. https://doi.org/10.1016/0165-1781(89)90047-4

    Article  Google Scholar 

  12. Carney CE, Buysse DJ, Ancoli-Israel S, Edinger JD, Krystal AD, Lichstein KL, Morin CM (2012) The consensus sleep diary: standardizing prospective sleep self-monitoring. Sleep 35(2):287–302. https://doi.org/10.5665/sleep.1642

    Article  Google Scholar 

  13. Hublin C, Partinen M, Koskenvuo M, Kaprio J (2007) Sleep and mortality: a population-based 22-year follow-up study. Sleep 30(10):1245–1253. https://doi.org/10.1093/sleep/30.10.1245

    Article  Google Scholar 

  14. Kushida CA, Littner MR, Morgenthaler T, Alessi CA, Bailey D, Coleman J Jr, Friedman L, Hirshkowitz M, Kapen S, Kramer M, Lee-Chiong T, Loube DL, Owens J, Pancer JP, Wise M (2005) Practice parameters for the indications for polysomnography and related procedures: an update for 2005. Sleep 28(4):499–519. https://doi.org/10.1093/sleep/28.4.499

    Article  Google Scholar 

  15. Ohayon M, Wickwire EM, Hirshkowitz M, Albert SM, Avidan A, Daly FJ, Dauvilliers Y, Ferri R, Fung C, Gozal D, Hazen N, Krystal A, Lichstein K, Mallampalli M, Plazzi G, Rawding R, Scheer FA, Somers V, Vitiello MV (2017) National sleep foundation’s sleep quality recommendations: first report. Sleep Health 3(1):6–19. https://doi.org/10.1016/j.sleh.2016.11.006

    Article  Google Scholar 

  16. Hirshkowitz M (2004) Normal human sleep: an overview. Med Clin North Am 88:51–65

    Article  Google Scholar 

  17. Keenan SA (1999) Normal human sleep. Respir Care Clin N Am 5:319–331

    Google Scholar 

  18. Sadeh A (2011) The role and validity of actigraphy in sleep medicine: an update. Sleep Med Rev 15(4):259–267. https://doi.org/10.1016/j.smrv.2010.10.001

    Article  Google Scholar 

  19. Ancoli-Israel S, Cole R, Alessi C, Chambers M, Moorcroft W, Pollak C (2003) The role of actigraphy in the study of sleep and circadian rhythms. American Academy of sleep medicine review paper. Sleep 26(3):342–392. https://doi.org/10.1093/sleep/26.3.342

    Article  Google Scholar 

  20. Littner M, Kushida CA, Anderson WM et al (2002) Practice parameters for the role of actigraphy in the study of sleep and circadian rhythms: an update for 2002. Sleep 26(3):337–341

    Article  Google Scholar 

  21. Vitiello MV, Larsen LH, Drolet G et al (2002) Gender differences in subjective-objective sleep relationships in non-complaining healthy older adults. Sleep 25S:A61

    Google Scholar 

  22. Van Ravesteyn LM, Tulen JH, Kamperman AM et al (2014) Perceived sleep quality is worse than objective parameters of sleep in pregnant women with a mental disorder. J Clin Sleep Med 10(10):1137–1141. https://doi.org/10.5664/jcsm.4118

    Google Scholar 

  23. Lund HG, Rybarczyk BD, Perrin PB, Leszczyszyn D, Stepanski E (2013) The discrepancy between subjective and objective measures of sleep in older adults receiving CBT for comorbid insomnia. J Clin Psychol 69(10):1108–1120. https://doi.org/10.1002/jclp.21938

    Article  Google Scholar 

  24. Most EIS, Aboudan S, Scheltens P, van Someren EJW (2012) Discrepancy between subjective and objective sleep disturbances in early- and moderate-stage Alzheimer disease. Am J Geriatr Psychiatry 20(6):460–467. https://doi.org/10.1097/JGP.0b013e318252e3ff

    Article  Google Scholar 

  25. O’Donnel D, Silva EJ, Munch M et al (2009) Comparison of subjective and objective assessments of sleep in healthy older subjectis without sleep complaints. J Sleep Res 18(2):254–263. https://doi.org/10.1111/j.1365-2869.2008.00719.x

    Article  Google Scholar 

  26. Tsuchiyama K, Nagayama H, Kudo K, Kojima K, Yamada K (2003) Discrepancy between subjective and objective sleep in patients with depression. Psychiatry Clin Neurosci 57(3):259–264. https://doi.org/10.1046/j.1440-1819.2003.01114.x

    Article  Google Scholar 

  27. Kolla BP, Mansukhani S, Mansukhani MP (2016) Consumer sleep tracking devices: a review of mechanisms, validity and utility. Expert Rev Med Devices 13(5):497–506. https://doi.org/10.1586/17434440.2016.1171708

    Article  Google Scholar 

  28. Shelgikar AV, Anderson PF, Stephens MR (2016) Sleep tracking, wearable technology, and opportunities for research and clinical care. Chest 150(3):732–743. https://doi.org/10.1016/j.chest.2016.04.016

    Article  Google Scholar 

  29. Ong AA, Gillespie MB (2016) Overview of smartphone applications for sleep analysis. World J Otorhinolaryngol Head Neck Surg 2(1):45–49. https://doi.org/10.1016/j.wjorl.2016.02.001

    Article  Google Scholar 

  30. Ko PT, Kientz JA, Choe EK, Kay M, Landis CA, Watson NF (2015) Consumer sleep technologies: a review of the landscape. J Clin Sleep Med 11(12):1455–1461. https://doi.org/10.5664/jcsm.5288

    Article  Google Scholar 

  31. Kelly JM, Strecker RE, Bianchi MT (2012) Recent developments in home sleep-monitoring devices. International Scholarly Research Network Neurology, Article ID 768794, 10 pages

  32. Montgomery-Downs HE, Insana SP, Bond JA (2012) Movement toward a novel activity monitoring device. Sleep Breath 6(3):913–917

    Article  Google Scholar 

  33. De Zambotti M, Claudatos S, Inkelis S et al (2015) Evaluation of a consumer fitness-tracking device to access sleep in adults. Chronobiol Int 32(7):1024–1028. https://doi.org/10.3109/07420528.2015.1054395

    Article  Google Scholar 

  34. Bhat S, Ferraris A, Gupta D, Mozafarian M, DeBari VA, Gushway-Henry N, Gowda SP, Polos PG, Rubinstein M, Seidu H, Chokroverty S (2015) Is there a clinical role for smartphone sleep apps? Comparison of sleep cycle detection by a smartphone application to polysomnography. J Clin Sleep Med 11(7):709–715. https://doi.org/10.5664/jcsm.4840

    Google Scholar 

  35. Shirazi AS, Clawson J, Hassanpour Y, Tourian MJ et al (2013) Already up? Using mobile phones to track & share sleep behavior. Int J Hum Comput Stud 71(9):878–888. https://doi.org/10.1016/j.ijhcs.2013.03.001

    Article  Google Scholar 

  36. Behar J, Roebuck A, Shahid M, Daly J, Hallack A, Palmius N, Stradling J, Clifford GD (2015) SleepAp: an automated obstructive sleep apnoea scorning application for smartphones. IEEE J Biomed Health Inform 19(1):325–331. https://doi.org/10.1109/JBHI.2014.2307913

    Article  Google Scholar 

  37. Liang Z, Ploderer B (2016) Sleep tracking in the real world: a qualitative study into barriers for improving sleep. In: Proceedings of OZCHI 2016: 537–541

  38. Liang Z, Ploderer B, Chapa-Martell MA (2017) Is Fitbit fit for sleep-tracking? Sources of measurement errors and proposed countermeasures. In: Proceedings of Pervasive Health 2017

  39. Natale V, Leger D, Martoni M et al (2014) The role of actigraphy in the assessment of primary insomnia: a retrospective study. Sleep Med 15(1):111–115. https://doi.org/10.1016/j.sleep.2013.08.792

    Article  Google Scholar 

  40. Suh S, Nowakowski S, Bernert RA, Ong JC, Siebern AT, Dowdle CL, Manber R (2012) Clinical significance of night-to-night sleep variability in insomnia. Sleep Med 13(5):469–475. https://doi.org/10.1016/j.sleep.2011.10.034

    Article  Google Scholar 

  41. Buysse DJ, Cheng Y, Germain A, Moul DE, Franzen PL, Fletcher M, Monk TH (2010) Night-to-night sleep variability in older adults with and without chronic insomnia. Sleep Med 11(1):56–64. https://doi.org/10.1016/j.sleep.2009.02.010

    Article  Google Scholar 

  42. Natale V, Plazzi G, Martoni M (2009) Actigraphy in the assessment of insomnia: a quantitative approach. Sleep 32(6):767–771. https://doi.org/10.1093/sleep/32.6.767

    Article  Google Scholar 

  43. Fitbit Charge 2: How do I track my sleep? https://help.fitbit.com/articles/en_US/Help_article/1314/?l=en_US&c=Topics%3ASleep&p=charge_2&fs=Search&pn=1#Whatisthedifference

  44. Neuroon sleep analytics. https://neuroon.jp/features/sleep-analytics/

  45. Yoshida M, Shinohara H, Kodama H (2015) Assessment of nocturnal sleep architecture by actigraphy and one-channel electroencephalography in early infancy. Early Hum Dev 91(9):519–526. https://doi.org/10.1016/j.earlhumdev.2015.06.005

    Article  Google Scholar 

  46. Rosenberg RS, Van Hout S (2013) The American Academy of Sleep Medicine inter-scorer reliability program: sleep stage scoring. J Clin Sleep Med 9(1):81–87. https://doi.org/10.5664/jcsm.2350

    Google Scholar 

  47. Liang Z, Ploderer B, Liu W, Nagata Y, Bailey J, Kulik L, Li X (2016) SleepExplorer: a visualization tool to make sense of correlations between personal sleep data and contextual factors. Pers Ubiquit Comput 20(6):985–1000. https://doi.org/10.1007/s00779-016-0960-6

    Article  Google Scholar 

  48. Iber C, Ancoli-Israel S, Chesson A et al (2007) The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications, 1st edn. American Academy of Sleep Medicine, Westchester

    Google Scholar 

  49. McCall C, McCall WV (2012) Objective vs. subjective measurements of sleep in depressed insomniacs: first night effect or reverse first night effect? J Clin Sleep Med 8(1):59–65

    Google Scholar 

  50. Ahmadi N, Shapiro GK, Chung SA, Shapiro CM (2009) Clinical diagnosis of sleep apnea based on single night of polysomnography vs. two nights of polysomnography. Sleep Breath 13(3):221–226. https://doi.org/10.1007/s11325-008-0234-2

    Article  Google Scholar 

  51. Tworoger SS, Davis S, Vitiello MV, Lentz MJ, McTiernan A (2005) Factors associated with objective (actigraphic) and subjective sleep quality in young adult women. J Psychosom Res 59(1):11–19. https://doi.org/10.1016/j.jpsychores.2005.03.008

    Article  Google Scholar 

  52. Evenson KR, Goto MM, Furberg RD (2015) Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutr Phys Act 12(1):159. https://doi.org/10.1186/s12966-015-0314-1

    Article  Google Scholar 

  53. Higgins PA, Straub AJ (2006) Understanding the error of our ways: mapping the concepts of validity and reliability. Nurs Outlook 54(1):23–29. https://doi.org/10.1016/j.outlook.2004.12.004

    Article  Google Scholar 

  54. Benjamini Y (1988) Opening the box of a boxplot. Am Stat 42(4):257–262

    Google Scholar 

  55. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83. https://doi.org/10.2307/3001968

    Article  Google Scholar 

  56. Fay MP, Proschan MA (2010) Wilcoson-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. Stat Surv 4(0):1–39. https://doi.org/10.1214/09-SS051

    Article  MathSciNet  MATH  Google Scholar 

  57. Bradley JV (1968) Distribution-free statistical tests. Prentice-Hall

  58. Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1(8476):307–310

    Article  Google Scholar 

  59. Meltzer L, Walsh C, Traylor J, Westin A (2012) Direct comparison of two new actigraphs and polysomnography in children and adolescents. Sleep 35(1):159–166. https://doi.org/10.5665/sleep.1608

    Google Scholar 

  60. Werner H, Molinari L, Guyer C, Jenni OG (2008) Agreement rates between actigraphy, diary, and questionnaire for children’s sleep patterns. Arch Pediatr Adolesc Med 162(4):350–358. https://doi.org/10.1001/archpedi.162.4.350

    Article  Google Scholar 

  61. Rosenberger ME, Buman MP, Haskell WL et al (2015) 24h or sleep, sedentary behavior, and physical activity with nine wearable devices. Med Sci Sports Exerc 48(3):457–465

    Article  Google Scholar 

  62. Kappenman ES, Luck SJ (2010) The effects of electrode impedance on data quality and statistical significance in ERP recordings. Psychophysiology 47(5):888–904. https://doi.org/10.1111/j.1469-8986.2010.01009.x

    Google Scholar 

  63. Duun-Henriksen J, Kjaer TW, Looney D, et al. (2015) EEG signal quality of a subcutaneous recording system compared to standard surface electrodes. J Sensors 2015: Article 341208, 9 pages

  64. Baroni A, Bruzzese JM, Di Bartolo CA, Shatkin JP (2016) Fitbit Flex: an unreliable device for longitudinal sleep measures in a non-clinical population. Sleep Breath 20(2):853–854. https://doi.org/10.1007/s11325-015-1271-2

    Article  Google Scholar 

  65. Ferguson T, Rowlands AV, Olds T, Maher C (2015) The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. Int J Behav Nutr Phys Act 12(1):42. https://doi.org/10.1186/s12966-015-0201-9

    Article  Google Scholar 

  66. Neuroon Open product overview. http://community.neuroonopen.com/product

  67. Hein M, Lanquart JP, Loas G, Hubain P, Linkowski P (2017) Similar polysomnographic pattern in primary insomnia and major depression with objective insomnia: a sign of common pathophysiology? BMC Psychiatry 17(1):273. https://doi.org/10.1186/s12888-017-1438-4

    Article  Google Scholar 

  68. Lucey BP, McLeland JS, Toedebusch CD et al (2016) Comparison of a single-channel EEG sleep study to polysomnography. J Sleep Res 25(6):625–635. https://doi.org/10.1111/jsr.12417

    Article  Google Scholar 

  69. De Zambotti M, Baker FC, Willoughby AR et al (2016) Measures of sleep and cardiac functioning during sleep using a multi-sensory commercially-available wristband in adolescents: wearable technology to measure sleep and cardiac functioning. Physiol Behav 158:143–149

    Article  Google Scholar 

  70. Taibi DM, Landis CA, Vitiello MV (2013) Concordance of polysomnographic and actigraphic measurement of sleep and wake in older women with insomnia. J Clin Sleep Med 9(3):217–225. https://doi.org/10.5664/jcsm.2482

    Google Scholar 

  71. Blackwell T, Redline S, Ancoli-Isreal S et al (2008) Comparison of sleep parameters from actigraphy and polysomnography in older women: the SOF study. Sleep 31(2):283–291. https://doi.org/10.1093/sleep/31.2.283

    Article  Google Scholar 

  72. Cellini N, Buman MP, McDevitt EA et al (2013) Direct comparison of two actigrapy devices with polysomnographically recorded naps in healthy young adults. Chronobiol Int 30(5):691–698. https://doi.org/10.3109/07420528.2013.782312

    Article  Google Scholar 

  73. Kosecki D (2017) Your heart rate is the key to smarter sleep stages. Here’s why. Fitbit News, https://blog.fitbit.com/heart-rate-during-sleep-stages/

Download references

Acknowledgements

This work was supported by JSPS KAKENHI Grant-in-Aid for Research Activity Start-up (Grant Number 16H07469) and an internal research grant from the National Institute of Advanced Industrial Science and Technology of Japan. The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of the paper.

Author information

Authors and Affiliations

  1. The University of Tokyo, 7-3-1 Hongo, Bunkyo-Ku, Tokyo, 113-8656, Japan

    Zilu Liang

  2. CAC Corporation, 24-1, Hakozaki-cho, Nihonbashi, Chuo-Ku, Tokyo, 103-0015, Japan

    Mario Alberto Chapa Martell

Authors
  1. Zilu Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mario Alberto Chapa Martell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zilu Liang or Mario Alberto Chapa Martell.

Ethics declarations

Conflict of Interest

The authors certify that there is no conflict of interest involved in this manuscript and this study. The opinions expressed in this paper are those of the authors and do not represent the views of the second author’s company.

Ethical Approval

Ethics approval was obtained from the Ethic Committee of the University of Tokyo (Ethics ID: KE16–83). All participants provided informed consent.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Z., Chapa Martell, M.A. Validity of Consumer Activity Wristbands and Wearable EEG for Measuring Overall Sleep Parameters and Sleep Structure in Free-Living Conditions. J Healthc Inform Res 2, 152–178 (2018). https://doi.org/10.1007/s41666-018-0013-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41666-018-0013-1

Keywords

Search

Navigation