Md Mahbubur Rahman
ABSTRACT
Proliferation of sensors embedded in smartphones and smartwatches helps capture rich dataset for machine learning algorithms to extract meaningful digital bio-markers on consumer devices for monitoring disease progression and treatment response. However, development and validation of machine learning algorithms depend on gathering high fidelity sensor data and reliable ground-truth. We conduct a study, called mLungStudy, with 131 subjects with varying pulmonary conditions to collect mobile sensor data including audio, accelerometer, gyroscope using a smartphone and a smartwatch, in order to extract pulmonary biomarkers such as breathing, coughs, spirometry, and breathlessness. Our study shows that commonly used breathing ground-truth data from chestband may not always be reliable as a gold-standard. Our analysis shows that breathlessness biomarkers such as pause time and pause frequency from 2.15 minutes of audio can be as reliable as those extracted from 5 minutes' worth of speech data. This finding can be useful for future studies to trade-off between the reliability of breathlessness data and patient comfort in generating continuous speech data. Furthermore, we use crowdsourcing techniques to annotate pulmonary sound events for developing signal processing and machine learning algorithms. In this paper, we highlight several practical challenges to collect and annotate physiological data and acoustic symptoms from chronic pulmonary patients and ways to improve data quality. We show that the waveform visualization of the audio signal improves annotation quality which leads to a 6.59% increase in cough classification accuracy and a 6% increase in spirometry event classification accuracy. Findings from this study inform future studies focusing on developing explainable machine learning models to extract pulmonary digital bio-markers using mobile sensors.
- Mohsin Ahmed, Md Mahbubur Rahman, Viswam Nathan, Ebrahim Nemati, Korosh Vatanparvar, and Jilong Kuang. 2019. mLung: Privacy-Preserving Naturally Windowed Lung Activity Detection for Pulmonary Patients. In IEEE BSN.Google Scholar
- ALA 2018. American Lung Association. Retrieved December 12, 2018 from http://www.lung.org/lung-health-and-diseases/lung-disease-lookup/copd/learn-about-copd/how-serious-is-copd.htmlGoogle Scholar
- Rummana Bari, Roy J Adams, Md Mahbubur Rahman, Megan Battles Parsons, Eugene H Buder, and Santosh Kumar. 2018. rConverse: Moment by Moment Conversation Detection Using a Mobile Respiration Sensor. ACM IMWUT (2018). Google ScholarDigital Library
- Andrew Bates, Martin J Ling, Janek Mann, and DK Arvind. 2010. Respiratory Rate and Flow Waveform Estimation from Tri-axial Accelerometer Data. In IEEE BSN. Google ScholarDigital Library
- Matthias Budde, Andrea Schankin, Julien Hoffmann, Marcel Danz, Till Riedel, and Michael Beigl. 2017. Participatory Sensing or Participatory Nonsense?: Mitigating the Effect of Human Error on Data Quality in Citizen Science. ACM IMWUT (2017). Google ScholarDigital Library
- S Burge and JA Wedzicha. 2003. COPD Exacerbations: Definitions and Classifications. European Respiratory Journal (2003).Google Scholar
- Pierre-Régis Burgel, Pascale Nesme-Meyer, Pascal Chanez, Denis Caillaud, Philippe Carré, Thierry Perez, Nicolas Roche, et al. 2009. Cough and Sputum Production are Associated with Frequent Exacerbations and Hospitalizations in COPD Subjects. Elsevier Chest (2009).Google Scholar
- Soujanya Chatterjee, Md Mahbubur Rahman, Ebrahim Nemati, and Jilong Kunag. 2019. WheezeD: Respiration Phase Based Wheeze Detection Using Acoustic Data From Pulmonary Patients Under Attack. In ACM Pervasive Computing Technologies for Healthcare (PervasiveHealth).Google Scholar
- Emil Chiauzzi, Carlos Rodarte, and Pronabesh DasMahapatra. 2015. Patient-centered activity monitoring in the self-management of chronic health conditions. BMC medicine (2015).Google Scholar
- Alvaro A Cruz. 2007. Global surveillance, prevention and control of chronic respiratory diseases: a comprehensive approach. World Health Organization.Google Scholar
- Emre Ertin, Nathan Stohs, Santosh Kumar, Andrew Raij, Mustafa al'Absi, and Siddharth Shah. 2011. AutoSense: Unobtrusively Wearable Sensor Suite for Inferring the Onset, Causality, and Consequences of Stress in the Field. In ACM SenSys. Google ScholarDigital Library
- Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. Opensmile: the Munich Versatile and Fast Open-source Audio Feature Extractor. In ACM conference on Multimedia. Google ScholarDigital Library
- Grant Fairbanks. 1960. Voice and Articulation Drillbook. Harper New York.Google Scholar
- FigureEight 2019. Figure Eight. Retrieved April 15, 2019 from https://www.figure-eight.com/Google Scholar
- Kevin E Forkheim, David Scuse, and Hans Pasterkamp. 1995. A comparison of neural network models for wheeze detection. In IEEE WESCANEX Communications, Power, and Computing.Google Scholar
- Javier Hernandez, Daniel McDuff, and Rosalind W Picard. 2015. Biowatch: estimation of heart and breathing rates from wrist motions. In IEEE Pervasive Computing Technologies for Healthcare (PervasiveHealth). Google ScholarDigital Library
- Javier Hernandez, Daniel J McDuff, and Rosalind W Picard. 2015. Biophone: Physiology monitoring from peripheral smartphone motions. (2015).Google Scholar
- Eric C Larson, Mayank Goel, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, and Shwetak N Patel. 2012. SpiroSmart: using a microphone to measure lung function on a mobile phone. In ACM UbiComp. Google ScholarDigital Library
- Daniyal Liaqat, Robert Wu, Andrea Gershon, Hisham Alshaer, Frank Rudzicz, and Eyal de Lara. 2018. Challenges with Real-world Smartwatch based Audio Monitoring. In ACM Workshop on Wearable Systems and Applications. Google ScholarDigital Library
- LungInstitute 2018. Lung Institute. Retrieved December 12, 2018 from https://lunginstitute.com/blog/the-cost-of-lung-disease/Google Scholar
- Marc Miravitlles. 2011. Cough and Sputum Production as Risk Factors for Poor Outcomes in Patients with COPD. Elsevier Respiratory Medicine (2011).Google Scholar
- Jimmy Moore, Pascal Goffin, Miriah Meyer, Philip Lundrigan, Neal Patwari, Katherine Sward, and Jason Wiese. 2018. Managing In-home Environments through Sensing, Annotating, and Visualizing Air Quality Data. ACM IMWUT (2018). Google ScholarDigital Library
- Bernard Munos, Pamela C Baker, Brian M Bot, Michelle Crouthamel, Glen Vries, Ian Ferguson, John D Hixson, Linda A Malek, John J Mastrototaro, Veena Misra, Aydogan Ozcan, Leonard Sacks, and Pei Wang. 2016. Mobile Health: the Power of Wearables, Sensors, and Apps to Transform Clinical Trials. Wiley Online Library Annals of the New York Academy of Sciences (2016).Google Scholar
- Viswam Nathan, Korosh Vatanparvar, Md Mahbubur Rahman, Ebrahim Nemati, and Jilong Kuang. 2019. Assessment of Chronic Pulmonary Disease Patients Using Biomarkers from Natural Speech Recorded by Mobile Devices. In IEEE BSN.Google Scholar
- Ebrahim Nemati, Christina Batteate, and Michael Jerrett. 2017. Opportunistic Environmental Sensing with Smartphones: a Critical Review of Current Literature and Applications. Springer Current environmental health reports (2017).Google Scholar
- Ebrahim Nemati, Md Mahbubur Rahman, Viswam Nathan, and Jilong Kuang. 2018. Private Audio-Based Cough Sensing for In-Home Pulmonary Assessment using Mobile Devices. In EAI International Conference on Body Area Networks.Google Scholar
- Ebrahim Nemati, Young Soo Suh, Babak Motamed, and Majid Sarrafzadeh. 2016. Gait Velocity Estimation for a Smartwatch Platform using Kalman Filter Peak Recovery. In IEEE BSN.Google Scholar
- Nuance 2019. Nuance PowerMic III. Retrieved January 15, 2019 from https://www.nuance.com/dragon/dragon-accessories.htmlGoogle Scholar
- M Pourhomayoun, E Nemati, B Mortazavi, and M Sarrafzadeh. 2015. Context-aware data analytics for activity recognition. In International Conference on Data Analytics.Google Scholar
- Public Good 2018. Private Data for Public Good. Retrieved December 18, 2018 from http://hdexplore.calit2.net/wp-content/uploads/2015/08/hdx_final_report_small.pdfGoogle Scholar
- Mahbubur Rahman, Nasir Ali, Rummana Bari, Nazir Saleheen, Mustafa al'Absi, Emre Ertin, Ashley Kennedy, Kenzie L Preston, and Santosh Kumar. 2017. mDebugger: Assessing and Diagnosing the Fidelity and Yield of Mobile Sensor Data. In Springer Mobile Health.Google Scholar
- Md Mahbubur Rahman, Rummana Bari, Amin Ahsan Ali, Moushumi Sharmin, Andrew Raij, Karen Hovsepian, Syed Monowar Hossain, Emre Ertin, Ashley Kennedy, David H Epstein, Kenzie L Preston, Michelle Jobes, Kenneth D Ward, Mustafa al'Absi, and Santosh Kumar. 2014. Are We There Yet? Feasibility of Continuous Stress Assessment via Wireless Physiological Sensors. In ACM Conference on Bioinformatics, Computational Biology, and Health Informatics. Google ScholarDigital Library
- Md Mahbubur Rahman, Ebrahim Nemati, Viswam Nathan, and Jilong Kuang. 2018. InstantRR: Instantaneous Respiratory Rate Estimation on Context-aware Mobile Devices. In EAI International Conference on Body Area Networks.Google Scholar
- Javier Ramırez, José C Segura, Carmen Benıtez, Angel De La Torre, and Antonio Rubio. 2004. Efficient Voice Activity Detection Algorithms Using Long-term Speech Information. Elsevier Speech communication (2004).Google Scholar
- Ronald W Schafer. 2011. What is a Savitzky-Golay Filter?{lecture notes}. IEEE Signal processing magazine (2011).Google Scholar
- sHealth 2018. Samsung Health. Retrieved December 18, 2018 from https://www.samsung.com/us/support/answer/ANS00062448/Google Scholar
- Germán D Sosa, Angel Cruz-Roa, and Fabio A González. 2015. Automatic Detection of Wheezes by Evaluation of Multiple Acoustic Feature Extraction Methods and C-weighted SVM. In Society for Optics and Photonics Symposium on Medical Information Processing and Analysis.Google Scholar
- Xiao Sun, Li Qiu, Yibo Wu, Yeming Tang, and Guohong Cao. 2017. Sleepmonitor: Monitoring respiratory rate and body position during sleep using smartwatch. ACM IMWUT (2017). Google ScholarDigital Library
- R Tehrany. 2015. Speech Breathing Patterns in Health and Chronic Respiratory Disease. Ph.D. Dissertation. University of Southampton.Google Scholar
- Korosh Vatanparvar, Viswam Nathan, Ebrahim Nemati, Md Mahbubur Rahman, and Jilong Kuang. 2019. A Generative Model for Speech Segmentation and Obfuscation for Remote Health Monitoring. In IEEE BSN.Google Scholar
- Varun Viswanath, Jake Garrison, and Shwetak Patel. 2018. SpiroConfidence: Determining the Validity of Smartphone Based Spirometry Using Machine Learning. In IEEE EMBC.Google Scholar
- Rui Wang, Fanglin Chen, Zhenyu Chen, Tianxing Li, Gabriella Harari, Stefanie Tignor, Xia Zhou, Dror Ben-Zeev, and Andrew T Campbell. 2014. StudentLife: Assessing Mental Health, Academic Performance and Behavioral Trends of College Students using Smartphones. In ACMUbiComp. Google ScholarDigital Library
- Shichao Yue, Hao He, Hao Wang, Hariharan Rahul, and Dina Katabi. 2018. Extracting Multi-Person Respiration from Entangled RF Signals. ACM IMWUT (2018). Google ScholarDigital Library
- Youwei Zeng, Dan Wu, Ruiyang Gao, Tao Gu, and Daqing Zhang. 2018. Full-Breathe: Full Human Respiration Detection Exploiting Complementarity of CSI Phase and Amplitude of WiFi Signals. ACM IMWUT (2018). Google ScholarDigital Library
- zephyr 2019. Zephyr Bioharness. Retrieved January 15, 2019 from https://www.zephyranywhere.com/Google Scholar
Index Terms
- Towards Reliable Data Collection and Annotation to Extract Pulmonary Digital Biomarkers Using Mobile Sensors
Recommendations
Assessing Severity of Pulmonary Obstruction from Respiration Phase-Based Wheeze-Sensing Using Mobile Sensors
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsObstructive pulmonary diseases cause limited airflow from the lung and severely affect patients' quality of life. Wheeze is one of the most prominent symptoms for them. High requirements imposed by traditional diagnosis methods make regular monitoring of ...
A Mobile Health System to Identify the Onset of Paroxysmal Atrial Fibrillation
ICHI '15: Proceedings of the 2015 International Conference on Healthcare InformaticsAtrial Fibrillation (AFib) is increasingly recognized as a risk factor for clots, strokes, heart failure and other complications. One estimate states that 2.7 million individuals are living in U.S. With AFib and this number may increase to 5.6 million ...
Comments