Robust respiration detection from remote photoplethysmography
Continuous monitoring of respiration is essential for early detection of critical illness. Current methods require sensors attached to the body and/or are not robust to subject motion. Alternative camera based solutions have been presented using motion vectors and remote photoplethysmography. In this work, we present a non contact camera based method to detect respiration, which can operate in both visible and dark lighting conditions by detecting the respiratory induced colour differences of the skin. We make use of the close similarity between skin colour variations caused by the beating of the heart and those caused by respiration, leading to a much improved signal quality compared to single channel approaches. Essentially, we propose to find the linear combination of colour channels which suppresses the distortions best in a frequency band including pulse rate, and subsequently we use this same linear combination to extract the respiratory signal in a lower frequency band. Evaluation results obtained from recordings on healthy subjects which perform challenging scenarios, including motion, show that respiration can be accurately detected over the entire range of respiratory frequencies, with a correlation coefficient of 0.96 in visible light and 0.98 in infrared, compared to 0.86 with the best performing non contact benchmark algorithm. Furthermore, evaluation on a set of videos recorded in a Neonatal Intensive Care Unit (NICU) shows that this technique looks promising as a future alternative to current contact sensors showing a correlation coefficient of 0.87. (c) 2016 Optical Society of America OCIS codes: (170.0170) Medical optics and biotechnology imaging system; (170.1470) Blood or tissue constituent monitoring; (280.0280) Remote sensing and sensors; (170.3660) Light propagation in tissues References and links 1. C. Seymour, J. Kahn, C. Cooke, T. Watkins, S. Heckbert, and T. Rea, "Prediction of critical illness during out of hospital emergency care," J. Am. Med. Assoc. 304(7) 747 754 (2010) 2. F. Q. Al Khalidi, R. Saatchi, D. Burke, H. Elphick, and S. Tan, "Respiration rate monitoring methods: a review," Pediatr. Pulmonol. 46(6) 523 529 (2011) 3. P. H. Charlton, T. Bonnici, L. Tarassenko, D. A. Clifton, R. Beale, and P. J. Watkinson, "An assessment of algorithms to estimate respiratory rate from the electrocardiogram and photoplethysmogram," Physiol. Meas. 37(4) 610 626 (2016) 4. W. Karlen, A. Garde, D. Myers, C. Scheffer, J. Ansermino, and G. Dumont, "Estimation of respiratory rate from photoplethysmographic imaging videos compared to pulse oximetry," IEEE J. Biomed. Health Inform. 19(4) 1331 1338 (2015) 5. F. Adib, H. Mao, Z. Kabelac, D. Katabi, and R. C. Miller, "Smart homes that monitor breathing and heart rate," in "Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems," pp. 837 846 (2015) 6. J. Fei and I. Pavlidis, "Thermistor at a distance: unobtrusive measurement of breathing," IEEE Trans. Biomed. Eng. 57(4) 988 998 (2010) 7. E. Greneker, "Radar sensing of heartbeat and respiration at a distance with applications of the technology," in "Radar 97 (Conf. Publ. No. 449)" pp. 150 154 (1997) 8. M. Bartula, T. Tigges, and J. Muehlsteff, "Camera based system for contactless monitoring of respiration," in "Engineering in Medicine and Biology Society, 2013 35th Annual International Conference of the IEEE," pp. 2672 2675 (2013) 9. A. Heinrich, F. van Heesch, B. Puvvula, and M. Rocque, "Video based actigraphy and breathing monitoring from the bedside table of shared beds," J. Ambient. Intell. Humaniz. Comput. 6(1) 107 120 (2015) 10. F. Zhao, M. Li, Y. Qian, and J. Z. Tsien, "Remote measurements of heart and respiration rates for telemedicine," PLOS ONE 8(10) e71384 (2013) Vol. 7, No. 12 1 Dec 2016 BIOMEDICAL OPTICS EXPRESS 4941 #270614 http:/dx.doi.org/10.1364/BOE.7.004941 Journal (c) 2016 Received 15 Jul 2016; revised 30 Sep 2016; accepted 6 Oct 2016; published 3 Nov 2016 11. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, "Remote plethysmographic imaging using ambient light," Opt. Express 16(26) 21434 21445 (2008) 12. M. Z. Poh, D. J. McDuff, and R.W. Picard, "Advancements in noncontact, multiparameter physiological measurements using a webcam," IEEE Trans. Biomed. Eng. 58(1) 7 11 (2011) 13. Y. Sun, S. Hu, V. Azorin Peris, S. Greenwald, J. Chambers, and Y. Zhu, "Motion compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise," J. Biomed. Opt. 16(7) 077010 077010 9 (2011) 14. F. Bousefsaf, C. Maaoui, and A. Pruski, "Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate," Biomed. Signal Process. Control 8(6) 568 574 (2013) 15. P. Leonard, N. R. Grubb, P. S. Addison, D. Clifton, and J. N. Watson, "An algorithm for the detection of individual breaths from the pulse oximeter waveform," J. Clin. Monit. Comput. 18(5 6) 309 312 (2004) 16. K. H. Chon, S. Dash, and K. Ju, "Estimation of respiratory rate from photoplethysmogram data using time frequency spectral estimation," IEEE Trans. Biomed. Eng. 56(8) 2054 2063 (2009) 17. S. G. Fleming and L. Tarassenko, "A comparison of signal processing techniques for the extraction of breathing rate from the photoplethysmogram," Int. J. Biol. Med. Sci. 2(4) 232 236 (2007) 18. A. Sikdar, S. K. Behera, and D. P. Dogra, "Computer vision guided human pulse rate estimation: A review," IEEE Rev. Biomed. Eng. 9, 91 105 (2016) 19. C. G. Caro, The mechanics of the circulation (Cambridge University Press, 2012) 20. G. de Haan and V. Jeanne, "Robust pulse rate from chrominance based rPPG," IEEE Trans. Biomed. Eng. 60(10) 2878 2886 (2013) 21. G. de Haan and A. van Leest, "Improved motion robustness of remote PPG by using the blood volume pulse signature," Physiol. Meas. 35(9) 1913 1926 (2014) 22. M. Hulsbusch and V. Blazek, "Contactless mapping of rhythmical phenomena in tissue perfusion using PPGI," Proc. SPIE 4683, 110 117 (2002) 23. L. F. Corral Martinez, G. Paez, and M. Strojnik, "Optimal wavelength selection for noncontact reflection photoplethysmography," 22nd Congress of the International Commission for Optics 8011, 801191 7 (2011) 24. S. Prahl, "Optical absorption of hemoglobin," Oregon Medical Laser Center, http:/omlc.ogi.edu/ spectra/hemoglobin/index.html 15 (1999) 25. C. Tomasi and T. Kanade, "Detection and tracking of point features," Tech. rep. Int. J. Comput. Vision (1991) 26. J. Shi and C. Tomasi, "Good features to track," in "Computer Vision and Pattern Recognition (CVPR) Proceedings CVPR'94. 1994 IEEE Computer Society Conference on," pp. 593 600 (1994) 27. E. Tur, M. Tur, H. I. Maibach, and R. H. Guy, "Basal perfusion of the cutaneous microcirculation: measurements as a function of anatomic position," J. Invest. Dermatol. 81(5) 442 446 (1983) 28. L. M. Nilsson, "Respiration signals from photoplethysmography," Anesth. Analg. 117(4) 859 865 (2013) 29. M. Lewandowska, J. Ruminski, T. Kocejko, and J. Nowak, "Measuring pulse rate with a webcam a non contact method for evaluating cardiac activity," in "Computer Science and Information Systems (FedCSIS) 2011 Federated Conference on" pp. 405 410 (2011) 30. M. van Gastel, S. Stuijk, and G. de Haan, "Motion robust remote PPG in infrared," IEEE Trans. Biomed. Eng. 62(5) 1425 1433 (2015) 31. L. A. Silvestri, Saunders comprehensive review for the NCLEX PN(r) examination (Elsevier Health Sciences, 2015) 32. L. G. Lindberg, H. Ugnell, and P. Oberg, "Monitoring of respiratory and heart rates using a fibre optic sensor," Med. Biol. Eng. Comput. 30(5) 533 537 (1992) 33. A. Johansson and P. Oberg, "Estimation of respiratory volumes from the photoplethysmographic signal. part i: experimental results," Med. Biol. Eng. Comput. 37(1) 42 47 (1999) 34. L. Nilsson, A. Johansson, and S. Kalman, "Respiratory variations in the reflection mode photoplethysmographic signal. Relationships to peripheral venous pressure," Med. Biol. Eng. Comput. 41(3) 249 254 (2003) 35. J. Li, J. Jin, X. Chen, W. Sun, and P. Guo, "Comparison of respiratory induced variations in photoplethysmographic signals," Physiol. Meas. 31(3) 415 425 (2010) 36. P. S. Addison, J. N. Watson, M. L. Mestek, and R. S. Mecca, "Developing an algorithm for pulse oximetry derived respiratory rate (rroxi) a healthy volunteer study," J. Clin. Monit. Comput. 26(1) 45 51 (2012)