Human activity recognition and fall detection using convolutional neural network and transformer-based architecture
DOI:
https://doi.org/10.1016/j.bspc.2024.106412الملخص
Human Activity Recognition (HAR) and fall detection, as applications within the field of biomedical signal processing, are increasingly pivotal in enhancing patient care, preventive healthcare, and rehabilitation. Fall therapy is one of the most expensive treatments, usually taking a long time to complete. A single fall accident might result in serious injuries, long-term incapacity, or even death. As a result, a reliable and cost-effective fall detection system is essential. Wearable sensors have received wide attention due to their availability and capability to capture different human motions. Thus, in the current study, we develop a comprehensive HAR system for multi-classification tasks to recognize several human actions, such as walking, sitting, standing, falling, and others. At the same time, a binary classification of this model is developed to recognize fall and non-fall actions, which can be used to track elderly actions and send an alert in case of falling to do necessary rescue actions. The developed system is built using a Parallel Convolutional Neural Network and Transformer-based architecture (PCNN-Transformer). PCNN-Transformer benefits from the parallel architecture and the residual mapping mechanism to learn temporal feature representations from the sensors’ data. The CNN blocks are aligned in parallel alongside several Transformer-based encoders, followed by a concatenation operation to sum up the extracted features from the input data ( sensors data). Moreover, the CNN blocks implement a residual mapping mechanism to reduce the model complexity and training time. The proposed model is tested using several open-source datasets: SisFall, UniMib-SHAR, and MobiAct. It recorded high accuracy rates compared to several deep learning models. For instance, in the binary classification (fall detection), the proposed model achieved an average accuracy of 99.95%, 98.68%, and 99.71% for SisFall, UniMib-SHAR, and MobiAct, respectively.المراجع
Al-qaness, M. A., et al. (2022). Multi-ResAtt: Multilevel residual network with attention for human activity recognition using wearable sensors. IEEE Transactions on Industrial Informatics, 18, 5521–5530.
Baek, W.-S., et al. (2018). Real life applicable fall detection system based on wireless body area network. IEEE Systems Journal, 12, 2673–2682.
Dahou, A., et al. (2022). Human activity recognition in IoHT applications using arithmetic optimization algorithm and deep learning. Measurement, 200, Article 111627.
Dang, L. M., et al. (2020). Sensor-based and vision-based human activity recognition: A comprehensive survey. Pattern Recognition, 105, Article 107325.
De Raeve, N., et al. (2022). Bluetooth-low-energy-based fall detection and warning system for elderly people in nursing homes. Journal of Sensors, 2022, Article 4270776.
Delgado-Escano, R., et al. (2020). A cross-dataset deep learning-based classifier for people fall detection and identification. Computer Methods and Programs in Biomedicine, 197, Article 105748.
Hu, L., et al. (2023). Activity recognition via correlation coefficients based graph with nodes updated by multi-aggregator approach. Biomedical Signal Processing and Control, 85, Article 104932.
Hussain, F., et al. (2019). Activity-aware fall detection and recognition based on wearable sensors. IEEE Sensors Journal, 19(22), 10728–10736.
Kuncan, F., et al. (2022). A new approach for physical human activity recognition from sensor signals based on motif patterns and long-short term memory. Biomedical Signal Processing and Control, 73, Article 103444.
Li, Y., et al. (2023). Sensor-based fall detection using a combination model of a temporal convolutional network and a gated recurrent unit. Future Generation Computer Systems, 141, 601–612.
Mastorakis, G., et al. (2014). Fall detection system using kinect’s infrared sensor. Journal of Real-Time Image Processing, 9, 575–586.
Nikpour, B., et al. (2023). Spatio-temporal hard attention learning for skeleton-based activity recognition. Pattern Recognition, 138, Article 109405.
Raj, M. S., et al. (2024). Leveraging spatio-temporal features using graph neural networks for human activity recognition. Pattern Recognition, 149, Article 110243.
Sadreazami, H., et al. (2021). Contactless fall detection using time-frequency analysis and convolutional neural networks. IEEE Transactions on Industrial Informatics, 17(10), 6842–6851.
Sezavar, A., et al. (2024). DCapsNet: Deep capsule network for human activity and gait recognition with smartphone sensors. Pattern Recognition, 145, Article 109961.
Shang, M., et al. (2023). Recurrent ConFormer for WiFi activity recognition. IEEE/CAA Journal of Automatica Sinica, 10(3), 732–744.
Wang, G., et al. (2020). CMFALL: A cascade and parallel multi-state fall detection algorithm using waist-mounted tri-axial accelerometer signals. IEEE Transactions on Consumer Electronics, 66(3), 254–263.
Wu, Y., et al. (2019). Wearable-sensor-based pre-impact fall detection system with a hierarchical classifier. Measurement, 145, 632–642.
Xu, T., et al. (2021). A fusion fall detection algorithm combining threshold-based method and convolutional neural network. Microprocessors and Microsystems, 85, Article 104313.
Yang, Y., et al. (2022). Fall detection system based on infrared array sensor and multi-dimensional feature fusion. Measurement, 190, Article 110685.
Salehzadeh, A., et al. (2020). Human activity recognition using deep electroencephalography learning. Biomedical Signal Processing and Control, 62, Article 102075.
