HARGAN: Generative Adversarial Network BasedDeep Learning Framework for Efficient Recognition of Human Actions from Surveillance Videos
DOI:
https://doi.org/10.22399/ijcesen.587Keywords:
Human Action Recognition, Artificial Intelligence, Deep Learning, Generative Adversarial Network, CNN, LSTMAbstract
Analyzing public surveillance videos has become an important research area as it is linked to different real-world applications. Video Analytics for human action recognition is given significance due to its utility. However, it is very challenging to analyze live-streaming videos to identify human actions across the frames in the video. The literature showed that Convolutional Neural Networks (CNNs) are among computer vision applications' most popular deep learning algorithms. Another important observation is that Generative Adversarial Network(GAN) architecture with deep learning has the potential to leverage effectiveness in applications using computer vision. Inspired by this finding, we created a GAN-based framework (called HARGAN) in this research for human activity identification from surveillance films. The framework exploits a retrained deep learning model known as ResNet50 and convolutional LSTM for better performance in action recognition. Our framework has two critical functionalities: feature learning and human action recognition. The ResNet50 model achieves the former, while the GAN-based convolutional LSTM model achieves the latter. We proposed an algorithm called the Generative Adversarial Approach for Human Action Recognition (GAA-HAR) to realize the framework. We used a benchmark dataset known as UCF50, which is extensively used in studies on human action identification. Based on our experimental findings, the suggested framework performs better than the current baseline models like CNN, LSTM, and convolutional LSTM, with the highest accuracy of 97.73%. Our framework can be used in video analytics applications linked to large-scale public surveillance.
References
V.K. Kambala, H. Jonnadula, (2022). Privacy preserving human activity recognition framework using an optimized prediction algorithm, IAES International Journal of Artificial Intelligence (IJ-AI),11(1);254-264.
Z. Wu, H. Wang, Z. Wang, H. Jin, Z. Wang,(2022). Privacy-Preserving Deep Action Recognition: An Adversarial Learning Framework and A New Dataset, IEEE Trans Pattern Anal Mach Intell 44(4):2126-2139. doi: 10.1109/TPAMI.2020.3026709
J. Liu, N. Akhtar, A. Mian, (2020). Adversarial Attack on Skeleton-Based Human Action Recognition, IEEE Transactions on Neural Networks and Learning Systems, 1–14. doi:10.1109/tnnls.2020.3043002
M. Sun, Q. Wang, Z. Liu, (2020). Human Action Image Generation with Differential Privacy, IEEE International Conference on Multimedia and Expo (ICME), 1-6. doi:10.1109/icme46284.2020.910276
K. Ahuja, Y. Jiang, M. Goel, C. Harrison,(2021). Synthesizing Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition, Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, Vid2Doppler: 1-10. https://doi.org/10.1145/3411764.3
M. Hou, S. Liu, J. Zhou, Y. Zhang, Z. Feng, (2021). Extreme Low-Resolution Activity Recognition Using a Super-Resolution-Oriented Generative Adversarial Network, Micromachines, 12(6);1-15. https://doi.org/10.3390/mi1206067
J. Jiang, G. Li, S. Wu, H. Zhang, Y. Nie, (2021). BPA-GAN: Human motion transfer using body-part-aware generative adversarial networks, Graphical Models, 1151-10. doi:10.1016/j.gmod.2021.101107
K. Gedamu, Y. Ji, Y. Yang, L. Gao, H.T. Shen, (2021). Arbitrary-view human action recognition via novel-view action generation, Pattern Recognition, 118;1-9. doi:10.1016/j.patcog.2021.108043
G. Pikramenos, E. Mathe, E. Vali, I. Vernikos, A. Papadakis, E. Spyrou, P. Mylonas, (2020) An adversarial semi-supervised approach for action recognition from pose information, Neural Computing and Applications, 1-15. doi:10.1007/s00521-020-05162-5
A.O. JIMALE, M.H.M. NOOR, (2022). Fully Connected Generative Adversarial Network for Human Activity Recognition, IEEE Access, 1-10.
T.H. Tan, L. Badarch, W.X. Zeng, M. Gochoo, (2021). Binary Sensors-Based Privacy-Preserved Activity Recognition of Elderly Living Alone Using an RNN, MDPI 1-18.
J. Yan, F. Angelini, S.M. Naqvi (2020), Image Segmentation Based Privacy-Preserving Human Action Recognition for Anomaly Detection, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 1-5. doi:10.1109/icassp40776.2020.9054
J. Imran, B. Raman, A.S. Rajput, Robust, (2020). Efficient and privacy-preserving violent activity recognition in videos Proceedings of the 35th Annual ACM Symposium on Applied Computing, doi:10.1145/3341105.3373942
C. Liang, D. Liu, L. Qi, L. Guan, (2020). Multi-Modal Human Action Recognition With Sub-Action Exploiting and Class-Privacy Preserved Collaborative Representation Learning, IEEE Access, 8 39920–39933. doi:10.1109/access.2020.2976496
A.S. Rajput, B. Raman, J. Imran, (2020), Privacy-preserving human action recognition as a remote cloud service using RGB-D sensors and deep CNN, Expert Systems with Applications 113349 (2020) 1-15. doi:10.1016/j.eswa.2020.113349
Z.W. Wang, V. Vineet, F. Pittaluga, S.N. Sinha, O. Cossairt, S.B. Kang, (2019). Privacy-Preserving Action Recognition Using Coded Aperture Videos, IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). doi:10.1109/cvprw.2019.00007
S. Chaudhary, A. Dudhane, P. Patil, S. Murala, (2019), Pose Guided Dynamic Image Network for Human Action Recognition in Person Centric Videos, IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) 1-8. doi:10.1109/avss.2019.8909835
Y. Hao, Z. Shi, Y. Liu, (2020) A Wireless-Vision Dataset for Privacy Preserving Human Activity Recognition, Fourth International Conference on Multimedia Computing, Networking and Applications (MCNA), 1-9. doi:10.1109/mcna50957.2020.926428
D. Wu, N. Sharma, M. Blumenstein, (2017) Recent advances in video-based human action recognition using deep learning: A review, International Joint Conference on Neural Networks (IJCNN), 1-8. doi:10.1109/ijcnn.2017.7966210
S.A. OBAIDI, H.A. KHAFAJI, C. ABHAYARATNE, (2020). Modeling Temporal Visual Salience for Human Action Recognition Enabled Visual Anonymity Preservation, IEEE Access,8;1-19.
T.N. Bach, D. Junger, C.Curio, O. Burgert, (2022) Towards Human Action Recognition during Surgeries using De-identified Video Data, Current Directions in Biomedical Engineering, 8(1);10 -112.
J. Dai, J. Wu, B. Saghafi, J. Konrad, P. Ishwar, (2015). Towards privacy-preserving activity recognition using extremely low temporal
and spatial resolution cameras, IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 1-9. doi:10.1109/cvprw.2015.7301356
A.G. Perera, Y.W. Law, T.T. Ogunwa, J. Chahl, (2020). A Multiviewpoint Outdoor Dataset for Human Action Recognition, IEEE Transactions on Human-Machine Systems, 1–9. doi:10.1109/thms.2020.2971958
J. Liu, R. Zhang, G. Han, N. Sun, S. Kwong,(2021) ,Video action recognition with visual privacy protection based on compressed sensing, Journal of Systems Architecture 113;1-14. doi:10.1016/j.sysarc.2020.101882
I.C. Duta, B. Ionescu, K. Aizawa, N. Sebe, (2016) Spatio-Temporal VLAD Encoding for Human Action Recognition in Videos, Lecture Notes in Computer Science, 365–378. doi:10.1007/978-3-319-51811-4_30
L. Wang, C. Zhao, K. Zhao, B. Zhang, S. Jing, Zhenxi, (2022). Privacy-Preserving Collaborative Computation for Human Activity Recognition, Hindawi Security and Communication Networks, 1-8.
F. Angelini, Z. Fu, Y. Long, L. Shao, S.M. Naqvi, (2019) 2D Pose-based Real-time Human Action Recognition with Occlusion-handling, IEEE Transactions on Multimedia, 1–14. doi:10.1109/tmm.2019.2944745
J.M. Roshtkhari, M.D. Levine, (2013). Human activity recognition in videos using a single example, Image and Vision Computing,
(11);864–876. doi:10.1016/j.imavis.2013.08.005
M. Farrajota, J.M.F. Rodrigues, J.M.H. du Buf, (2018) Human action recognition in videos with articulated pose information by deep networks, Pattern Analysis and Applications, 1-12. doi:10.1007/s10044-018-0727-y
Y. Yanga, P. Hua, J. Shenb, H. Chengc, Z.A. Xiulo, (2024). Elsevier, Privacy-preserving human activity sensing: A survey,1-35.
J. Xu, R. Song, H. Wei, J. Guo, Y. Zhou, X. Huang, (2021), A fast human action recognition network based on spatio-temporal features, Neurocomputing 441;350–358. doi:10.1016/j.neucom.2020.04.150
M. Ramanathan, W.Y. Yau, E.K. Teoh, (2014), Human Action Recognition With Video Data: Research and Evaluation Challenges, IEEE Transactions on Human-Machine Systems 44(5);650–663. doi:10.1109/thms.2014.2325871
M.V.D. Silva, A.N. Marana, (2020), Human action recognition in videos based on spatiotemporal features and bag-of-poses, Applied Soft Computing. 95;106513 doi:10.1016/j.asoc.2020.106513
C.J. Dhamsania, T.V. Ratanpara, (2016), A survey on Human action recognition from videos, Online International Conference on Green Engineering and Technologies (IC-GET) 1-5. doi:10.1109/get.2016.7916717
A. Abdelbaky, S. Aly, (2020), Human action recognition using short-time motion energy template images and PCANet features, Neural Computing and Applications. 32;12561–12574 doi:10.1007/s00521-020-04712-1
S. Agahian, F. Negin, C. Köse, (2019), An efficient human action recognition framework with pose-based spatiotemporal features, Engineering Science and Technology, an International Journal. doi:10.1016/j.jestch.2019.04.014
M. Gutoski, A.E. Lazzaretti, H.S. Lopes, (2020) Deep metric learning for open-set human action recognition in videos, Neural Computing and Applications, 33(4);1207–1220. https://doi.org/10.1007/s00521-020-05009-z
A. Kar, N. Rai, K. Sikka, G. Sharma, (2017). Adaptive Scan Pooling in Deep Convolutional Neural Networks for Human Action Recognition in Videos, AdaScan: 1-10. doi:10.1109/cvpr.2017.604
H. Yang, C. Yuan, J. Xing, W. Hu, (2017) SCNN: Sequential convolutional neural network for human action recognition in videos, IEEE International Conference on Image Processing (ICIP), 1-5. doi:10.1109/icip.2017.8296302
P. Pareek, A. Thakkar, (2020), A survey on video-based Human Action Recognition: recent updates, datasets, challenges, and applications, Artificial Intelligence Review 1-64. doi:10.1007/s10462-020-09904-8
D. Wu, N. Sharma, M. Blumenstein, (2017) , Recent advances in video-based human action recognition using deep learning: A review, International Joint Conference on Neural Networks (IJCNN) 1-8. doi:10.1109/ijcnn.2017.7966210
UCF50 - Action Recognition Data Set. Retrieved from https://www.crcv.ucf.edu/data/UCF50.php
M. Husain Bathushaw, & S. Nagasundaram. (2024). The Role of Blockchain and AI in Fortifying Cybersecurity for Healthcare Systems. International Journal of Computational and Experimental Science and Engineering, 10(4);1120-1129. https://doi.org/10.22399/ijcesen.596
AY, S. (2024). Vehicle Detection And Vehicle Tracking Applications On Traffic Video Surveillance Systems: A systematic literature review. International Journal of Computational and Experimental Science and Engineering, 10(4);1059-1068. https://doi.org/10.22399/ijcesen.629
Rama Lakshmi BOYAPATI, & Radhika YALAVARTHI. (2024). RESNET-53 for Extraction of Alzheimer’s Features Using Enhanced Learning Models. International Journal of Computational and Experimental Science and Engineering, 10(4);879-889. https://doi.org/10.22399/ijcesen.519
Sheela Margaret D, Elangovan N, Sriram M, & Vedha Balaji. (2024). The Effect of Customer Satisfaction on Use Continuance in Bank Chatbot Service. International Journal of Computational and Experimental Science and Engineering, 10(4);1069-1077. https://doi.org/10.22399/ijcesen.410
jaber, khalid, Lafi, M., Alkhatib, A. A., AbedAlghafer, A. K., Abdul Jawad, M., & Ahmad, A. Q. (2024). Comparative Study for Virtual Personal Assistants (VPA) and State-of-the-Art Speech Recognition Technology. International Journal of Computational and Experimental Science and Engineering, 10(3);427-433. https://doi.org/10.22399/ijcesen.383
Guven, M. (2024). A Comprehensive Review of Large Language Models in Cyber Security. International Journal of Computational and Experimental Science and Engineering, 10(3);507-516. https://doi.org/10.22399/ijcesen.469
M, V., V, J., K, A., Kalakoti, G., & Nithila, E. (2024). Explainable AI for Transparent MRI Segmentation: Deep Learning and Visual Attribution in Clinical Decision Support. International Journal of Computational and Experimental Science and Engineering, 10(4);575-584. https://doi.org/10.22399/ijcesen.479
ÖZNACAR, T., & ERGENE, N. (2024). A Machine Learning Approach to Early Detection and Malignancy Prediction in Breast Cancer. International Journal of Computational and Experimental Science and Engineering, 10(4);911-917 https://doi.org/10.22399/ijcesen.516
Venkatraman Umbalacheri Ramasamy. (2024). Overview of Anomaly Detection Techniques across Different Domains: A Systematic Review. International Journal of Computational and Experimental Science and Engineering, 10(4);898-910. https://doi.org/10.22399/ijcesen.522
Türkmen, G., Sezen, A., & Şengül, G. (2024). Comparative Analysis of Programming Languages Utilized in Artificial Intelligence Applications: Features, Performance, and Suitability. International Journal of Computational and Experimental Science and Engineering, 10(3);461-469. https://doi.org/10.22399/ijcesen.342
Jafar Ismail, R., Samar Jaafar Ismael, Dr. Sara Raouf Muhamad Amin, Wassan Adnan Hashim, & Israa Tahseen Ali. (2024). Survey of Multiple Destination Route Discovery Protocols. International Journal of Computational and Experimental Science and Engineering, 10(3);420-426. https://doi.org/10.22399/ijcesen.385
guven, mesut. (2024). Dynamic Malware Analysis Using a Sandbox Environment, Network Traffic Logs, and Artificial Intelligence. International Journal of Computational and Experimental Science and Engineering, 10(3);480-490. https://doi.org/10.22399/ijcesen.460
Serap ÇATLI DİNÇ, AKMANSU, M., BORA, H., ÜÇGÜL, A., ÇETİN, B. E., ERPOLAT, P., … ŞENTÜRK, E. (2024). Evaluation of a Clinical Acceptability of Deep Learning-Based Autocontouring: An Example of The Use of Artificial Intelligence in Prostate Radiotherapy. International Journal of Computational and Experimental Science and Engineering, 10(4);1181-1186. https://doi.org/10.22399/ijcesen.386
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
