Adaptive Transformer-Based Multi-Modal Image Fusion for Real-Time Medical Diagnosis and Object Detection
DOI:
https://doi.org/10.22399/ijcesen.562Keywords:
Multi-Modal Image Fusion, Adaptive Transformer, Real-Time Medical Diagnosis, Attention Mechanism, Medical ImagingAbstract
In recent years, medical diagnosis and object detection have been significantly enhanced by the integration of multi-modal image fusion techniques. This study proposes an Adaptive Transformer-Based Multi-Modal Image Fusion (AT-MMIF) framework designed for real-time medical diagnosis and object detection. The framework employs a Transformer architecture to capture both global and local feature correlations across multiple imaging modalities, including MRI, CT, PET, and X-ray, for more accurate diagnostic results and faster object detection in medical imagery. The fusion process incorporates spatial and frequency-domain information to improve the clarity and detail of the output images, enhancing diagnostic accuracy. The adaptive attention mechanism within the Transformer dynamically adjusts to the relevant features of different image types, optimizing fusion in real time. This leads to an improved sensitivity (98.5%) and specificity (96.7%) in medical diagnosis. Additionally, the model significantly reduces false positives and negatives, with an F1 score of 97.2% in object detection tasks. The AT-MMIF framework is further optimized for real-time processing with an average inference time of 120 ms per image and a model size reduction of 35% compared to existing multi-modal fusion models. By leveraging the strengths of Transformer architectures and adaptive learning, the proposed framework offers a highly efficient and scalable solution for real-time medical diagnosis and object detection in various clinical settings, including radiology, oncology, and pathology.
References
Ahmed, F.Y.H.; Masli, A.A.; Khassawneh, B.; Yousif, J.H.; Zebari, D.A. (2023). Optimized Downlink Scheduling over LTE Network Based on Artificial Neural Network. Computers 12;179. https://doi.org/10.3390/computers12090179
Stojčić, M.; Banjanin, M.K.; Vasiljević, M.; Nedić, D.; Stjepanović, A.; Danilović, D.; Puzić, G. (2023) Predictive Modeling of Delay in an LTE Network by Optimizing the Number of Predictors Using Dimensionality Reduction Techniques. Appl. Sci. 13;8511. https://doi.org/10.3390/app13148511
Mao, Jingxuan, (2022). Machine Learning Based Energy Efficient Bandwidth Optimization, Electrical Engineering, Electronic Engineering, Information Engineering, 2024, p. 52.
Yang, H., Zhao, J., Lam, K., Xiong, Z., Wu, Q. & Xiao, L. (2022). Distributed deep reinforcement learning‑based spectrum and power allocation for heterogeneous networks. IEEE Transactions on Wireless Communications, 21(9);6935‑6948. https://dx.doi.org/10.1109/TWC.2022.3153175.
Yang, Y., Li, F., Zhang, X., Liu, Z., & Chan, K. Y. (2022). Dynamic power allocation in cellular network based on multi-agent double deep reinforcement learning. Computer Networks, 217, 109342. https://doi.org/10.1016/j.comnet.2022.109342
Chol Jong, Jae-Hyon Kim, Chang-Sop Pak, Chol-Man Nam, (2022). A Study on the Resource Block Allocation Method to Enhance the Total Energy Efficiency for LTE-A Networks, Wireless Personal Communications 123(11), DOI:10.1007/s11277-021-09260-y.
Z. Ali, S. Khaf, Z. H. Abbas, G. Abbas, F. Muhammad, and S. Kim, (2020). A Deep Learning Approach for Mobility-Aware and Energy-Efficient Resource Allocation in MEC, IEEE Access, 8;179530-179546, doi: 10.1109/ACCESS.2020.3028240.
R. Ruby, H. Yang, F. A. P. de Figueiredo, T. Huynh-The and K. Wu, (2023). Energy-Efficient Multiprocessor-Based Computation and Communication Resource Allocation in Two-Tier Federated Learning Networks, IEEE Internet of Things Journal, 10(7);5689-5703, doi: 10.1109/JIOT.2022.3153996.
N. Sharma and K. Kumar, (2023). Energy Efficient Clustering and Resource Allocation Strategy for Ultra-Dense Networks: A Machine Learning Framework, IEEE Transactions on Network and Service Management, 20(2);1884-1897, doi: 10.1109/TNSM.2022.3218819.
M. Merluzzi, P. D. Lorenzo and S. Barbarossa, (2021). Wireless Edge Machine Learning: Resource Allocation and Trade-Offs," IEEE Access, 9;45377-45398, doi: 10.1109/ACCESS.2021.3066559.
H. Dai, Y. Huang, Y. Xu, C. Li, B. Wang and L. Yang, (2019). Energy-Efficient Resource Allocation for Energy Harvesting-Based Device-to-Device Communication, IEEE Transactions on Vehicular Technology, 68(1);509-524, doi: 10.1109/TVT.2018.2881545.
X. Hou, J. Wang, C. Jiang, Z. Meng, J. Chen and Y. Ren, (2024). Efficient Federated Learning for Metaverse via Dynamic User Selection, Gradient Quantization, and Resource Allocation, IEEE Journal on Selected Areas in Communications, 42(4);850-866, doi: 10.1109/JSAC.2023.3345393.
A. Mughees, M. Tahir, M. A. Sheikh, and A. Ahad, (2021). Energy-Efficient Ultra-Dense 5G Networks: Recent Advances, Taxonomy and Future Research Directions, IEEE Access, 9;147692-147716, doi: 10.1109/ACCESS.2021.3123577.
C. He, Y. Zhou, G. Qian, X. Li and D. Feng, (2019). Energy Efficient Power Allocation Based on Machine Learning Generated Clusters for Distributed Antenna Systems, IEEE Access, 7;59575-59584, doi: 10.1109/ACCESS.2019.2914159.
A. B. M. Adam, Z. Wang, X. Wan, Y. Xu and B. Duo, (2022). Energy-Efficient Power Allocation in Downlink Multi-Cell Multi-Carrier NOMA: Special Deep Neural Network Framework, IEEE Transactions on Cognitive Communications and Networking, 8(4);1770-1783, doi: 10.1109/TCCN.2022.3198652.
Q. Zeng, Y. Du, K. Huang, and K. K. Leung, (2021). Energy-Efficient Resource Management for Federated Edge Learning With CPU-GPU Heterogeneous Computing, IEEE Transactions on Wireless Communications, 20(12);7947-7962, doi: 10.1109/TWC.2021.3088910.
M. Poposka, B. Jovanovski, V. Rakovic, D. Denkovski and Z. Hadzi-Velkov, (2023). Resource Allocation of NOMA Communication Systems for Federated Learning IEEE Communications Letters, 27(8);2108-2112, doi: 10.1109/LCOMM.2023.3286909.
J. Lin, D. Cui, Z. Peng, Q. Li, and J. He, (2020). A Two-Stage Framework for the Multi-User Multi-Data Center Job Scheduling and Resource Allocation, IEEE Access, 8;197863-197874, doi: 10.1109/ACCESS.2020.3033557.
P. Biswas, M. S. Akhtar, S. Saha, S. Majhi and A. Adhya, (2023). Q-Learning-Based Energy-Efficient Network Planning in IP-Over-EON, IEEE Transactions on Network and Service Management, 20(1);3-13, doi: 10.1109/TNSM.2022.3197329.
M. G. Brahmam and V. A. R, (2024). VMMISD: An Efficient Load Balancing Model for Virtual Machine Migrations via Fused Metaheuristics with Iterative Security Measures and Deep Learning Optimizations, IEEE Access, 12;39351-39374, doi: 10.1109/ACCESS.2024.3373465.
P, P., P, D., R, V., A, Y., & Natarajan, V. P. (2024). Chronic Lower Respiratory Diseases detection based on Deep Recursive Convolutional Neural Network. International Journal of Computational and Experimental Science and Engineering, 10(4);744-752. https://doi.org/10.22399/ijcesen.513
Er, H., Kantar, D., Acun, A. D., Gemici, A., Derin, N., & Ercan Kelek, S. (2024). Effects of Acetyl-L-Carnitine Administration on Auditory Evoked Potentials in Rats Exposed to Chronic Ethanol. International Journal of Computational and Experimental Science and Engineering, 10(1)6-10. https://doi.org/10.22399/ijcesen.252
M, V., V, J., K, A., Kalakoti, G., & Nithila, E. (2024). Explainable AI for Transparent MRI Segmentation: Deep Learning and Visual Attribution in Clinical Decision Support. International Journal of Computational and Experimental Science and Engineering, 10(4)575-584. https://doi.org/10.22399/ijcesen.479
Rama Lakshmi BOYAPATI, & Radhika YALAVAR. (2024). RESNET-53 for Extraction of Alzheimer’s Features Using Enhanced Learning Models. International Journal of Computational and Experimental Science and Engineering, 10(4)879-889. https://doi.org/10.22399/ijcesen.519
U. S. Pavitha, S. Nikhila, & Mohan, M. (2024). Hybrid Deep Learning Based Model for Removing Grid-Line Artifacts from Radiographical Images. International Journal of Computational and Experimental Science and Engineering, 10(4763-774). https://doi.org/10.22399/ijcesen.514
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
