MedFusionAI: A Deep Learning Framework for Multi-Modal Health Data Fusion to Predict Chronic Disease Risks
DOI:
https://doi.org/10.22399/ijcesen.2159Keywords:
Multi-Modal Data Fusion, Deep Learning, Chronic Disease Prediction, Attention Mechanism, Clinical Decision SupportAbstract
Chronic diseases are still one of the most common causes of death worldwide, so early and precise predictive models should be developed to help improve patient disease management and healthcare service delivery. With multi-modal medical data, including more prevalent structured sources like Electronic Health Records (EHR) and lab tests, to unstructured sources like clinical notes, wearable sensor streams, and medical imaging, the potential for AI-driven health analytics is enormous. Current methods, however, are plagued with limitations including: (1) dependence on single-modality data; (2) poor consideration of the missing data problem; and (3) suboptimal modelling of the inter-modality relationship, which can lead to suboptimal performance. These problems demonstrate the necessity for a standard and solid mechanism to integrate multiple disparate data sources effectively. This paper presents MedFusionAI, a novel deep learning framework for multi-modal medical data fusion in the chronic disease risk prediction task context. The proposed model utilizes dedicated modality-specific encoders: MLP for EHR, LSTM for sequential lab and measurements from wearable devices, CNN models for various medical images, and ClinicalBERT for text to extract salient features. Experiments on benchmark healthcare datasets show that MedFusionAI effectively outperforms previous baselines and fusion models with a 98.76% accuracy and high precision, recall, and AUC-ROC across all risk classes. The framework also provides interpretability features to enable clinicians to understand the contributions of features. MedFusionAI provides an interpretable, scalable, and reliable solution for clinical decision support systems for the timely visibility of chronic disease risks, improving preventive care.
References
[1] Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J. (2022). Multimodal deep learning for biomedical data fusion: A review. Briefings in Bioinformatics, 23(2). https://doi.org/10.1093/bib/bbab569 DOI: https://doi.org/10.1093/bib/bbab569
[2] Steyaert, S., Pizurica, M., Nagaraj, D., et al. (2023). Multimodal data fusion for cancer biomarker discovery with deep learning. Nature Machine Intelligence, 5, 351–362. https://doi.org/10.1038/s42256-023-00633-5 DOI: https://doi.org/10.1038/s42256-023-00633-5
[3] Fan, Z., Yan, Z., & Wen, S. (2023). Deep learning and artificial intelligence in sustainability: A review of SDGs, renewable energy, and environmental health. Sustainability, 15(18), 13493. DOI: https://doi.org/10.3390/su151813493
[4] Cardoso, M. J., Li, W., Brown, R., Ma, N., Kerfoot, E., Wang, Y., Murrey, B., et al. (2022). Monai: An open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.02701.
[5] Siddique, S., & Chow, J. C. L. (2021). Machine learning in healthcare communication. Encyclopedia, 1(1), 220–239. DOI: https://doi.org/10.3390/encyclopedia1010021
[6] Bharadwaj, H. K., et al. (2021). A review on the role of machine learning in enabling IoT based healthcare applications. IEEE Access, 9, 38859–38890. https://doi.org/10.1109/ACCESS.2021.3059858 DOI: https://doi.org/10.1109/ACCESS.2021.3059858
[7] van der Schaar, M., Alaa, A. M., Floto, A., et al. (2021). How artificial intelligence and machine learning can help healthcare systems respond to COVID-19. Machine Learning, 110, 1–14. https://doi.org/10.1007/s10994-020-05928-x DOI: https://doi.org/10.1007/s10994-020-05928-x
[8] Ghazal, T. M., Hasan, M. K., Alshurideh, M. T., Alzoubi, H. M., Ahmad, M., Akbar, S. S., ... & Akour, I. A. (2021). IoT for smart cities: Machine learning approaches in smart healthcare—A review. Future Internet, 13(8), 218. DOI: https://doi.org/10.3390/fi13080218
[9] Munirathnam, R., & Kanchetti, D. (2024). Artificial intelligence (AI)-powered predictive models in chronic disease management: A data-driven approach. International Journal of Computer Science and Information Technology Research, 5(1), 42–54.
[10] Xie, Y., Lu, L., Gao, F., et al. (2021). Integration of artificial intelligence, blockchain, and wearable technology for chronic disease management: A new paradigm in smart healthcare. Current Medical Science, 41, 1123–1133. https://doi.org/10.1007/s11596-021-2485-0 DOI: https://doi.org/10.1007/s11596-021-2485-0
[11] Decharatanachart, P., Chaiteerakij, R., Tiyarattanachai, T., et al. (2021). Application of artificial intelligence in chronic liver diseases: A systematic review and meta-analysis. BMC Gastroenterology, 21, 10. https://doi.org/10.1186/s12876-020-01585-5 DOI: https://doi.org/10.1186/s12876-020-01585-5
[12] Feng, Y., Wang, Y., Zeng, C., & Mao, H. (2021). Artificial intelligence and machine learning in chronic airway diseases: Focus on asthma and chronic obstructive pulmonary disease. International Journal of Medical Sciences, 18(13), 2871–2889. https://doi.org/10.7150/ijms.58191 DOI: https://doi.org/10.7150/ijms.58191
[13] Singh, V., Asari, V. K., & Rajasekaran, R. (2022). A deep neural network for early detection and prediction of chronic kidney disease. Diagnostics, 12(1), 116. DOI: https://doi.org/10.3390/diagnostics12010116
[14] Akter, S., et al. (2021). Comprehensive performance assessment of deep learning models in early prediction and risk identification of chronic kidney disease. IEEE Access, 9, 165184–165206. https://doi.org/10.1109/ACCESS.2021.3129491 DOI: https://doi.org/10.1109/ACCESS.2021.3129491
[15] Sarp, S., Kuzlu, M., Wilson, E., Cali, U., & Guler, O. (2021). The enlightening role of explainable artificial intelligence in chronic wound classification. Electronics, 10(12), 1406. DOI: https://doi.org/10.3390/electronics10121406
[16] Krishnamurthy, S., Ks, K., Dovgan, E., Luštrek, M., Gradišek Piletič, B., Srinivasan, K., ... & Syed-Abdul, S. (2021, May). Machine learning prediction models for chronic kidney disease using national health insurance claim data in Taiwan. In Healthcare, 9(5), 546. MDPI. DOI: https://doi.org/10.3390/healthcare9050546
[17] Ayesha, S., Hanif, M. K., & Talib, R. (2022). Performance enhancement of predictive analytics for health informatics using dimensionality reduction techniques and fusion frameworks. IEEE Access, 10, 753–769. https://doi.org/10.1109/ACCESS.2021.3139123 DOI: https://doi.org/10.1109/ACCESS.2021.3139123
[18] Khan, F., & Jack, H. (2025). From data streams to decisions: Enhancing healthcare analytics with AI and machine learning. [Journal Name Missing].
[19] Zhou, X., Leung, C. K., Wang, K. I.-K., & Fortino, G. (2024). Editorial: Deep learning-empowered big data analytics in biomedical applications and digital healthcare. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 21(4), 516–520. https://doi.org/10.1109/TCBB.2024.3371808 DOI: https://doi.org/10.1109/TCBB.2024.3371808
[20] Rehman, A., Naz, S., & Razzak, I. (2022). Leveraging big data analytics in healthcare enhancement: Trends, challenges and opportunities. Multimedia Systems, 28, 1339–1371. https://doi.org/10.1007/s00530-020-00736-8 DOI: https://doi.org/10.1007/s00530-020-00736-8
[21] Ahmed, A., Xi, R., Hou, M., Shah, S. A., & Hameed, S. (2023). Harnessing big data analytics for healthcare: A comprehensive review of frameworks, implications, applications, and impacts. IEEE Access, 11, 112891–112928. https://doi.org/10.1109/ACCESS.2023.3323574 DOI: https://doi.org/10.1109/ACCESS.2023.3323574
[22] Rehan, H. (2023). Artificial intelligence and machine learning: The impact of machine learning on predictive analytics in healthcare. Innovative Computer Sciences Journal, 9(1), 1–20.
[23] Ye, J. (2021). The impact of electronic health record–integrated patient-generated health data on clinician burnout. Journal of the American Medical Informatics Association, 28(5), 1051–1056. DOI: https://doi.org/10.1093/jamia/ocab017
[24] Koren, A., & Prasad, R. (2022). IoT health data in electronic health records (EHR): Security and privacy issues in era of 6G. Journal of ICT Standardization, 10(1), 63–84. https://doi.org/10.13052/jicts2245-800X.1014 DOI: https://doi.org/10.13052/jicts2245-800X.1014
[25] Espinoza, J. C., Yeung, A. M., Huang, J., Seley, J. J., Longo, R., & Klonoff, D. C. (2023). The need for data standards and implementation policies to integrate insulin delivery data into the electronic health record. Journal of Diabetes Science and Technology, 17(5), 1376–1386. https://doi.org/10.1177/19322968231178549 DOI: https://doi.org/10.1177/19322968231178549
[26] Zainab, H., et al. (2024). Integration of AI and wearable devices for continuous cardiac health monitoring. International Journal of Multidisciplinary Sciences and Arts, 3(4), 123–139.
[27] Reda, R., Piccinini, F., Martinelli, G., et al. (2022). Heterogeneous self-tracked health and fitness data integration and sharing according to a linked open data approach. Computing, 104, 835–857. https://doi.org/10.1007/s00607-021-00988-w DOI: https://doi.org/10.1007/s00607-021-00988-w
[28] Melstrom, L. G., Rodin, A. S., Rossi, L. A., Fu, P., Fong, Y., & Sun, V. (2021). Patient-generated health data and electronic health record integration in oncologic surgery: A call for artificial intelligence and machine learning. Journal of Surgical Oncology, 123(1), 52–60. DOI: https://doi.org/10.1002/jso.26232
[29] Aaron, R. E., Tian, T., Yeung, A. M., Huang, J., Klonoff, D. C., & Espinoza, J. C. (2023). The launch of the iCoDE-2 standard project: Integration of connected diabetes device data into the electronic health record. Journal of Diabetes Science and Technology, 18(1), 82–88. https://doi.org/10.1177/19322968231207888 DOI: https://doi.org/10.1177/19322968231207888
[30] Kormiltsyn, A., Udokwu, C., Dwivedi, V., Norta, A., & Nisar, S. (2023). Privacy-conflict resolution for integrating personal and electronic health records in blockchain-based systems. Blockchain in Healthcare Today, 6. https://doi.org/10.30953/bhty.v6.276
[31] Wang, W.-H., & Hsu, W.-S. (2023). Integrating artificial intelligence and wearable IoT system in long-term care environments. Sensors, 23(13), 5913. DOI: https://doi.org/10.3390/s23135913
[32] Kormiltsyn, A., & Norta, A. (2022). Formal evaluation of privacy-conflict resolution for integrating personal-and electronic health records in blockchain-based systems. [Technical Report]. DOI: https://doi.org/10.30953/bhty.v6.276
[33] Sattar, N., McGuire, D. K., Pavo, I., et al. (2022). Tirzepatide cardiovascular event risk assessment: A pre-specified meta-analysis. Nature Medicine, 28, 591–598. https://doi.org/10.1038/s41591-022-01707-4 DOI: https://doi.org/10.1038/s41591-022-01707-4
[34] Yi, J. K., Rim, T. H., Park, S., Kim, S. S., Kim, H. C., Lee, C. J., ... & Cheng, C. Y. (2023). Cardiovascular disease risk assessment using a deep-learning-based retinal biomarker: A comparison with existing risk scores. European Heart Journal – Digital Health, 4(3), 236–244. https://doi.org/10.1093/ehjdh/ztad023 DOI: https://doi.org/10.1093/ehjdh/ztad023
[35] Qiu, S., Miller, M. I., Joshi, P. S., et al. (2022). Multimodal deep learning for Alzheimer’s disease dementia assessment. Nature Communications, 13, 3404. https://doi.org/10.1038/s41467-022-31037-5 DOI: https://doi.org/10.1038/s41467-022-31037-5
[36] Nancy, A. A., Ravindran, D., Raj Vincent, P. D., Srinivasan, K., & Gutierrez Reina, D. (2022). IoT-cloud-based smart healthcare monitoring system for heart disease prediction via deep learning. Electronics, 11(15), 2292. DOI: https://doi.org/10.3390/electronics11152292
[37] Wang, D. D., Nguyen, L. H., Li, Y., et al. (2021). The gut microbiome modulates the protective association between a Mediterranean diet and cardiometabolic disease risk. Nature Medicine, 27, 333–343. https://doi.org/10.1038/s41591-020-01223-3 DOI: https://doi.org/10.1038/s41591-020-01223-3
[38] de Rojas, I., Moreno-Grau, S., Tesi, N., et al. (2021). Common variants in Alzheimer’s disease and risk stratification by polygenic risk scores. Nature Communications, 12, 3417. https://doi.org/10.1038/s41467-021-22491-8 DOI: https://doi.org/10.1038/s41467-021-22491-8
[39] Garbarino, S., Lanteri, P., Bragazzi, N. L., et al. (2021). Role of sleep deprivation in immune-related disease risk and outcomes. Communications Biology, 4, 1304. https://doi.org/10.1038/s42003-021-02825-4 DOI: https://doi.org/10.1038/s42003-021-02825-4
[40] Ahmad, W., Alharthy, R. D., Zubair, M., et al. (2021). Toxic and heavy metals contamination assessment in soil and water to evaluate human health risk. Scientific Reports, 11, 17006. https://doi.org/10.1038/s41598-021-94616-4 DOI: https://doi.org/10.1038/s41598-021-94616-4
[41] Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L. W. H., Feng, M., Ghassemi, M., ... & Mark, R. G. (2021). MIMIC-IV (version 1.0). PhysioNet. https://doi.org/10.13026/s6n6-xd98
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
