A Cross-Attention CNN Framework for Spectral–Spatial Fusion of HSI and LiDAR Data
DOI:
https://doi.org/10.22399/ijcesen.4474Keywords:
Hyperspectral imaging (HSI), LIDAR, Deep learning, Data fusion, Land-cover classificationAbstract
Hyperspectral imagery (HSI) and Light Detection and Ranging (LiDAR) data furnish complementary spectral and structural information essential for precise land-cover classification. This paper presents a Cross-Attention Fusion Network (CAFN) that proficiently amalgamates spectral–spatial features from Hyperspectral Imaging (HSI) with elevation-based indicators from LiDAR. The suggested framework uses dual-branch Convolutional Neural Network (CNN) encoders to get representations that are specific to each modality. Then, it uses a cross-attention fusion mechanism that learns how modalities are related to each other in real time. This attention-driven interaction lets the model focus on important spectral-structural relationships without using sequential recurrent units like GRUs. This makes it more efficient and better at generalising. For final classification, fully connected layers further improve the fused features. Tests done on two well-known multimodal datasets, Houston 2013 and Trento, show that the proposed method is strong and better than others, with overall accuracies (OA) of 84.03% and 97.55%, respectively. Ablation studies validate that cross-attention-based fusion significantly surpasses single-modality and basic concatenation methods, affirming the proposed architecture's efficacy for multimodal land-cover classification.
References
[1] Cao, M., Zhao, G., Lv, G., Dong, A., Guo, Y., & Dong, X. (2024). Spectral–spatial–language fusion network for hyperspectral, LiDAR, and text data classification. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–15.
[2] Dong, W., Qu, J., Zhang, T., Xiao, S., & Li, Y. (2024). Contrastive constrained cross-scene model-informed interpretable classification strategy for hyperspectral and LiDAR data. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–14.
[3] Guo, F., Meng, Q., Li, Z., Wang, L., Zhang, J., & Hu, Y. (2024). Multisource feature embedding and interaction fusion network for coastal wetland classification with hyperspectral and LiDAR data. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–16.
[4] Han, W., Miao, W., Geng, J., & Jiang, W. (2024). CMSE: Cross-modal semantic enhancement network for classification of hyperspectral and LiDAR data. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–14.
[5] Huang, W., Wu, T., Zhang, X., Li, L., Lv, M., Jia, Z., Zhao, X., Ma, H., & Vivone, G. (2025). MCFTNet: Multimodal cross-layer fusion transformer network for hyperspectral and LiDAR data classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 18, 12803–12818.
[6] Jia, S., Zhan, Z., & Xu, M. (2021). Shearlet-based structure-aware filtering for hyperspectral and LiDAR data classification. Journal of Remote Sensing, 2021, 1–25.
[7] Li, Z., Sui, H., Luo, C., & Guo, F. (2023). Morphological convolution and attention calibration network for hyperspectral and LiDAR data classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16, 5728–5740.
[8] Li, Z., Wu, J., Zhang, Y., & Yan, Y. (2025). CMFNet: Cross-Mamba fusion network for hyperspectral and LiDAR data classification. IEEE Transactions on Geoscience and Remote Sensing, 63, 1–14.
[9] Lin, J., Gao, F., Qi, L., Dong, J., Du, Q., & Gao, X. (2025). Dynamic cross-modal feature interaction network for hyperspectral and LiDAR data classification. IEEE Transactions on Geoscience and Remote Sensing, 63, 1–16.
[10] Liu, G., Song, J., Chu, Y., Zhang, L., Li, P., & Xia, J. (2025). Deep fuzzy fusion network for joint hyperspectral and LiDAR data classification. Remote Sensing, 2025, 1–15.
[11] Wang, A., Lei, G., Dai, S., Wu, H., & Iwahori, Y. (2025). Multiscale attention feature fusion based on improved transformer for hyperspectral image and LiDAR data classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 18, 4124–4140.
[12] Xu, K., Wang, B., Zhu, Z., Jia, Z., & Fan, C. (2025). A contrastive learning enhanced adaptive multimodal fusion network for hyperspectral and LiDAR data classification. IEEE Transactions on Geoscience and Remote Sensing, 63, 1–19.
[13] Zhao, G., Ye, Q., Sun, L., Wu, Z., Pan, C., & Jeon, B. (2023). Joint classification of hyperspectral and LiDAR data using a hierarchical CNN and transformer. IEEE Transactions on Geoscience and Remote Sensing, 61, 1–16.
[14] Cao, M., Zhao, G., Lv, G., Dong, A., Guo, Y., & Dong, X. (2024). Spectral–Spatial–Language Fusion Network for Hyperspectral, LiDAR, and Text Data Classification. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–15.
[15] Ding, K., Lu, T., Fu, W., Li, S., & Ma, F. (2022). Global–Local Transformer Network for HSI and LiDAR Data Joint Classification. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–13.
[16] Gao, H., Feng, H., Zhang, Y., Fei, S., Shen, R., Xu, S., & Zhang, B. (2024). Interactive Enhanced Network Based on Multihead Self-Attention and Graph Convolution for Classification of Hyperspectral and LiDAR Data. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–16.
[17] Han, W., Miao, W., Geng, J., & Jiang, W. (2024). CMSE: Cross-Modal Semantic Enhancement Network for Classification of Hyperspectral and LiDAR Data. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–14.
[18] Huang, W., Wu, T., Zhang, X., Li, L., Lv, M., Jia, Z., Zhao, X., Ma, H., &Vivone, G. (2025). MCFTNet: Multimodal Cross-Layer Fusion Transformer Network for Hyperspectral and LiDAR Data Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 18, 12803–12818.
[19] Jia, S., Zhan, Z., & Xu, M. (2021). Shearlet-Based Structure-Aware Filtering for Hyperspectral and LiDAR Data Classification. Journal of Remote Sensing, 2021, 1–25.
[20] Li, Z., Sui, H., Luo, C., & Guo, F. (2023). Morphological Convolution and Attention Calibration Network for Hyperspectral and LiDAR Data Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16, 5728–5740.
[21] Li, Z., Wang, Y., Wang, L., Guo, F., Yang, Y., & Wei, J. (2024). Pseudolabeling Contrastive Learning for Semisupervised Hyperspectral and LiDAR Data Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 17099–17116.
[22] Lin, J., Gao, F., Qi, L., Dong, J., Du, Q., & Gao, X. (2025). Dynamic Cross-Modal Feature Interaction Network for Hyperspectral and LiDAR Data Classification. IEEE Transactions on Geoscience and Remote Sensing, 63, 1–16.
[23] Liu, G., Song, J., Chu, Y., Zhang, L., Li, P., & Xia, J. (2025). Deep Fuzzy Fusion Network for Joint Hyperspectral and LiDAR Data Classification. Remote Sensing, 2025.
[24] Song, Q., Mo, F., Ding, K., Xiao, L., Dian, R., Kang, X., & Li, S. (2025). MCFNet: Multiscale Cross-Domain Fusion Network for HSI and LiDAR Data Joint Classification. IEEE Transactions on Geoscience and Remote Sensing, 63, 1–12.
[25] Tang, X., Zou, Y., Ma, J., Zhang, X., Liu, F., & Jiao, L. (2024). Multiple Information Collaborative Fusion Network for Joint Classification of Hyperspectral and LiDAR Data. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–16.
[26] Wang, X., Song, L., Feng, Y., & Zhu, J. (2025). S3F2Net: Spatial-Spectral-Structural Feature Fusion Network for Hyperspectral Image and LiDAR Data Classification. IEEE Transactions on Circuits and Systems for Video Technology, 35, 4801–4815.
[27] Xu, K., Wang, B., Zhu, Z., Jia, Z., & Fan, C. (2025). A Contrastive Learning Enhanced Adaptive Multimodal Fusion Network for Hyperspectral and LiDAR Data Classification. IEEE Transactions on Geoscience and Remote Sensing, 63, 1–19.
[28] Yang, B., Wang, X., Xing, Y., Cheng, C., Jiang, W., & Feng, Q. (2024). Modality Fusion Vision Transformer for Hyperspectral and LiDAR Data Collaborative Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 17052–17065.
[29] Zhao, Y., Bao, W., Xu, J., & Xu, X. (2024). BIHAF-Net: Bilateral Interactive Hierarchical Adaptive Fusion Network for Collaborative Classification of Hyperspectral and LiDAR Data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 15971–15988.
[30] Berger, C., Riedel, F., Rosentreter, J., Stein, E., Hese, S., & Schmullius, C. (2025). Data fusion of hyperspectral and LiDAR data for urban microclimate modeling. Remote Sensing, 17, 1324.
[31] Zhang, Y., Peng, Y., & Zhou, C. (2023). Hyperspectral and LiDAR data classification for urban environment monitoring. Remote Sensing, 15, 4232.
[32] Roy, S. K., Deria, A., Hong, D., & Plaza, A. J. (2022). Hyperspectral and LiDAR data classification using joint CNNs and morphological feature learning. IEEE Transactions on Geoscience and Remote Sensing, 60, 12345.
[33] Yang, B., Wang, X., Xing, Y., Cheng, C., Jiang, W., & Feng, Q. (2024). Multimodal fusion models for land cover classification: A review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 17, 17052–17065.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
