Land cover classification in multi-modal remote sensing images using dual attention and multi-branch losses

doi:10.13474/j.cnki.11-2246.2025.1113

Abstract

Abstract: Existing land cover classification methods still face numerous challenges in feature extraction and fusion quality.This paper proposes a multimodal image classification method named dual-attentive triple-branch fusionNet (DATF-Net),which integrate dual-attention collaboration and multi-branch joint loss.A cross-correlation feature enhancement strategy is adopted,and a channel-spatial attention collaboration mechanism is introduced to achieve comprehensive fusion of complementary multimodal features.The consistency of different branch decisions is ensured under the joint constraint of dice loss and cross-entropy loss.Ablation and comparative experiments are conducted on the Dongying multi-modal image dataset.The results demonstrate that the dual-attention collaboration mechanism and multi-branch joint loss function both contribute to improving land cover classification accuracy.Compared with other methods,DATF-Net achieves optimal performance in precision across various land cover categories and multiple overall classification metrics.Notably,The OA and FWIoU of DATF-Net outperform the second-best method (VFesuNet) by 7.9% and 12.66% respectively.The proposed method effectively mitigate the speckle noise interference in SAR images,enhance the coherence of classification boundaries,and improve classification accuracy and robustness in complex urban scenarios.

Key words: multi-modal remote sensing imagery, land cover classification, convolutional neural network, attention mechanism, loss function

CLC Number:

P237

YU Xiaowei, ZHENG Yadong, LIANG Li. Land cover classification in multi-modal remote sensing images using dual attention and multi-branch losses[J]. Bulletin of Surveying and Mapping, 2025, 0(11): 84-90,153.

References

[1] 叶沅鑫,徐其志,胡忠文.多模态遥感图像配准[M].北京:科学出版社,2022.
[2] ZHANG L,LI H,WANG Q.A review of optical and SAR image fusion techniques[J/OL].IEEE Transactions on Geoscience and Remote Sensing,2013,60(5):1793-1800.
[3] CHARBUTY B,ABDULAZEEZ A.Classification based on decision tree algorithm for machine learning[J].Journal of Applied Science and Technology Trends,2021,2(1):20-28.
[4] SAMANIEGO L,SCHULZ K.Supervised classification of agricultural land cover using a modified k-NN technique (MNN)and landsat remote sensing imagery[J].Remote Sensing,2009,1(4):875-895.
[5] ZHANG Fang,YANG Xiaojun.Improving land cover classification in an urbanized coastal area by random forests:the role of variable selection[J].Remote Sensing of Environment,2020,251:112105.
[6] PANEQUE-GÁLVEZ J,MAS J F,MORÉ G,et al.Enhanced land use/cover classification of heterogeneous tropical landscapes using support vector machines and textural homogeneity[J].International Journal of Applied Earth Observation and Geoinformation,2013,23:372-383.
[7] LI Qingyun,CHEN Yushi,ZENG Ying.Transformer with transfer CNN for remote-sensing-image object detection[J].Remote Sensing,2022,14(4):984.
[8] YUE Kai,YANG Lei,LI Ruirui,et al.TreeUNet:adaptive tree convolutional neural networks for subdecimeter aerial image segmentation[J].ISPRS Journal of Photogrammetry and Remote Sensing,2019,156:1-13.
[9] LI Xiao,LEI Lin,SUN Yuli,et al.Multimodal bilinear fusion network with second-order attention-based channel selection for land cover classification[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2020,13:1011-1026.
[10] LI Xiao,LEI Lin,SUN Yuli,et al.Collaborative attention-based heterogeneous gated fusion network for land cover classification[J].IEEE Transactions on Geoscience and Remote Sensing,2021,59(5):3829-3845.
[11] KANG Wenchao,XIANG Yuming,WANG Feng,et al.CFNet:a cross fusion network for joint land cover classification using optical and SAR images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2022,15:1562-1574.
[12] REN Bo,MA Shibin,HOU Biao,et al.A dual-stream high resolution network:deep fusion of GF-2 and GF-3 data for land cover classification[J].International Journal of Applied Earth Observation and Geoinformation,2022,112:102896.
[13] LI Xue,ZHANG Guo,CUI Hao,et al.MCANet:a joint semantic segmentation framework of optical and SAR images for land use classification[J].International Journal of Applied Earth Observation and Geoinformation,2022,106:102638.
[14] 刘潇,官恺,金飞,等.面向深度学习密集匹配的无监督损失函数[J].测绘通报,2024(11):133-139.
[15] MILLETARI F,NAVAB N,AHMADI S A.V-net:fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of 2016 Fourth International Conference on 3D Vision (3DV).Stanford,CA:IEEE,2016:565-571.
[16] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[17] TAN M,LE Q V.EfficientNet:Rethinking model scaling for convolutional neural networks[C]//Proceedings of the 36th International Conference on Machine Learning.Long Beach:PMLR,2019:6105-6114.
[18] ZHOU Zongwei,RAHMAN SIDDIQUEE M M,TAJBAKHSH N,et al.UNet++:a nested U-Net architecture for medical image segmentation[C]//Proceedings of 2018 Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support.Cham:Springer,2018:3-11.
[19] AUDEBERT N,LE SAUX B,LEFÈVRE S.Beyond RGB:very high resolution urban remote sensing with multimodal deep networks[J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,140:20-32.
[20] HUGHES L H,SCHMITT M,MOU Lichao,et al.Identifying corresponding patches in SAR and optical images with a pseudo-Siamese CNN[J].IEEE Geoscience and Remote Sensing Letters,2018,15(5):784-78
[21] XU Xiaodong,LI Wei,RAN Qiong,et al.Multisource remote sensing data classification based on convolutional neural network[J].IEEE Transactions on Geoscience and Remote Sensing,2018,56(2):937-949.