联合双注意力和多分支损失的多模态遥感影像分类方法

doi:10.13474/j.cnki.11-2246.2025.1113

摘要/Abstract

摘要： 现有的多模态遥感影像土地覆盖分类方法在特征提取、融合质量等方面仍面临众多挑战。基于此,本文提出了一种联合双注意力协同与多分支联合损失的多模态影像分类方法(DATF-Net)。该方法采取互相关特征增强策略,引入通道-空间注意力协同机制,实现了多模态互补特征的充分融合;在Dice损失和交叉熵损失的联合约束下,保证不同分支决策的一致性。在东营多模态影像数据集上进行了消融和对比试验。结果表明,双注意力协同机制和多分支联合损失函数均有助于提升地物分类的精度。同时,相较其他方法,DATF-Net在各类地物精确度及多个总体分类指标上均为最优,尤其在总体分类正确率(OA)和加权平均交并比(FWIoU)两个关键指标上,DATF-Net较次优方法VFesuNet分别提高了7.9%和12.66%。本文方法不仅缓解了SAR影像的斑点噪声干扰,增强了分类边界的连贯性,同时提升了在城市复杂场景下的分类精度与稳健性。

关键词: 多模态遥感影像, 土地覆盖分类, 卷积神经网络, 注意力机制, 损失函数

Abstract: Existing land cover classification methods still face numerous challenges in feature extraction and fusion quality.This paper proposes a multimodal image classification method named dual-attentive triple-branch fusionNet (DATF-Net),which integrate dual-attention collaboration and multi-branch joint loss.A cross-correlation feature enhancement strategy is adopted,and a channel-spatial attention collaboration mechanism is introduced to achieve comprehensive fusion of complementary multimodal features.The consistency of different branch decisions is ensured under the joint constraint of dice loss and cross-entropy loss.Ablation and comparative experiments are conducted on the Dongying multi-modal image dataset.The results demonstrate that the dual-attention collaboration mechanism and multi-branch joint loss function both contribute to improving land cover classification accuracy.Compared with other methods,DATF-Net achieves optimal performance in precision across various land cover categories and multiple overall classification metrics.Notably,The OA and FWIoU of DATF-Net outperform the second-best method (VFesuNet) by 7.9% and 12.66% respectively.The proposed method effectively mitigate the speckle noise interference in SAR images,enhance the coherence of classification boundaries,and improve classification accuracy and robustness in complex urban scenarios.

Key words: multi-modal remote sensing imagery, land cover classification, convolutional neural network, attention mechanism, loss function

中图分类号:

P237

禹小伟, 郑亚东, 梁莉. 联合双注意力和多分支损失的多模态遥感影像分类方法[J]. 测绘通报, 2025, 0(11): 84-90,153.

YU Xiaowei, ZHENG Yadong, LIANG Li. Land cover classification in multi-modal remote sensing images using dual attention and multi-branch losses[J]. Bulletin of Surveying and Mapping, 2025, 0(11): 84-90,153.

参考文献

[1] 叶沅鑫,徐其志,胡忠文.多模态遥感图像配准[M].北京:科学出版社,2022.
[2] ZHANG L,LI H,WANG Q.A review of optical and SAR image fusion techniques[J/OL].IEEE Transactions on Geoscience and Remote Sensing,2013,60(5):1793-1800.
[3] CHARBUTY B,ABDULAZEEZ A.Classification based on decision tree algorithm for machine learning[J].Journal of Applied Science and Technology Trends,2021,2(1):20-28.
[4] SAMANIEGO L,SCHULZ K.Supervised classification of agricultural land cover using a modified k-NN technique (MNN)and landsat remote sensing imagery[J].Remote Sensing,2009,1(4):875-895.
[5] ZHANG Fang,YANG Xiaojun.Improving land cover classification in an urbanized coastal area by random forests:the role of variable selection[J].Remote Sensing of Environment,2020,251:112105.
[6] PANEQUE-GÁLVEZ J,MAS J F,MORÉ G,et al.Enhanced land use/cover classification of heterogeneous tropical landscapes using support vector machines and textural homogeneity[J].International Journal of Applied Earth Observation and Geoinformation,2013,23:372-383.
[7] LI Qingyun,CHEN Yushi,ZENG Ying.Transformer with transfer CNN for remote-sensing-image object detection[J].Remote Sensing,2022,14(4):984.
[8] YUE Kai,YANG Lei,LI Ruirui,et al.TreeUNet:adaptive tree convolutional neural networks for subdecimeter aerial image segmentation[J].ISPRS Journal of Photogrammetry and Remote Sensing,2019,156:1-13.
[9] LI Xiao,LEI Lin,SUN Yuli,et al.Multimodal bilinear fusion network with second-order attention-based channel selection for land cover classification[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2020,13:1011-1026.
[10] LI Xiao,LEI Lin,SUN Yuli,et al.Collaborative attention-based heterogeneous gated fusion network for land cover classification[J].IEEE Transactions on Geoscience and Remote Sensing,2021,59(5):3829-3845.
[11] KANG Wenchao,XIANG Yuming,WANG Feng,et al.CFNet:a cross fusion network for joint land cover classification using optical and SAR images[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2022,15:1562-1574.
[12] REN Bo,MA Shibin,HOU Biao,et al.A dual-stream high resolution network:deep fusion of GF-2 and GF-3 data for land cover classification[J].International Journal of Applied Earth Observation and Geoinformation,2022,112:102896.
[13] LI Xue,ZHANG Guo,CUI Hao,et al.MCANet:a joint semantic segmentation framework of optical and SAR images for land use classification[J].International Journal of Applied Earth Observation and Geoinformation,2022,106:102638.
[14] 刘潇,官恺,金飞,等.面向深度学习密集匹配的无监督损失函数[J].测绘通报,2024(11):133-139.
[15] MILLETARI F,NAVAB N,AHMADI S A.V-net:fully convolutional neural networks for volumetric medical image segmentation[C]//Proceedings of 2016 Fourth International Conference on 3D Vision (3DV).Stanford,CA:IEEE,2016:565-571.
[16] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks[J].Communications of the ACM,2017,60(6):84-90.
[17] TAN M,LE Q V.EfficientNet:Rethinking model scaling for convolutional neural networks[C]//Proceedings of the 36th International Conference on Machine Learning.Long Beach:PMLR,2019:6105-6114.
[18] ZHOU Zongwei,RAHMAN SIDDIQUEE M M,TAJBAKHSH N,et al.UNet++:a nested U-Net architecture for medical image segmentation[C]//Proceedings of 2018 Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support.Cham:Springer,2018:3-11.
[19] AUDEBERT N,LE SAUX B,LEFÈVRE S.Beyond RGB:very high resolution urban remote sensing with multimodal deep networks[J].ISPRS Journal of Photogrammetry and Remote Sensing,2018,140:20-32.
[20] HUGHES L H,SCHMITT M,MOU Lichao,et al.Identifying corresponding patches in SAR and optical images with a pseudo-Siamese CNN[J].IEEE Geoscience and Remote Sensing Letters,2018,15(5):784-78
[21] XU Xiaodong,LI Wei,RAN Qiong,et al.Multisource remote sensing data classification based on convolutional neural network[J].IEEE Transactions on Geoscience and Remote Sensing,2018,56(2):937-949.