关键词: 水稻分类, 多模态融合, 时间序列, 置信度引导, 自监督学习

Abstract: [Purposes]Optical remote sensing images of cloudy areas are susceptible to cloud contamination,leading to decreased crop classification accuracy.Synthetic aperture radar (SAR)possesses all-weather imaging capabilities and can complement optical data.This paper proposes a confidence-guided multi-modal Transformer rice extraction method.[Methods]Firstly,based on HLS optical and Sentinel-1 radar time-series data,This method uses cloud masks to estimate the confidence of optical data at each time phase.Then,a Transformer encoder extracts the temporal features of both optical and radar data separately,and a confidence-guided gating fusion mechanism is designed to adaptively fuse the two.Furthermore,a self-supervised temporal reconstruction strategy is introduced,which enhances the model's ability to compensate for SAR information when optical data is missing by performing mask reconstruction on some optical data,thus improving model robustness.[Findings] Experiments show that the proposed method achieves an overall accuracy of 88.00% and a F1 score of 88.25%,outperforming comparative models such as random forest,LSTM,and Transformer.[Conclusions] It effectively improves the rice extraction accuracy in cloudy areas and provides a reference for crop classification under complex climatic conditions.

Key words: rice classification, mult-imodal fusion, time series, confidence-guide, self-supervised learning