基于非局部挖掘再现网络的自监督无人机航拍图像超分辨率重建

doi:10.13474/j.cnki.11-2246.2025.0104

摘要/Abstract

摘要： 近年来,深度卷积神经网络在无人机航拍图像超分辨率任务中应用,使得无人机航拍图像超分辨率的性能取得了巨大的提升。然而,基于卷积神经网络的超分辨率方法依赖于特定的训练数据集,这种数据集通常是通过固定双三次采用内核对图像进行下采样构建的。当处理后的图像不满足这种“理想”情况时,其性能将急剧下降。因此,本文提出一种基于非局部挖掘再现网络的自监督无人机航拍图像超分辨率重建方法(NLMRN)。NLMRN只需要一张输入图像,无需对外部数据集进行预训练。它利用无人机航拍图像内部信息的非局部再现性,对输入图像本身进行下采样,以获得较低分辨率的图像并进行训练。为了更好地学习非局部重复特征,使用非局部上下文挖掘块(NLCM)建立非局部区域间的关系,并选择全局特征图的子集补充每个特定位置,以获得精确细节和纹理的重建。NLCM有效弥补了卷积运算一次只能处理一个局部邻域的不足。通过大量的试验验证, NLMRN 在处理“非理想”条件下的无人机航拍图像时,明显优于其他先进的超分辨率方法。

关键词: 无人机航拍, 超分辨率, 非局部再现性, 全局特征图

Abstract: In recent years, the application of deep convolutional neural networks in UAV aerial image super-resolution tasks has made a huge leap in the performance of UAV aerial image super-resolution. However,super-resolution methods based-on convolutional neural network rely on specific training datasets, which are typically constructed by downsampling images using a fixed bicubic kernel. When the processed image does not meet this “ideal” situation, its performance will drop dramatically. This paper proposes a self-supervised UAV aerial image super-resolution reconstruction method based on non-local mining reconstruction network (NLMRN) to solve this problem. NLMRN does not require external datasets for pre-training, only an input image. It exploits the non-local reproducibility of the internal information of UAV aerial images by downsampling the input image itself to obtain a lower resolution image for training. To better learn non-local repeated features, we use the non-local context mining block (NLCM) to establish relationships between non-local regions and select a subset of global feature maps to supplement each specific location to obtain precise details and texture reconstruction. NLCM effectively makes up for the shortcoming that the convolution operation can only process one local neighborhood at a time. Through extensive experimental verification, NLMRN is significantly better than other advanced super-resolution methods when processing UAV aerial images under “non-ideal” conditions.

Key words: UAV aerial, super-resolution, non-local reproducibility, global feature map

中图分类号:

P237

张永挺, 林江涛, 谢绍敏, 刘剑. 基于非局部挖掘再现网络的自监督无人机航拍图像超分辨率重建[J]. 测绘通报, 2025, 0(1): 16-21.

ZHANG Yongting, LIN Jiangtao, XIE Shaomin, LIU Jian. Super-resolution reconstruction of aerial images from self-supervised UAV based on non-local mining reconstruction network[J]. Bulletin of Surveying and Mapping, 2025, 0(1): 16-21.

参考文献

[1] SUN Xian,WANG Bing,WANG Zhirui,et al. Research progress on few-shot learning for remote sensing image interpretation[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2021,14: 2387-2402.
[2] 张中方,李金屏,拜佩.基于多颜色空间融合的移动目标检测算法[J].济南大学学报(自然科学版),2011,25(2):191-195.
[3] LIN Haoning,SHI Zhenwei,ZOU Zhengxia.Fully convolutional network with task partitioning for inshore ship detection in optical remote sensing images[J].IEEE Geoscience and Remote Sensing Letters,2017,14(10): 1665-1669.
[4] 李庆武,朱国庆,周妍,等.基于特征在线选择的目标压缩跟踪算法[J].自动化学报,2015,41(11):1961-1970.
[5] ZHANG Lei,WU Xiaolin.An edge-guided image interpolation algorithm via directional filtering and data fusion[J].IEEE Transactions on Image Processing,2006,15(8): 2226-2238.
[6] ZHANG Kaibing,GAO Xinbo,TAO Dacheng,et al.Single image super-resolution with non-local means and steering kernel regression[J].IEEE Transactions on Image Processing,2012,21(11): 4544-4556.
[7] ZHANG Kai,ZUO Wangmeng,ZHANG Lei.Learning a single convolutional super-resolution network for multiple degradations[C]//Praceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT:IEEE,2018:3262-3271.
[8] DONG Chao,LOY C C,HE Kaiming,et al.Learning a deep convolutional network for image super-resolution[M]//Lecture Notes in Computer Science.Cham: Springer International Publishing,2014: 184-199.
[9] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Honolulu,HI:IEEE,2017: 136-144.
[10] YEO Y J,SAGONG M C,SHIN Y G,et al.Simple yet effective way for improving the performance of depth map super-resolution[J].IEEE Signal Processing Letters,2020,27: 2099-2103.
[11] HUANG Junjie,DRAGOTTI P L.Learning deep analysis dictionaries for image super-resolution[J].IEEE Transactions on Signal Processing,2020,68: 6633-6648.
[12] HUANG Liqin,ZHANG Jianjia,ZUO Yifan,et al.Pyramid-structured depth MAP super-resolution based on deep dense-residual network[J].IEEE Signal Processing Letters,2019,26(12): 1723-1727.
[13] HU Jie,SHEN Li,SUN Gang.Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT: IEEE,2018: 7132-7141.
[14] WOO S,PARK J,LEE J Y,et al.CBAM: convolutional block attention module[M]//Lecture Notes in Computer Science.Cham: Springer International Publishing,2018: 3-19.
[15] DAI Tao,CAI Jianrui,ZHANG Yongbing,et al.Second-order attention network for single image super-resolution[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach,CA:IEEE,2019:11065-11074.
[16] LEI Sen,SHI Zhenwei,ZOU Zhengxia.Super-resolution for remote sensing images via local-global combined network[J].IEEE Geoscience and Remote Sensing Letters,2017,14(8): 1243-1247.
[17] HAUT J M,PAOLETTI M E,FERNANDEZ-BELTRAN R,et al.Remote sensing single-image superresolution based on a deep compendium model[J].IEEE Geoscience and Remote Sensing Letters,2019,16(9): 1432-1436.
[18] AGUSTSSON E,TIMOFTE R.NTIRE 2017 challenge on single image super-resolution: dataset and study[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Honolulu,HI: IEEE,2017: 126-135.
[19] GLASNER D,BAGON S,IRANI M.Super-resolution from a single image[C]//Proceedings of the 12th International Conference on Computer Vision:Kyoto: IEEE,2009: 349-356.
[20] FREEMAN W T,JONES T R,PASZTOR E C.Example-based super-resolution[J].IEEE Computer Graphics and Applications,2002,22(2): 56-65.
[21] HUANG Jiabin,SINGH A,AHUJA N.Single image super-resolution from transformed self-exemplars[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA: IEEE,2015: 5197-5206.
[22] WANG Xiaolong,GIRSHICK R,GUPTA A,et al.Non-local neural networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,UT:IEEE,2018:7794-7803.
[23] LIU D,WEN B,FAN Y,et al.Non-local recurrent network for image restoration[J].Advances in Neural Information Processing Systems,2018:47007607.
[24] ZHANG Yulun,LI Kunpeng,LI Kai,et al.Image super-resolution using very deep residual channel attention networks[M]//Lecture Notes in Computer Science.Cham: Springer International Publishing,2018: 294-310.
[25] WANG Zhou,BOVIK A C,SHEIKH H R,et al.Image quality assessment: from error visibility to structural similarity[J].IEEE Transactions on Image Processing,2004,13(4): 600-612.
[26] ZHAO Lijun,TANG Ping,HUO Lianzhi.Feature significance-based multibag-of-visual-words model for remote sensing image scene classification[J].Journal of Applied Remote Sensing,2016,10(3): 035004.
[27] YANG Yi,NEWSAM S.Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems.San Jose California: ACM Press,2010: 270-279.
[28] CHENG Gong,HAN Junwei,LU Xiaoqiang.Remote sensing image scene classification: benchmark and state of the art[J].Proceedings of the IEEE,2017,105(10): 1865-1883.
[29] BELL-KLIGLER S,SHOCHER A,IRANI M.Blind super-resolution kernel estimation using an internal-GAN[EB/OL].[2024-09-01].https://arxiv.org/abs/1909.06581v6.
[30] MEI Yiqun,FAN Yuchen,ZHOU Yuqian,et al.Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle,WA: IEEE,2020: 5690-5699.