面向深度学习密集匹配的无监督损失函数

doi:10.13474/j.cnki.11-2246.2024.1123

摘要/Abstract

摘要： 随着深度学习的发展,监督型密集匹配网络取得了瞩目的成就。然而,密集匹配的真实标签制作困难,获取成本高,基于无监督深度学习的密集匹配方法是未来趋势。目前,面向无监督密集匹配提出了众多损失函数,但各种组合损失复杂且存在效果不明的问题。为此,本文针对密集匹配无监督损失函数开展研究,分析各类损失的精度和匹配效果,并考证组合应用的有效性。结果表明,重构相似性约束是令无监督密集匹配网络匹配精度收敛的关键项,组合使用重构损失和左右一致损失有助于无纹理、弱纹理区域匹配,在此基础上加入相对平滑损失能更好适应阴暗环境。

关键词: 密集匹配, 深度学习, 无监督, 损失函数

Abstract: With the advancement of deep learning, supervised dense matching networks have achieved remarkable progress. However, obtaining real annotations for dense matching is challenging and costly, making unsupervised deep learning-based methods the future trend. Recently, numerous loss functions have been proposed for unsupervised dense matching. However, their combinations are complex and effects remain unknown. Therefore, this study investigates unsupervised loss functions for dense matching, analyzes the accuracy and matching performance of various losses, and validates the effectiveness of combined applications. The results demonstrate that the appearance matching loss plays a pivotal role in achieving convergence in accuracy for unsupervised dense matching networks. Combining appearance matching loss with left-right disparity consistency loss facilitates accurate non and weak textured region matches. Then, adding relative smoothing loss can better adapt to dark environments.

Key words: dense matching, deep learning, unsupervised, loss function

中图分类号:

P237

刘潇, 官恺, 金飞, 芮杰, 王淑香, 林雨准, 程传祥. 面向深度学习密集匹配的无监督损失函数[J]. 测绘通报, 2024, 0(11): 133-139.

LIU Xiao, GUAN Kai, JIN Fei, RUI Jie, WANG Shuxiang, LIN Yuzhun, CHENG Chuanxiang. Unsupervised loss function for dense matching in deep learning[J]. Bulletin of Surveying and Mapping, 2024, 0(11): 133-139.

参考文献

[1] 赵晨园, 李文新, 张庆熙.双目视觉的立体匹配算法研究进展[J].计算机科学与探索, 2020, 14(7):1104-1113.
[2] 季顺平, 罗冲, 刘瑾.基于深度学习的立体影像密集匹配方法综述[J].武汉大学学报(信息科学版), 2021, 46(2):193-202.
[3] 金飞, 官恺, 刘智, 等.无监督密集匹配特征提取网络性能分析[J].测绘学报, 2022, 51(3):426-436.
[4] POGGI M, TOSI F, BATSOS K, et al.On the synergies between machine learning and binocular stereo for depth estimation from images:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9):5314-5334.
[5] ZBONTAR J, LECUN Y.Computing the stereo matching cost with a convolutional neural network[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston, MA:IEEE, 2015:1592-1599.
[6] SEKI A, POLLEFEYS M.SGM-nets:semi-global matching with neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI:IEEE, 2017:231-240.
[7] MAYER N, ILG E, HAUSSER P, et al.A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas, NV:IEEE, 2016:4040-4048.
[8] ZBONTAR J, LECUN Y.Stereo matching by training a convolutional neural network to compare image patches[J].CoRR, 2015:abs/1510.05970.
[9] LIANG Zhengfa, FENG Yiliu, GUO Yulan, et al.Learning for disparity estimation through feature constancy[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake city, UT:IEEE, 2018:2811-2820.
[10] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al.End-to-end learning of geometry and context for deep stereo regression[C]// Proceedings of 2017 IEEE International Conference on Computer Vision.Venice:IEEE, 2017:66-75.
[11] CHANG Jiaren, CHEN Yongsheng.Pyramid stereo matching network[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT:IEEE, 2018:5410-5418.
[12] GUO Xiaoyang, YANG Kai, YANG Wukui, et al.Group-wise correlation stereo network[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach, CA:IEEE, 2019:3273-3282.
[13] XIE Junyuan, GIRSHICK R, FARHADI A.Deep3D:fully automatic 2D-to-3D video conversion with deep convolutional neural networks[M]//Computer Vision-ECCV 2016.Cham:Springer International Publishing, 2016:842-857.
[14] GARG R, VIJAY KUMAR B G, CARNEIRO G, et al.Unsupervised CNN for single view depth estimation:geometry to the rescue[M]//Computer Vision-ECCV 2016.Cham:Springer International Publishing, 2016:740-756.
[15] GODARD C, MAC AODHA O, BROSTOW G J.Unsupervised monocular depth estimation with left-right consistency[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI:IEEE, 2017:270-279.
[16] WANG Zhou, BOVIK A C, SHEIKH H R, et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[17] ZHONG Yiran, DAI Yuchao, LI Hongdong.Self-supervised learning for stereo matching with self-improving ability[J].Computer Science, Engineering, 2017:31613165.
[18] 王玉锋, 王宏伟, 吴晨, 等.基于共同视域的自监督立体匹配算法[J].光学学报, 2019, 39(2):0215004.
[19] GEIGER A, LENZ P, STILLER C, et al.Vision meets robotics:the KITTI dataset[J].International Journal of Robotics Research, 2013, 32(11):1231-1237.
[20] GEIGER A, LENZ P, URTASUN R.Are we ready for autonomous driving? The KITTI vision benchmark suite[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence, RI:IEEE, 2012:3354-3361.
[21] MENZE M, GEIGER A.Object scene flow for autonomous vehicles[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston, MA:IEEE, 2015:3061-3070.