Unsupervised loss function for dense matching in deep learning

doi:10.13474/j.cnki.11-2246.2024.1123

Abstract

Abstract: With the advancement of deep learning, supervised dense matching networks have achieved remarkable progress. However, obtaining real annotations for dense matching is challenging and costly, making unsupervised deep learning-based methods the future trend. Recently, numerous loss functions have been proposed for unsupervised dense matching. However, their combinations are complex and effects remain unknown. Therefore, this study investigates unsupervised loss functions for dense matching, analyzes the accuracy and matching performance of various losses, and validates the effectiveness of combined applications. The results demonstrate that the appearance matching loss plays a pivotal role in achieving convergence in accuracy for unsupervised dense matching networks. Combining appearance matching loss with left-right disparity consistency loss facilitates accurate non and weak textured region matches. Then, adding relative smoothing loss can better adapt to dark environments.

Key words: dense matching, deep learning, unsupervised, loss function

CLC Number:

P237

LIU Xiao, GUAN Kai, JIN Fei, RUI Jie, WANG Shuxiang, LIN Yuzhun, CHENG Chuanxiang. Unsupervised loss function for dense matching in deep learning[J]. Bulletin of Surveying and Mapping, 2024, 0(11): 133-139.

References

[1] 赵晨园, 李文新, 张庆熙.双目视觉的立体匹配算法研究进展[J].计算机科学与探索, 2020, 14(7):1104-1113.
[2] 季顺平, 罗冲, 刘瑾.基于深度学习的立体影像密集匹配方法综述[J].武汉大学学报(信息科学版), 2021, 46(2):193-202.
[3] 金飞, 官恺, 刘智, 等.无监督密集匹配特征提取网络性能分析[J].测绘学报, 2022, 51(3):426-436.
[4] POGGI M, TOSI F, BATSOS K, et al.On the synergies between machine learning and binocular stereo for depth estimation from images:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9):5314-5334.
[5] ZBONTAR J, LECUN Y.Computing the stereo matching cost with a convolutional neural network[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston, MA:IEEE, 2015:1592-1599.
[6] SEKI A, POLLEFEYS M.SGM-nets:semi-global matching with neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI:IEEE, 2017:231-240.
[7] MAYER N, ILG E, HAUSSER P, et al.A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas, NV:IEEE, 2016:4040-4048.
[8] ZBONTAR J, LECUN Y.Stereo matching by training a convolutional neural network to compare image patches[J].CoRR, 2015:abs/1510.05970.
[9] LIANG Zhengfa, FENG Yiliu, GUO Yulan, et al.Learning for disparity estimation through feature constancy[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake city, UT:IEEE, 2018:2811-2820.
[10] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al.End-to-end learning of geometry and context for deep stereo regression[C]// Proceedings of 2017 IEEE International Conference on Computer Vision.Venice:IEEE, 2017:66-75.
[11] CHANG Jiaren, CHEN Yongsheng.Pyramid stereo matching network[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT:IEEE, 2018:5410-5418.
[12] GUO Xiaoyang, YANG Kai, YANG Wukui, et al.Group-wise correlation stereo network[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach, CA:IEEE, 2019:3273-3282.
[13] XIE Junyuan, GIRSHICK R, FARHADI A.Deep3D:fully automatic 2D-to-3D video conversion with deep convolutional neural networks[M]//Computer Vision-ECCV 2016.Cham:Springer International Publishing, 2016:842-857.
[14] GARG R, VIJAY KUMAR B G, CARNEIRO G, et al.Unsupervised CNN for single view depth estimation:geometry to the rescue[M]//Computer Vision-ECCV 2016.Cham:Springer International Publishing, 2016:740-756.
[15] GODARD C, MAC AODHA O, BROSTOW G J.Unsupervised monocular depth estimation with left-right consistency[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu, HI:IEEE, 2017:270-279.
[16] WANG Zhou, BOVIK A C, SHEIKH H R, et al.Image quality assessment:from error visibility to structural similarity[J].IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[17] ZHONG Yiran, DAI Yuchao, LI Hongdong.Self-supervised learning for stereo matching with self-improving ability[J].Computer Science, Engineering, 2017:31613165.
[18] 王玉锋, 王宏伟, 吴晨, 等.基于共同视域的自监督立体匹配算法[J].光学学报, 2019, 39(2):0215004.
[19] GEIGER A, LENZ P, STILLER C, et al.Vision meets robotics:the KITTI dataset[J].International Journal of Robotics Research, 2013, 32(11):1231-1237.
[20] GEIGER A, LENZ P, URTASUN R.Are we ready for autonomous driving? The KITTI vision benchmark suite[C]// Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence, RI:IEEE, 2012:3354-3361.
[21] MENZE M, GEIGER A.Object scene flow for autonomous vehicles[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston, MA:IEEE, 2015:3061-3070.