双分支网络架构下的图像相似度学习

doi:10.13474/j.cnki.11-2246.2019.0385

摘要/Abstract

摘要： 图像相似度学习是指通过网络学习图像内容信息来预测两张图像是否匹配。迄今为止，基于卷积神经网络改进的变体网络有效提升了学习效率，但由于提取特征比较单一无法准确描述图像特征，导致相似度学习效率较低。为此，本文提出一种基于卷积神经网络结构的双分支网络。该网络为左右分支网络结构相同，但权值不共享，网络输入为双分支输入。首先由左右分支网络分别提取单通道图像特征；然后通过特征融合层进行特征融合；最后将融合特征直接输入全连接层进行相似度学习，既改善了提取的图像特征多样性，又加快了模型训练速度。在实验室工业相机拍摄的芯片卡槽图像数据集上进行对比试验，结果表明，相比其他模型，本文提出的模型具有较强的网络学习能力和模型泛化能力，准确率高达97.96%。

关键词: 图像相似度学习, 卷积神经网络, 双分支网络, 权值不共享, 特征融合

Abstract: Image similarity learning is to predict whether two images match by learning image content information through the network. Until now, the improved variant network based on convolutional neural network has effectively improved the learning efficiency. However, because the extracted features are relatively single and cannot accurately describe the image features, the similarity learning efficiency is low. To this end, a two-branch network based on a convolutional neural network structure is proposed. The network is a left and right branch network, the structure is the same but the weights are not shared, and the network input is a two-branch input. Firstly, the single channel image features are extracted by the left and right branch networks respectively. Then the features fused through the feature fusion layer. Finally, the fusion feature is directly input into the fully connected layer for similarity learning. It not only improves the feature diversity of the extracted images, but also speeds up the training of the model. Comparative experiments were carried out on the chip card slot image dataset taken by the laboratory industrial camera. The experimental results show that compared with other models, the proposed model has strong network learning ability and model generalization ability, and the accuracy rate is 97.96%.

Key words: image similarity learning, convolutional neural network, two-branch network, weight not shared, feature fusion

中图分类号:

P234

卢健, 马成贤, 周嫣然, 李哲. 双分支网络架构下的图像相似度学习[J]. 测绘通报, 2019, 0(12): 50-55.

LU Jian, MA Chengxian, ZHOU Yanran, LI Zhe. Image similarity learning via two-branch network architecture[J]. Bulletin of Surveying and Mapping, 2019, 0(12): 50-55.

参考文献

[1] NOWAK E, JURIE F. Learning visual similarity measures for comparing never seen objects[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis:IEEE, 2007:1-8.
[2] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[3] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF)[J]. Computer Vision & Image Understanding, 2008, 110(3):346-359.
[4] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.[S.l.]:IEEE, 2018:586-595.
[5] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[6] ZHANG L, ZHANG L, MOU X, et al. FSIM:a feature similarity index for image quality assessment[J]. IEEE Transactions on Image Processing, 2011, 20(8):2378-2386.
[7] WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale structural similarity for image quality assessment[C]//The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers. Pacific Grove:IEEE, 2003, 2:1398-1402.
[8] 解天鹏,许妙忠,丛铭,等.基于Contourlet-GSSIM的无参考遥感图像质量评价研究[J].测绘通报,2015(1):59-62.
[9] 李鸣,张鸿.基于深度特征分析的双线性图像相似度匹配算法[J].计算机应用,2016,36(10):2822-2825,2831
[10] LITJENS G, KOOI T, BEJNORDI B E, et al. A survey on deep learning in medical image analysis[J]. Medical Image Analysis, 2017, 42:60-88.
[11] FISCHER P, DOSOVITSKIY A, BROX T. Descriptor matching with convolutional neural networks:a comparison to SIFT[J]. Computer Science, 2014:1867-1874.
[12] ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:4353-4361.
[13] HAN X, LEUNG T, JIA Y, et al. Matchnet:unifying feature and metric learning for patch-based matching[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:3279-3286.
[14] SIMO-SERRA E, TRULLS E, FERRAZ L, et al. Discrimin-ative learning of deep convolutional feature point descriptors[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago:IEEE, 2015:118-126.
[15] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//IEEE Computer Society Conference on Computer Vision & Pattern Recognition.[S.l.]:IEEE, 2005:539-546.
[16] NOROUZI M, FLEET D J, SALAKHUTDINOV R R. Hamming distance metric learning[C]//Advances in Neural Information Processing Systems. Lake Tohoe:[s.n.], 2012:1061-1069.
[17] SIMONYAN K, VEDALDI A, ZISSERMAN A. Learning local feature descriptors using convex optimisation[J]. IEEE Transactions Pattern Analysis Machine Intelligence, 2014, 36(8):1573-1585.
[18] ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:4353-4361.
[19] APPALARAJU S, CHAOJI V. Image similarity using Deep CNN and Curriculum Learning[J]. arXiv preprint arXiv,2017:1709.08761.
[20] SUBRAMANIAM A, BALASUBRAMANIAN P, MITTAL A. NCC-net:normalized cross correlation based deep matcher with robustness to illumination variations[C]//2018 IEEE Winter Conference on Applications of Computer Vision (WACV).Lake Tahoe:IEEE, 2018:1944-1953.
[21] BROWN M, HUA G, WINDER S. Discriminative learning of local image descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(1):43-57.
[22] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Lake Tohoe:[s.n.], 2012:1097-1105.
[23] ZBONTAR J, LECUN Y. Computing the stereo matching cost with a convolutional neural network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. Boston:IEEE, 2015:1592-1599.
[24] 朱玉玲,王建步,王安东,等.融合浅层特征的深度卷积神经网络互花米草遥感监测方法[J].海洋科学,2019,43(7):12-22.
[25] BROMLEY J, GUYON I, LECUN Y, et al. Signature verification using a "siamese" time delay neural network[C]//Advances in Neural Information Processing Systems. Denver:[s.n.], 1994:737-744.
[26] 卢健,马成贤,杨腾飞,等.Text-CRNN+Attention架构下的多类别文本信息分类[J/OL].计算机应用研究:1-6[2019-09-21].https://doi.org/10.19734/j.issn.1001-3695.2018.12.0858.