一种基于域适应的动态商品视觉识别方法

doi:10.13474/j.cnki.11-2246.2023.0064

测绘通报 ›› 2023, Vol. 0 ›› Issue (3): 10-15.doi: 10.13474/j.cnki.11-2246.2023.0064

一种基于域适应的动态商品视觉识别方法

雷洋洋¹, 李礼¹, 孙飞¹, 姚剑^1,2

1. 武汉大学遥感信息工程学院, 湖北武汉 430079;
2. 广东开放大学人工智能应用创新研究中心, 广东广州 510091

收稿日期:2022-03-31 发布日期:2023-04-04
通讯作者: 姚剑。E-mail:jian.yao@whu.edu.cn
作者简介:雷洋洋(1997-),女,硕士生,研究方向为图像检测与识别。E-mail:leiyangyang@whu.edu.cn
基金资助:
CCF-百度松果基金（OF2021023）；深圳市中央引导地方科技发展专项资金（2021Szvup100）

A dynamic commodity visual recognition method based on domain adaptation

LEI Yangyang¹, LI Li¹, SUN Fei¹, YAO Jian^1,2

1. School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China;
2. AI Application and Innovation Research Center, The Open University of Guangdong, Guangzhou 510091, China

Received:2022-03-31 Published:2023-04-04

摘要/Abstract

摘要： 由于变形、遮挡、运动模糊、商品之间外观的相似性及真实场景中未知的分布偏差，商品动态视觉识别在实际应用中仍存在巨大的挑战。本文提出了一种面向智能零售的动态商品视觉识别方法，即首先通过目标检测网络实时检测商品的外接矩形框，然后在此基础上识别商品的类别并给予推荐，辅助完成消费结算。针对商品拿取视频与商品库图像、训练图像之间的跨域差异，引入邻域风格自适应模型（IBN）和卷积注意力模块（CBAM），提升模型的域适应能力。为了验证该方法的有效性，构建了一个真实的场景数据集Commodity247，数据由智能货柜的顶视摄像头采集，包含247类常见的零售商品，以及37 050张带标注框和商品类别的图片。试验结果表明，在Commodity247数据集上，商品识别的准确率（mAP）可达96.84%，第一推荐正确率（Rank1）可达98.41%，最难样本检索准确率（mINP）可达85.24%；与基于ResNet搭建的基础模型相比，mAP提升了2.91%，Rank1提升了0.60%，mINP提升了10.86%，有效降低了多角度、多光线、多背景的影响。

关键词: 动态商品视觉识别, 邻域风格自适应, 注意力机制, 商品识别数据集

Abstract: Due to large deformation, occlusion, motion blur, similarity in appearance between items, and unknown distribution deviation in real scenes, item dynamic visual recognition still has huge challenges in practical applications. To this end, this paper proposes a dynamic commodity visual recognition method for smart retail. First, the bounding rectangle of the commodity is detected in real time through the target detection network, and then the category of the commodity is identified on this basis and recommendations are given to assist in the completion of consumer settlement. At the same time, in view of the cross-domain difference between the product picking video, the product library image, and the training image, this paper introduces a neighborhood style adaptive model (IBN) and a convolutional attention module (CBAM) to improve the domain adaptability of the model. In order to verify the effectiveness of this method, this paper constructs a real scene dataset Commodity247. The data is collected by the top-view camera of the smart container, including 247 common retail commodities and 37 050 pictures with annotated boxes and commodity categories. The experimental results show that on the Commodity247 dataset, the accuracy rate of product recognition(mAP) can reach 96.84%, the accuracy rate of the first recommendation(Rank1) can reach 98.41%, and the accuracy rate of the most difficult sample retrieval(mINP) can reach 85.24%, which is better than the one based on ResNet. For the basic model, mAP increases by 2.91%, Rank1 increases by 0.60%, and mINP increases by 10.86%, effectively reducing the influence of multi-angle, multi-light, and multi-background.

Key words: dynamic commodity recognition, instance batch normalization, attention, commodity recognition dataset

中图分类号:

P237

雷洋洋, 李礼, 孙飞, 姚剑. 一种基于域适应的动态商品视觉识别方法[J]. 测绘通报, 2023, 0(3): 10-15.

LEI Yangyang, LI Li, SUN Fei, YAO Jian. A dynamic commodity visual recognition method based on domain adaptation[J]. Bulletin of Surveying and Mapping, 2023, 0(3): 10-15.

参考文献

[1] 梁英男. 智能货柜场景下的商品识别算法的研究与系统设计[D]. 广州:华南理工大学, 2020.
[2] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas:IEEE, 2016:770-778.
[3] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:7132-7141.
[4] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu:IEEE, 2017:2261-2269.
[5] SANDLER M, HOWARD A, ZHU Menglong, et al. MobileNetV2:inverted residuals and linear bottlenecks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:4510-4520.
[6] ZHANG Xiangyu, ZHOU Xinyu, LIN Mengxiao, et al. ShuffleNet:an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:6848-6856.
[7] XIE Saining, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu:IEEE, 2017:5987-5995.
[8] ZHANG Hang, WU Chongruo, ZHANG Zhongyue, et al. ResNeSt:split-attention networks[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).New Orleans:IEEE, 2022:2735-2745.
[9] 张文强. 基于深度学习的商品检测和识别研究[D]. 成都:电子科技大学, 2019.
[10] ZHAN Xunlin, WU Yangxin, DONG Xiao, et al. Product1M:towards weakly supervised instance-level product retrieval via cross-modal pretraining[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal:IEEE, 2021:11762-11771.
[11] 李昊璇, 闫新艳. 基于深度残差收缩网络的商品图像识别[J]. 测试技术学报, 2021, 35(4):294-299.
[12] ZENG Weiyu, WANG Tianlei, CAO Jiuwen, et al. Clustering-guided pairwise metric triplet loss for person reidentification[J]. IEEE Internet of Things Journal, 2022, 9(16):15150-15160.
[13] KOMATSU R, GONSALVES T. Multi-CartoonGAN with conditional adaptive instance-layer normalization for conditional artistic face translation[J]. AI, 2022, 3(1):37-52.
[14] ZHONG Zhun, ZHENG Liang, CAO Donglin, et al. Re-ranking person re-identification with k-reciprocal encoding[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu:IEEE, 2017:3652-3661.

一种基于域适应的动态商品视觉识别方法

A dynamic commodity visual recognition method based on domain adaptation

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 9

编辑推荐

Metrics

本文评价

[1]	宋伟东, 毕春杨, 赵奉书. 改进Mask R-CNN公路病害检测算法[J]. 测绘通报, 2023, 0(1): 58-64.
[2]	唐菲菲, 唐天俊, 朱洪洲, 胡川, 马英, 李昕. 结合注意力机制和Bi-LSTM的降雨型滑坡位移预测[J]. 测绘通报, 2022, 0(9): 74-79,104.
[3]	潘建平, 李鑫, 孙博文, 胡勇, 李明明. 基于注意力密集连接金字塔网络的新增建设用地变化检测[J]. 测绘通报, 2022, 0(3): 41-46,59.
[4]	杨勇, 崔丽珍, 郭倩倩, 薛中浩. 煤矿井下基于Attention机制的LSTM地磁定位算法[J]. 测绘通报, 2022, 0(1): 72-78.
[5]	吴永静, 吴锦超, 林超, 窦宝成, 黎珂. 基于深度学习的高分辨率遥感影像光伏用地提取[J]. 测绘通报, 2021, 0(5): 96-101.
[6]	刘广进, 王光辉, 毕卫华, 刘慧杰, 杨化超. 改进D-LinkNet模型在国产卫星影像云检测中的应用[J]. 测绘通报, 2021, 0(11): 54-58,64.
[7]	马林飞, 倪欢, 周子涵. 基于注意力机制的高分遥感图像采石场识别[J]. 测绘通报, 2021, 0(11): 96-100.
[8]	胡建朗, 罗亚荣, 郭迟. 一种用于运动姿态评估的视觉伺服机器人[J]. 测绘通报, 2020, 0(8): 33-38.
[9]	张朕通, 单玉刚, 袁杰. 改进注意力机制的遥感地貌识别算法[J]. 测绘通报, 2020, 0(10): 93-96,100.