基于图像和点云融合的三维小目标检测方法

doi:10.13474/j.cnki.11-2246.2025.0306

测绘通报 ›› 2025, Vol. 0 ›› Issue (3): 33-38.doi: 10.13474/j.cnki.11-2246.2025.0306

• 学术研究 • 上一篇

基于图像和点云融合的三维小目标检测方法

郝佳¹, 姚国英¹, 周剑², 王斯远³, 肖进胜³

1. 92942部队, 北京 100161;
2. 武汉大学测绘遥感信息工程国家重点实验室, 湖北武汉 430072;
3. 武汉大学电子信息学院, 湖北武汉 430072

收稿日期:2024-08-27 发布日期:2025-04-03
通讯作者: 肖进胜。E-mail:xiaojs@whu.edu.cn
作者简介:郝佳(1987—),男,硕士,工程师,主要研究方向为航空保障系统。E-mail:472497914@qq.com
基金资助:
国家自然科学基金(42101448);2023年湖北省重大攻关项目(JD)(2023BAA02604)

3D small object detection method based on image and point cloud fusion

HAO Jia¹, YAO Guoying¹, ZHOU Jian², WANG Siyuan³, XIAO Jinsheng³

1. Troops 92942, Beijing 100161, China;
2. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430072, China;
3. School of Electronic Information, Wuhan University, Wuhan 430072, China

Received:2024-08-27 Published:2025-04-03

摘要/Abstract

摘要： 目标检测技术在人工智能、人脸识别、自动驾驶等关键领域发挥着至关重要的作用。三维点云目标检测,特别是对小目标的识别,仍然是技术发展中的一个难点。针对该问题,本文提出了一种新的三维检测网络,该网络融合了图像与点云数据,以显著提高三维小目标的检测精度。首先,利用YOLOv5进行精确的二维目标检测,并利用相机和激光雷达的坐标映射关系建立三维约束,从原始点云中提取出锥形感兴趣区域;然后,针对远处的点云小目标,提出了一种基于聚类优化的三维目标检测网络架构,将感兴趣区域的点云同时输入PointNet及聚类模块中,并对两者的检测结果进行融合判别,提升三维小目标检测精度。在KITTI数据集上的测试结果表明:与现有技术相比,本文算法在中等难度条件下,两种小目标物体的平均精度(AP)分别提升了15.94%、2.29%;在高难度条件下,分别提升了13.34%、2.86%。证明了本文算法在提升三维小目标检测精度方面的显著效果和实际应用潜力。

关键词: 三维目标检测, 小目标, 感兴趣区域, 点云聚类, 点云图像融合

Abstract: Object detection technology plays a pivotal role in key fields such as artificial intelligence, facial recognition, and autonomous driving. 3D point cloud object detection, especially for small objects, remains a significant challenge in technological development. To address this challenge, this paper proposes a novel 3D detection network that integrates image and point cloud data to significantly enhance the accuracy of 3D small object detection. The approach begins by leveraging Yolov5 for precise 2D object detection and establishing a 3D constraint using the coordinate mapping relationship between cameras and LiDAR to extract conical regions of interest from the raw point cloud data. Furthermore, to tackle the issue of detecting small objects in distant point clouds, a cluster-optimized 3D detection network architecture is introduced. This architecture simultaneously inputs the point clouds of the regions of interest into both the PointNet and clustering modules, and then fuses their detection results to improve the accuracy of 3D small object detection. Testing on the KITTI dataset shows that, compared to existing techniques, the proposed algorithm improves the average precision (AP) for two small object categories by 15.94% and 2.29% under moderate difficulty conditions, and by 13.34% and 2.86% under high difficulty conditions. These results underscore the significant impact and practical application potential of this algorithm in enhancing the accuracy of 3D small object detection.

Key words: 3D object detection, small object, area of interest, point cloud clustering, image and point cloud fusion

中图分类号:

P237

郝佳, 姚国英, 周剑, 王斯远, 肖进胜. 基于图像和点云融合的三维小目标检测方法[J]. 测绘通报, 2025, 0(3): 33-38.

HAO Jia, YAO Guoying, ZHOU Jian, WANG Siyuan, XIAO Jinsheng. 3D small object detection method based on image and point cloud fusion[J]. Bulletin of Surveying and Mapping, 2025, 0(3): 33-38.

参考文献

[1] 周燕,蒲磊,林良熙,等.激光点云的三维目标检测研究进展[J].计算机科学与探索,2022,16(12):2695-2717.
[2] DING M Y,HUO Y Q,YI H W,et al. Learning depth-guided convolutions for monocular 3D object detection[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:11669-11678.
[3] MA X Z,WANG Z H,LI H J,et al. Accurate monocular 3D object detection via color-embedded 3D reconstruction for autonomous driving[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul:IEEE,2019:6850-6859.
[4] WANG L,DU L,YE X Q,et al. Depth-conditioned dynamic message propagation for monocular 3D object detection[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville:IEEE,2021:454-463.
[5] MA X,LIU S,XIA Z,et al. Rethinking pseudo-lidar representation[C]//Proceedings of 2020 European Conference on Computer Vision.Glasgow:Springer Science and Business Media Deutschland GmbH,2020:311-327.
[6] CHARLES R Q,HAO S,MO K C,et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE,2017:77-85.
[7] QI C R,YI L,SU H,et al.PointNet++:Deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of 2017 Advances in Neural Information Processing Systems.Long Beach:NIPS,2017:5099-5108.
[8] LI Y Y,BU R,SUN M C,et al. PointCNN:convolution on X-transformed points[C] //Proceedings of 2018 Advances in Neural Information Processing Systems.Montreal:NIPS,2018:820-830.
[9] LIU H J,DU J X,ZHANG Y,et al. Extracting geometric and semantic point cloud features with gateway attention for accurate 3D object detection[J]. Engineering Applications of Artificial Intelligence,2023,123:106227.
[10] ZHOU Y,TUZEL O. VoxelNet:end-to-end learning for point cloud based 3D object detection[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake city:IEEE,2018:4490-4499.
[11] YAN Y,MAO Y X,LI B. SECOND:sparsely embedded convolutional detection[J].Sensors,2018,18(10):3337-3354.
[12] 黄远宪,李必军,黄琦,等.融合相机与激光雷达的目标检测、跟踪与预测[J/OL].武汉大学学报(信息科学版):1-8[2023-02-26].https://doi.org/10.13203/j.whugis20210614.
[13] CHEN X Z,MA H M,WAN J,et al. Multi-view 3D object detection network for autonomous driving[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu:IEEE,2017:6526-6534.
[14] KU J,MOZIFIAN M,LEE J,et al. Joint 3D proposal generation and object detection from view aggregation[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Madrid:IEEE,2018:1-8.
[15] QI C R,LIU W,WU C X,et al. Frustum PointNets for 3D object detection from RGB-D data[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake city:IEEE,2018:918-927.
[16] ZHU Y J,XIE J M,LIU M Y,et al. BF3D:bi-directional fusion 3D detector with semantic sampling and geometric mapping[J]. Image and Vision Computing,2023,139:104835.
[17] TAO B,YAN F W,YIN Z S,et al. A multimodal 3-D detector with attention from the corresponding modal[J]. IEEE Sensors Journal,2023,23(8):8581-8590.
[18] WANG Z X,JIA K. Frustum ConvNet:sliding Frustums to aggregate local point-wise features for amodal 3D object detection[C]//Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Macau:IEEE,2019:1742-1749.
[19] VORA S,LANG A H,HELOU B,et al. PointPainting:sequential fusion for 3D object detection[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle:IEEE,2020:4603-4611.
[20] 肖进胜,赵陶,周剑,等.基于上下文增强和特征提纯的小目标检测网络[J].计算机研究与发展,2023,60(2):465-474.
[21] WOO S,PARK J,LEE J Y,et al. CBAM:convolutional block attention module[M]//Lecture Notes in Computer Science. Cham:Springer International Publishing,2018:3-19.

基于图像和点云融合的三维小目标检测方法

3D small object detection method based on image and point cloud fusion

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 3

编辑推荐

Metrics

本文评价

[1]	王步云, 李宏伟, 赵姗. 融合点云与全景影像的路侧多目标识别[J]. 测绘通报, 2023, 0(10): 40-46.
[2]	江珊, 吕京国, 李现虎. 改进蚁群算法的三维激光点云聚类方法[J]. 测绘通报, 2018, 0(3): 38-42.
[3]	吴飞龙, 濮国梁, 程承旗, 王焕炯, 李滨. 面向移动服务的遥感影像感兴趣区提取压缩方法[J]. 测绘通报, 2016, 0(11): 35-38,54.