3D small object detection method based on image and point cloud fusion

doi:10.13474/j.cnki.11-2246.2025.0306

Abstract

Abstract: Object detection technology plays a pivotal role in key fields such as artificial intelligence, facial recognition, and autonomous driving. 3D point cloud object detection, especially for small objects, remains a significant challenge in technological development. To address this challenge, this paper proposes a novel 3D detection network that integrates image and point cloud data to significantly enhance the accuracy of 3D small object detection. The approach begins by leveraging Yolov5 for precise 2D object detection and establishing a 3D constraint using the coordinate mapping relationship between cameras and LiDAR to extract conical regions of interest from the raw point cloud data. Furthermore, to tackle the issue of detecting small objects in distant point clouds, a cluster-optimized 3D detection network architecture is introduced. This architecture simultaneously inputs the point clouds of the regions of interest into both the PointNet and clustering modules, and then fuses their detection results to improve the accuracy of 3D small object detection. Testing on the KITTI dataset shows that, compared to existing techniques, the proposed algorithm improves the average precision (AP) for two small object categories by 15.94% and 2.29% under moderate difficulty conditions, and by 13.34% and 2.86% under high difficulty conditions. These results underscore the significant impact and practical application potential of this algorithm in enhancing the accuracy of 3D small object detection.

Key words: 3D object detection, small object, area of interest, point cloud clustering, image and point cloud fusion

CLC Number:

P237

HAO Jia, YAO Guoying, ZHOU Jian, WANG Siyuan, XIAO Jinsheng. 3D small object detection method based on image and point cloud fusion[J]. Bulletin of Surveying and Mapping, 2025, 0(3): 33-38.

References

[1] 周燕,蒲磊,林良熙,等.激光点云的三维目标检测研究进展[J].计算机科学与探索,2022,16(12):2695-2717.
[2] DING M Y,HUO Y Q,YI H W,et al. Learning depth-guided convolutions for monocular 3D object detection[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle:IEEE,2020:11669-11678.
[3] MA X Z,WANG Z H,LI H J,et al. Accurate monocular 3D object detection via color-embedded 3D reconstruction for autonomous driving[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul:IEEE,2019:6850-6859.
[4] WANG L,DU L,YE X Q,et al. Depth-conditioned dynamic message propagation for monocular 3D object detection[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville:IEEE,2021:454-463.
[5] MA X,LIU S,XIA Z,et al. Rethinking pseudo-lidar representation[C]//Proceedings of 2020 European Conference on Computer Vision.Glasgow:Springer Science and Business Media Deutschland GmbH,2020:311-327.
[6] CHARLES R Q,HAO S,MO K C,et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu:IEEE,2017:77-85.
[7] QI C R,YI L,SU H,et al.PointNet++:Deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of 2017 Advances in Neural Information Processing Systems.Long Beach:NIPS,2017:5099-5108.
[8] LI Y Y,BU R,SUN M C,et al. PointCNN:convolution on X-transformed points[C] //Proceedings of 2018 Advances in Neural Information Processing Systems.Montreal:NIPS,2018:820-830.
[9] LIU H J,DU J X,ZHANG Y,et al. Extracting geometric and semantic point cloud features with gateway attention for accurate 3D object detection[J]. Engineering Applications of Artificial Intelligence,2023,123:106227.
[10] ZHOU Y,TUZEL O. VoxelNet:end-to-end learning for point cloud based 3D object detection[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake city:IEEE,2018:4490-4499.
[11] YAN Y,MAO Y X,LI B. SECOND:sparsely embedded convolutional detection[J].Sensors,2018,18(10):3337-3354.
[12] 黄远宪,李必军,黄琦,等.融合相机与激光雷达的目标检测、跟踪与预测[J/OL].武汉大学学报(信息科学版):1-8[2023-02-26].https://doi.org/10.13203/j.whugis20210614.
[13] CHEN X Z,MA H M,WAN J,et al. Multi-view 3D object detection network for autonomous driving[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu:IEEE,2017:6526-6534.
[14] KU J,MOZIFIAN M,LEE J,et al. Joint 3D proposal generation and object detection from view aggregation[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Madrid:IEEE,2018:1-8.
[15] QI C R,LIU W,WU C X,et al. Frustum PointNets for 3D object detection from RGB-D data[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake city:IEEE,2018:918-927.
[16] ZHU Y J,XIE J M,LIU M Y,et al. BF3D:bi-directional fusion 3D detector with semantic sampling and geometric mapping[J]. Image and Vision Computing,2023,139:104835.
[17] TAO B,YAN F W,YIN Z S,et al. A multimodal 3-D detector with attention from the corresponding modal[J]. IEEE Sensors Journal,2023,23(8):8581-8590.
[18] WANG Z X,JIA K. Frustum ConvNet:sliding Frustums to aggregate local point-wise features for amodal 3D object detection[C]//Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Macau:IEEE,2019:1742-1749.
[19] VORA S,LANG A H,HELOU B,et al. PointPainting:sequential fusion for 3D object detection[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle:IEEE,2020:4603-4611.
[20] 肖进胜,赵陶,周剑,等.基于上下文增强和特征提纯的小目标检测网络[J].计算机研究与发展,2023,60(2):465-474.
[21] WOO S,PARK J,LEE J Y,et al. CBAM:convolutional block attention module[M]//Lecture Notes in Computer Science. Cham:Springer International Publishing,2018:3-19.