Road side multi-object recognition by integrating point cloud and panoramic image

doi:10.13474/j.cnki.11-2246.2023.0293

Abstract

Abstract: In order to automatically and accurately identify objects such as vehicles,garbage cans and rod-shaped traffic facilities from point clouds,this paper proposes a roadside multi-object recognition method that integrates point clouds and panoramic images,and makes full use of spatial geometry information in point cloud data and semantic information in panoramic images to improve the accuracy of target recognition. Firstly,the panoramic image is segmented to obtain the two-dimensional mask of the object in the image. Then,the laser point cloud is projected to generate a panoramic depth map,and the corresponding candidate point cloud is obtained using the depth image as the medium. Finally,for the problem of angle occlusion and occlusion caused by camera shooting,through analyzing the continuity and integrity of the target in 3D space,secondary clustering of candidate point clouds is carried out,and finally the classification of the target is completed. Experimental results show that the accuracy rates of the three types of targets are 96.64%,92.68% and 90.74% respectively,which prove that the proposed method can effectively identify vehicles,garbage cans and rod-shaped traffic facilities in urban scenes by integrating semantic information in images with spatial geometry information in point clouds.

Key words: LiDAR point cloud, panoramic image, target recognition, point cloud clustering

CLC Number:

P237

WANG Buyun, LI Hongwei, ZHAO Shan. Road side multi-object recognition by integrating point cloud and panoramic image[J]. Bulletin of Surveying and Mapping, 2023, 0(10): 40-46.

References

[1] 李德仁,姚远,邵振峰.智慧城市中的大数据[J].武汉大学学报(信息科学版),2014,39(6):631-640.
[2] 杨必胜,梁福逊,黄荣刚.三维激光扫描点云数据处理研究进展、挑战与趋势[J].测绘学报,2017,46(10):1509-1516.
[3] PUENTE I,GONZÁLEZ-JORGE H,MARTÍNEZ-SÁNCHEZ J,et al.Review of mobile mapping and surveying technologies[J].Measurement,2013,46(7):2127-2145.
[4] GUO Yulan,WANG Hanyun,HU Qingyong,et al.Deep learning for 3D point clouds:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(12):4338-4364.
[5] CHEN Xiaozhi,MA Huimin,WAN Ji,et al.Multi-view 3D object detection network for autonomous driving[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,USA:IEEE,2017:6526-6534.
[6] SU Hang,MAJI S,KALOGERAKIS E,et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV).Santiago,Chile:IEEE,2016:945-953.
[7] MATURANA D,SCHERER S.VoxNet:a 3D Convolutional Neural Network for real-time object recognition[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Hamburg,Germany:IEEE,2015:922-928.
[8] RIEGLER G,ULUSOY A O,GEIGER A.OctNet:learning deep 3D representations at high resolutions[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,USA:IEEE,2017:6620-6629.
[9] CHARLES R Q,HAO Su,MO Kaichun,et al.PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,USA:IEEE,2017:77-85.
[10] QI C R,YI Li,SU Hao,et al.PointNet++:deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York:ACM Press,2017:5105-5114.
[11] JIANG Mingyang,WU Yiran,ZHAO Tianqi,et al.PointSIFT:a SIFT-like network module for 3D point cloud semantic segmentation[EB/OL].[2023-01-01].https://arxiv.org/abs/1807.00652.
[12] LI Y,BU R,SUN M,et al.PointCNN [J].Computer Vision and Pattern Recognition,2018:1801.
[13] HU Qingyong,YANG Bo,XIE Linhai,et al.RandLA-net:efficient semantic segmentation of large-scale point clouds[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle,USA:IEEE,2020:11105-11114.
[14] 杨必胜,董震.点云智能研究进展与趋势[J].测绘学报,2019,48(12):1575-1585.
[15] KU J,MOZIFIAN M,LEE J,et al.Joint 3D proposal generation and object detection from view aggregation[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Madrid,Spain:IEEE,2019:1-8.
[16] QI C R,LIU Wei,WU Chenxia,et al.Frustum PointNets for 3D object detection from RGB-D data[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA:IEEE,2018:918-927.
[17] WULFF F,SCHÄUFELE B,SAWADE O,et al.Early Fusion of Camera and Lidar for robust road detection based on U-Net FCN[C]//Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV).Changshu,China:IEEE,2018:1426-1431.
[18] HE Kaiming,GKIOXARI G,DOLLÁR P,et al.Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV).Venice,Italy:IEEE,2017:2980-2988.
[19] REN Shaoqing,HE Kaiming,GIRSHICK R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[C]//Proceedings of 2016 IEEE Transactions on Pattern Analysis and Machine Intelligence.[S.l.]:IEEE,2016:1137-1149.
[20] ZHANG Wuming,QI Jianbo,WAN Peng,et al.An easy-to-use airborne LiDAR data filtering method based on cloth simulation[J].Remote Sensing,2016,8(6):501.