融合点云与全景影像的路侧多目标识别

doi:10.13474/j.cnki.11-2246.2023.0293

摘要/Abstract

摘要： 为了从点云中自动准确识别车辆、垃圾桶与杆状交通设施等目标,本文提出了一种融合点云与全景影像的路侧多目标识别方法,充分利用点云数据中的空间几何信息与全景影像中的语义信息提升目标的识别准确度。首先,对全景影像进行实例分割,获取图像中目标的二维掩码(mask);然后,将激光点云投影生成全景深度图后,以深度图像为媒介获得对应的候选目标点云;最后,对于相机拍摄时产生的视角遮挡与被遮挡问题,通过分析目标在三维空间中的连续性与完整性,对候选点云进行二次聚类,最终完成对目标的分类。试验结果表明,3类目标的准确率分别为96.64%、92.68%及90.74%,证明本文方法可有效识别城市场景中的车辆、垃圾桶与杆状等交通设施。

关键词: 激光点云, 全景影像, 目标识别, 点云聚类

Abstract: In order to automatically and accurately identify objects such as vehicles,garbage cans and rod-shaped traffic facilities from point clouds,this paper proposes a roadside multi-object recognition method that integrates point clouds and panoramic images,and makes full use of spatial geometry information in point cloud data and semantic information in panoramic images to improve the accuracy of target recognition. Firstly,the panoramic image is segmented to obtain the two-dimensional mask of the object in the image. Then,the laser point cloud is projected to generate a panoramic depth map,and the corresponding candidate point cloud is obtained using the depth image as the medium. Finally,for the problem of angle occlusion and occlusion caused by camera shooting,through analyzing the continuity and integrity of the target in 3D space,secondary clustering of candidate point clouds is carried out,and finally the classification of the target is completed. Experimental results show that the accuracy rates of the three types of targets are 96.64%,92.68% and 90.74% respectively,which prove that the proposed method can effectively identify vehicles,garbage cans and rod-shaped traffic facilities in urban scenes by integrating semantic information in images with spatial geometry information in point clouds.

Key words: LiDAR point cloud, panoramic image, target recognition, point cloud clustering

中图分类号:

P237

王步云, 李宏伟, 赵姗. 融合点云与全景影像的路侧多目标识别[J]. 测绘通报, 2023, 0(10): 40-46.

WANG Buyun, LI Hongwei, ZHAO Shan. Road side multi-object recognition by integrating point cloud and panoramic image[J]. Bulletin of Surveying and Mapping, 2023, 0(10): 40-46.

参考文献

[1] 李德仁,姚远,邵振峰.智慧城市中的大数据[J].武汉大学学报(信息科学版),2014,39(6):631-640.
[2] 杨必胜,梁福逊,黄荣刚.三维激光扫描点云数据处理研究进展、挑战与趋势[J].测绘学报,2017,46(10):1509-1516.
[3] PUENTE I,GONZÁLEZ-JORGE H,MARTÍNEZ-SÁNCHEZ J,et al.Review of mobile mapping and surveying technologies[J].Measurement,2013,46(7):2127-2145.
[4] GUO Yulan,WANG Hanyun,HU Qingyong,et al.Deep learning for 3D point clouds:a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,43(12):4338-4364.
[5] CHEN Xiaozhi,MA Huimin,WAN Ji,et al.Multi-view 3D object detection network for autonomous driving[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,USA:IEEE,2017:6526-6534.
[6] SU Hang,MAJI S,KALOGERAKIS E,et al.Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV).Santiago,Chile:IEEE,2016:945-953.
[7] MATURANA D,SCHERER S.VoxNet:a 3D Convolutional Neural Network for real-time object recognition[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Hamburg,Germany:IEEE,2015:922-928.
[8] RIEGLER G,ULUSOY A O,GEIGER A.OctNet:learning deep 3D representations at high resolutions[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,USA:IEEE,2017:6620-6629.
[9] CHARLES R Q,HAO Su,MO Kaichun,et al.PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,USA:IEEE,2017:77-85.
[10] QI C R,YI Li,SU Hao,et al.PointNet++:deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.New York:ACM Press,2017:5105-5114.
[11] JIANG Mingyang,WU Yiran,ZHAO Tianqi,et al.PointSIFT:a SIFT-like network module for 3D point cloud semantic segmentation[EB/OL].[2023-01-01].https://arxiv.org/abs/1807.00652.
[12] LI Y,BU R,SUN M,et al.PointCNN [J].Computer Vision and Pattern Recognition,2018:1801.
[13] HU Qingyong,YANG Bo,XIE Linhai,et al.RandLA-net:efficient semantic segmentation of large-scale point clouds[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle,USA:IEEE,2020:11105-11114.
[14] 杨必胜,董震.点云智能研究进展与趋势[J].测绘学报,2019,48(12):1575-1585.
[15] KU J,MOZIFIAN M,LEE J,et al.Joint 3D proposal generation and object detection from view aggregation[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).Madrid,Spain:IEEE,2019:1-8.
[16] QI C R,LIU Wei,WU Chenxia,et al.Frustum PointNets for 3D object detection from RGB-D data[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City,USA:IEEE,2018:918-927.
[17] WULFF F,SCHÄUFELE B,SAWADE O,et al.Early Fusion of Camera and Lidar for robust road detection based on U-Net FCN[C]//Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV).Changshu,China:IEEE,2018:1426-1431.
[18] HE Kaiming,GKIOXARI G,DOLLÁR P,et al.Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV).Venice,Italy:IEEE,2017:2980-2988.
[19] REN Shaoqing,HE Kaiming,GIRSHICK R,et al.Faster R-CNN:towards real-time object detection with region proposal networks[C]//Proceedings of 2016 IEEE Transactions on Pattern Analysis and Machine Intelligence.[S.l.]:IEEE,2016:1137-1149.
[20] ZHANG Wuming,QI Jianbo,WAN Peng,et al.An easy-to-use airborne LiDAR data filtering method based on cloth simulation[J].Remote Sensing,2016,8(6):501.