Real-time semantic SLAM algorithm based on dynamic scenes

doi:10.13474/j.cnki.11-2246.2025.0405

Abstract

Abstract: Aiming at the problems of traditional visual SLAM (simultaneous localization and mapping) in indoor dynamic environment with low localization accuracy, poor robustness and poor real-time performance after combining with deep learning as well as the inability to construct dense maps, this paper proposes an improved algorithm based on ORB-SLAM3. First, a lightweight SegFormer semantic segmentation network is used to identify dynamic objects present in the image, and then a mask image adaptive expansion method is added to automatically adjust the mask expansion range according to the total number of feature points to more effectively remove potential dynamic object feature points; second, the bag-of-words model is improved to enhance the loading and matching speed of the algorithm; and lastly, a dense map building thread is added to construct a map for removing dynamic features according to the mask information and keyframes to construct a dense point cloud map after removing dynamic features. The experimental results show that the algorithm in this paper can effectively remove dynamic object feature points in highly dynamic scenes, and improve the localization accuracy and robustness of the system,and the average processing speed is 20.3FPS, which basically meets the requirements of real-time operation.

Key words: VSLAM, ORB-SLAM3, semantic segmentation, dense mapping

CLC Number:

FU Qiang, ZHONG Zhen, JI Yuanfa, REN Fenghua. Real-time semantic SLAM algorithm based on dynamic scenes[J]. Bulletin of Surveying and Mapping, 2025, 0(4): 27-33.

References

[1] 赵洋,刘国良,田国会,等. 基于深度学习的视觉SLAM综述[J].机器人,2017,39(6):889-896.
[2] MUR-ARTAL R,TARDÓS J D.ORB-SLAM2:an open-source SLAM system for monocular,stereo,and RGB-D cameras[J].IEEE Transactions on Robotics,2017,33(5):1255-1262.
[3] CAMPOS C,ELVIRA R,RODRÍGUEZ J J G,et al.ORB-SLAM3:an accurate open-source library for visual,visual-inertial,and multimap SLAM[J].IEEE Transactions on Robotics,2021,37(6):1874-1890.
[4] ROSTEN E,PORTER R,DRUMMOND T.Faster and better:a machine learning approach to corner detection[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2010,32(1):105-119.
[5] CALONDER M,LEPETIT V,ÖZUYSAL M,et al.BRIEF:computing a local binary descriptor very fast[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(7):1281-1298.
[6] PALAZZOLO E,BEHLEY J,LOTTES P,et al.ReFusion:3D reconstruction in dynamic environments for RGB-D cameras exploiting residuals[C]//Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems.[S.l.]:IEEE,2019:7855-7862.
[7] LI S L,LEE D.RGB-D SLAM in dynamic environments using static point weighting[J].IEEE Robotics and Automation Letters,2017,2(4):2263-2270.
[8] YU C,LIU Z X,LIU X J,et al.DS-SLAM:a semantic visual SLAM towards dynamic environments[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems.Madrid:IEEE,2018:1168-1174.
[9] BADRINARAYANAN V,KENDALL A,CIPOLLA R.SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[10] BESCOS B,FÁCIL J M,CIVERA J,et al.DynaSLAM:tracking,mapping,and inpainting in dynamic scenes[J].IEEE Robotics and Automation Letters,2018,3(4):4076-4083.
[11] HE K M,GKIOXARI G,DOLLÁR P,et al.Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Venice:IEEE,2017:2980-2988.
[12] CHENG S H,SUN C H,ZHANG S J,et al.SG-SLAM:a real-time RGB-D visual SLAM toward dynamic scenes with semantic and geometric information[J].IEEE Transactions on Instrumentation and Measurement,2023,72:7501012.
[13] XIE E Z,WANG W H,YU Z D,et al.SegFormer:simple and efficient design for semantic segmentation with transformers[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems.[S.l.]:ACM Press,2021.
[14] VASWANI A,SHAZEER N M,PARMAR N,et al.Attention is All you Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.[S.l.]:ACM Press,2017.
[15] TOLSTIKHIN I,HOULSBY N,KOLESNIKOV A,et al.MLP-Mixer:An all-MLP Architecture for Vision[EB/OL].2021-05-04[2024-07-02].https://arxiv.org/abs/2105.01601.
[16] QADER W A,AMEEN M M,AHMED B I.An overview of bag of Words;Importance,implementation,applications,and challenges[C]//Proceedings of 2019 International Engineering Conference.Erbil:IEEE,2019:200-204.
[17] STURM J,ENGELHARD N,ENDRES F,et al.A benchmark for the evaluation of RGB-D SLAM systems[C]//Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.Vilamoura-Algarve:IEEE,2012:573-580.