Semantic mask segmentation enhanced indoor visual global localization

doi:10.13474/j.cnki.11-2246.2025.0712

Abstract

Abstract: In recent years,with the emergence of new technologies in the field of computer vision in image processing and feature extraction,visual indoor positioning has attracted extensive attention.Different from the relative positioning method of visual odometry,visual global positioning methods fuse images with the visual map to provide absolute pose with geographic reference data.There are various movable targets and a large number of repetitive similar textures in indoor structured scenes,which pose a challenge to visual positioning in indoor environment.In this paper,an indoor positioning method combining structure from motion (SfM) and panoptic segmentation is proposed.An environmental semantic mask is designed to remove the interference of movable targets and repeated textures in the image,and improve the accuracy and reliability of global positioning.The experimental results show that the SfM method with semantic mask can reconstruct 3D point clouds with less noise and clearer indoor structure,the positioning accuracy reaches 0.84 m(1σ).

Key words: indoor positioning, visual positioning, semantic mask, SfM, global localization

CLC Number:

P228

LI Jie, YIN Fei, LIU Jingbin, LI Mengxiang, ZHANG Wei. Semantic mask segmentation enhanced indoor visual global localization[J]. Bulletin of Surveying and Mapping, 2025, 0(7): 73-79.

References

[1] LIU Hui,DARABI H,BANERJEE P,et al.Survey of wireless indoor positioning techniques and systems[J].IEEE Transactions on Systems,Man and Cybernetics,Part C (Applications and Reviews),2007,37(6):1067-1080.
[2] NIU Xiaoguang,ZHANG Zejun,WANG Ankang,et al.Online learning-based WIFI radio map updating considering high-dynamic environmental factors[J].IEEE Access,2019,7:110074-110085.
[3] LIU J,CHEN R,PEI L,et al.A hybrid smartphone indoor positioning solution for mobile LBS[J].Sensors (Basel),2012,12(12):17208-17233.
[4] KENDALL A,GRIMES M,CIPOLLA R.PoseNet:a convolutional network for real-time 6-DOF camera relocalization[C]//Proceedings of 2015 IEEE International Conference on Computer Vision.Santiago:IEEE,2015:2938-2946.
[5] LASKAR Z,MELEKHOV I,KALIA S,et al.Camera relocalization by computing pairwise relative poses using convolutional neural network[C]//Proceedings of 2017 IEEE International Conference on Computer Vision Workshops.Venice:IEEE,2017:920-929.
[6] SATTLER T,ZHOU Qunjie,POLLEFEYS M,et al.Understanding the limitations of CNN-based absolute camera pose regression[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE,2019:3297-3307.
[7] ARANDJELOVIĆ R,ZISSERMAN A.DisLocation:scalable descriptor distinctiveness for location recognition[C]//Proceedings of 2015 Computer Vision-ACCV.Cham:Springer International Publishing,2015:188-204.
[8] SATTLER T,HAVLENA M,SCHINDLER K,et al.Large-scale location recognition and the geometric burstiness problem[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:1582-1590.
[9] TORRI A,ARANDJELOVIĆ R,SIVIC J,et al.24/7 place recognition by view synthesis[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE,2015:1808-1817.
[10] ZAMIR A R,SHAH M.Image geo-localization based on multiplenearest neighbor feature matching usinggeneralized graphs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2014,36(8):1546-1558.
[11] TORII A,TAIRA H,SIVIC J,et al.Are large-scale 3D models really necessary for accurate visual localization?[J].IEEE Trans Pattern Anal Mach Intell,2021,43(3):814-829.
[12] ZAMIR A R,SHAH M.Accurate image localization based on google maps street view[C]//Proceedings of 2010 Computer Vision.Berlin,Heidelberg:Springer Berlin Heidelberg,2010:255-268.
[13] SNAVELY N,SEITZ S M,SZELISKI R.Photo tourism[J].ACM Transactions on Graphics,2006,25(3):835-846.
[14] WU T,LIU J,LI Z,et al.Accurate smartphone indoor visual positioning based on a high-precision 3D photorealistic map[J].Sensors (Basel),2018,18(6):E1974.
[15] SATTLER T,LEIBE B,KOBBELT L.Fast image-based localization using direct 2D-to-3D matching[C]//Proceedings of 2011 International Conference on Computer Vision.Barcelona:IEEE,2011:667-674.
[16] SCHONBERGER J L,FRAHM J M.Structure-from-motion revisited[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas:IEEE,2016:4104-4113.
[17] MOULON P,MONASSE P,PERROT R,et al.OpenMVG:open multiple view geometry[C]//Proceedings of 2017 Reproducible Research in Pattern Recognition.Cham:Springer International Publishing,2017:60-74.
[18] KIRILLOV A,GIRSHICK R,HE Kaiming,et al.Panoptic feature pyramid networks[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE,2019:6392-6401.
[19] CHENG Bowen,COLLINS M D,ZHU Yukun,et al.Panoptic-DeepLab:a simple,strong,and fast baseline for bottom-up panoptic segmentation[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Seattle:IEEE,2020:12472-12482.
[20] HE Kaiming,GKIOXARI G,DOLLAR P,et al.Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Venice:IEEE,2017:2980-2988.
[21] LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004,60(2):91-110.
[22] SNAVELY N,SEITZ S M,SZELISKI R.Modeling the world from Internet photo collections[J].International Journal of Computer Vision,2008,80(2):189-210.
[23] AGARWAL S,FURUKAWA Y,SNAVELY N,et al.Building Rome in a day[J].Communications of the ACM,2011,54(10):105-112.
[24] CRANDALL D,OWENS A,SNAVELY N,et al.Discrete-continuous optimization for large-scale structure from motion[C]//Proceedings of 2011 CVPR.Colorado:IEEE,2011:3001-3008.
[25] INDYK P,MOTWANI R.Approximate nearest neighbors:towards removing the curse of dimensionality[C]//Proceedings of the 30th Annual ACM Symposium on Theory of Computing-STOC '98.Dallas:ACM Press,1998:604-613.
[26] CHENG Jian,LENG Cong,WU Jiaxiang,et al.Fast and accurate image matching with cascade hashing for 3D reconstruction[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Columbus:IEEE,2014:1-8.
[27] STRECHA C,BRONSTEIN A M,BRONSTEIN M M,et al.LDAHash:improved matching with smaller descriptors[J].IEEE Trans Pattern Anal Mach Intell,2012,34(1):66-78.
[28] FISCHLER M A,BOLLES R C.Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography[J].Communications of the ACM,1981,24(6):381-395.
[29] MOISAN L,MOULON P,MONASSE P.Automatic homographic registration of a pair of images,with A contrario elimination of outliers[J].Image Processing on Line,2012,2:56-73.
[30] 张顺,龚怡宏,王进军.深度卷积神经网络的发展及其在计算机视觉领域的应用[J].计算机学报,2019,42(3):453-482.
[31] 周飞燕,金林鹏,董军.卷积神经网络研究综述[J].计算机学报,2017,40(6):1229-1251.
[32] 吴帅,徐勇,赵东宁.基于深度卷积网络的目标检测综述[J].模式识别与人工智能,2018,31(4):335-346.
[33] 田萱,王亮,丁琪.基于深度学习的图像语义分割方法综述[J].软件学报,2019,30(2):440-468.
[34] 徐鹏斌,瞿安国,王坤峰,等.全景分割研究综述[J].自动化学报,2021,47(3):549-568.
[35] HAFIZ A M,BHAT G M.A survey on instance segmentation:state of the art[J].International Journal of Multimedia Information Retrieval,2020,9(3):171-189.
[36] KIRILLOV A,HE Kaiming,GIRSHICK R,et al.Panoptic segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach:IEEE,2019:9396-9405.
[37] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:770-778.
[38] LIN T Y,MAIRE M,BELONGIE S,et al.Microsoft COCO:common objects in context[C]//Proceedings of 2014 Computer Vision-ECCV.Cham:Springer International Publishing,2014:740-755.
[39] CORDTS M,OMRAN M,RAMOS S,et al.The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE,2016:3213-3223.