面向无人平台的视觉空间关系模型

doi:10.13474/j.cnki.11-2246.2023.0292

摘要/Abstract

摘要： 视觉空间关系是指通过视觉传感器获得的空间关系。当认知主体由人转变为无人平台时,显式地描述与记录空间关系是无人平台实现场景理解和空间推理的关键。本文聚焦无人平台视觉空间关系模型缺失现状,将地图学和机器视觉中关于空间关系的分类、模型和算法进行融合,提出了一种面向无人平台的视觉空间关系模型;构建了融合视觉空间关系模型的数据集,训练视觉空间关系预测模型,比较不同模型的视觉空间关系检测能力,验证了基于视觉空间关系模型所构建数据集的完备性及模型提升视觉关系检测能力的有效性。本文能够改善当前机器视觉领域空间关系不统一的问题,对提高无人平台视觉关系检测、实现空间关系记录和地图模型构建等方面具有一定的研究意义。

关键词: 视觉空间关系, 无人平台, 视觉关系检测, 时空Transformer

Abstract: Visual spatial relationship refer to the spatial relations obtained through visual sensors. When the cognitive subject changes from a human to an unmanned platform, describing and recording spatial relationships explicitly is the key for unmanned platform to realize scene understanding and spatial reasoning. Focusing on the current situation of the lack of visual spatial relationship model for unmanned platform, we integrate the classification, model and algorithm about spatial relationship in cartography and machine vision, propose a visual spatial relationship model for unmanned platform. We construct datasets incorporating the visual spatial relationship model, train the visual spatial relationship prediction model, and compare the visual spatial relationship detection capabilities of different models, which verified the completeness of the dataset constructed based on the visual spatial relationship model as well as the validity of the model in enhancing the visual spatial relationship detection capabilities. This study can improve the current problem of spatial relationship inconsistency in the field of machine vision, and has certain research significance in improving the visual relationship detection of unmanned platform, realizing the spatial relationship record and map model construction.

Key words: visual spatial relationship, unmanned platform, visual relationship detection, ST-Transformer

中图分类号:

P208

皇甫润南, 田江鹏, 游雄, 屠铱成. 面向无人平台的视觉空间关系模型[J]. 测绘通报, 2023, 0(10): 34-39,66.

HUANGFU Runnan, TIAN Jiangpeng, YOU Xiong, TU Yicheng. Visual spatial relationship model for unmanned platform[J]. Bulletin of Surveying and Mapping, 2023, 0(10): 34-39,66.

参考文献

[1] BADUE C,GUIDOLINI R,CARNEIRO R V,et al. Self-driving cars: a survey[J]. Expert Systems with Applications,2021,165: 113816.
[2] SHAKHATREH H,SAWALMEH A H,AL-FUQAHA A,et al. Unmanned aerial vehicles (UAVs): a survey on civil applications and key research challenges[J]. IEEE Access,2019,7: 48572-48634.
[3] 汪连贺. 无人艇集群在海道测量中协同控制的实现[J]. 测绘通报,2022(5): 145-149.
[4] 高俊. 图到用时方恨少,重绘河山待后生: 《测绘学报》60年纪念与前瞻[J]. 测绘学报,2017,46(10): 1219-1225.
[5] SEIF H G,HU Xiaolong. Autonomous driving in the iCity—HD maps as a key challenge of the automotive industry[J]. Engineering,2016,2(2): 159-162.
[6] 刘经南,詹骄,郭迟,等. 智能高精地图数据逻辑结构与关键技术[J]. 测绘学报,2019,48(8): 939-953.
[7] 杨振凯,华一新,訾璐,等. 浅析高精度地图发展现状及关键技术[J]. 测绘通报,2021(6): 54-60.
[8] 游雄. 机器地图的基本问题及研究思考[R]. 第八届全国虚拟地理环境学术会议(CVGE 2021). 赣州:江西理工大学,2022.
[9] MACEACHREN A M. How maps work: representation,visualization,and design[M]. New York: Guilford Press,1995.
[10] LU Cewu,KRISHNA R,BERNSTEIN M,et al. Visual relationship detection with language priors[C]//Proceedings of 2016 Computer Vision-ECCV. Cham: Springer International Publishing,2016: 852-869.
[11] JOHNSON J,KRISHNA R,STARK M,et al. Image retrieval using scene graphs[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE,2015: 3668-3678.
[12] ZAREIAN A,KARAMAN S,CHANG S F. Bridging knowledge graphs to generate scene graphs[C]//Proceedings of 2020 European Conference on Computer Vision. Cham: Springer,2020: 606-623.
[13] CONG Yuren,LIAO Wentong,ACKERMANN H,et al. Spatial-temporal transformer for dynamic scene graph generation[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE,2022: 16352-16362.
[14] 郭仁忠. 空间分析[M]. 2版. 北京: 高等教育出版社,2001.
[15] EGENHOFER M J,FRANZOSA R D.Point-set topological spatial relations[J].International Journal of Geographical Information Systems,1991,5(2):161-174.
[16] 王家耀. 普通地图制图综合原理[M]. 北京: 测绘出版社,1993.
[17] 邓敏,李成名,刘晓丽. 论GIS空间关系描述中存在的几个基本问题[J]. 中国工程科学,2013,15(5): 20-24.
[18] KRISHNA R,ZHU Yuke,GROTH O,et al. Visual genome: connecting language and vision using crowdsourced dense image annotations[J]. International Journal of Computer Vision,2017,123(1): 32-73.
[19] JI Jingwei,KRISHNA R,LI Feifei,et al. Action genome: actions As compositions of spatio-temporal scene graphs[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE,2020: 10233-10244.
[20] WIECZOREK J,GUO Qinghua,HIJMANS R. The point-radius method for georeferencing locality descriptions and calculating associated uncertainty[J]. International Journal of Geographical Information Science,2004,18(8): 745-767.
[21] 刘瑜,龚咏喜,张晶,等. 地理空间中的空间关系表达和推理[J]. 地理与地理信息科学,2007,23(5): 1-7.
[22] REN Shaoqing,HE Kaiming,GIRSHICK R,et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6): 1137-1149.
[23] ZELLERS R,YATSKAR M,THOMSON S,et al. Neural motifs: scene graph parsing with global context[C]///Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE,2018: 5831-5840.