Visual spatial relationship model for unmanned platform

doi:10.13474/j.cnki.11-2246.2023.0292

Abstract

Abstract: Visual spatial relationship refer to the spatial relations obtained through visual sensors. When the cognitive subject changes from a human to an unmanned platform, describing and recording spatial relationships explicitly is the key for unmanned platform to realize scene understanding and spatial reasoning. Focusing on the current situation of the lack of visual spatial relationship model for unmanned platform, we integrate the classification, model and algorithm about spatial relationship in cartography and machine vision, propose a visual spatial relationship model for unmanned platform. We construct datasets incorporating the visual spatial relationship model, train the visual spatial relationship prediction model, and compare the visual spatial relationship detection capabilities of different models, which verified the completeness of the dataset constructed based on the visual spatial relationship model as well as the validity of the model in enhancing the visual spatial relationship detection capabilities. This study can improve the current problem of spatial relationship inconsistency in the field of machine vision, and has certain research significance in improving the visual relationship detection of unmanned platform, realizing the spatial relationship record and map model construction.

Key words: visual spatial relationship, unmanned platform, visual relationship detection, ST-Transformer

CLC Number:

P208

HUANGFU Runnan, TIAN Jiangpeng, YOU Xiong, TU Yicheng. Visual spatial relationship model for unmanned platform[J]. Bulletin of Surveying and Mapping, 2023, 0(10): 34-39,66.

References

[1] BADUE C,GUIDOLINI R,CARNEIRO R V,et al. Self-driving cars: a survey[J]. Expert Systems with Applications,2021,165: 113816.
[2] SHAKHATREH H,SAWALMEH A H,AL-FUQAHA A,et al. Unmanned aerial vehicles (UAVs): a survey on civil applications and key research challenges[J]. IEEE Access,2019,7: 48572-48634.
[3] 汪连贺. 无人艇集群在海道测量中协同控制的实现[J]. 测绘通报,2022(5): 145-149.
[4] 高俊. 图到用时方恨少,重绘河山待后生: 《测绘学报》60年纪念与前瞻[J]. 测绘学报,2017,46(10): 1219-1225.
[5] SEIF H G,HU Xiaolong. Autonomous driving in the iCity—HD maps as a key challenge of the automotive industry[J]. Engineering,2016,2(2): 159-162.
[6] 刘经南,詹骄,郭迟,等. 智能高精地图数据逻辑结构与关键技术[J]. 测绘学报,2019,48(8): 939-953.
[7] 杨振凯,华一新,訾璐,等. 浅析高精度地图发展现状及关键技术[J]. 测绘通报,2021(6): 54-60.
[8] 游雄. 机器地图的基本问题及研究思考[R]. 第八届全国虚拟地理环境学术会议(CVGE 2021). 赣州:江西理工大学,2022.
[9] MACEACHREN A M. How maps work: representation,visualization,and design[M]. New York: Guilford Press,1995.
[10] LU Cewu,KRISHNA R,BERNSTEIN M,et al. Visual relationship detection with language priors[C]//Proceedings of 2016 Computer Vision-ECCV. Cham: Springer International Publishing,2016: 852-869.
[11] JOHNSON J,KRISHNA R,STARK M,et al. Image retrieval using scene graphs[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston: IEEE,2015: 3668-3678.
[12] ZAREIAN A,KARAMAN S,CHANG S F. Bridging knowledge graphs to generate scene graphs[C]//Proceedings of 2020 European Conference on Computer Vision. Cham: Springer,2020: 606-623.
[13] CONG Yuren,LIAO Wentong,ACKERMANN H,et al. Spatial-temporal transformer for dynamic scene graph generation[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal: IEEE,2022: 16352-16362.
[14] 郭仁忠. 空间分析[M]. 2版. 北京: 高等教育出版社,2001.
[15] EGENHOFER M J,FRANZOSA R D.Point-set topological spatial relations[J].International Journal of Geographical Information Systems,1991,5(2):161-174.
[16] 王家耀. 普通地图制图综合原理[M]. 北京: 测绘出版社,1993.
[17] 邓敏,李成名,刘晓丽. 论GIS空间关系描述中存在的几个基本问题[J]. 中国工程科学,2013,15(5): 20-24.
[18] KRISHNA R,ZHU Yuke,GROTH O,et al. Visual genome: connecting language and vision using crowdsourced dense image annotations[J]. International Journal of Computer Vision,2017,123(1): 32-73.
[19] JI Jingwei,KRISHNA R,LI Feifei,et al. Action genome: actions As compositions of spatio-temporal scene graphs[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle: IEEE,2020: 10233-10244.
[20] WIECZOREK J,GUO Qinghua,HIJMANS R. The point-radius method for georeferencing locality descriptions and calculating associated uncertainty[J]. International Journal of Geographical Information Science,2004,18(8): 745-767.
[21] 刘瑜,龚咏喜,张晶,等. 地理空间中的空间关系表达和推理[J]. 地理与地理信息科学,2007,23(5): 1-7.
[22] REN Shaoqing,HE Kaiming,GIRSHICK R,et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6): 1137-1149.
[23] ZELLERS R,YATSKAR M,THOMSON S,et al. Neural motifs: scene graph parsing with global context[C]///Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE,2018: 5831-5840.