A distributed storage and index method of trajectory big data based on the Hilbert curve

doi:10.13474/j.cnki.11-2246.2024.0619

Abstract

Abstract: In response to the rapid growth of trajectory big data with spatio-temporal characteristics and the urgent need for its fast query, traditional relational databases have certain limitations on the storage of massive trajectory data and specific query requirements, while non-relational databases are difficult to meet the efficient indexing requirements of massive data, and the efficiency of the storage and indexing of trajectory data is still in urgent need of improvement. In this paper, a framework for storage and retrieval based on HBase database is designed and implemented to cope with the efficient management of spatio-temporal trajectory data. Firstly, a novel Rowkey structure is designed, and the GeoMesa-HBase underlying storage model is constructed by combining spatio-temporal indexing tools. Secondly, a Hilbert curve-based coding technique is integrated to construct the spatial index, which improves the storage and retrieval efficiency of trajectory data. In order to evaluate the effectiveness of the proposed method, this paper compares its storage and query performance with traditional storage databases (HBase and MySQL) and Geohash index. The experimental results show that the scheme is able to achieve effective storage of trajectory data and improve the retrieval efficiency, which is of great practical significance in addressing the challenges associated with trajectory big data management.

Key words: trajectory data, spatio-temporal index, spatio-temporal storage, HBase, Hilbert curve

CLC Number:

P208

CHEN Kai, SONG Weiwei, JIN Baoxuan, LI Yongning, PU Hongxun. A distributed storage and index method of trajectory big data based on the Hilbert curve[J]. Bulletin of Surveying and Mapping, 2024, 0(6): 109-114,138.

References

[1] YUAN Jing,ZHENG Yu,ZHANG Chengyang,et al. T-drive: driving directions based on taxi trajectories[C]//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems.San Jose,Californi:ACM Press,2010:99-108.
[2] JIANG Weiwei,LUO Jiayun. Big data for traffic estimation and prediction: a survey of data and tools[J]. Applied System Innovation,2022,5(1): 23.
[3] ZHENG Yu. Trajectory data mining: an overview[J].ACM Transactions on Intelligent Systems and Technology,6(3):29.
[4] STONEBRAKER M. SQL databases v. NoSQL databases[J]. Communications of the ACM,2010,53(4): 10-11.
[5] DING X,CHEN L,GAO Y,et al. UlTraMan: a unified platform for big trajectory data management and analytics[J]. Proceedings of the VLDB Endowment,2018,11(7): 787-799.
[6] HE Huajun,LI Ruiyuan,BAO Jie,et al. JUST-traj: a distributed and holistic trajectory data management system[C]//Proceedings of the 29th International Conference on Advances in Geographic Information Systems.Beijing:ACM Press,2021: 403-406.
[7] TONG Xiaochong,CHENG Chengqi,WANG Rong,et al. An efficient integer coding index algorithm for multi-scale time information management[J]. Data & Knowledge Engineering,2019,119(C): 123-138.
[8] KWON D,LEE Sangjun,LEE S. Indexing the current positions of moving objects using the lazy update R-tree[C]//Proceedings of the 3rd International Conference on Mobile Data Management MDM. Singapore:IEEE,2002: 113-120.
[9] GUAN Xuefeng,BO Cheng,LI Zhenqiang,et al. ST-hash: an efficient spatiotemporal index for massive trajectory data in a NoSQL database[C]//Proceedings of 2017 International Conference on Geoinformatics.Buffalo,NY:IEEE,2017: 1-7.
[10] HUGHES J N,ANNEX A,EICHELBERGER C N,et al. GeoMesa: a distributed architecture for spatio-temporal fusion[C]//Proceedings of 2015 Geospatial Informatics,Fusion,and Motion Video Analytics V. Baltimore,MD:[s.n.],2015:128-140.
[11] LI Ruiyuan,HE Huajun,WANG Rubin,et al. TrajMesa: a distributed NoSQL storage engine for big trajectory data[C]//Proceedings of 2020 IEEE 36th International Conference on Data Engineering (ICDE). Dallas,TX:IEEE,2020: 2002-2005.
[12] LAWDER J K,KING P J H. Using space-filling curves for multi-dimensional indexing[C]//Proceedings of 2000 British National Conference on Databases. Berlin,Heidelberg: Springer Berlin Heidelberg,2000: 20-35.
[13] MOKBEL M F,AREF W G,KAMEL I. Analysis of multi-dimensional space-filling curves[J]. GeoInformatica,2003,7(3): 179-209.
[14] WU Y H,CAO X,AN Z. A spatiotemporal trajectory data index based on the Hilbert curve code[C]//Proceedings of 2020 IOP Conference Series: Earth and Environmental Science. [S.l.]:IOP Publishing,2020,502(1): 012005.
[15] 曹布阳,冯华森,梁峻浩,等. 利用Hilbert曲线与Cassandra技术实现时空大数据存储与索引[J]. 武汉大学学报(信息科学版),2021,46(5): 620-629.
[16] STONEBRAKER M,ABADI D J,BATKIN A,et al. C-store:a column-oriented DBMS[M]//Making Databases Work:the Pragmatic Wisdom of Michael Stonebraker.[S.l.]:Association for Computing Machinery,2018:491-518.
[17] GAO Fan,YUE Peng,WU Zhaoyan,et al. Geospatial data storage based on HBase and MapReduce[C]//Proceedings of the 6th International Conference on Agro-Geoinformatics. Fairfax,VA:IEEE,2017: 1-4.
[18] XIONG Shuyi,OUYANG Xue,XIONG Wei. Distributed or centralized: an experimental study on spatial database systems for processing big trajectory data[C]//Proceedings of 2023 IEEE International Conference on Big Data Analytics (ICBDA). Harbin:IEEE,2023:8-13.