文档-关系数据查询执行技术研究与实现

doi:10.3778/j.issn.1673-9418.1908023

摘要/Abstract

摘要：

随着大数据时代的到来，各类互联网应用产生了丰富的数据类型。将多种多样结构的数据进行一体化存储、查询和组织是新时代下的大数据管理系统的研究热点。对关系数据库和NoSQL文档数据库加以统一管理，将支持结构化数据和半结构化数据的两种不同的数据库引擎集成在大数据管理系统中，实现了查询引擎ENTIA来执行查询处理。基于全局视图对用户提供统一的查询接口，终端用户无需关心数据的类型、结构以及物理存储位置，只需根据业务需求向ENTIA发出请求即可。进行了大量的前期实验，基于启发式规则进行查询优化，单个查询被重写为可以并行执行的多个查询子任务，将计算推向合适的数据库引擎，充分利用系统计算资源，大大提高了系统的查询性能。以关系数据库PostgreSQL和文档数据库MongoDB两个对等引擎为代表，实现了ENTIA对多数据类型的查询能力以及查询优化能力。通过功能符合实验测试了ENTIA能够正确地执行混合查询，以多组性能对比实验证明了优化方法的有效性。

关键词: 关系数据库, 文档数据库, 混合查询, 查询优化

Abstract:

With the arrival of the era of big data, Internet applications have produced abundant data types. Integ-rating storage, query and organization of data with various structures is a research hotspot of large data management system. The relational database and NoSQL document database are performed unified management. Two different database engines supporting structured and semi-structured data are integrated into the large data management system. The query engine ENTIA is implemented to perform query processing. Based on the global view, a unified query interface is provided to users. The end user does not need to care about the type and structure of the data, and the physical storage location. It only needs to send a request to ENTIA according to the business requirements. A large number of preliminary experiments are carried out to optimize the query based on heuristic rules. The single query is rewritten into multiple query sub-tasks that can be executed in parallel. The calculation is pushed to the appropriate database engine, which makes full use of the computing resources of the system and greatly improves the query performance of the system. Represented by two peer-to-peer engines of relational database PostgreSQL and document database MongoDB, ENTIA’s query ability for multiple data types and query optimization are realized. ENTIA can correctly execute mixed queries through functional coincidence experiments. The effectiveness of the optimization method is proven by a number of performance comparison experiments.

Key words: relation database, document database, hybrid query, query optimization

马志程，袁海峰，谷洋，刘亚茹，张孝. 文档-关系数据查询执行技术研究与实现[J]. 计算机科学与探索, 2020, 14(8): 1315-1326.

MA Zhicheng, YUAN Haifeng, GU Yang, LIU Yaru, ZHANG Xiao. Research and Implementation of Document-Relational Data Query Execution Tech-nology[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(8): 1315-1326.

参考文献

[1] Strang K D. Do the critical success factors from learning analytics predict student outcomes[J]. Journal of Educational Technology Systems, 2016, 44(3): 273-299.
[2] Hecht R, Jablonski S. NoSQL evaluation: a use case oriented survey[C]//Proceedings of the 2011 International Conference on Cloud and Service Computing, Hong Kong, China, Dec 12-14, 2011. Washington: IEEE Computer Society, 2011: 336-341.
[3] Okman L, Galoz N, Gonen Y, et al. Security issues in NoSQL databases[C]//Proceedings of the 10th International Con-ference on Trust, Security and Privacy in Computing and Communications, Changsha, Nov 16-18, 2011. Washington: IEEE Computer Society, 2011: 541-547.
[4] Parker Z, Poe S, Vrbsky S V, et al. Comparing NoSQL Mongo-DB to an SQL DB[C]//Proceedings of the ACM Southeast Regional Conference 2013, Savannah, Apr 4-6, 2013. New York: ACM, 2013: 5.
[5] Dede E, Sendir B, Kuzlu P, et al. An evaluation of cassandra for Hadoop[C]//Proceedings of the 2013 IEEE International Conference on Cloud Computing, Santa Clara, Jun 28-Jul 3, 2013. Washington: IEEE Computer Society, 2013: 494-501.
[6] Tahara D, Diamond T, Abadi D J, et al. Sinew: a SQL system for multi-structured data[C]//Proceedings of the 2014 Inter-national Conference on Management of Data, Snowbird, Jun 22-27, 2014. New York: ACM, 2014: 815-826.
[7] Melton J, Michels J, Josifovski V, et al. SQL/MED: a status report[J]. SIGMOD Record, 2002, 31(3): 81-89.
[8] Stonebraker M. SQL databases v. NoSQL databases[J]. Com-munications of the ACM, 2010, 53(4): 10-11.
[9] Floratou A, Teletia N, Dewitt D J, et al. Can the elephants handle the NoSQL onslaught[J]. Proceedings of the VLDB Endowment, 2012, 5(12): 1712-1723.
[10] Tatemura J, Po O, Hsiung W, et al. Partiqle: an elastic SQL engine over key-value stores[C]//Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, Scottsdale, May 20-24, 2012. New York: ACM, 2012: 629-632.
[11] Vila?a R M, Cruz F, Pereira J, et al. An effective scalable SQL engine for NoSQL databases[C]//LNCS 7891: Proceedings of the 8th International Federated Conference on Distributed Computing Techniques, Florence, Jun 3-5, 2013. Berlin, Hei-delberg: Springer, 2013: 155-168.
[12] Armbrust M, Lanham N, Tu S, et al. The case for PIQL: a performance insightful query language[C]//Proceedings of the 1st ACM Symposium on Cloud Computing, Indianapolis, Jun 10-11, 2010. New York: ACM, 2010: 131-136.
[13] Brantner M, Florescu D, Graf D A, et al. Building a data-base on S3[C]//Proceedings of the 2008 ACM SIGMOD Inter-national Conference on Management of Data, Vancouver, Jun 10-12, 2008. New York: ACM, 2008: 251-264.
[14] Mason T, Lawrence R. Dynamic database integration in a JDBC driver[C]//Proceedings of the 7th International Con-ference on Enterprise Information Systems, Miami, May 25-28, 2005. Berlin, Heidelberg: Springer, 2005: 326-333.
[15] Roijackers J, Fletcher G H L. On bridging relational and document-centric data stores[C]//LNCS 7968: Proceedings of the British National Conference on Databases, Oxford, Jul 8-10, 2013. Berlin, Heidelberg: Springer, 2013: 135-148.
[16] Wang S, Sa S X. Introduction to database systems[M]. 5th ed. Beijing: Higher Education Press, 2014.王珊, 萨师煊. 数据库系统概论[M]. 5版. 北京: 高等教育出版社, 2014.
[17] Yang D Q, Wu Y Q, Bao X Y, et al. Database system imp-lementation[M]. 2nd ed. Beijing: Mechanical Industry Press, 2015.杨冬青, 吴愈青, 包小源, 等. 数据库系统实现[M]. 2版. 北京: 机械工业出版社, 2015.