计算机科学与探索 ›› 2013, Vol. 7 ›› Issue (3): 236-246.DOI: 10.3778/j.issn.1673-9418.1211006

• 学术研究 • 上一篇    下一篇

HBase数据库迁移工具的设计与实现

杨寒冰,赵  龙,贾金原+   

  1. 同济大学 软件学院,上海 201804
  • 出版日期:2013-03-01 发布日期:2013-03-05

Design and Realization of HBase Migration Tool

YANG Hanbing, ZHAO Long, JIA Jinyuan+   

  1. School of Software Engineering, Tongji University, Shanghai 201804, China
  • Online:2013-03-01 Published:2013-03-05

摘要: 当今互联网中的巨大信息量使得整个应用朝着数据海量化,用户访问高并行化的方向发展,这对在应用中承受巨大负荷的数据库层提出了更高的性能要求,应用分布式数据库HBase可以解决这一问题。以分布式数据库HBase为基础,提出了在存储海量数据的前提下如何将数据库从关系型数据库向HBase迁移的方法,并依据所提出的四种表模式变换方法实现了半自动化迁移工具。以HouseDIY家居设计系统这个Web3D可视化电子商务类应用为例,阐述了该半自动化迁移工具的具体工作过程及细节,并证明了该迁移工具能够有效地把原有基于SQL Server数据库的Web应用简单、快速地移植到HBase数据库上。最后,该工具与Kettle进行了比较,证明了经该工具迁移后数据库在连接查询下的高效性。

关键词: 数据库迁移, SQL Server, HBase, 分布式数据库模式

Abstract: Nowadays, the large amount of information generated in Internet makes the entire applications become larger in the scale of data and have higher visit capacity. And it requires better performance in the database layer of an application which bears enormous loads, while the distributed database HBase can solve this problem. Based on HBase which is a very popular distributed database, this paper proposes the methods to design a tool to migrate database which is on the huge amounts of data from the relational database to HBase, and realizes this semi-automated migration tool according to the four ways of converting table patterns mentioned here. Then this paper chooses the HouseDIY which is a typical E-commerce Web application as an example, shows how this tool works in detail, and demonstrates that this tool can do migration from the original SQL Server database of an application to HBase easily and quickly. Finally, this paper compares the tool with Kettle and explains that the tool has high performance in joins query.

Key words: database migration, SQL Server, HBase, distributed database pattern