关键词: 大数据系统, 构件选型, 决策树模型, 参数配置, 性能模型

Abstract: Big data applications include data collection, storage, analysis, mining, visualization, and other technical    aspects. Every aspect has a variety of solutions, involves several hundred application systems and the system configuration is complicated, which has brought great challenges for a company to construct big data applications. To solve the problem of component selection in the development of application system, this paper establishes standardized requirement norms and achieves automatic component selection by using the components selection decision tree. This paper embarks from the several mainstream distributed storage systems, takes Cassandra as an example, conducts experiments and uses multiple regression method to calculate the performance model for hardware parameters. Then, this    paper uses the performance model to help user configure hardware parameters under the input of user’s requirements. Finally, this paper studies the system’s principle, structure and characteristics and constructs a knowledge base of software parameters configuration to help configure software parameters. In these ways the problem of component selection and parameter configuration in the development of big data system can be solved.

Key words: big data system, component selection, decision tree model, parameter configuration, performance model