关键词: 大数据, 数据挖掘, 分类算法, 概念漂移

Abstract: With the deepening research of the application on big data and the emergence of more and more distributed computing framework, the research on concept drift in data stream becomes one of the research highlights in data mining for big data.The existing research on concept drift mainly depends on the data structure and algorithm optimization, the calculation mainly depends on the sole computer and limited resources to complete concept drift detection. Thus,this paper proposes a classification mining algorithm and system for big data based on Storm to resist concept drift. The S-CVFDT (Storm-concept very fast decision tree) algorithm system uses the parallel window mechanism to detect mutant concept drift in data stream and adaptively changes the parallel window size so as to update S-CVFDT algorithm model. The experimental analysis and results show that the algorithm can effectively detect mutant concept drift and lower the system resources waste. Not only the modeling is more efficient, but also the classification accuracy is improved.

Key words: big data, data mining, classification algorithm, concept drift