关键词: 大数据, 负载均衡, 实时分析处理

Abstract: The system must have high data processing efficiency during processing big data. This paper designs a data processing model based on receiving and processing separation. Using the method the system can real-time, efficiently, stably process data. The model includes the data receiving unit, the memory database, the original data distribution unit, the data processing unit, the processed data distribution unit and the data merging unit. The receiving unit receives and integrates structure and unstructured data to a whole structure datum and puts it into the memory database. The original data distribution unit checks data from the memory database and dispatches the data to the data processing unit using the big data load balancing method. The data processing unit processes data and puts the result data into the memory database again. The processed data distribution unit gets data from the memory database and dispatches the data to the data merging unit using the big data load balancing method. The experimental results show that the system keeps with the high efficiency using the model.

Key words: big data, load balancing, real-time analysis and processing