关键词: 错误管理, 日志分析, 聚类

Abstract: Online system has to serve massive users’ requests in real time. Various issues will cause request failure during the system running. Due to large scale of the requests and logs, manual effort of failed requests management is highly time-consuming. This paper proposes a new problem management approach based on clustering, by mining the logs generated from the system and performing classification on the failed requests. Engineers/administrators can get an overview of the issues of the online system and know the distribution and trend of failed requests with the help of the proposed approach. Moreover, this approach can identify the code defect automatically and reduce the cost of debugging by using voting approach to locate the cause of each type of issue into a small range in the source code. Besides, this approach realizes a problem management tool, and deploys the tool on a real enterprise online system. The experimental results turn out that the proposed approach can manage failed requests efficiently and reduce the cost of maintenance and debugging. Moreover, this approach can be combined with the MapReduce technique to deal with large scale logs.

Key words: problem management, log analysis, clustering