Abstract: The publishing freedom of users on Internet poses new challenges in Web content filtering. This paper presents a self-study algorithm, called SAFE (self-study algorithm of filtering Chinese text content), for Chinese content filtering through two layers. It processes texts in the form of data stream. Based on Apriori property, SAFE filters Chinese text content through two layers by mining key characters and keywords without manual dictionary. The per-formance research of SAFE on the real-world data shows that for the given theme, the recall of SAFE is greater than 93.75% and the precision is 100%. The runtime of SAFE satisfies the real-time requirement of Web applications.

Key words: data mining, text content filtering, keywords mining

摘要： 用户在互联网发布信息的自由性对Web信息内容过滤提出新的挑战。为此, 给出一种自学习的两级内容过滤算法SAFE (self-study algorithm of filtering Chinese text content)。SAFE以数据流的方式处理文本, 并根据Apriori性质, 在不依赖词典的情况下, 通过挖掘关键字和关键词实现对文档的两级内容过滤。利用真实世界Web文档验证了SAFE的有效性, 实验表明对给定的主题进行文本内容过滤, SAFE的查全率达到93.75%以上, 查准率达到100%, 执行时间能够满足Web应用的实时性要求。

关键词: 数据挖掘, 文本内容过滤, 关键词挖掘