关键词: 主题模型, 数据挖掘, 文本数据库

Abstract:

It's prevalent to use topic model to analyze documents in data mining at present. LDA (latent Dirichlet   allocation), as a simple topic model, has received much attention. However, LDA assumes the generating process of each document to be independent, which neglects the connection between documents. By modeling the connection between documents, this paper develops a new topic model DbLDA (LDA over text database). DbLDA explores the partitional structure of text databases (e.g., time, location), utilizes the commonalities inside each subset and thus is more expressive than original LDA. Due to the complexity of DbLDA, this paper uses partial collapsed variational Bayesian method to perform the model inference task, which has a fast training speed. For experiments, this paper trains DbLDA and LDA on news datasets. The experimental results justify that DbLDA yields a better performance than LDA.

Key words: topic model, data mining, text database