Abstract: In recent years, text information has experienced explosive growth, and people haven??t enough time to read all these texts. Therefore, how to automatically extract key information from massive texts is particularly important. Text summarization technology can solve this problem. At present, sequence to sequence model with attention mechanism is usually used to generate text summary. However, the attention mechanism is independent at  each moment, and the text information generated at the previous moment is not taken into account. This results in the term repetition in the summary. In order to solve this problem, this paper proposes new coverage models. In these models, coverage vector is developed to record the historical attention weight distribution and adjust the current attention weight distribution, making the summarization model pay more attention to information that is not used. The experiment conducted on the Sina Weibo data set shows that the proposed model with coverage mechanism performs better than the normal sequence to sequence model.

Key words: text summarization, deep learning, recurrent neural network, coverage mechanism

摘要： 近年来文本信息出现了爆炸式增长，人们没有足够的精力去阅读这些文本，因此如何自动地从文本中提取关键信息就显得尤为重要，而文本摘要技术可以很好地解决这个问题。目前的文本摘要技术主要是利用带有注意力（attention）机制的序列到序列模型（sequence to sequence）对文本生成摘要，但是注意力机制在每个时刻的计算是独立的，没有考虑到之前时刻生成的文本信息，导致模型在生成文本时忽略了之前生成的内容，导致重复生成部分信息。针对这一问题，在文本摘要模型中引入了一种新的覆盖率（coverage）机制，通过覆盖向量记录历史时刻的注意力权重分布信息，并用来改变当前时刻注意力机制的权重分布，使模型更多地关注没有利用到的信息。改进后的模型在新浪微博数据集上进行了实验，实验结果表明，基于新提出的覆盖率机制的文本摘要模型的准确度高于普通的序列到序列模型。

关键词: 文本摘要, 深度学习, 循环神经网络, 覆盖率机制