关键词: 视频, 标题生成, GoogLeNet, ResNet, 长短时记忆（LSTM）, 密集帧率采样

Abstract: The method of picking frames at regular fixed intervals for video captioning leads to missing much static and dynamic information easily, thereby, the quality of generation sentences can't be further improved. Facing the challenge, this paper proposes a model named dense frame rate sampling based captioning model (DFS-CM). The video is first split into many fragments with fixed length, then, all the frames in each fragment are fed into the pre-trained CNN (convolutional neural network) model for feature. Next, the average or maximum CNN features from each fragment are calculated in line with the corresponding dimension. Additionally, the training strategy of the pre-trained CNN model is improved and the stability and generalization of the model are strengthened. Finally, based on S2VT framework, the model is evaluated with GoogLeNet and ResNet-152 features. The experimental results on Youtube2Text dataset show that the performance is improved greatly compared to the baseline model. Particularly, the B@4 and CIDEr reach to 47.1% and 34.1% respectively with ResNet-152 and maximum mode.

Key words: video, caption generation, GoogLeNet, ResNet, long short term memory (LSTM), dense frame rate sampling