Survey of Affective-Based Dialogue System

doi:10.3778/j.issn.1673-9418.2012012

Abstract

Abstract:

As an important way of human-computer interaction, the dialogue system has broad application prospects. Existing dialogue systems focus on solving problems such as semantic consistency and content richness and paying little attention to improving human-computer interaction and human-computer resonance. How to make the generated sentences communicate with users more naturally on the basis of semantic relevance is one of the main problems in current dialogue system. First, it summarizes the overall situation of the dialogue system. Then it introduces the two major tasks of dialogue emotion perception and emotional dialogue generation in the emotional dialogue system. And further it investigates and summarizes related methods respectively. Dialogue emotion perception tasks are roughly divided into context-based and user-based methods. The emotional dialogue generation methods include rule matching algorithms, specified emotional response generation models, and non-specified emotional response generation models. The models are compared and analyzed in terms of emotional data categories and model methods. Next, for subsequent research, it summarizes characteristics and links of the data sets under the two major tasks. Further, different evaluation methods in the current emotional dialogue system are summarized. Finally, the work of the emotional dialogue system is summarized and prospected.

Key words: emotional dialogue system, dialogue emotional perception, generation of emotional dialogue

摘要：

对话系统作为人机交互的重要方式，有着广泛的应用前景。现有的对话系统专注于解决语义一致性和内容丰富性等问题，对于提高人机交互以及产生人机共鸣方向的研究关注度不高。如何让生成的语句在具有语义相关性的基础上更自然地与用户交流是当前对话系统面临的主要问题之一。首先对对话系统进行了整体情况的概括。接着介绍了情感对话系统中的对话情绪感知和情感对话生成两大任务，并分别调研归纳了相关方法。对话情绪感知任务大致分为基于上下文和基于用户信息两类方法。情感对话生成的方法包括规则匹配算法、指定情感回复的生成模型和不指定情感回复的生成模型，并从情绪数据类别和模型方法等方面进行了对比分析。然后总结整理了两大任务下数据集的特点和链接便于后续的研究，并归纳了当前情感对话系统中不同的评估方法。最后对情感对话系统的工作进行了总结和展望。

关键词: 情感对话系统, 对话情绪感知, 情感对话生成

ZHUANG Yin, LIU Zhen, LIU Tingting, WANG Yuanyi, LIU Cuijuan, CHAI Yanjie. Survey of Affective-Based Dialogue System[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(5): 825-837.

庄寅, 刘箴, 刘婷婷, 王媛怡, 刘翠娟, 柴艳杰. 文本情感对话系统研究综述[J]. 计算机科学与探索, 2021, 15(5): 825-837.

References

[1] WILENSKY R, CHIN D N, LURIA M, et al. The Berkeley UNIX consultant project[M]//BRAUER W, WAHLSTER W. Wissensbasierte Systeme. Berlin, Heidelberg: Springer, 1987.
[2] WALLACE R S. The Anatomy of A. L. I. C. E.[M]//EPSTEIN R, ROBERTS G, BEBER G. Parsing the Turing Test. Berlin, Heidelberg: Springer, 2009.
[3] MARIETTO M G B, DE AGUIAR R V, BARBOSA G O, et al. Artificial intelligence markup language: a brief tutorial[J]. arXiv:1307.3091, 2013.
[4] GOODRICH M A, SCHULTZ A C. Human-robot interaction: a survey[J]. Human-Computer Interaction, 2007, 1(3): 203-275.
[5] SALOVEY P, MAYER J D. Emotional intelligence[J]. Imagination, Cognition and Personality, 1990, 9(3): 185-211.
[6] PRENDINGER H, MORI J, ISHIZUKA M. Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game[J]. International Journal of Human-Computer Studies, 2005, 62(2): 231-245.
[7] CHE W X, ZHANG W N. Survey of human-machine dialogue system[J]. Artificial Intelligence VIEW, 2018(1): 76-82.
车万翔, 张伟男. 人机对话系统综述[J]. 人工智能, 2018(1): 76-82.
[8] CHEN C, ZHU Q Q, YAN R, et al. Survey on deep learning based open domain dialogue system[J]. Chinese Journal of Computers, 2019, 42(7): 1439-1466.
陈晨, 朱晴晴, 严睿, 等. 基于深度学习的开放领域对话系统研究综述[J]. 计算机学报, 2019, 42(7): 1439-1466.
[9] PAMUNGKAS E W. Emotionally-aware chatbots: a survey[J]. arXiv:1906.09774, 2019.
[10] MARSELLA S, GRATCH J. Computationally modeling human emotion[J]. Communications of the ACM, 2014, 57(12): 56-67.
[11] CHEN J P, MA J H, WANG Y J. A survey of human-computer dialogue system based on multiple-round interaction[J]. Journal of Nanjing University of Information Science & Technology, 2019, 11(3): 256-268.
陈健鹏, 马建辉, 王怡君. 基于多轮交互的人机对话系统综述[J]. 南京信息工程大学学报, 2019, 11(3): 256-268.
[12] BORDES A, BOUREAU Y L, WESTON J. Learning end-to-end goal-oriented dialog[J]. arXiv:1605.07683, 2017.
[13] ZHOU H, HUANG M, ZHANG T, et al. Emotional chatting machine: emotional conversation generation with internal and external memory[C]//Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 730-738.
[14] PORIA S, CAMBRIA E, HAZARIKA D, et al. Context-dependent sentiment analysis in user-generated videos[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Jul 31-Aug 4, 2017. Stroudsburg: ACL, 2017: 873-883.
[15] CHEN S Y, HSU C C, KUO C C, et al. Emotionlines: an emotion corpus of multi-party conversations[J]. arXiv:1802. 08379, 2018.
[16] CERISARA C, JAFARITAZEHJANI S, OLUOKUN A, et al. Multi-task dialog act and sentiment recognition on Mastodon[C]//Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, Aug 20-26, 2018. Stroudsburg: ACL, 2018: 745-754.
[17] WANG J C, XU Y, LIU Q Y, et al. Dialog sentiment analysis with neural topic model[J]. Journal of Chinese Information Processing, 2020, 34(1): 106-112.
王建成, 徐扬, 刘启元, 等. 基于神经主题模型的对话情感分析[J]. 中文信息学报, 2020, 34(1): 106-112.
[18] HAZARIKA D, PORIA S, ZADEH A, et al. Conversational memory network for emotion recognition in dyadic dialogue videos[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 2122-2132.
[19] HAZARIKA D, PORIA S, MIHALCEA R, et al. ICON: interactive conversational memory network for multimodal emotion detection[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 2594-2604.
[20] MAJUMDER N, PORIA S, HAZARIKA D, et al. Dialogue-RNN: an attentive RNN for emotion detection in conversations[C]//Proceedings of the AAAI Conference on Artificial Intelligence, Hilton Hawaiian Village, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 6818-6825.
[21] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv:1609.02907, 2016.
[22] ZHANG D, WU L, SUN C, et al. Modeling both context-and speaker-sensitive dependence for emotion detection in multi-speaker conversations[C]//Proceedings of the 2019 International Joint Conference on Artificial Intelligence, Macao, China, Jul 13-19, 2019. San Mateo: Morgan Kaufmann, 2019: 5415-5421.
[23] GHOSAL D, MAJUMDER N, PORIA S, et al. DialogueGCN: a graph convolutional neural network for emotion recognition in conversation[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 154-164.
[24] WEIZENBAUM J. ELIZA—a computer program for the study of natural language communication between man and machine[J]. Communications of the ACM, 1966, 9(1): 36-45.
[25] COLBY K M. Modeling a paranoid mind[J]. Behavioral and Brain Sciences, 1981, 4(4): 515-534.
[26] KESHTKAR F, INKPEN D. A pattern-based model for generating text to express emotion[C]//LNCS 6975: Proceedings of the 2011 International Conference on Affective Computing and Intelligent Interaction, Memphis, Oct 9-12, 2011. Berlin, Heidelberg: Springer, 2011: 11-21.
[27] SKOWRON M. Affect listeners: acquisition of affective states by means of conversational systems[C]//LNCS 5967: Development of Multimodal Interfaces: Active Listening and Synchrony, Dublin, Mar 23-27, 2009. Berlin, Heidelberg: Springer, 2010: 169-181.
[28] HE Y S, LI L. The application of “emotional guidance” in physical education[J]. China School Physical Education, 2004(3): 12-13.
郝延省, 李丽. “情绪辅导”在体育教学中的运用[J]. 中国学校体育, 2004(3): 12-13.
[29] SHEN L, FENG Y. CDL: curriculum dual learning for emotion-controllable response generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 556-566.
[30] HUANG C, ZAIANE O R, TRABELSI A, et al. Automatic dialogue generation with expressed emotions[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 49-54.
[31] XIE Y, SVIKHNUSHINA E, PU P. A multi-turn emotionally engaging dialog model[J]. arXiv:1908.07816, 2019.
[32] SONG Z, ZHENG X, LIU L, et al. Generating responses with a specific emotion in dialog[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 3685-3695.
[33] MA Z, YANG R, DU B, et al. A control unit for emotional conversation generation[J]. IEEE Access, 2020, 8: 43168-43176.
[34] LUBIS N, SAKTI S, YOSHINO K, et al. Eliciting positive emotion through affect-sensitive dialogue response generation: a neural network approach[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th Innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 5293-5300.
[35] COLOMBO P, WITON W, MODI A, et al. Affect-driven dialog generation[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 3734-3743.
[36] MOHAMMAD S. Obtaining reliable human ratings of valence, arousal, and dominance for 20, 000 English words[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Jul 15-20, 2018. Stroudsburg: ACL, 2018: 174-184.
[37] ZHOU X D, WANG W Y. MojiTalk: generating emotional responses at scale[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Jul 15-20, 2018. Stroudsburg: ACL, 2018: 1128-1137.
[38] YANG F R, HUO N, ZHANG X H, et al. Topic-expanded emotional conversation generation with attention mechanism[J/OL]. Journal of Computer Applications[2021-01-29]. http://kns.cnki.net/kcms/detail/51.1307.TP.20201011.1715.014.html.
杨丰瑞, 霍娜, 张许红, 等. 基于注意力机制的主题扩展情感对话生成[J/OL]. 计算机应用[2021-01-29]. http://kns.cnki.net/kcms/detail/51.1307.TP.20201011.1715.014.html.
[39] SUN X, LI J, WEI X. Emotional editing constraint conversation generation based on reinforcement learning[J]. Acta Automatica Sinica[2021-01-29]. https://doi.org/10.16383/j.aas.c190058.
孙晓, 李佳, 卫星. 基于强化学习的情感编辑约束对话内容生成[J]. 自动化学报[2021-01-29]. https://doi.org/10.16383/ j.aas.c190058.
[40] PENG Y, FANG Y, XIE Z, et al. Topic-enhanced emotional conversation generation with attention mechanism[J]. Knowledge-Based Systems, 2019, 163: 429-437.
[41] ASGHAR N, POUPART P, HOEY J, et al. Affective neural response generation[C]//LNCS 10772: Proceedings of the 40th European Conference on IR Research on Advances in Information Retrieval, Grenoble, Mar 26-29, 2018. Berlin, Heidelberg: Springer, 2018: 154-166.
[42] SHI W Y, YU Z. Sentiment adaptive end-to-end dialog systems[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Jul 15-20, 2018. Stroudsburg: ACL, 2018: 1509-1519.
[43] LIANG Y, MENG F, ZHANG Y, et al. Infusing multi-source knowledge with heterogeneous graph neural network for emotional conversation generation[J]. arXiv:2012.04882, 2020.
[44] ZHANG C X, SONG D J, HUANG C, et al. Heterogeneous graph neural network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, Aug 4-8, 2019. New York: ACM, 2019: 793-803.
[45] SHANTALA R, KYSELOV G, KYSELOVA A. Neural dialogue system with emotion embeddings[C]//Proceedings of the 2018 IEEE 1st International Conference on System Analysis & Intelligent Computing, Ukraine, Oct 8-12, 2018. Piscataway: IEEE, 2018: 1-4.
[46] GHOSH S, CHOLLET M, LAKSANA E, et al. Affect-LM: a neural language model for customizable affective text generation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 634-642.
[47] BUSSO C, BULUT M, LEE C C, et al. IEMOCAP: interactive emotional dyadic motion capture database[J]. Language Resources and Evaluation, 2008, 42(4): 335-359.
[48] MCKEOWN G, VALSTAR M, COWIE R, et al. The semaine database: annotated multimodal records of emotionally colored conversations between a person and a limited agent[J]. IEEE Transactions on Affective Computing, 2011, 3(1): 5-17.
[49] PORIA S, HAZARIKA D, MAJUMDER N, et al. MELD: a multimodal multi-party dataset for emotion recognition in conversations[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 527-536.
[50] LI Y R, SU H, SHEN X Y, et al. DailyDialog: a manually labelled multi-turn dialogue dataset[C]//Proceedings of the 8th International Joint Conference on Natural Language Processing, Taipei, China, Nov 27-Dec 1, 2017. Asian Federation of Natural Language Processing, 2017: 986-995.
[51] CHATTERJEE A, GUPTA U, CHINNAKOTLA M K, et al. Understanding emotions in text using deep learning and big data[J]. Computers in Human Behavior, 2019, 93: 309-317.
[52] SHANG L F, LU Z D, LI H. Neural responding machine for short-text conversation[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 1577-1586.
[53] DANESCU-NICULESCU-MIZIL C, LEE L. Chameleons in imagined conversations: a new approach to understanding coordination of linguistic style in dialogs[C]//Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics, Portland, Jun 23, 2011. Stroudsburg: ACL, 2011: 76-87.
[54] TIEDEMANN J. News from OPUS—a collection of multilingual parallel corpora with tools and interfaces[C]//Proceedings of the Advances in Natural Language Processing, Bulgaria, Sep 14-16, 2009. Stroudsburg: ACL, 2009: 237-248.
[55] HUANG C Y, KU L W. Emotionpush: emotion and response time prediction towards human-like chatbots[C]//Proceedings of the 2018 IEEE Global Communications Conference, Abu Dhabi, Dec 9-13, 2018. Piscataway: IEEE, 2018: 206-212.
[56] FIRDAUS M, CHAUHAN H, EKBAL A, et al. EmoSen: generating sentiment and emotion controlled responses in a multimodal dialogue system[J]. IEEE Transactions on Affective Computing, 2020, 14(8): 1-12.
[57] SHAO Y L, GOUWS S, BRITZ D, et al. Generating high-quality and informative conversation responses with sequence-to-sequence models[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Sep 9-11, 2017. Stroudsburg: ACL, 2017: 2210-2219.
[58] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003, 3(2): 1137-1155.
[59] PAPINENI K, ROUKOS S, WARD T, et al. BLEU: a method for automatic evaluation of machine translation[C]//Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Jul 7-12, 2002. Stroudsburg: ACL, 2002: 311-318.
[60] LI J, GALLEY M, BROCKETT C, et al. A diversity-promoting objective function for neural conversation models[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, California, Jun 12-17, 2016. Stroudsburg: ACL, 2016: 110-119.
[61] VINYALS O, LE Q. A neural conversational model[J]. arXiv:1506.05869, 2015.
[62] LIU C W, LOWE R, SERBAN I V, et al. How NOT to evaluate your dialogue system: an empirical study of unsupervised evaluation metrics for dialogue response generation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 2122-2132.