关键词: 模仿学习, 行为克隆, 逆强化学习, 对抗模仿学习

Abstract: The problem of optimal decision has a long history in the field of machine learning. Imitation learning, originating from reinforcement learning, is studied to reconstruct the expected policy from expert data and learn the optimal decision-making. In recent years, imitation learning has been successfully applied in both theoretical research and computer vision, as well as in various applications such as autonomous driving and robotics. The origin of imitation learning and the two traditional research methods, namely behavior cloning and inverse reinforcement learning, are introduced. With the development of adversarial training structures, generative adversarial imitation learning has become a key research direction, and its subsequent improvement work is collectively referred to as adversarial imitation learning. The research content of adversarial imitation learning combined with visual demonstrations is analyzed, along with summarizing common issues like suboptimal expert demonstrations, limited data, and low sample utilization efficiency, and the existing corresponding solutions. Then, the performance of different methods in addressing these problems is compared and analyzed based on experimental results. Finally, practical applications of adversarial visual imitation learning in scenarios such as autonomous driving and industrial robotics are discussed, and this paper is concluded by pointing out future research directions, as well as the potential prospects and challenges in applications.

Key words: imitation learning, behavior cloning, inverse reinforcement learning, adversarial imitation learning