Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (9): 2151-2162.DOI: 10.3778/j.issn.1673-9418.2102070

• Theory and Algorithm • Previous Articles     Next Articles

Particle Swarm Optimization Combined with Q-learning of Experience Sharing Strategy

LUO Yixuan1,2, LIU Jianhua1,2,+(), HU Renyuan1,2, ZHANG Dongyang1,2, BU Guannan1,2   

  1. 1. College of Computer Science and Mathematics, Fujian University of Technology, Fuzhou 350118, China
    2. Fujian Provincial Key Laboratory of Big Data Mining and Applications, Fuzhou 350118, China
  • Received:2021-03-01 Revised:2021-05-08 Online:2022-09-01 Published:2021-05-18
  • About author:LUO Yixuan, born in 1996, M.S. candidate,member of CCF. His research interests include computational intelligence and reinforcement learning.
    LIU Jianhua, born in 1967, Ph.D., professor,member of CCF. His research interests include computational intelligence, big data analysis and IoT technology.
    HU Renyuan, born in 1997, M.S. candidate,member of CCF. His research interests include natural language processing and deep learning.
    ZHANG Dongyang, born in 1994, M.S. candidate. Her research interests include computational intelligence and machine learning.
    BU Guannan, born in 1994, M.S. candidate,member of CCF. His research interests include computational intelligence and machine learning.
  • Supported by:
    Natural Science Foundation of Fujian Province(2019J01061137);Development Foundation for Fujian University of Technology(GY-Z17150)


罗逸轩1,2, 刘建华1,2,+(), 胡任远1,2, 张冬阳1,2, 卜冠南1,2   

  1. 1.福建工程学院 计算机科学与数学学院,福州 350118
    2.福建省大数据挖掘与应用技术重点实验室,福州 350118
  • 通讯作者: + E-mail:
  • 作者简介:罗逸轩(1996—),男,福建福州人,硕士研究生,CCF会员,主要研究方向为智能计算、强化学习。
  • 基金资助:


Particle swarm optimization (PSO) has shortcomings such as easy to fall into local optimum, insufficient diversity and low precision. Recently, adopting the strategy of combining the reinforcement learning method like Q-learning to improve the PSO algorithm has become a new idea. However, this method has been proven to suffer the insufficient objectiveness of parameter selection and the limited strategy is not capable of coping with various situations. This paper proposes a Q-learning PSO with experience sharing (QLPSOES). The algorithm combines the PSO algorithm with the reinforcement learning method to construct a Q-table for each particle for dynamic selection of particle parameter settings. At the same time, an experience sharing strategy is designed, in which the particles share the “behavior experience” of the optimal particle through the Q-table. This method can accelerate the convergence of Q-table, enhance the learning ability between particles, and balance the global and local search ability of the algorithm. In addition, this paper uses orthogonal analysis experiments to find reinforcement learning methods for the selection of state, action parameters and reward functions in the PSO algorithm. The experiment is tested on the CEC2013 test function. The results show that the convergence speed and convergence accuracy of the QLPSOES algorithm are significantly improved compared with other algorithms, which verifies that the algorithm has better performance.

Key words: particle swarm optimization (PSO), reinforcement learning, experience sharing strategy, Q-table, orthogonal experiment



关键词: 粒子群算法(PSO), 强化学习, 经验共享策略, Q表, 正交实验

CLC Number: