计算机科学与探索

• 学术研究 •    

车联网区块链吞吐量优化的强化学习方法研究

张立, 段明达, 万剑雄,李雷孝,刘楚仪   

  1. 1.内蒙古自治区气象信息中心,呼和浩特 010051
    2.内蒙古工业大学,呼和浩特 010051
    3.内蒙古自治区基于大数据的软件服务工程技术研究中心, 呼和浩特 100080

Research on Deep Reinforcement Learning Method for Throughput Optimization of Internet of Vehicles Blockchain

ZHANG Li, DUAN Mingda, WAN Jianxiong, LI Leixiao, LIU Chuyi   

  1. 1.Inner Mongolia Meteorological Information Center, Hohhot 010051, China
    2.Inner Mongolia University of Technology, Hohhot 010051, China
    3.Inner Mongolia Autonomous Region Engineering & Technology Research Center of Big Data Based Software Service, Hohhot 010080, China

摘要: 区块链应用于车联网(Internet of Vehicles ,IoV)可以有效解决车联网数据安全和隐私等问题。但是,区块链吞吐量低的问题阻碍了其在车联网中的广泛应用。已有的区块链吞吐量优化研究大都存在决策行为空间爆炸的问题,可扩展性较差。针对上述问题,提出了一种基于深度强化学习(Deep Reinforcement Learning,DRL)的区块链车联网吞吐量优化方法,通过选择区块生产者、共识算法、调整区块大小和区块间隔优化区块链的吞吐量,同时保证IoV区块链的去中心化、延迟和安全性。该方法通过引入BDQ(Branching Dueling Q-Network,BDQ)框架将行为空间进行细粒度划分,解决了区块链使用传统深度强化学习方法对吞吐量进行优化时出现的行为空间爆炸问题。仿真结果表明,提出的方法可以有效地提高IoV区块链系统的吞吐量。

关键词: 车联网, 区块链, 吞吐量, 深度强化学习

Abstract: The rapid development of Internet of Vehicles (IoV) depends on the safe and reliable infrastructure for storing and sharing large amounts of data. Blockchain, a kind of distributed data storage technology that cannot be forged and tampered with, can solve the security and privacy issues of IoV. However, the low throughput of blockchain hinders its wide application in IoV. The current research on blockchain throughput optimization has poor scalability because of its action space explosion. As the problems above, a blockchain throughput optimization method in IoV based on Deep Reinforcement Learning (DRL) is proposed to maximize the transaction throughput, optimize the throughput of the blockchain by choosing block producers, consensus algorithms, adjusting block size and block interval while ensuring the decentralization, low delay and high security of the underlying blockchain system. This method introduces the Branching Dueling Q-network (BDQ) framework in DRL, divides the action space into fine-grained, and solves the problem of traditional deep reinforcement learning methods action space explosion. Simulation results show that the proposed method can improve the throughput of blockchain in IoV effectively.

Key words: IoV, Blockchain, Throughput, Deep Reinforcement Learning