车联网区块链吞吐量优化的深度强化学习方法研究

doi:10.3778/j.issn.1673-9418.2205019

摘要/Abstract

摘要： 区块链应用于车联网（IoV）可以有效解决车联网数据安全和隐私等问题。但是，区块链吞吐量低的问题阻碍了其在车联网中的广泛应用。已有的区块链吞吐量优化研究大都存在决策行为空间爆炸的问题，可扩展性较差。针对上述问题，提出了一种基于深度强化学习（DRL）的区块链车联网吞吐量优化方法，通过选择区块生产者和共识算法，调整区块大小和区块间隔优化区块链的吞吐量，同时保证IoV区块链的去中心化、延迟和安全性。该方法通过引入BDQ框架将行为空间进行细粒度划分，解决了区块链使用传统深度强化学习方法对吞吐量进行优化时出现的行为空间爆炸问题。仿真结果表明，提出的方法可以有效地提高IoV区块链系统的吞吐量。

关键词: 车联网（IoV）, 区块链, 吞吐量, 深度强化学习（DRL）

Abstract: The rapid development of Internet of vehicles (IoV) depends on the safe and reliable infrastructure for storing and sharing large amounts of data. Blockchain, a kind of distributed data storage technology that cannot be forged and tampered with, can solve the security and privacy issues of IoV. However, the low throughput of blockchain hinders its wide application in IoV. The current research on blockchain throughput optimization has poor scalability because of its action space explosion. Aiming at the above problems, a blockchain throughput optimi-zation method in IoV based on deep reinforcement learning (DRL) is proposed to maximize the transaction throughput, and optimize the throughput of the blockchain by choosing block producers and consensus algorithms, adjusting block size and block interval while ensuring the decentralization, low delay and high security of the underlying blockchain system. This method introduces the branching dueling Q-network (BDQ) framework in DRL, carries out fine-grained division for action space, and solves the problem of action space explosion of traditional deep reinforcement learning methods. Simulation results show that the proposed method can improve the throughput of blockchain in IoV effectively.

Key words: Internet of vehicles (IoV), blockchain, throughput, deep reinforcement learning (DRL)

张立, 段明达, 万剑雄, 李雷孝, 刘楚仪. 车联网区块链吞吐量优化的深度强化学习方法研究[J]. 计算机科学与探索, 2023, 17(7): 1708-1718.

ZHANG Li, DUAN Mingda, WAN Jianxiong, LI Leixiao, LIU Chuyi. Research on Deep Reinforcement Learning Method for Throughput Optimization of Internet of Vehicles Blockchain[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(7): 1708-1718.

参考文献

[1] YANG F, WANG S, LI J, et al. An overview of internet of vehicles[J]. China communications, 2014, 11(10): 1-15.
[2] KAIWARTYA O, ABDULLAH A H, CAO Y, et al. Internet of vehicles: motivation, layered architecture, network model, challenges, and future aspects[J]. IEEE Access, 2016, 4: 5356-5373.
[3] YANG Z, YANG K, LEI L, et al. Blockchain-based decen-tralized trust management in vehicular networks[J]. IEEE Internet of Things Journal, 2018, 6(2): 1495-1505.
[4] NAKAMOTO S. Bitcoin: a peer-to-peer electronic cash sys-tem[J]. Decentralized Business Review, 2008: 21260.
[5] YU F R. vDLT: a service-oriented blockchain system with virtualization and decoupled management/control and exec-ution[J]. arXiv:1809.00290, 2018.
[6] LIU M, YU F R, TENG Y, et al. Performance optimization for blockchain-enabled industrial Internet of Things (IIoT) systems: a deep reinforcement learning approach[J]. IEEE Transactions on Industrial Informatics, 2019, 15(6): 3559-3570.
[7] 喻辉, 张宗洋, 刘建伟. 比特币区块链扩容技术研究[J]. 计算机研究与发展, 2017, 54(10): 2390-2403.
YU H, ZHANG Z Y, LIU J W. Research on scaling technology of bitcoin blockchain[J]. Journal of Computer Research and Development, 2017, 54(10): 2390-2403.
[8] POON J, DRYJA T. The bitcoin lightning network: scalable off-chain instant payments: Draft Version 0.5.9.2[R]. 2016.
[9] DECKER C, WATTENHOFER R. A fast and scalable payment network with bitcoin duplex micropayment channels[C]//LNCS 9212: Proceedings of the 17th International Sym-posium on Stabilization, Safety, and Security of Distributed Systems, Edmonton, Aug 18-21, 2015. Cham: Springer, 2015: 3-18.
[10] LUU L, NARAYANAN V, ZHENG C, et al. A secure sharding protocol for open blockchains[C]//Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Oct 24-28, 2016. New York: ACM, 2016: 17-30.
[11] KOKORIS-KOGIAS E, JOVANOVIC P, GASSER L, et al. Omniledger: a secure, scale-out, decentralized ledger via sha-rding[C]//Proceedings of the 2018 IEEE Symposium on Security and Privacy, San Francisco, May 21-23, 2018. Washington: IEEE Computer Society, 2018: 583-598.
[12] EYAL I, GENCER A E, SIRER E G, et al. Bitcoin-NG: a scalable blockchain protocol[C]//Proceedings of the 13th USENIX Symposium on Networked Systems Design and Implementation, Santa Clara, Mar 16-18, 2016. Berkeley: USENIX Association, 2016: 45-59.
[13] CACHIN C. Architecture of the hyperledger blockchain fabric: CH-8803[R]. Zurich, 2016.
[14] KOGIAS E K, JOVANOVIC P, GAILLY N, et al. Enhan-cing bitcoin security and performance with strong consis-tency via collective signing[C]//Proceedings of the 25th USENIX Security Symposium, Austin, Aug 10-12, 2016. Berkeley: USENIX Association, 2016: 279-296.
[15] ABRAHAM I, MALKHI D, NAYAK K, et al. Solida: a blockchain protocol based on reconfigurable Byzantine consensus[J]. arXiv:1612.02916, 2016.
[16] CASTRO M, LISKOV B. Practical Byzantine fault toler-ance[C]//Proceedings of the 3rd Symposium on Operating Systems Design and Implementation, New Orleans, Feb 22-25, 1999. Berkeley: USENIX Association, 1999: 173-186.
[17] QIU C, YU F R, XU F, et al. Blockchain-based distributed software-defined vehicular networks via deep Q-learning[C]// Proceedings of the 8th ACM Symposium on Design and Analysis of Intelligent Vehicular Networks and Applic-ations, Montreal, Oct 28-Nov 2, 2018. New York: ACM, 2018: 8-14.
[18] FU X, YU F R, WANG J, et al. Performance optimization for blockchain-enabled distributed network function virtual-ization management and orchestration[J]. IEEE Transactions on Vehicular Technology, 2020, 69(6): 6670-6679.
[19] LIN H, GARG S, HU J, et al. Blockchain and deep reinfo-rcement learning empowered spatial crowdsourcing in software-defined Internet of vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(6): 3755-3764.
[20] MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.
[21] TAVAKOLI A, PARDO F, KORMUSHEV P. Action bran-ching architectures for deep reinforcement learning[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th Innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 4131-4138.
[22] CASTRO M, LISKOV B. Practical Byzantine fault toler-ance and proactive recovery[J]. ACM Transactions on Computer Systems, 2002, 20(4): 398-461.
[23] JAVARONE M A, WRIGHT C S. From bitcoin to bitcoin cash: a network analysis[C]//Proceedings of the 1st Work-shop on Cryptocurrencies and Blockchains for Distributed Systems, Munich, Jun 15, 2018. New York: ACM, 2018: 77-81.
[24] KOTLA R, ALVISI L, DAHLIN M, et al. Zyzzyva: spec-ulative Byzantine fault tolerance[C]//Proceedings of the 21st ACM Symposium on Operating Systems Principles, Stevenson, Oct 14-17, 2007. New York: ACM, 2007: 45-58.
[25] GUERRAOUI R, KNE?EVI? N, QUéMA V, et al. The next 700 BFT protocols[C]//Proceedings of the 5th European Conference on Computer Systems, Paris, Apr 13-16, 2010. New York: ACM, 2010: 363-376.
[26] GINI C. Variability and mutability[J]. Journal of the Royal Statistical Society, 1913, 76: 619-622.