高精度可扩展的密态隐私保护语音分类

doi:10.3778/j.issn.1673-9418.2311085

摘要/Abstract

摘要： 为解决现有全同态加密技术在语音分类任务中计算效率和分类准确率均较低的挑战，提出一种高精度可扩展的密态隐私保护语音分类PPSC方案。设计基于CKKS全同态加密技术两方服务器协同的安全乘法协议，因避免使用昂贵的自举操作，可有效提升深层次密文乘法的计算效率，从而使得方案可扩展到更深层次的神经网络；基于上述架构设计安全指数、安全倒数以及安全比较等安全非多项式协议，相较于多项式近似拟合非多项式运算的方法，在提高计算精度的同时，降低了计算开销。安全实现卷积层、ReLU层、平均池化层、全连接层、Softmax层等PPSC方案的基本模块，确保语音数据、语音模型和中间计算结果的隐私性。从有效性和安全性等维度对PPSC方案进行了详尽的理论分析，证明安全乘法协议在更深层次的乘法运算中具有更高的运算效率。在Speech Command Database语音数据集上的实验结果表明，PPSC方案可以在保护数据和模型参数隐私的情况下实现有效的语音分类，其准确率相比于HEKWS方案提高3.57个百分点。

关键词: 全同态加密, 隐私保护, 两方协同, 语音分类, 安全计算协议

Abstract: To address the challenges of low computational efficiency and classification accuracy in existing fully homomorphic encryption technology for speech classification tasks, a high-precision and scalable encrypted privacy-preserving speech classification (PPSC) scheme is proposed. First of all, a secure multiplication protocol based on CKKS fully homomorphic encryption technology is designed to avoid the use of expensive bootstrapping operations, which can effectively improve the computational efficiency of deep ciphertext multiplication, so that the scheme can be extended to deeper neural networks. Based on the above architecture, secure non-polynomial protocols such as secure exponent, secure reciprocal and secure comparison are designed. Compared with the method of polynomial approximate fitting of non-polynomial operations, the protocols improve computation accuracy and reduce computation overhead. Secondly, the PPSC scheme securely implements the fundamental modules such as the convolutional layer, ReLU layer, average pooling layer, fully connected layer, and Softmax layer. This ensures the privacy of speech data, speech classification models, and intermediate computing results. Finally, a detailed theoretical analysis of the PPSC scheme is conducted to evaluate its effectiveness and security. The analysis demonstrates that the secure multiplication protocol exhibits higher computational efficiency in deeper multiplication operations. Experimental results on the Speech Command Database validate the effectiveness of the PPSC scheme in achieving accurate speech classification while preserving the privacy of data and model parameters. Furthermore, the proposed scheme achieves an accuracy that is 3.57 percentage points higher than that of the HEKWS scheme.

Key words: fully homomorphic encryption, privacy-preserving, two-party collaboration, speech classification, secure computing protocol

王雷蕾, 宋考, 张媛媛, 毕仁万, 熊金波. 高精度可扩展的密态隐私保护语音分类[J]. 计算机科学与探索, 2025, 19(2): 528-538.

WANG Leilei, SONG Kao, ZHANG Yuanyuan, BI Renwan, XIONG Jinbo. PPSC: High-Precision and Scalable Encrypted Privacy-Preserving Speech Classification[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(2): 528-538.

参考文献

[1] LI Q, LIN X D. Efficient and privacy-preserving speaker recognition for cybertwin-driven 6G[J]. IEEE Internet of Things Journal, 2021, 8(22): 16195-16206.
[2] ABDEL-HAMID O, MOHAMED A R, JIANG H, et al. Convolutional neural networks for speech recognition[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014, 22(10): 1533-1545.
[3] BI R W, XIONG J B, TIAN Y L, et al. Achieving lightweight and privacy-preserving object detection for connected autonomous vehicles[J]. IEEE Internet of Things Journal, 2023, 10(3): 2314-2329.
[4] XIONG J B, BI R W, TIAN Y L, et al. Toward lightweight, privacy-preserving cooperative object classification for connected autonomous vehicles[J]. IEEE Internet of Things Journal, 2022, 9(4): 2787-2801.
[5] 张晓旭, 马志强, 刘志强, 等. Transformer在语音识别任务中的研究现状与展望[J]. 计算机科学与探索, 2021, 15(9): 1578-1594.
ZHANG X X, MA Z Q, LIU Z Q, et al. Research status and prospect of transformer in speech recognition[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(9): 1578-1594.
[6] GARTENBERG C. Apple apologizes for Siri audio recordings, announces privacy changes going forward[Z]. 2019.
[7] TUROW J. Voice intelligence and the future of engagement metrics on commercial platforms[M]//The economic lives of platforms. [S.l.]: Bristol University Press, 2024: 73-87.
[8] 陈学先, 李宏发, 李霄铭, 等. 基于同态加密的能源大数据安全系统[J]. 福建师范大学学报(自然科学版), 2023, 39(2): 9-25.
CHEN X X, LI H F, LI X M, et al. Secure energy big data system based on homomorphic encryption[J]. Journal of Fujian Normal University (Natural Science Edition), 2023, 39(2): 9-25.
[9] 赵敏, 田有亮, 熊金波, 等. 基于同态加密的神经网络模型训练方法[J]. 计算机科学, 2023, 50(5): 372-381.
ZHAO M, TIAN Y L, XIONG J B, et al. Neural network model training method based on homomorphic encryption[J]. Computer Science, 2023, 50(5): 372-381.
[10] 钟洋, 毕仁万, 颜西山, 等. 支持隐私保护训练的高效同态神经网络[J]. 计算机应用, 2022, 42(12): 3792-3800.
ZHONG Y, BI R W, YAN X S, et al. Efficient homomorphic neural network supporting privacy-preserving training[J]. Journal of Computer Applications, 2022, 42(12): 3792-3800.
[11] 王馨雅, 华光, 江昊, 等. 深度学习模型的版权保护研究综述[J]. 网络与信息安全学报, 2022, 8(2): 1-14.
WANG X Y, HUA G, JIANG H, et al. Survey on intellectual property protection for deep learning model[J]. Chinese Journal of Network and Information Security, 2022, 8(2): 1-14.
[12] DOWLIN N, GILAD-BACHRACH R, LAINE K, et al. Cryptonets: applying neural networks to encrypted data with high throughput and accuracy[C]//Proceedings of the 33rd International Conference on Machine Learning. New York: PMLR, 2016: 201-210.
[13] ZHANG S X, GONG Y F, YU D. Encrypted speech recognition using deep polynomial networks[C]//Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2019: 5691-5695.
[14] RAHULAMATHAVAN Y. Privacy-preserving similarity calculation of speaker features using fully homomorphic encryption[EB/OL]. [2023-10-08]. https://arxiv.org/abs/2202.07994.
[15] ELWORTH D L, KIM S. HEKWS: privacy-preserving convolutional neural network-based keyword spotting with a ciphertext packing technique[C]//Proceedings of the 2022 IEEE 24th International Workshop on Multimedia Signal Processing. Piscataway: IEEE, 2022: 1-6.
[16] BOURA C, GAMA N, GEORGIEVA M, et al. CHIMERA: combining ring-LWE-based fully homomorphic encryption schemes[J]. Journal of Mathematical Cryptology, 2020,14(1): 316-338.
[17] CHILLOTTI I, GAMA N, GEORGIEVA M, et al. Faster packed homomorphic operations and efficient circuit bootstrapping for TFHE[C]//Advances in Cryptology: Proceedings of the 23rd International Conference on the Theory and Application of Cryptology and Information Security. Cham: Springer, 2017: 377-408.
[18] DUCAS L, MICCIANCIO D. FHEW: bootstrapping homomorphic encryption in less than a second[C]//Advances in Cryptology-EUROCRYPT 2015: Proceedings of the 34th Annual International Conference on the Theory and Applications of Cryptographic Techniques. Berlin, Heidelberg: Springer, 2015: 617-640.
[19] FAN J F, VERCAUTEREN F. Somewhat practical fully homomorphic encryption[J]. IACR Cryptol EPrint Arch, 2024, 2012: 144.
[20] BRAKERSKI Z, GENTRY C, VAIKUNTANATHAN V. (leve-led) fully homomorphic encryption without bootstrapping[J]. ACM Transactions on Computation Theory, 2014, 6(3): 1-36.
[21] CHEON J H, KIM A, KIM M,et al. Homomorphic encryption for arithmetic of approximate numbers[C]//Proceedings of the 23rd International Conference on the Theory and Applications of Cryptology and Information Security.Cham: Springer, 2017: 409-437.
[22] LU W J, HUANG Z C, HONG C, et al. PEGASUS: bridging polynomial and non-polynomial evaluations in homomorphic encryption[C]//Proceedings of the 2021 IEEE Symposium on Security and Privacy. Piscataway: IEEE, 2021: 1057-1073.
[23] CHOU E, BEAL J, LEVY D, et al. Faster CryptoNets: leveraging sparsity for real-world encrypted inference[EB/OL]. [2023-10-08]. https://arxiv.org/abs/1811.09953.
[24] AL BADAWI A, JIN C, LIN J, et al. Towards the AlexNet moment for homomorphic encryption: HCNN, the first homomorphic CNN on encrypted data with GPUs[J]. IEEE Transactions on Emerging Topics in Computing, 2021, 9(3): 1330-1343.
[25] HESAMIFARD E, TAKABI H, GHASEMI M, et al. Privacy-preserving machine learning in cloud[C]//Proceedings of the 2017 on Cloud Computing Security Workshop. New York: ACM, 2017: 39-43.
[26] LIU X M, QIN B D, DENG R H, et al. An efficient privacy-preserving outsourced computation over public data[J]. IEEE Transactions on Services Computing, 2017, 10(5): 756-770.
[27] XIONG J B, BI R W, ZHAO M F, et al. Edge-assisted privacy-preserving raw data sharing framework for connected autonomous vehicles[J]. IEEE Wireless Communications, 2020, 27(3): 24-30.
[28] BI R W, GUO D L, ZHANG Y Y, et al. Outsourced and privacy-preserving collaborative k-prototype clustering for mixed data via additive secret sharing[J]. IEEE Internet of Things Journal, 2023, 10(18): 15810-15821.
[29] 毕仁万, 陈前昕, 熊金波, 等. 面向深度神经网络的安全计算协议设计方法[J]. 网络与信息安全学报, 2020, 6(4): 130-139.
BI R W, CHEN Q X, XIONG J B, et al. Design method of secure computing protocol for deep neural network[J]. Chinese Journal of Network and Information Security, 2020, 6(4): 130-139.
[30] HAN K, HONG S, CHEON J H, et al. Logistic regression on homomorphic encrypted data at scale[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI, 2019: 9466-9471.
[31] VASUDEVAN A, ANDERSON A, GREGG D. Parallel multi channel convolution using general matrix multiplication[C]//Proceedings of the 2017 IEEE 28th International Conference on Application-Specific Systems, Architectures and Processors. Piscataway: IEEE, 2017: 19-24.
[32] TANG R, LIN J. Honk: a PyTorch reimplementation of convolutional neural networks for keyword spotting[EB/OL]. [2023-10-08]. https://arxiv.org/abs/1710.06554.