Abstract: Machine learning has already become one of the most widely used techniques in the field of computer science, and it has been widely applied in image processing, natural language processing, network security and other fields. However, there has been many security threats that need to be overcome on current machine learning algorithms and training data set, which will affect the security of several practical applications, such as facial detection, malware detection and automatic driving, etc. According to the known security threats, which aim to a variety of machine learning algorithms, such as the support vector machine (SVM) classifier, clustering and deep neural networks, this paper introduces the issues that happen in the training, testing/inference phase of machine learning, which include privacy leaking and attacks of poisoning, evasion, impersonate and inversion based on the adversarial samples. Then, this paper sums up the machine learning adversary model as well as its safety assessment mechanism and concludes a certain number of countermeasures and privacy protection techniques on training and testing processes. Finally, this paper looks forward some correlative problems worthy of further discussion.

Key words: machine learning, adversarial sample, security threats, countermeasuring techniques

摘要： 机器学习已经成为当前计算机领域研究和应用最广泛的技术之一，在图像处理、自然语言处理、网络安全等领域被广泛应用。然而，一些机器学习算法和训练数据本身还面临着诸多安全威胁，进而影响到基于机器学习的面部检测、恶意程序检测、自动驾驶汽车等实际应用系统的安全性。由目前已知的针对支持向量机（support vector machine，SVM）分类器、聚类、深度神经网络（deep neural networks，DNN）等多种机器学习算法的安全威胁为出发点，介绍了在机器学习的训练阶段和测试/推理阶段中出现的基于对抗样本的投毒、逃逸、模仿、逆向等攻击和隐私泄露等问题，归纳了针对机器学习的敌手模型及其安全评估机制，总结了训练过程和测试过程中的若干防御技术和隐私保护技术，最后展望了下一步机器学习安全研究的发展趋势。

关键词: 机器学习, 对抗样本, 安全威胁, 防御技术