Journal of Frontiers of Computer Science and Technology ›› 2015, Vol. 9 ›› Issue (1): 1-13.DOI: 10.3778/j.issn.1673-9418.1410046

Previous Articles     Next Articles

Survey on PAC-Bayes Theory and Application Research

TANG Li1,2, GONG Xiujun2,3+, HE Li1   

  1. 1. Department of Information Science and Technology, School of Science and Technology, Tianjin University of Finance and Economics, Tianjin 300222, China
    2. School of Computer Science and Technology, Tianjin University, Tianjin 300072, China
    3. Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin 300072, China
  • Online:2015-01-01 Published:2014-12-31


汤  莉1,2,宫秀军2,3+,何  丽1   

  1. 1. 天津财经大学 理工学院 信息科学与技术系,天津 300222
    2. 天津大学 计算机科学与技术学院,天津 300072
    3. 天津市认知计算与应用重点实验室,天津 300072

Abstract: PAC-Bayes theory integrating theories of Bayesian paradigm and structure risk minimization for stochastic classifiers has been considered as a framework for deriving some of the tightest generalization bounds. This paper analyzes the research background and profound significance of PAC-Bayes theory, and introduces the framework of PAC-Bayes theory and its application to support vector machine (SVM). Then, this paper discusses PAC-Bayes bound of many machine learning algorithms, and specially analyzes the bound with the non-IID data. Furthermore, this paper elaborates research status and development of the PAC-Bayes bound application from four directions, and compares different research methods and features. Finally, this paper draws the research prospect of the PAC-Bayes bound.

Key words: PAC-Bayes bound, support vector machine (SVM), generalization capability, classifier

摘要: PAC-Bayes理论融合了贝叶斯定理和随机分类器的结构风险最小化原理,它作为一个理论框架,可得到最紧的泛化风险边界。分析了PAC-Bayes理论的研究背景和重要意义,介绍了PAC-Bayes理论框架及其在支持向量机上的应用,分别探讨了多种机器学习算法的PAC-Bayes边界,并特别对非独立同分布数据的PAC-Bayes边界进行了分析。从4个方面深入阐述了PAC-Bayes边界应用的研究现状及进展,并对不同的研究方法和特点进行了比较。最后展望了PAC-Bayes边界未来的研究发展方向。

关键词: PAC-Bayes边界, 支持向量机, 泛化能力, 分类器