Abstract: PAC-Bayes theory integrating theories of Bayesian paradigm and structure risk minimization for stochastic classifiers has been considered as a framework for deriving some of the tightest generalization bounds. This paper analyzes the research background and profound significance of PAC-Bayes theory, and introduces the framework of PAC-Bayes theory and its application to support vector machine (SVM). Then, this paper discusses PAC-Bayes bound of many machine learning algorithms, and specially analyzes the bound with the non-IID data. Furthermore, this paper elaborates research status and development of the PAC-Bayes bound application from four directions, and compares different research methods and features. Finally, this paper draws the research prospect of the PAC-Bayes bound.

Key words: PAC-Bayes bound, support vector machine (SVM), generalization capability, classifier

摘要： PAC-Bayes理论融合了贝叶斯定理和随机分类器的结构风险最小化原理，它作为一个理论框架，可得到最紧的泛化风险边界。分析了PAC-Bayes理论的研究背景和重要意义，介绍了PAC-Bayes理论框架及其在支持向量机上的应用，分别探讨了多种机器学习算法的PAC-Bayes边界，并特别对非独立同分布数据的PAC-Bayes边界进行了分析。从4个方面深入阐述了PAC-Bayes边界应用的研究现状及进展，并对不同的研究方法和特点进行了比较。最后展望了PAC-Bayes边界未来的研究发展方向。

关键词: PAC-Bayes边界, 支持向量机, 泛化能力, 分类器