关键词: 机器学习, 集成多样性, 信息论, 联结树

Abstract: Ensemble diversity, that is, the difference among the individual learners, is a fundamental issue in ensemble learning. Multi-information diversity measures ensemble diversity based on information theory and provides a promising direction for understanding ensemble diversity. In practice, the high-order information is usually hard to be estimated. This paper proposes to estimate high-order information based on a particular kind of k-order t-cherry junction tree so as to get approximate multi-information diversity. The method includes approximating multi-information directly as well as approximating its mathematical components by the junction tree, and the relevance between the two cases is discussed. Experiments show that this estimation method is superior to established ones under the same low-order approximation.

Key words: machine learning, ensemble diversity, information theory, junction tree