Robust Probabilistic Matrix Tri-factorization

doi:10.3778/j.issn.1673-9418.1905065

Abstract

Abstract:

Matrix factorization is a commonly-used data analysis tool in computer vision, machine learning and data mining. In recent years, the probabilistic models of matrix factorization have become the focus of attention. Existing probabilistic matrix factorization models generally decompose a given data matrix into the product of two low rank matrices, which probably limits the flexibility and practicability of these models. To address this issue, this paper proposes a model of robust probabilistic matrix tri-factorization (RPMTF). This model factorizes the data matrix into the product of three matrices and the robustness is taken into account simultaneously. A hierarchical representation of the Laplace distribution is firstly adopted for solving the proposed model. Then an expectation maximization algorithm is designed based on a strategy of maximum a posterior estimation. In the experiment, RPMTF is applied to image denoising and video background modeling, and the results verify the feasibility and effectiveness of the proposed method.

Key words: matrix tri-factorization, probabilistic matrix factorization, expectation maximization, maximum a posterior estimation

摘要：

矩阵分解是计算机视觉、机器学习和数据挖掘中经常使用的数据分析工具。近年来，矩阵分解的概率模型已成为人们关注的焦点。现有的概率矩阵分解一般将数据矩阵分解为两个低秩矩阵之积，这可能会限制该模型的灵活性和实用性。为此，提出了鲁棒概率矩阵三分解模型（RPMTF）。该模型将数据矩阵分解为三个矩阵的乘积，并考虑了其鲁棒性。在模型求解时，先将拉普拉斯分布进行分层表示；再采用基于极大后验估计的策略，设计了一种条件期望最大化算法。在实验中，将鲁棒概率矩阵三分解应用到图像去噪和视频背景建模中，结果证实了所提方法的可行性与有效性。

关键词: 矩阵三分解, 概率矩阵分解, 期望最大化, 极大后验估计

SHI Jiarong, CHEN Jiaojiao. Robust Probabilistic Matrix Tri-factorization[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(7): 1251-1260.

史加荣，陈姣姣. 鲁棒概率矩阵三分解[J]. 计算机科学与探索, 2020, 14(7): 1251-1260.

References

[1] Jolliffe I T, Cadima J. Principal component analysis: a review and recent developments[J]. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2016, 374(2065): 20150202.
[2] Derksen H. On the nuclear norm and the singular value decomposition of tensors[J]. Foundations of Computation Mathematics, 2016, 16(3): 779-811.
[3] Hernando A, Bobadilla J, Ortega F. A non-negative matrix factorization for collaborative filtering recommender systems based on a Bayesian probabilistic model[J]. Knowledge- Based Systems, 2016, 97: 188-202.
[4] Kwak N. Principal component analysis based on L1-norm maximization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(9): 1672-1680.
[5] Lin Z, Chen M, Ma Y. The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices[J]. arXiv:1009.5055, 2010.
[6] Chandrasekaran V, Sanghavi S, Parrilo P A, et al. Sparse and low-rank matrix decompositions[J]. IFAC Proceedings Volumes, 2009, 42(10): 1493-1498.
[7] Mu Y D, Dong J, Yuan X T, et al. Accelerated low-rank visual recovery by random projection[C]//Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, Jun 20-25, 2011. Washington: IEEE Computer Society, 2011: 2609-2616.
[8] Zhou T Y, Tao D C. GoDec: randomized lowrank & sparse matrix decomposition in noisy case[C]//Proceedings of the 28th International Conference on Machine Learning, Bellevue, Jun 28-Jul 2, 2011. Madison: Omni Press, 2011: 33-40.
[9] Candès E J, Li X D, Ma Y, et al. Robust principal component analysis?[J]. Journal of the ACM, 2011, 58(3): 11.
[10] Shi J R, Zheng X Y, Yang W. Survey on probabilistic models of low-rank matrix factorizations[J]. Entropy, 2017, 19(8): 424.
[11] McLachlan G, Krishnan T. The EM algorithm and extensions[M]. Hoboken: John Wiley & Sons, 2007.
[12] Bishop C M. Pattern recognition and machine learning[M]. Berlin, Heidelberg: Springer, 2006.
[13] Tzikas D G, Likas A C, Galatsanos N P. The variational approximation for Bayesian inference[J]. IEEE Signal Processing Magazine, 2008, 25(6): 131-146.
[14] Mantz A B. A Gibbs sampler for multivariate linear regression[J]. Monthly Notices of the Royal Astronomical Society, 2016, 457(2): 1279-1288.
[15] Martino L, Read J, Luengo D. Independent doubly adaptive rejection Metropolis sampling within Gibbs sampling[J]. IEEE Transactions on Signal Processing, 2015, 63(12): 3123- 3138.
[16] Bernardo J M, Smith A F M. Bayesian theory[M]. Hoboken: John Wiley & Sons, 2009.
[17] Beal M J. Variational algorithms for approximate Bayesian inference[M]. London: University of London, 2003.
[18] Blei D M, Jordan M I. Variational inference for Dirichlet process mixtures[J]. Bayesian Analysis, 2006, 1(1): 121-143.
[19] Bernardo J M, Bayarri M J, Berger J O, et al. The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures[J]. Bayesian Statistics, 2003, 7: 453-464.
[20] Chen Z, Babacan S D, Molina R, et al. Variational Bayesian methods for multimedia problems[J]. IEEE Transactions on Multimedia, 2014, 16(4): 1000-1017.
[21] Gao J B. Robust L1 principal component analysis and its Bayesian variational inference[J]. Neural Computation, 2008, 20(2): 555-572.
[22] Luttinen J, Ilin A, Karhunen J. Bayesian robust PCA of incomplete data[J]. Neural Processing Letters, 2012, 36(2): 189-202.
[23] Mnih A, Salakhutdinov R. Probabilistic matrix factorization[C]//Proceedings of the 21st Annual Conference on Neural Information Processing Systems, Vancouver, Dec 3-5, 2007. Red Hook: Curran Associates, 2008: 1257-1264.
[24]Salakhutdinov R, Mnih A. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo[C]//Proceedings of the 25th International Conference on Machine Learning, Helsinki, Jul 5-9, 2008. New York: ACM, 2008: 880-887.
[25] Lakshminarayanan B, Bouchard G, Archambeau C. Robust Bayesian matrix factorisation[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, Apr 11-13, 2011: 425-433.
[26] Wang N Y, Yeung D Y. Bayesian robust matrix factorization for image and video processing[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Washington: IEEE Computer Society, 2013: 1785-1792.
[27] Yang Z J, Zhang F L, Zhang H, et al. Three-decomposition model and algorithm and its application in image restoration[J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(12): 1940-1949. 杨章静, 张凡龙, 张辉, 等. 三分解模型与算法及其在图像恢复中的应用[J]. 计算机科学与探索, 2018, 12(12): 1940-1949.
[28] Eltoft T, Kim T, Lee T W. On the multivariate Laplace distribution[J]. IEEE Signal Processing Letters, 2006, 13(5): 300-303.
[29] Salojärvi J, Puolamäki K, Kaski S. Expectation maximization algorithms for conditional likelihoods[C]//Proceedings of the 22nd International Conference on Machine Lear-ning, Bonn, Aug 7-11, 2005. New York: ACM, 2005: 752-759.
[30] Ding X H, He L H, Carin L. Bayesian robust principal component analysis[J]. IEEE Transactions on Image Processing, 2011, 20(12): 3419-3430.
[31] Shi J R, Zhang A Y. Review on probabilistic tensor decompositions[J]. Journal of Shaanxi University of Technology (Natural Science Edition), 2018, 34(4): 70-79. 史加荣, 张安银. 概率张量分解综述[J]. 陕西理工大学学报(自然科学版), 2018, 34(4): 70-79.
[32] Hayashi K, Takenouchi T, Shibata T, et al. Exponential family tensor factorization for missing-values prediction and anomaly detection[C]//Proceedings of the 10th International Conference on Data Mining, Sydney, Dec 14-17, 2010. Washington: IEEE Computer Society, 2010: 216-225.
[33] Wang N Y, Yao T S, Wang J D, et al. A probabilistic approach to robust matrix factorization[C]//LNCS 7578: Proceedings of the 12th European Conference on Computer Vision, Florence, Oct 7-13, 2012. Berlin, Heidelberg: Springer, 2012: 126-139.
[34] Yong H W, Meng D Y, Zuo W M, et al. Robust online matrix factorization for dynamic background subtraction[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(7): 1726-1740.