Multi-grained Fusion Image Feature Learning with Fuzzy Rule System

doi:10.3778/j.issn.1673-9418.1911057

Abstract

Abstract:

Currently, the most popular image feature learning method is deep neural network, which can automatically extract efficient features through feature learning for tasks such as classification and recognition without human involvement. However, the deep neural network image feature extraction method currently faces many challenges. Its effectiveness relies heavily on large data, and it is usually regarded as a black box model with poor interpretability. Aiming at the above challenges, a more interpretable and scalable feature learning method, multi-grained fusion image feature learning with fuzzy rule system, is proposed in the paper. The method is based on Takagi-Sugeno-Kang fuzzy system (TSK-FS) with fuzzy rule inference. This method extracts image features through rule-based TSK-FS, so the feature learning process can be explained by rules. Then, multi-granularity scanning also further enhances its feature learning ability. Extensive experiments have been conducted on image datasets of different scales, and the results show the proposed method is effective on image datasets.

Key words: feature learning, fuzzy system, non-linear model, image classification, interpretability

摘要：

当前最流行的图像特征学习方法是深度神经网络，该类方法无需人工参与即可自动地通过特征学习提取高效的特征，用于分类识别等任务。然而，深度神经网络图像特征抽取方法目前也面临着诸多挑战，其有效性严重依赖大规模的数据，且通常被视为黑盒模型，解释性较差。针对上述挑战，以基于模糊规则推理的TSK模糊系统（TSK-FS）为基础，提出了一种适用于不同规模数据集且易于理解的特征学习方法——多粒度融合的模糊规则系统图像特征学习算法。该方法通过基于规则的TSK-FS抽取图像特征，因而特征学习过程是可以利用规则进行解释的。其次，多粒度扫描也使得其特征学习能力进一步提升。在不同规模的图像数据集上进行了充分的实验，实验结果表明该方法在图像数据集上具有较好的有效性。

关键词: 特征学习, 模糊系统, 非线性模型, 图像分类, 可解释性

MA Xiang, DENG Zhaohong, WANG Shitong. Multi-grained Fusion Image Feature Learning with Fuzzy Rule System[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(1): 173-184.

马翔, 邓赵红, 王士同. 多粒度融合的模糊规则系统图像特征学习[J]. 计算机科学与探索, 2021, 15(1): 173-184.

References

[1] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, Jul 2005. Washington: IEEE Computer Society, 2005: 886-893.
[2] OJALA T, PIETIKAINEN M, MAENPAA T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971-987.
[3] LIU F, WANG T, ZHOU D W. Image retrieval based on color-space 2D histogram[J]. Computer Engineering and Applications, 2002, 38(12): 85-88.
刘芳, 王涛, 周登文. 基于颜色-空间二维直方图的图象检索[J]. 计算机工程与应用, 2002, 38(12): 85-88.
[4] LI T, MEI T, KWEON I S, et al. Contextual bag-of-words for visual categorization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2011, 21(4): 381-392.
[5] LIU L Q, WANG L, LIU X W. In defense of soft-assignment coding[C]//Proceedings of the 2011 IEEE International Conference on Computer Vision, Barcelona, Nov 6-13, 2011. Washington: IEEE Computer Society, 2011: 2486-2493.
[6] PHILBIN J, CHUM O, ISARD M, et al. Lost in quantization: improving particular object retrieval in large scale image databases[C]//Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Jun 24-26, 2008. Washington: IEEE Computer Society, 2008: 1-8.
[7] WANG J J, YANG J C, YU K, et al. Locality-constrained linear coding for image classification[C]//Proceedings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, Jun 13-18, 2010. Washington: IEEE Computer Society, 2010: 3360-3367.
[8] GRAUMAN K, DARRELL T. The pyramid match kernel: efficient learning with sets of features[J]. Journal of Machine Learning Research, 2007, 8(2): 725-760.
[9] Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1798- 1828.
[10] WOLD S. Principal component analysis[J]. Chemometrics & Intelligent Laboratory Systems, 1987, 2(1): 37-52.
[11] LU J, PLATANIOTIS K N, VENETSANOPOULOS A N. Face recognition using LDA-based algorithms[J]. IEEE Transactions on Neural Networks, 2003, 14(1): 195-200.
[12] HYVARINEN A, OJA E. Independent component analysis: algorithms and applications[J]. Neural Networks, 2000, 13(4): 411-430.
[13] LAST M. Kernel methods for pattern analysis[J]. Journal of the American Statistical Association, 2006, 101(476): 1730-1730.
[14] HU Y G, REN C X, YAO Y F, et al. Face recognition using nonlinear partial least squares in reproducing kernel Hilbert space[J]. Communications in Computer & Information Science, 2012, 321: 316-323.
[15] SCH?LKOPF B, SMOLA A J, MüLLER K R. Kernel principal component analysis[C]//LNCS 1327: Proceedings of the 7th International Conference on Artificial Neural Networks, Lausanne, Oct 8-10, 1997. Berlin, Heidelberg: Springer, 1997: 583-588.
[16] MIKA S, RATSCH G, WESTON J, et al. Fisher discriminant analysis with kernels[C]//Proceedings of the Neural Networks for Signal Processing IX, Proceedings of the 1999 IEEE Signal Processing Society Workshop, Madison, Aug 1999. Piscataway: IEEE, 1999: 41-48.
[17] BENGIO Y, DELALLEAU O. On the expressive power of deep architectures[C]//LNCS 6925: Proceedings of the 22nd International Conference on Algorithmic Learning Theory, Oct 5-7, 2011. Berlin, Heidelberg: Springer, 2011: 18-36.
[18] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7): 1527-1554.
[19] LU H T, ZHANG Q C. Applications of deep convolutional neural network in computer vision[J]. Journal of Data Acquisition and Processing, 2016, 31(1): 1-17.
卢宏涛, 张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述[J]. 数据采集与处理, 2016, 31(1): 1-17.
[20] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[21] DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, Jun 20-25, 2009. Piscataway: IEEE, 2009: 248-255.
[22] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015: 3431-3440.
[23] RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6088): 533-536.
[24] BOTTOU L. Large-scale machine learning with stochastic gradient descent[C]//Proceedings of the 19th International Conference on Computational Statistics, Paris, Aug 22-27, 2010. Heidelberg: Physica-Verlag, 2010: 177-186.
[25] ALCALá R, ALCALá-FDEZ J, CASILLAS J, et al. Local identification of prototypes for genetic learning of accurate TSK fuzzy rule-based systems[J]. International Journal of Intelligent Systems, 2007, 22(9): 909-941.
[26] ZHANG Y, ISHIBUCHI H, WANG S. Deep Takagi-Sugeno- Kang fuzzy classifier with shared linguistic fuzzy rules[J]. IEEE Transactions on Fuzzy Systems, 2018, 26(3): 1535-1549.
[27] CHANG P C, LIU C H. A TSK type fuzzy rule based system for stock price prediction[J]. Expert Systems with Applications, 2008, 34(1): 135-144.
[28] TAKAGI T, SUGENO M. Fuzzy identification of systems and its applications to modeling and control[J]. Readings in Fuzzy Sets for Intelligent Systems, 1993, 15(1): 387-403.
[29] CARTER H. Fuzzy sets and systems—theory and appli-cations[J]. Journal of the Operational Research Society, 1982, 33(2): 198-198.
[30] SUGENO M, KANG G T. Structure identification of fuzzy model[J]. Fuzzy Sets and Systems, 1988, 28(1): 15-33.
[31] GUPTA M M. Fuzzy set theory and its applications[J]. Fuzzy Sets & Systems, 1992, 47(3): 396-397.
[32] PAL N R, BEZDEK J C. On cluster validity for the fuzzy C-means model[J]. IEEE Transactions on Fuzzy Systems, 1995, 3(3): 370-379.
[33] SU T, DY J G. In search of deterministic methods for initiali-zing K-means and Gaussian mixture clustering[J]. Intelligent Data Analysis, 2006, 11(4): 319-338.
[34] JANG J-S R. ANFIS: adaptive-network-based fuzzy inference system[J]. IEEE Transactions on Systems, Man and Cybernetics, 1993, 23(3): 665-685.
[35] DENG Z, JIANG Y, CHOI K S, et al. Knowledge-leverage-based TSK fuzzy system modeling[J]. IEEE Transactions on Neural Networks and Learning Systems, 2013, 24(8): 1200-1212.
[36] GUAN Q, DENG Z H, WANG S T. Improved fuzzy C-means clustering algorithm[J]. Computer Engineering and Applications, 2011, 47(10): 27-29.
关庆, 邓赵红, 王士同. 改进的模糊C-均值聚类算法[J]. 计算机工程与应用, 2011, 47(10): 27-29.
[37] CHANG C C, LIN C J. LIBSVM: a library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 1-27.
[38] CAI D, HE X, HAN J, et al. Orthogonal Laplacianfaces for face recognition[J]. IEEE Transactions on Image Processing, 2006, 15(11): 3608-3614.
[39] CAI D, HE X F, HAN J W. Spectral regression for efficient regularized subspace learning[C]//Proceedings of the IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Oct 14-20, 2007. Washington: IEEE Computer Society, 2007: 1-8.
[40] CAI D, HE X F, HU Y X, et al. Learning a spatially smooth subspace for face recognition[C]//Proceedings of the Computer Vision and Pattern Recognition, Minneapolis, Jun 18-23, 2007. Washington: IEEE Computer Society, 2007: 1-7.
[41] HE X F, YAN S C, HU Y X, et al. Face recognition using Laplacianfaces[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(3): 328-340.
[42] SALAKHUTDINOV R, HINTON G. An efficient learning procedure for deep Boltzmann machines[J]. Neural Computation, 2012, 24(8): 1967-2006.
[43] VISIN F, KASTNER K, CHO K, et al. ReNet: a recurrent neural network based alternative to convolutional networks[J]. arXiv:1505.00393, 2015.