Abstract:

Factorization machine (FM) has been widely applied in recommender systems since its publication, and to capture feature interactions, FM represents its 2nd-order coefficient of any two features as inner product of the corresponding embedding vectors in Euclidean space. Considering that objects in recommender systems such as items, users, properties and contexts can be described as a heterogeneous network exhibiting hierarchical structures, whereas flat Euclidean space is not able to capture this kind of structure, restricting the feature representation ability of FM, this paper proposes hyperbolic FM (HFM). It represents each feature as a vector in hyperbolic space rather than in Euclidean space, and evaluates the 2nd-order feature interaction strength with hyperbolic distance measure. The reason for adopting hyperbolic geometry is that it has been shown to be the underlying embedding space of hierarchical structures, like trees, graphs and vocabulary. This paper designs two HFMs based on Poincaré ball model and hyperboloid model, respectively, and derives the corresponding Riemannian gradient descent algorithm for optimization. Experiments conducted on various datasets indicate that HFM achieves better performance than original FM with identical number of trainable parameters, and reveals the hierarchical structure of features which is missing in FM, offering explanability to some extent.

Key words: factorization machine, hyperbolic space, recommender system, representation learning, manifold learning

摘要：

因子分解机（FM）自提出以来已被广泛用于推荐系统，为了捕捉特征间的二阶交互，FM将任意两个特征的二阶系数表示成欧氏空间中对应嵌入向量的内积。考虑到推荐场景中的对象如商品、用户、属性、上下文信息等，可用具有层次结构的异构网络进行表达，而平坦的欧氏空间无法刻画这种层次结构，限制了FM的特征表示能力，为此提出了双曲因子分解机（HFM）。它将每维特征表示为双曲空间而非欧氏空间中的向量，并利用双曲距离度量评估特征间的二阶交互强度。选择双曲空间是因为其被证明更适合树、图和词汇等具有层次结构的对象嵌入。分别设计了基于庞加莱球和基于双曲面两种双曲空间模型的HFM，并导出了对应的黎曼梯度下降优化算法。在多个数据集上的实验结果表明，HFM在等量参数的情形下，获得了比FM更优的性能，同时揭示出了在FM中欠缺的特征间的层次关系，使之具有部分可解释性。

关键词: 因子分解机, 双曲空间, 推荐系统, 表示学习, 流形学习