关键词: 表示学习, 图嵌入, 异构信息网络, 跨模态融合, 对抗攻击, 节点分类

Abstract: Current cooking recipe representation learning methods primarily depend on aligning recipe texts with corresponding images or using adjacency matrix to capture relationships between cooking recipes and their ingredients for embedding learning. However, these methods are relatively rough in information fusion processing, fail to deeply mine the interaction information between different modalities, and face challenges in effectively and dynamically evaluating the strength of correlations between cooking recipe components, restricting the model??s representational capacity. To address these problems, this paper proposes a heterogeneous information network-based multimodal cooking recipe representation learning model (CookRec2vec) that integrates visual, textual, and relational information into cooking recipe embedding and fully mines and quantifies the correlation between the major components of the cooking recipes through adaptive adjacency relationships. At the same time, an explicit modeling approach based on high-order co-occurrence matrices provides complementary information while preserving original characteristics, which significantly improves the expression ability of cooking recipe features. Experimental results show that the proposed model outperforms the existing mainstream methods in cooking recipe classification performance and has made significant progress in the field of innovative dish embedding prediction.

Key words: representation learning, graph embedding, heterogeneous information network, cross-modal fusion, adversarial attack, node classification