Abstract: Low-level semantic features (such as color and texture, etc.) are widely used to compare the similarity of images for the existing image feature representation. However doctors usually judge the similarity of medical images based on the differences between high-level semantic features (such as lesions and lesion types, etc.) in the local region. This paper proposes a novel multimodal features medical image clustering method which combines the medical image texture features and morphological features. This method can solve the problem that traditional methods ignore the high-level semantic features of medical images, which leads to the poor clustering effect of medical images. Firstly, a texture feature fusion method is proposed to represent the global underlying semantic features of medical images, on the other hand, the morphological description of the region of interest (ROI) is proposed as the local high-level semantics of medical images feature. Secondly, the similarity measure is used to calculate the similarity between the two types of features for CT images. Finally, the learning of the feature fusion weight is studied by using the multi-kernel learning method, and the validity of the method is verified by using the multi-kernel spectral clustering experiment.

Key words: multimodal, feature extraction, image clustering, medical images

摘要： 现有的图像特征表达大多使用低层语义特征（如颜色、纹理等）细粒度地比较图像的相似度，然而医生就诊更多依据图像在局部区域高层语义特征（如是否病变、病变类型等）的差异粗粒度地判断图像的相似程度。针对现有的医学图像特征表达忽略了医学图像特有的高层语义特征，致使医学图像聚类效果不佳的问题，提出了一种融合医学图像纹理特征和特有形态学特征的多模态特征医学图像聚类方法。首先一方面提出使用纹理特征融合方法表示医学图像全局底层语义特征；另一方面提出使用图像分割的感兴趣区域（region of interest，ROI）的形态学描述作为形态学特征表示医学图像的局部高层语义信息。其次结合提出的相似性度量方法分别计算脑CT图像两类特征间的相似度。最后利用多核学习方法学习特征融合权重，并在多核谱聚类实验上验证了该方法的有效性。

关键词: 多模态, 特征抽取, 图像聚类, 医学图像