Abstract: To alleviate the noises of low quality modalities and improve the efficiency in multimodal object tracking, this paper proposes an adaptive modal selection based method to effectively select most reliable modality to track object. Specifically, for each modal, the clustering algorithm is utilized to divide the object region and its surrounding background region into several sub-clusters, and then the discriminative ability between object and background, which measures the modal quality, is computed by the feature difference between their respective sub-clusters. The most reliable modality is thus selected based on the defined discriminative ability to track object by using the correlation filter algorithm. For maintaining the effective object model, this paper employs double-threshold strategy to update all modal models. Experiments on the collected thermal-visible video pairs demonstrate the effectiveness of the proposed method. In addition, the runtime of the proposed tracker achieves 141 f/s.

Key words: modal selection, adaptive tracking, real-time processing, thermal information

摘要： 在多模态跟踪过程中，为了避免低质量模态的噪声影响和提高跟踪方法的效率，提出了一种基于自适应模态选择的目标跟踪方法，能够选择较好的模态进行跟踪。具体地，对于每个模态，使用聚类方法将目标区域及其周围背景区域各聚为若干个子类，然后通过它们子类之间的特征差异衡量目标和周围背景的判别性（即模态质量），选择判别性最大的模态对目标使用相关性滤波算法进行跟踪。同时，为了维持各个模态的目标模型的有效性，提出了一种双阈值策略更新选择和未被选择模态的跟踪模型。在7组热红外和可见光视频对上进行了实验，验证了该方法的有效性，且跟踪速度达到141 f/s。

关键词: 模态选择, 自适应跟踪, 实时处理, 热红外信息