计算机科学与探索 ›› 2023, Vol. 17 ›› Issue (2): 409-418.DOI: 10.3778/j.issn.1673-9418.2105113

• 图形·图像 • 上一篇    下一篇

双流网络的水下视频客观质量评价模型

宋巍,肖毅,杜艳玲,张明华   

  1. 上海海洋大学 信息学院,上海 201306
  • 出版日期:2023-02-01 发布日期:2023-02-01

Objective Underwater Video Quality Assessment Model via Two-Stream Networks

SONG Wei, XIAO Yi, DU Yanling, ZHANG Minghua   

  1. College of Information, Shanghai Ocean University, Shanghai 201306, China
  • Online:2023-02-01 Published:2023-02-01

摘要: 水下拍摄的视频存在质量退化效应。一方面,光线在水中传播时呈指数衰减导致水下视频质量损失;另一方面,水下复杂拍摄环境(例如水流等)造成视频的不稳定性。为此,综合考虑水下视频时空特征和运动特征对视频质量的影响,提出一种针对水下场景的客观无参考视频质量评价模型(TS-UVQA)。TS-UVQA采用双流网络结构:设计了由三维卷积、自适应正则化和慢融合策略组成的时空特征提取网络(Spatial-temporal Net),从视频原始帧中学习时空特征;设计了由二维卷积层和自适应正则化堆叠的运动特征提取网络(Motion Net),从光流场块中学习水下视频的相关运动特征;使用决策级融合实现高精度的水下视频质量评价。以模型的视频质量评价结果与主观质量分数的皮尔森线性相关系数(PLCC)和斯皮尔曼秩序相关系数(SROCC)为指标,通过实验验证了TS-UVQA中运动网络对于水下视频质量评价的性能提升效果,同时与13种图像和视频客观质量评价方法相比,在水下视频数据集上取得最佳性能。此外,TS-UVQA在3个自然场景视频数据集(ECVQ、EVVQ、LIVE)也取得了与最先进方法接近的相关系数,表明方法具有良好的泛化性能。

关键词: 质量评价, 无参考视频质量评价, 双流法, 水下视频

Abstract: Underwater videos often suffer from quality degradation. On the one hand, the exponential attenuation of the natural light in water media leads to the loss of underwater video quality. On the other hand, the video is unstable due to the underwater shooting environment such as flow and pressure. Considering the influence of underwater video spatial-temporal features and motion features on video quality, this paper proposes an objective no-reference video quality assessment (VQA) method for underwater scenes, named TS-UVQA (two-stream underwater video quality assessment). TS-UVQA adopts two-stream network structure: Spatial-temporal Net including three-dimensional convolution,  switchable normalization and slow fusion strategy is designed to extract spatial-temporal features from the original video frames; Motion Net including two-dimensional convolution and switchable norm-alization is designed to extract motion features from the optical flow fields. Then, decision-level linear fusion is used to aggregate video features and realize high-precision underwater VQA. This paper takes the Pearson linear correlation coefficient (PLCC) and Spearman order correlation coefficient (SROCC) of the video quality assessment result and subjective quality score as indicators. Experimental results show that the motion characteristics extracted from the optical flow can enhance the performance of underwater VQA. Compared with 13 image/video quality assessment methods in an underwater video dataset, TS-UVQA  achieves the best performance in terms of PLCC and SROCC. Further experiments on the natural scene video datasets, ECVQ, EVVQ and LIVE achieve a close perfor-mance to the state-of-the-art VQA method, and show that TS-UVQA has good generalization to nature scene VQA.

Key words: quality assessment, no-reference video quality assessment, two-stream network, underwater video