MSV-Net: Visual Super-Resolution Reconstruction for Scientific Simulated Data of Mixed Surface-Volume

doi:10.3778/j.issn.1673-9418.2112069

Abstract

Abstract: High fidelity visual analysis usually relies on high-resolution grid data of coupled geometric models generated by large-scale scientific simulation, which brings great challenges to data storage and smooth interaction. Therefore, this paper proposes a super-resolution reconstruction method for large-scale scientific simulated data of mixed surface-volume, which is MSV-Net. The network is an end-to-end deep neural network, which realizes the joint learning of hybrid rendering mapping from low resolution data to high resolution data through multi-layer nonlinear transformation. The network without the fully connected layer, can not only reduce the network parameters, but also improve the flexibility and reusability of the network. In addition, MSV-Dataset, a surface-volume mixed dataset for large-scale electromagnetic simulation application is constructed for model training and verification. This dataset consists of mixed rendered images using nontransparent geometric model rendering coupled with semitransparent volume rendering. The proposed method is compared with a variety of traditional and deep learning methods. The quantitative analysis results show that the MOS absolute evaluation index of this method reaches 4.1, and the reconstruction accuracy is second only to the real image; it takes 66.28 s to directly draw mixed data with 1500×1500 image resolution, but only 4.14 s with the proposed method. The interaction performance is improved about 15 times.

Key words: scientific data visualization, hybrid rendering, large-scale simulation, super-resolution reconstruction; , deep learning

摘要： 高保真度的可视分析通常依赖大规模科学模拟产生的耦合几何模型的高分辨率网格数据，这对数据存储和流畅交互均提出了巨大挑战。提出了MSV-Net，一个面向大规模科学模拟面体混合数据的超分辨率重建方法。该网络为端到端的深度神经网络，通过多层非线性变换实现从低分辨数据到高分辨数据的混合绘制映射的联合学习；该网络舍弃了全连接层，不仅可以减少网络参数，而且能够提升网络的灵活性与可复用性。此外，构建了面向大规模电磁模拟应用的面体混合数据集MSV-Dataset，用于模型训练和验证。该数据集由采用不透明几何模型绘制耦合半透明体绘制的混合绘制的图像构成。与多种传统方法和深度学习方法进行了对比，定量分析结果显示，MOS绝对评价指标达到了4.1，重建准确率仅次于真实图像；基于混合数据绘制1 500×1 500分辨率的图像，采用直接绘制需要66.28 s，而采用MSV-Net则仅需要4.14 s，交互性能提升了约15倍。

关键词: 科学数据可视化, 混合绘制, 大规模模拟, 超分辨率重建, 深度学习

CAO Siming, WANG Xiaohua, WANG Hongkun, CAO Yi. MSV-Net: Visual Super-Resolution Reconstruction for Scientific Simulated Data of Mixed Surface-Volume[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1321-1328.

曹斯铭, 王晓华, 王弘堃, 曹轶. MSV-Net：面向科学模拟面体混合数据的超分重建方法[J]. 计算机科学与探索, 2023, 17(6): 1321-1328.

References

[1] NOWELL L T. Science at extreme scale: challenges in data management, analysis, and visualization[R]. 2013-05-06.
[2] MCCORMICK B H, DEFANTI T A, BROWN M D. Visu-alization in scientific computing[J]. ACM SIGBIO Newsletter, 1988, 10(1): 15-21.
[3] AHRENS J, JOURDAIN S, O??LEARY P, et al. An image-based approach to extreme scale in Situ visualization and analysis[C]//Proceedings of the 2014 International Confe-rence for High Performance Computing, Networking, Storage and Analysis, New Orleans, Nov 16-21, 2014. Piscataway: IEEE, 2014: 424-434.
[4] ZHOU Z L, HOU Y L, WANG Q R, et al. Volume up-scaling with convolutional neural networks[C]//Proceeding of the 2017 Computer Graphics International Conference, Yokohama, Jun 27-30, 2017. New York: ACM, 2017: 38.
[5] BLU T, THéVENAZ P, UNSER M. Linear interpolation revitalized[J]. IEEE Transactions on Image Processing, 2004, 13(5): 710-719.
[6] MEIJERING E, UNSER M. A note on cubic convolution interpolation[J]. IEEE Transactions on Image Processing, 2003, 12(4): 477-479.
[7] DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2): 295-307.
[8] DONG C, LOY C C, TANG X O. Accelerating the super-resolution convolutional neural network[C]//LNCS 9906:Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 391-407.
[9] SHI W Z, CABALLERO J, HUSZáR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 1874-1883.
[10] BERGER M, LI J X, LEVINE J A. A generative model for volume rendering[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 25(4): 1636-1650.
[11] WEISS S, CHU M Y, THUEREY N, et al. Volumetric isosurface rendering with deep learning-based super-resolution [J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 27(6): 3064-3078.
[12] HAN J, WANG C L. TSR-TVD: temporal super-resolution for time-varying data analysis and visualization[J]. IEEE Tran-sactions on Visualization and Computer Graphics, 2020, 26(1): 205-215.
[13] HAN J, WANG C. SSR-TVD: spatial super-resolution for time-varying data analysis and visualization[J]. IEEE Trans-actions on Visualization and Computer Graphics, 2022, 28(6): 2445-2456 .
[14] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[15] MO Z Y, ZHANG A Q, CAO X L, et al. JASMIN: a parallel software infrastructure for scientific computing[J]. Frontiers of Computer Science in China, 2010, 4(4): 480-488.
[16] 肖丽, 曹小林, 王华维, 等. 激光聚变数值模拟中的大规模数据可视分析[J]. 计算机辅助设计与图形学学报, 2014, 26(5): 675-686.
XIAO L, CAO X L, WANG H W, et al. Large-scale data visual analysis for numerical simulation of laser fusion[J]. Journal of Computer-Aided Design & Computer Graphics, 2014,26(5): 675-686.
[17] 王弘堃, 曹轶, 肖丽. 基于图像的大规模数据集交互可视化[J]. 计算机研究与发展, 2017, 54(4): 855-860.
WANG H K, CAO Y, XIAO L. Image-based interactive visualization of large-scale data sets[J]. Journal of Computer Research and Development, 2017, 54(4): 855-860.
[18] 王晓华, 曹轶, 王华维, 等. 面向效应场数值模拟的颜色映射自动调优框架[J]. 计算机辅助设计与图形学学报, 2022, 34(4): 623-635.
WANG X H, CAO Y, WANG H W, et al. A color mapping automatic tuning framework for complex simulation datasets[J]. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(4): 623-635.
[19] PASZKE A, GROSS S, MASSA F, et al. Pytorch: an impe-rative style, high-performance deep learning library[C]//Pro-ceedings of the Annual Conference on Neural Information Processing Systems 2019, Vancouver, Dec 8-14, 2019: 8024-8035.