Journal Article

Video supervised for 3D reconstruction from single image

Wang, Yi, Luo, Shoutong, Sun, Zhengxing, Zhong, Yijie, Sun, Yunhan

2022

Overview

As a long-standing ill-posed problem, 3D reconstruction from a single image is an important research topic in computer vision. The information in a single image can represent an infinite number of possible three-dimensional shapes. To recover reasonable object geometry from a single image requires a correct shape prior. Thus, using what kind of supervision and how to make better use of training data are key issues. In this paper, we propose a framework for 3D reconstruction from single image with video supervision. On the one hand, we build a temporal network to generate fine 3D structure from video input benefiting from its temporal correlation. On the other hand, we introduce the knowledge distillation to transfer the shape prior extracted from the video. Also the mechanism ensures that the student network which for single image reconstruction can make full use of the knowledge learned from the teacher network which receives video input. In the inference phase, we can use the student network independently. Extensive experiments on ShapeNet show the superiority of our method.

• Publisher: Springer US,Springer Nature B.V
• Subject: Ablation / Computer Communication Networks / Computer Science / Computer vision / Data Structures and Information Theory / Deep learning / Distillation / Ill posed problems / Image reconstruction / Knowledge / Knowledge management / Multimedia / Multimedia Information Systems / Neural networks / Special Purpose and Application-Based Systems / Supervision