Journal Article

Utilizing the Neural Renderer for Accurate 3D Face Reconstruction from a Single Image

Wei, Wei, Duan, Xiaodong, Guo, Chen, Wang, Huichen, Zhang, Danni

2023

Overview

Recently, deep learning-based methods have shown significant results in 3D face reconstruction. By harnessing the power of convolutional neural networks, significant progress has been made in recovering 3D face shapes from single images using the 3D Morphable Model approach. However, training neural networks typically requires a large amount of data, while face images with ground-truth 3D face shapes are scarce. In this paper, we propose an unsupervised learning framework for accurate 3D face reconstruction from a single image. Our key idea is to process the images generated by a differentiable renderer and leverage the advantages of Generative Adversarial Networks to train a powerful neural renderer that produces highly realistic face images resembling the input image. We then modify traditional fitting methods to exploit the advantages of the neural renderer in finding optimal face parameters for improved 3D face reconstruction. The experimental results demonstrate that our method is capable of generating more accurate 3D face reconstruction results.

• Publisher: Springer US,Springer Nature B.V
• Subject: Artificial Intelligence / Artificial neural networks / Complex Systems / Computational Intelligence / Computer Science / Deep learning / Generative adversarial networks / Image reconstruction / Machine learning / Methods / Optimization / Parameter estimation / Renderers / Three dimensional models / Unsupervised learning