Journal Article

SPRITZ-PS: validation of synthetic face images using a large dataset of printed documents

Habibi, Yoosef, Nowroozi, Ehsan, Conti, Mauro

2024

Overview

Due to the appeal of Generative Adversarial Networks (GANs), synthetic face images generated with GAN models are difficult to differentiate from genuine human faces and may be used to create counterfeit identities. However, these face images contain artifacts presented in irises owing to the irregularity of highlights between the left and right irises. Adversaries can utilize PS documents of these images trying to conceal the artifacts of images, which makes it more challenging to distinguish between pristine and fake images. In order to tackle this, our research introduces a complete dataset and analytical tools that make a substantial contribution to multimedia forensics. This allows for the authentication of documents despite common alterations in the PS documents. Owing to the lack of large-scale reference IRIS datasets in the PS scenario, this study provided a pioneering dataset aiming to set a standard for multimedia forensic investigations. Given face images, we extracted iris images using the Dlib (King, J Mach Learn Res 10(60):1755–1758, 2020 ) and EyeCool (Wang et al. 2021 ) models, as described in Guo et al. ( 2022 ). However, in some cases, the potential eyelid occlusion phenomenon resulted in incomplete iris images. We utilized a hypergraph convolution-based image inpainting technique to complete the missing pixels in the extracted iris images, thus uncovering the intricate relationships within the iris data. To evaluate the IRIS image dataset and highlight associated issues, we conducted a series of analyses using Siamese Neural Networks, including ResNet50, Xception, VGG16, and MobileNet-v2, to measure the similarities between authentic and synthetic human iris images. Our SNN model, with four different backbones, effectively differentiated between genuine and synthetic iris images. For instance, using the Xception network, we achieved 56.76% similarity in IRISes for synthetic images and 92.77% similarity in IRISes for real images. The effectiveness of our approach was demonstrated by the similarity scores obtained from all SNN architectures, which showed a significant difference between the GAN-generated images from ProGAN or StyleGAN and the original PS iris photos. The similarity scores resulting from StyleGAN are higher than those of the ProGAN architecture, but at its highest, it is 76%, while for the pristine images, it ranges from 85% to 95%. This discrepancy can be utilized in order to distinguish between pristine and GAN-generated images.

• Publisher: Springer US,Springer Nature B.V
• Subject: Computer Communication Networks / Computer Science / Counterfeit / Data Structures and Information Theory / Datasets / Deception / Documents / Forensic computing / Generative adversarial networks / Image manipulation / Multimedia / Multimedia Information Systems / Neural networks / Occlusion / Similarity / Special Purpose and Application-Based Systems / Synthetic data