Explore the vast range of titles available.

The dependability of visual information on the web and the authenticity of digital media appearing virally in social media platforms has been raising unprecedented concerns. As a result, in the last years the multimedia forensics research community pursued the ambition to scale the forensic analysis to real-world web-based open systems. This survey aims at describing the work done so far on the analysis of shared data, covering three main aspects: forensics techniques performing source identification and integrity verification on media uploaded on social networks, platform provenance analysis allowing to identify sharing platforms, and multimedia verification algorithms assessing the credibility of media objects in relation to its associated textual information. The achieved results are highlighted together with current open issues and research challenges to be addressed in order to advance the field in the next future.

Due to the appeal of Generative Adversarial Networks (GANs), synthetic face images generated with GAN models are difficult to differentiate from genuine human faces and may be used to create counterfeit identities. However, these face images contain artifacts presented in irises owing to the irregularity of highlights between the left and right irises. Adversaries can utilize PS documents of these images trying to conceal the artifacts of images, which makes it more challenging to distinguish between pristine and fake images. In order to tackle this, our research introduces a complete dataset and analytical tools that make a substantial contribution to multimedia forensics. This allows for the authentication of documents despite common alterations in the PS documents. Owing to the lack of large-scale reference IRIS datasets in the PS scenario, this study provided a pioneering dataset aiming to set a standard for multimedia forensic investigations. Given face images, we extracted iris images using the Dlib (King, J Mach Learn Res 10(60):1755–1758, 2020 ) and EyeCool (Wang et al. 2021 ) models, as described in Guo et al. ( 2022 ). However, in some cases, the potential eyelid occlusion phenomenon resulted in incomplete iris images. We utilized a hypergraph convolution-based image inpainting technique to complete the missing pixels in the extracted iris images, thus uncovering the intricate relationships within the iris data. To evaluate the IRIS image dataset and highlight associated issues, we conducted a series of analyses using Siamese Neural Networks, including ResNet50, Xception, VGG16, and MobileNet-v2, to measure the similarities between authentic and synthetic human iris images. Our SNN model, with four different backbones, effectively differentiated between genuine and synthetic iris images. For instance, using the Xception network, we achieved 56.76% similarity in IRISes for synthetic images and 92.77% similarity in IRISes for real images. The effectiveness of our approach was demonstrated by the similarity scores obtained from all SNN architectures, which showed a significant difference between the GAN-generated images from ProGAN or StyleGAN and the original PS iris photos. The similarity scores resulting from StyleGAN are higher than those of the ProGAN architecture, but at its highest, it is 76%, while for the pristine images, it ranges from 85% to 95%. This discrepancy can be utilized in order to distinguish between pristine and GAN-generated images.

Information fusion, i.e., the combination of expert systems, has a huge potential to improve the accuracy of pattern recognition systems. During the last decades, various application fields started to use different fusion concepts extensively. The forensic sciences are still hesitant if it comes to blindly applying information fusion. Here, a potentially negative impact on the classification accuracy, if wrongly used or parameterized, as well as the increased complexity (and the inherently higher costs for plausibility validation) of fusion is in conflict with the fundamental requirements for forensics.The goals of this paper are to explain the reasons for this reluctance to accept such a potentially very beneficial technique and to illustrate the practical issues arising when applying fusion. For those practical discussions the exemplary application scenario of morphing attack detection (MAD) is selected with the goal to facilitate the understanding between the media forensics community and forensic practitioners.As general contributions, it is illustrated why the naive assumption that fusion would make the detection more reliable can fail in practice, i.e., why fusion behaves in a field application sometimes differently than in the lab. As a result, the constraints and limitations of the application of fusion are discussed and its impact to (media) forensics is reflected upon.As technical contributions, the current state of the art of MAD is expanded by: The introduction of the likelihood-based fusion and an fusion ensemble composition experiment to extend the set of methods (majority voting, sum-rule, and Dempster-Shafer Theory of evidence) used previouslyThe direct comparison of the two evaluation scenarios “MAD in document issuing” and “MAD in identity verification” using a realistic and some less restrictive evaluation setupsA thorough analysis and discussion of the detection performance issues and the reasons why fusion in a majority of the test cases discussed here leads to worse classification accuracy than the best individual classifier

Detecting and localizing image manipulation are necessary to counter malicious use of image editing techniques. Accordingly, it is essential to distinguish between authentic and tampered regions by analyzing intrinsic statistics in an image. We focus on JPEG compression artifacts left during image acquisition and editing. We propose a convolutional neural network that uses discrete cosine transform (DCT) coefficients, where compression artifacts remain, to localize image manipulation. Standard CNNs cannot learn the distribution of DCT coefficients because the convolution throws away the spatial coordinates, which are essential for DCT coefficients. We illustrate how to design and train a neural network that can learn the distribution of DCT coefficients. Furthermore, we introduce Compression Artifact Tracing Network that jointly uses image acquisition artifacts and compression artifacts. It significantly outperforms traditional and deep neural network-based methods in detecting and localizing tampered regions.

Forensic research community keeps proposing new techniques to analyze digital images and videos. However, the performance of proposed tools are usually tested on data that are far from reality in terms of resolution, source device, and processing history. Remarkably, in the latest years, portable devices became the preferred means to capture images and videos, and contents are commonly shared through social media platforms (SMPs, for example, Facebook, YouTube, etc.). These facts pose new challenges to the forensic community: for example, most modern cameras feature digital stabilization, that is proved to severely hinder the performance of video source identification technologies; moreover, the strong re-compression enforced by SMPs during upload threatens the reliability of multimedia forensic tools. On the other hand, portable devices capture both images and videos with the same sensor, opening new forensic opportunities. The goal of this paper is to propose the VISION dataset as a contribution to the development of multimedia forensics. The VISION dataset is currently composed by 34,427 images and 1914 videos, both in the native format and in their social version (Facebook, YouTube, and WhatsApp are considered), from 35 portable devices of 11 major brands. VISION can be exploited as benchmark for the exhaustive evaluation of several image and video forensic tools.

We present in this paper an algorithm that is capable of clustering images taken by an unknown number of unknown digital cameras into groups, such that each contains only images taken by the same source camera. It first extracts a sensor pattern noise (SPN) from each image, which serves as the fingerprint of the camera that has taken the image. The image clustering is performed based on the pairwise correlations between camera fingerprints extracted from images. During this process, each SPN is treated as a random variable and a Markov random field (MRF) approach is employed to iteratively assign a class label to each SPN (i.e., random variable). The clustering process requires no a priori knowledge about the dataset from the user. A concise yet effective cost function is formulated to allow different “neighbors” different voting power in determining the class label of the image in question depending on their similarities. Comparative experiments were carried out on the Dresden image database to demonstrate the advantages of the proposed clustering algorithm.

The rise of AI-generated synthetic media, or deepfakes, has introduced unprecedented opportunities and challenges across various fields, including entertainment, cybersecurity, and digital communication. Using advanced frameworks such as Generative Adversarial Networks (GANs) and Diffusion Models (DMs), deepfakes are capable of producing highly realistic yet fabricated content, while these advancements enable creative and innovative applications, they also pose severe ethical, social, and security risks due to their potential misuse. The proliferation of deepfakes has triggered phenomena like “Impostor Bias”, a growing skepticism toward the authenticity of multimedia content, further complicating trust in digital interactions. This paper is mainly based on the description of a research project called FF4ALL (FF4ALL-Detection of Deep Fake Media and Life-Long Media Authentication) for the detection and authentication of deepfakes, focusing on areas such as forensic attribution, passive and active authentication, and detection in real-world scenarios. By exploring both the strengths and limitations of current methodologies, we highlight critical research gaps and propose directions for future advancements to ensure media integrity and trustworthiness in an era increasingly dominated by synthetic media.

Deepfakes are realistic-looking fake media generated by deep-learning algorithms that iterate through large datasets until they have learned how to solve the given problem (i.e., swap faces or objects in video and digital content). The massive generation of such content and modification technologies is rapidly affecting the quality of public discourse and the safeguarding of human rights. Deepfakes are being widely used as a malicious source of misinformation in court that seek to sway a court’s decision. Because digital evidence is critical to the outcome of many legal cases, detecting deepfake media is extremely important and in high demand in digital forensics. As such, it is important to identify and build a classifier that can accurately distinguish between authentic and disguised media, especially in facial-recognition systems as it can be used in identity protection too. In this work, we compare the most common, state-of-the-art face-detection classifiers such as Custom CNN, VGG19, and DenseNet-121 using an augmented real and fake face-detection dataset. Data augmentation is used to boost performance and reduce computational resources. Our preliminary results indicate that VGG19 has the best performance and highest accuracy of 95% when compared with other analyzed models.

Detecting DeepFake videos has become a central task in modern multimedia forensics applications. This article presents a method to detect face swapped videos when the portrayed person in the video is known. We propose using a threshold classifier based on similarity scores obtained from a Deep Convolutional Neural Network (DCNN) trained for facial recognition. We compute a set of similarity scores between faces extracted from questioned videos and reference materials of the person depicted. We use the highest score to classify the questioned videos as authentic or fake, depending on the threshold chosen. We validate our method on the Celeb-DF (v2) dataset (Li et al., 2020) [13]. Using the training and testing splits specified on the dataset, we obtained an HTER of 0.020 and an AUC of 0.994, surpassing the most robust approaches against this dataset (Tran et al., 2021) [37]. Additionally, a logistic regression model was used to convert the highest score into a likelihood ratio for greater applicability in forensic analyses. •Use of high-level cues based in biometrics characteristics for deepfake detection.•State of the art method for deepfake detection on CelebDF using face recognition scores.•A forensic evaluation method for deepfake detection using the likelihood ratio paradigm.

The recent pace of generative media synthesis methods has greatly enhanced the credibility and accessibility of deepfake content, which causes serious risks to digital trust, authenticity of media, and forensic security. Existing deepfake detection methods are usually limited to either spatial domain visual cues or frequency domain artifacts, which leads to their limited robustness, poor generalization under realistic compression and poor interpretability. To overcome them, this paper will introduce a generalizable hybrid spatial-frequency deepfake detector, the proposed scheme combines both RGB-based visual representations with discrete cosine transform (DCT) frequency elements into a high-capacity convolutional network with attention-based refining. The suggested framework uses an EfficientNet-B7 backbone to identify rich hierarchical features and a convolutional block attention module (CBAM) to adaptively highlight information that is of interest to manipulation including spatial and channel-wise information. The early combination of spatial and frequency information allows the model to mutually exploit semantic inconsistencies and fine-scale high-frequency distortions added in the process of generating synthetic content. Comprehensive experiments of the FaceForensics + + C23 data set show that the proposed methodology has state-of-the-art performance with a ROC-AUC of 0.997, as well as high precision-recall balance and convergence of the training process. Further class separability is supported by feature-space analysis and prediction probability distributions and more complex CAM-based visualizations give significant forensic descriptions by identifying manipulation-prone regions of the faces. The high detection accuracy, the increased potential of generalization, and the greater interpretability are the factors that underline the efficiency of the suggested hybrid framework and confirm its appropriateness to the use in the field of the real-life deepfake forensics.