Researchers at NTU Singapore Propose Seq-DeepFake Transformer (SeqFakeFormer) for Detecting and Recovering Sequential DeepFake Manipulations

Deep generative models have recently been used to create hyper-realistic facial images that are virtually indistinguishable from real photographs. However, the substantial advancement in image synthesis raises security concerns about the potential malicious use of these methods to fabricate and spread false information (deepfake).

Several deepfake detection techniques have been proposed to identify such fabricated faces in order to address this security concern. In order to successfully complete the deepfake detection challenge, the model must correctly predict the binary labels (Real/Fake) given the original face image and the altered face image produced by the face swap technique.

People may easily modify face photographs in daily life because of the rising popularity of easily available facial editing programs. Users can now easily edit face photos utilizing multi-step operations in a sequential manner, as opposed to existing deepfake systems that mainly carry out the one-step facial modification.

The initial image can be changed by successively eliminating the beard, increasing the smile, and adding spectacles. Including sequential modification, information broadens the scope of already-existing deepfake issues and presents a new obstacle for one-step deepfake detection techniques. The researchers used this insight as inspiration to develop a brand-new research challenge, Detecting Sequential Deepfake Manipulation (Seq-Deepfake).

Researchers from NTU in Singapore recently submitted the first Seq-Deepfake dataset to help in the study of identifying Seq-Deepfake. The dataset performs sequential facial image alteration with a range of steps (from a minimum of 1 step to a maximum of 5 steps), resulting in sequences of varying lengths. Finding the precise alteration sequences is nearly as difficult as telling the difference between the original and altered facial photos.

The majority of the most recent applications for facial manipulation are based on the Generative Adversarial Network (GAN). It is well known that it is challenging to completely untangle the latent semantic space learned by GAN. Researchers asserted that this flaw is likely to reveal successive face manipulation traces that are both spatial and sequential. Based on this discovery, the team projected identifying Seq-Deepfake as a specific image-to-sequence task to detect these two forms of modification traces, and they then presented a short but potent Seq-DeepFake Transformer (SeqFakeFormer).

Spatial Relation Extraction and Sequential Relation Modeling with Spatially Enhanced Cross-attention are the two main components of SeqFakeFormer. SeqFakeFormer sends an altered image into a deep convolutional neural network (CNN) to learn its feature maps in order to adaptively capture delicate spatial manipulation regions.

Following thorough testing, the researchers found that the proposed SeqFakeFormer outperforms all baselines in detecting face manipulation sequences in both the modification of facial components and the manipulation of facial attributes. SeqFakeFormer outperforms other baselines with both CNNs, showing that the suggested technique is compatible with various feature extractors. According to two evaluation measures, the proposed strategy has specifically produced improvements of 1-2 and 3-4 percent in the sequential manipulation of facial features and components.

Conclusion 

A recent article by researchers at NTU, Singapore, aimed to detect a consecutive vector of multi-step facial manipulation operations. The research problem was called “Detecting Sequential DeepFake Manipulation.” The group unveiled the initial Seq-DeepFake dataset that offered sequentially altered facial photos. They projected detecting Seq-DeepFake manipulation as a specialized image-to-sequence task, supported by this new dataset, and presented a Seq-DeepFake Transformer (SeqFakeFormer). Extensive experimental results show SeqFakeFormer’s supremacy, and insightful discoveries open the door to further study of deeper deepfake detection.

This Article is written as a summary article by Marktechpost Staff based on the research paper 'Detecting and Recovering Sequential DeepFake Manipulation'. All Credit For This Research Goes To Researchers on This Project. Checkout the paper, project and github link.

Please Don't Forget To Join Our ML Subreddit