GLFF: Global and Local Feature Fusion for Face Forgery Detection

Published

November 16, 2022

Author(s)

Haiying Guan, Yan Ju, Shan Jia, Jialing Cai, Siwei Lyu

Abstract

With the rapid development of the deep generative models (such as Generative Adversarial Networks and Auto-encoders), AI-synthesized images of human face are now of such high qualities that humans can hardly distinguish them from pristine ones. Although existing detection methods have shown high performance in specific evaluation settings, e.g., on images with observable artifacts or from seen models, they tend to suffer serious performance degradation in real-world scenarios where testing images can be generated by more powerful generation models or combined with various post-processing operations. To address this issue, we propose a Global and Local Feature Fusion (GLFF) framework to learn rich and discriminative representations by combining multi-scale global features from the whole image with refined local features from informative patches for face forgery detection. Specifically, the global branch extracts and fuses low-level textual features and high-level semantic information to exploit the artifacts from multiple scale of the entire image. The fused features are utilized to guide the proposed patch selection module to select patches with more information for classification. These selected patches are then fed into the local branch to extract more detailed local artifacts. Finally, features extracted by two branches are combined using attention mechanism for binary classification. Due to the lack of comprehensive face forgery dataset for evaluation, we further create a challenging face forgery dataset, named DeepFakeFaceForensics (DF^3), which contains 6 state-of-the-art generation models and a variety of post-processing techniques to approach the real-world applications. Experimental results demonstrate the superiority of our method to the state-of-the-art methods on the proposed DF^3 dataset and three other open-source datasets.

Citation

arxiv

Pub Weblink

https://doi.org/10.48550/arXiv.2211.08615

Pub Type

Websites

Keywords

Deepfakes, Forgery Detection, Artificial Intelligence

Citation

Guan, H. , Ju, Y. , Jia, S. , Cai, J. and Lyu, S. (2022), GLFF: Global and Local Feature Fusion for Face Forgery Detection, arxiv, [online], https://doi.org/10.48550/arXiv.2211.08615 (Accessed October 31, 2024)