Optical Flow based CNN for detection of unlearnt deepfake manipulations

A new phenomenon named Deepfakes constitutes a serious threat in video manipulation. AI-based tech-nologies have provided easy-to-use methods to create extremely realistic videos. On the side of multi-media forensics, being able to individuate this kind of fake contents becomes ever more crucial. In this work, a new forensic technique able to detect fake and original video sequences is proposed; it is based on the use of CNNs trained to distinguish possible motion dissimilarities in the temporal structure of a video sequence by exploiting optical flow fields. The results obtained highlight comparable performances with the state-of-the-art methods which, in general, only resort to single video frames. Furthermore, the proposed optical flow based detection scheme also provides a superior robustness in the more realistic cross-forgery operative scenario and can even be combined with frame-based approaches to improve their global effectiveness. (c) 2021 Elsevier B.V. All rights reserved.

Automated summary

Deepfakes pose a serious threat in video manipulation, utilizing AI-based technologies to create realistic videos. A new forensic technique based on optical flow fields is proposed to detect fake and original video sequences, showing comparable performance with state-of-the-art methods and superior robustness in more realistic scenarios. The proposed technique can also be combined with frame-based approaches to enhance overall effectiveness.