  Since photorealistic faces can be readily generated by facial manipulation
technologies nowadays, potential malicious abuse of these technologies has
drawn great concerns. Numerous deepfake detection methods are thus proposed.
However, existing methods only focus on detecting one-step facial manipulation.
As the emergence of easy-accessible facial editing applications, people can
easily manipulate facial components using multi-step operations in a sequential
manner. This new threat requires us to detect a sequence of facial
manipulations, which is vital for both detecting deepfake media and recovering
original faces afterwards. Motivated by this observation, we emphasize the need
and propose a novel research problem called Detecting Sequential DeepFake
Manipulation (Seq-DeepFake). Unlike the existing deepfake detection task only
demanding a binary label prediction, detecting Seq-DeepFake manipulation
requires correctly predicting a sequential vector of facial manipulation
operations. To support a large-scale investigation, we construct the first
Seq-DeepFake dataset, where face images are manipulated sequentially with
corresponding annotations of sequential facial manipulation vectors. Based on
this new dataset, we cast detecting Seq-DeepFake manipulation as a specific
image-to-sequence (e.g. image captioning) task and propose a concise yet
effective Seq-DeepFake Transformer (SeqFakeFormer). Moreover, we build a
comprehensive benchmark and set up rigorous evaluation protocols and metrics
for this new research problem. Extensive experiments demonstrate the
effectiveness of SeqFakeFormer. Several valuable observations are also revealed
to facilitate future research in broader deepfake detection problems.

Computer Vision and Pattern Recognition