title: Drag Your GAN: Interactive Point-based Manipulation on the Generative
    Image Manifold
summary: Synthesizing visual content that meets users' needs often requires flexible
  and precise controllability of the pose, shape, expression, and layout of the
  generated objects. Existing approaches gain controllability of generative
  adversarial networks (GANs) via manually annotated training data or a prior 3D
  model, which often lack flexibility, precision, and generality. In this work,
  we study a powerful yet much less explored way of controlling GANs, that is, to
  "drag" any points of the image to precisely reach target points in a
  user-interactive manner, as shown in Fig.1. To achieve this, we propose
  DragGAN, which consists of two main components: 1) a feature-based motion
  supervision that drives the handle point to move towards the target position,
  and 2) a new point tracking approach that leverages the discriminative
  generator features to keep localizing the position of the handle points.
  Through DragGAN, anyone can deform an image with precise control over where
  pixels go, thus manipulating the pose, shape, expression, and layout of diverse
  categories such as animals, cars, humans, landscapes, etc. As these
  manipulations are performed on the learned generative image manifold of a GAN,
  they tend to produce realistic outputs even for challenging scenarios such as
  hallucinating occluded content and deforming shapes that consistently follow
  the object's rigidity. Both qualitative and quantitative comparisons
  demonstrate the advantage of DragGAN over prior approaches in the tasks of
  image manipulation and point tracking. We also showcase the manipulation of
  real images through GAN inversion.
paper_authors._id: 64670317cdea48159aa5d7a9
paper_authors.name: Xingang Pan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-19T11:11:53.469Z
paper_authors._id: 646758d6a9b4610868a138ff
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
paper_authors.fullname: Xingang Pan
paper_authors.type: user
paper_authors.user: Xingang-Pan
paper_authors._id: 64670317cdea48159aa5d7aa
paper_authors.name: Ayush Tewari
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2023-05-19T05:03:20.900Z
paper_authors._id: 6386e304b3ef0523c6419a05
paper_authors.avatarUrl: /avatars/3e6d292962dd27c8b299e3c96697061b.svg
paper_authors.fullname: Ayush Tewari
paper_authors.type: user
paper_authors.user: ayusht
paper_authors._id: 64670317cdea48159aa5d7ab
paper_authors.name: Thomas Leimkühler
paper_authors._id: 64670317cdea48159aa5d7ac
paper_authors.name: Lingjie Liu
paper_authors._id: 64670317cdea48159aa5d7ad
paper_authors.name: Abhimitra Meka
paper_authors._id: 64670317cdea48159aa5d7ae
paper_authors.name: Christian Theobalt
paper_title: Drag Your GAN: Interactive Point-based Manipulation on the Generative
    Image Manifold
paper_summary: Synthesizing visual content that meets users' needs often requires flexible
  and precise controllability of the pose, shape, expression, and layout of the
  generated objects. Existing approaches gain controllability of generative
  adversarial networks (GANs) via manually annotated training data or a prior 3D
  model, which often lack flexibility, precision, and generality. In this work,
  we study a powerful yet much less explored way of controlling GANs, that is, to
  "drag" any points of the image to precisely reach target points in a
  user-interactive manner, as shown in Fig.1. To achieve this, we propose
  DragGAN, which consists of two main components: 1) a feature-based motion
  supervision that drives the handle point to move towards the target position,
  and 2) a new point tracking approach that leverages the discriminative
  generator features to keep localizing the position of the handle points.
  Through DragGAN, anyone can deform an image with precise control over where
  pixels go, thus manipulating the pose, shape, expression, and layout of diverse
  categories such as animals, cars, humans, landscapes, etc. As these
  manipulations are performed on the learned generative image manifold of a GAN,
  they tend to produce realistic outputs even for challenging scenarios such as
  hallucinating occluded content and deforming shapes that consistently follow
  the object's rigidity. Both qualitative and quantitative comparisons
  demonstrate the advantage of DragGAN over prior approaches in the tasks of
  image manipulation and point tracking. We also showcase the manipulation of
  real images through GAN inversion.
paper_ai_summary: DragGAN enables precise user-controlled manipulation of image content through interactive point dragging, improving realism and flexibility over existing GAN-based methods.
paper_ai_keywords: generative adversarial networks feature-based motion supervision point tracking discriminative generator features generative image manifold image manipulation GAN inversion