title: Control4D: Dynamic Portrait Editing by Learning 4D GAN from 2D
    Diffusion-based Editor
summary: Recent years have witnessed considerable achievements in editing images with
  text instructions. When applying these editors to dynamic scene editing, the
  new-style scene tends to be temporally inconsistent due to the frame-by-frame
  nature of these 2D editors. To tackle this issue, we propose Control4D, a novel
  approach for high-fidelity and temporally consistent 4D portrait editing.
  Control4D is built upon an efficient 4D representation with a 2D
  diffusion-based editor. Instead of using direct supervisions from the editor,
  our method learns a 4D GAN from it and avoids the inconsistent supervision
  signals. Specifically, we employ a discriminator to learn the generation
  distribution based on the edited images and then update the generator with the
  discrimination signals. For more stable training, multi-level information is
  extracted from the edited images and used to facilitate the learning of the
  generator. Experimental results show that Control4D surpasses previous
  approaches and achieves more photo-realistic and consistent 4D editing
  performances. The link to our project website is
  https://control4darxiv.github.io.
paper_authors._id: 6477f638e5ddd7d2a1739207
paper_authors.name: Ruizhi Shao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-12-06T10:29:08.808Z
paper_authors._id: 63f52856b51da4d61da7aa21
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63f52856b51da4d61da7aa21/qSvSk1TnRp6DzdRzcoC8F.jpeg
paper_authors.fullname: Ruizhi Shao
paper_authors.type: user
paper_authors.user: Saurus
paper_authors._id: 6477f638e5ddd7d2a1739208
paper_authors.name: Jingxiang Sun
paper_authors._id: 6477f638e5ddd7d2a1739209
paper_authors.name: Cheng Peng
paper_authors._id: 6477f638e5ddd7d2a173920a
paper_authors.name: Zerong Zheng
paper_authors._id: 6477f638e5ddd7d2a173920b
paper_authors.name: Boyao Zhou
paper_authors._id: 6477f638e5ddd7d2a173920c
paper_authors.name: Hongwen Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T12:29:51.421Z
paper_authors._id: 638715a1aaee5cd69dc0c401
paper_authors.avatarUrl: /avatars/f5976b3b02fe86191cb7f59c3dae102b.svg
paper_authors.fullname: Hongwen Zhang
paper_authors.type: user
paper_authors.user: HongwenZhang
paper_authors._id: 6477f638e5ddd7d2a173920d
paper_authors.name: Yebin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-02T12:30:21.828Z
paper_authors._id: 62e14dbe4db2175cd2735a80
paper_authors.avatarUrl: /avatars/e6385dcedcb97e1b36281b49210321aa.svg
paper_authors.fullname: Yebin Liu
paper_authors.type: user
paper_authors.user: YebinLiu
paper_title: Control4D: Dynamic Portrait Editing by Learning 4D GAN from 2D
    Diffusion-based Editor
paper_summary: Recent years have witnessed considerable achievements in editing images with
  text instructions. When applying these editors to dynamic scene editing, the
  new-style scene tends to be temporally inconsistent due to the frame-by-frame
  nature of these 2D editors. To tackle this issue, we propose Control4D, a novel
  approach for high-fidelity and temporally consistent 4D portrait editing.
  Control4D is built upon an efficient 4D representation with a 2D
  diffusion-based editor. Instead of using direct supervisions from the editor,
  our method learns a 4D GAN from it and avoids the inconsistent supervision
  signals. Specifically, we employ a discriminator to learn the generation
  distribution based on the edited images and then update the generator with the
  discrimination signals. For more stable training, multi-level information is
  extracted from the edited images and used to facilitate the learning of the
  generator. Experimental results show that Control4D surpasses previous
  approaches and achieves more photo-realistic and consistent 4D editing
  performances. The link to our project website is
  https://control4darxiv.github.io.
paper_ai_summary: Control4D uses a 2D diffusion-based editor within a 4D GAN framework to achieve high-fidelity and temporally consistent 4D portrait editing.
paper_ai_keywords: 4D representation 2D diffusion-based editor 4D GAN discriminator generator multi-level information