title: TextToon: Real-Time Text Toonify Head Avatar from Single Video
summary: We propose TextToon, a method to generate a drivable toonified avatar. Given
  a short monocular video sequence and a written instruction about the avatar
  style, our model can generate a high-fidelity toonified avatar that can be
  driven in real-time by another video with arbitrary identities. Existing
  related works heavily rely on multi-view modeling to recover geometry via
  texture embeddings, presented in a static manner, leading to control
  limitations. The multi-view video input also makes it difficult to deploy these
  models in real-world applications. To address these issues, we adopt a
  conditional embedding Tri-plane to learn realistic and stylized facial
  representations in a Gaussian deformation field. Additionally, we expand the
  stylization capabilities of 3D Gaussian Splatting by introducing an adaptive
  pixel-translation neural network and leveraging patch-aware contrastive
  learning to achieve high-quality images. To push our work into consumer
  applications, we develop a real-time system that can operate at 48 FPS on a GPU
  machine and 15-18 FPS on a mobile machine. Extensive experiments demonstrate
  the efficacy of our approach in generating textual avatars over existing
  methods in terms of quality and real-time animation. Please refer to our
  project page for more details: https://songluchuan.github.io/TextToon/.
paper_authors._id: 67077a46be29abe37696f33a
paper_authors.name: Luchuan Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-10T14:03:10.468Z
paper_authors._id: 6707dd1b5370cb53e70d673d
paper_authors.avatarUrl: /avatars/795f443fa1f433a275e89e8ecd4cbfef.svg
paper_authors.fullname: Song
paper_authors.type: user
paper_authors.user: LuLuchuan
paper_authors._id: 67077a46be29abe37696f33b
paper_authors.name: Lele Chen
paper_authors._id: 67077a46be29abe37696f33c
paper_authors.name: Celong Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:35:08.788Z
paper_authors._id: 64364ba31adb261e94e0e5b4
paper_authors.avatarUrl: /avatars/cb6ef2155d88b1e64c4ff36bd3a8f255.svg
paper_authors.fullname: Celong Liu
paper_authors.type: user
paper_authors.user: goddice
paper_authors._id: 67077a46be29abe37696f33d
paper_authors.name: Pinxin Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-10T09:35:03.031Z
paper_authors._id: 62eb469dade76f18dd4f0dea
paper_authors.avatarUrl: /avatars/c558254a0352d115a73febb90bb9370f.svg
paper_authors.fullname: Pinxin Liu
paper_authors.type: user
paper_authors.user: pliu23
paper_authors._id: 67077a46be29abe37696f33e
paper_authors.name: Chenliang Xu
paper_title: TextToon: Real-Time Text Toonify Head Avatar from Single Video
paper_summary: We propose TextToon, a method to generate a drivable toonified avatar. Given
  a short monocular video sequence and a written instruction about the avatar
  style, our model can generate a high-fidelity toonified avatar that can be
  driven in real-time by another video with arbitrary identities. Existing
  related works heavily rely on multi-view modeling to recover geometry via
  texture embeddings, presented in a static manner, leading to control
  limitations. The multi-view video input also makes it difficult to deploy these
  models in real-world applications. To address these issues, we adopt a
  conditional embedding Tri-plane to learn realistic and stylized facial
  representations in a Gaussian deformation field. Additionally, we expand the
  stylization capabilities of 3D Gaussian Splatting by introducing an adaptive
  pixel-translation neural network and leveraging patch-aware contrastive
  learning to achieve high-quality images. To push our work into consumer
  applications, we develop a real-time system that can operate at 48 FPS on a GPU
  machine and 15-18 FPS on a mobile machine. Extensive experiments demonstrate
  the efficacy of our approach in generating textual avatars over existing
  methods in terms of quality and real-time animation. Please refer to our
  project page for more details: https://songluchuan.github.io/TextToon/.
paper_ai_keywords: conditional embedding Tri-plane Gaussian deformation field 3D Gaussian Splatting adaptive pixel-translation neural network patch-aware contrastive learning