YouDream: Generating Anatomically Controllable Consistent Text-to-3D
  Animals
3D generation guided by text-to-image diffusion models enables the creation
of visually compelling assets. However previous methods explore generation
based on image or text. The boundaries of creativity are limited by what can be
expressed through words or the images that can be sourced. We present YouDream,
a method to generate high-quality anatomically controllable animals. YouDream
is guided using a text-to-image diffusion model controlled by 2D views of a 3D
pose prior. Our method generates 3D animals that are not possible to create
using previous text-to-3D generative methods. Additionally, our method is
capable of preserving anatomic consistency in the generated animals, an area
where prior text-to-3D approaches often struggle. Moreover, we design a fully
automated pipeline for generating commonly found animals. To circumvent the
need for human intervention to create a 3D pose, we propose a multi-agent LLM
that adapts poses from a limited library of animal 3D poses to represent the
desired animal. A user study conducted on the outcomes of YouDream demonstrates
the preference of the animal models generated by our method over others.
Turntable results and code are released at https://youdream3d.github.io/
667b8d11bfe251ebedad28aa
Sandeep Mishra
extracted_confirmed
2024-06-26T04:05:41.987Z
6635048a7523759ca1355b22
/avatars/644dfa9eb489229c5e809d574099d759.svg
Sandeep Mishra
user
battleMaster
667b8d11bfe251ebedad28ab
Oindrila Saha
extracted_pending
2024-06-26T03:37:58.089Z
6635368639a7401f16fdec55
/avatars/5a1790b5e1e54b18363f14ef699a8b21.svg
Oindrila Saha
user
oindrila13saha
667b8d11bfe251ebedad28ac
Alan C. Bovik
YouDream: Generating Anatomically Controllable Consistent Text-to-3D
  Animals
3D generation guided by text-to-image diffusion models enables the creation
of visually compelling assets. However previous methods explore generation
based on image or text. The boundaries of creativity are limited by what can be
expressed through words or the images that can be sourced. We present YouDream,
a method to generate high-quality anatomically controllable animals. YouDream
is guided using a text-to-image diffusion model controlled by 2D views of a 3D
pose prior. Our method generates 3D animals that are not possible to create
using previous text-to-3D generative methods. Additionally, our method is
capable of preserving anatomic consistency in the generated animals, an area
where prior text-to-3D approaches often struggle. Moreover, we design a fully
automated pipeline for generating commonly found animals. To circumvent the
need for human intervention to create a 3D pose, we propose a multi-agent LLM
that adapts poses from a limited library of animal 3D poses to represent the
desired animal. A user study conducted on the outcomes of YouDream demonstrates
the preference of the animal models generated by our method over others.
Turntable results and code are released at https://youdream3d.github.io/
YouDream generates high-quality, anatomically consistent 3D animals using a text-to-image diffusion model guided by 2D pose priors, showcasing superior creativity and quality compared to prior methods.
text-to-image diffusion models 3D generation 2D views 3D pose prior anatomical consistency multi-agent LLM user study turntable results