Text2Layer: Layered Image Generation using Latent Diffusion Model
Layer compositing is one of the most popular image editing workflows among
both amateurs and professionals. Motivated by the success of diffusion models,
we explore layer compositing from a layered image generation perspective.
Instead of generating an image, we propose to generate background, foreground,
layer mask, and the composed image simultaneously. To achieve layered image
generation, we train an autoencoder that is able to reconstruct layered images
and train diffusion models on the latent representation. One benefit of the
proposed problem is to enable better compositing workflows in addition to the
high-quality image output. Another benefit is producing higher-quality layer
masks compared to masks produced by a separate step of image segmentation.
Experimental results show that the proposed method is able to generate
high-quality layered images and initiates a benchmark for future work.
64b8a631cf14c2fabe980782
Xinyang Zhang
64b8a631cf14c2fabe980783
Wentian Zhao
64b8a631cf14c2fabe980784
Xin Lu
64b8a631cf14c2fabe980785
Jeff Chien
Text2Layer: Layered Image Generation using Latent Diffusion Model
Layer compositing is one of the most popular image editing workflows among
both amateurs and professionals. Motivated by the success of diffusion models,
we explore layer compositing from a layered image generation perspective.
Instead of generating an image, we propose to generate background, foreground,
layer mask, and the composed image simultaneously. To achieve layered image
generation, we train an autoencoder that is able to reconstruct layered images
and train diffusion models on the latent representation. One benefit of the
proposed problem is to enable better compositing workflows in addition to the
high-quality image output. Another benefit is producing higher-quality layer
masks compared to masks produced by a separate step of image segmentation.
Experimental results show that the proposed method is able to generate
high-quality layered images and initiates a benchmark for future work.
A method using autoencoders and diffusion models generates high-quality layered images, improving compositing workflows and mask quality.
layer compositing diffusion models layered image generation autoencoder latent representation layer mask image segmentation benchmark