PixNerd: Pixel Neural Field Diffusion
The current success of diffusion transformers heavily depends on the
compressed latent space shaped by the pre-trained variational autoencoder(VAE).
However, this two-stage training paradigm inevitably introduces accumulated
errors and decoding artifacts. To address the aforementioned problems,
researchers return to pixel space at the cost of complicated cascade pipelines
and increased token complexity. In contrast to their efforts, we propose to
model the patch-wise decoding with neural field and present a single-scale,
single-stage, efficient, end-to-end solution, coined as pixel neural field
diffusion~(PixelNerd). Thanks to the efficient neural field representation in
PixNerd, we directly achieved 2.15 FID on ImageNet 256times256 and 2.84 FID
on ImageNet 512times512 without any complex cascade pipeline or VAE. We also
extend our PixNerd framework to text-to-image applications. Our PixNerd-XXL/16
achieved a competitive 0.73 overall score on the GenEval benchmark and 80.9
overall score on the DPG benchmark.
688c1ec68c434640078cc386
Shuai Wang
688c1ec68c434640078cc387
Ziteng Gao
688c1ec68c434640078cc388
Chenhui Zhu
688c1ec68c434640078cc389
Weilin Huang
688c1ec68c434640078cc38a
Limin Wang
PixNerd: Pixel Neural Field Diffusion
The current success of diffusion transformers heavily depends on the
compressed latent space shaped by the pre-trained variational autoencoder(VAE).
However, this two-stage training paradigm inevitably introduces accumulated
errors and decoding artifacts. To address the aforementioned problems,
researchers return to pixel space at the cost of complicated cascade pipelines
and increased token complexity. In contrast to their efforts, we propose to
model the patch-wise decoding with neural field and present a single-scale,
single-stage, efficient, end-to-end solution, coined as pixel neural field
diffusion~(PixelNerd). Thanks to the efficient neural field representation in
PixNerd, we directly achieved 2.15 FID on ImageNet 256times256 and 2.84 FID
on ImageNet 512times512 without any complex cascade pipeline or VAE. We also
extend our PixNerd framework to text-to-image applications. Our PixNerd-XXL/16
achieved a competitive 0.73 overall score on the GenEval benchmark and 80.9
overall score on the DPG benchmark.
Pixel Neural Field Diffusion (PixNerd) achieves high-quality image generation in a single-scale, single-stage process without VAEs or complex pipelines, and extends to text-to-image applications with competitive performance.
diffusion transformers compressed latent space pre-trained variational autoencoder pixel space patch-wise decoding neural field single-scale single-stage end-to-end solution pixel neural field diffusion PixNerd FID ImageNet text-to-image GenEval benchmark DPG benchmark