Wavelets Are All You Need for Autoregressive Image Generation
In this paper, we take a new approach to autoregressive image generation that
is based on two main ingredients. The first is wavelet image coding, which
allows to tokenize the visual details of an image from coarse to fine details
by ordering the information starting with the most significant bits of the most
significant wavelet coefficients. The second is a variant of a language
transformer whose architecture is re-designed and optimized for token sequences
in this 'wavelet language'. The transformer learns the significant statistical
correlations within a token sequence, which are the manifestations of
well-known correlations between the wavelet subbands at various resolutions. We
show experimental results with conditioning on the generation process.
6682e225f80f97532edd4a00
Wael Mattar
admin_assigned
2024-07-03T15:05:51.162Z
65faee3e9e96d231ddcf5183
/avatars/6530a3c79655509fdde83ff7c694603e.svg
Wael Mattar
user
WaelMattar
6682e225f80f97532edd4a01
Idan Levy
claimed_verified
2024-07-02T07:39:14.384Z
64257699120a3ed323326b8c
/avatars/2b8ee4feb99700791a7aa045506518ef.svg
Idan Levy
user
idanlevy
6682e225f80f97532edd4a02
Nir Sharon
admin_assigned
2024-07-03T15:05:37.290Z
65732a9714bfb7d56f80d352
/avatars/b89073e4cd8e1e6ee458399e13936b05.svg
King
user
NirSharon
6682e225f80f97532edd4a03
Shai Dekel
Wavelets Are All You Need for Autoregressive Image Generation
In this paper, we take a new approach to autoregressive image generation that
is based on two main ingredients. The first is wavelet image coding, which
allows to tokenize the visual details of an image from coarse to fine details
by ordering the information starting with the most significant bits of the most
significant wavelet coefficients. The second is a variant of a language
transformer whose architecture is re-designed and optimized for token sequences
in this 'wavelet language'. The transformer learns the significant statistical
correlations within a token sequence, which are the manifestations of
well-known correlations between the wavelet subbands at various resolutions. We
show experimental results with conditioning on the generation process.
Wavelet-based image generation uses a modified language transformer to model wavelet subband correlations for generating images conditioned on specific inputs.
wavelet image coding wavelet coefficients language transformer wavelet subbands token sequences wavelet language