Open-MAGVIT2: An Open-Source Project Toward Democratizing
  Auto-regressive Visual Generation
We present Open-MAGVIT2, a family of auto-regressive image generation models
ranging from 300M to 1.5B. The Open-MAGVIT2 project produces an open-source
replication of Google's MAGVIT-v2 tokenizer, a tokenizer with a super-large
codebook (i.e., 2^{18} codes), and achieves the state-of-the-art
reconstruction performance (1.17 rFID) on ImageNet 256 times 256.
Furthermore, we explore its application in plain auto-regressive models and
validate scalability properties. To assist auto-regressive models in predicting
with a super-large vocabulary, we factorize it into two sub-vocabulary of
different sizes by asymmetric token factorization, and further introduce "next
sub-token prediction" to enhance sub-token interaction for better generation
quality. We release all models and codes to foster innovation and creativity in
the field of auto-regressive visual generation.
66de6c935af9428df34917b4
Zhuoyan Luo
admin_assigned
2024-09-09T08:23:00.548Z
657187de7644d1128571495e
/avatars/89412c94fd6136f6680055551de3ddc4.svg
Zhuoyan Luo
user
RobertLuo1
66de6c935af9428df34917b5
Fengyuan Shi
admin_assigned
2024-09-09T08:26:16.830Z
666c1232a7ca4800af0545d7
/avatars/b4713c86ee57f30aae478262634692fe.svg
Fengyuan Shi
user
shifengyuan
66de6c935af9428df34917b6
Yixiao Ge
admin_assigned
2024-09-09T08:26:23.973Z
640e9762b03f4cd29f58d982
/avatars/81da37d628163fe3e094b247c7c3a3b5.svg
Yixiao Ge
user
yxgeee
66de6c935af9428df34917b7
Yujiu Yang
66de6c935af9428df34917b8
Limin Wang
66de6c935af9428df34917b9
Ying Shan
admin_assigned
2024-09-09T08:27:34.082Z
63ca3ddc04c979828310bfcb
/avatars/615e0d8622950b4408b40d550f02a894.svg
Ying Shan
user
yshan2u
Open-MAGVIT2: An Open-Source Project Toward Democratizing
  Auto-regressive Visual Generation
We present Open-MAGVIT2, a family of auto-regressive image generation models
ranging from 300M to 1.5B. The Open-MAGVIT2 project produces an open-source
replication of Google's MAGVIT-v2 tokenizer, a tokenizer with a super-large
codebook (i.e., 2^{18} codes), and achieves the state-of-the-art
reconstruction performance (1.17 rFID) on ImageNet 256 times 256.
Furthermore, we explore its application in plain auto-regressive models and
validate scalability properties. To assist auto-regressive models in predicting
with a super-large vocabulary, we factorize it into two sub-vocabulary of
different sizes by asymmetric token factorization, and further introduce "next
sub-token prediction" to enhance sub-token interaction for better generation
quality. We release all models and codes to foster innovation and creativity in
the field of auto-regressive visual generation.
Open-MAGVIT2 models, ranging from 300M to 1.5B parameters, achieve state-of-the-art image reconstruction and exploration in auto-regressive models with super-large token vocabularies.
auto-regressive image generation Open-MAGVIT2 MAGVIT-v2 tokenizer super-large codebook reconstruction performance asymmetric token factorization next sub-token prediction