Step-Video-T2V Technical Report: The Practice, Challenges, and Future of
  Video Foundation Model
We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model
with 30B parameters and the ability to generate videos up to 204 frames in
length. A deep compression Variational Autoencoder, Video-VAE, is designed for
video generation tasks, achieving 16x16 spatial and 8x temporal compression
ratios, while maintaining exceptional video reconstruction quality. User
prompts are encoded using two bilingual text encoders to handle both English
and Chinese. A DiT with 3D full attention is trained using Flow Matching and is
employed to denoise input noise into latent frames. A video-based DPO approach,
Video-DPO, is applied to reduce artifacts and improve the visual quality of the
generated videos. We also detail our training strategies and share key
observations and insights. Step-Video-T2V's performance is evaluated on a novel
video generation benchmark, Step-Video-T2V-Eval, demonstrating its
state-of-the-art text-to-video quality when compared with both open-source and
commercial engines. Additionally, we discuss the limitations of current
diffusion-based model paradigm and outline future directions for video
foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval
available at https://github.com/stepfun-ai/Step-Video-T2V. The online version
can be accessed from https://yuewen.cn/videos as well. Our goal is to
accelerate the innovation of video foundation models and empower video content
creators.
67b2a72e7a49eaea082b9dcf
Guoqing Ma
67b2a72e7a49eaea082b9dd0
Haoyang Huang
67b2a72e7a49eaea082b9dd1
Kun Yan
67b2a72e7a49eaea082b9dd2
Liangyu Chen
67b2a72e7a49eaea082b9dd3
Nan Duan
67b2a72e7a49eaea082b9dd4
Shengming Yin
67b2a72e7a49eaea082b9dd5
Changyi Wan
67b2a72e7a49eaea082b9dd6
Ranchen Ming
67b2a72e7a49eaea082b9dd7
Xiaoniu Song
67b2a72e7a49eaea082b9dd8
Xing Chen
67b2a72e7a49eaea082b9dd9
Yu Zhou
67b2a72e7a49eaea082b9dda
Deshan Sun
67b2a72e7a49eaea082b9ddb
Deyu Zhou
67b2a72e7a49eaea082b9ddc
Jian Zhou
67b2a72e7a49eaea082b9ddd
Kaijun Tan
67b2a72e7a49eaea082b9dde
Kang An
67b2a72e7a49eaea082b9ddf
Mei Chen
67b2a72e7a49eaea082b9de0
Wei Ji
67b2a72e7a49eaea082b9de1
Qiling Wu
67b2a72e7a49eaea082b9de2
Wen Sun
67b2a72e7a49eaea082b9de3
Xin Han
67b2a72e7a49eaea082b9de4
Yanan Wei
67b2a72e7a49eaea082b9de5
Zheng Ge
67b2a72e7a49eaea082b9de6
Aojie Li
67b2a72e7a49eaea082b9de7
Bin Wang
67b2a72e7a49eaea082b9de8
Bizhu Huang
67b2a72e7a49eaea082b9de9
Bo Wang
67b2a72e7a49eaea082b9dea
Brian Li
67b2a72e7a49eaea082b9deb
Changxing Miao
67b2a72e7a49eaea082b9dec
Chen Xu
67b2a72e7a49eaea082b9ded
Chenfei Wu
67b2a72e7a49eaea082b9dee
Chenguang Yu
67b2a72e7a49eaea082b9def
Dapeng Shi
67b2a72e7a49eaea082b9df0
Dingyuan Hu
67b2a72e7a49eaea082b9df1
Enle Liu
67b2a72e7a49eaea082b9df2
Gang Yu
67b2a72e7a49eaea082b9df3
Ge Yang
67b2a72e7a49eaea082b9df4
Guanzhe Huang
67b2a72e7a49eaea082b9df5
Gulin Yan
67b2a72e7a49eaea082b9df6
Haiyang Feng
67b2a72e7a49eaea082b9df7
Hao Nie
67b2a72e7a49eaea082b9df8
Haonan Jia
67b2a72e7a49eaea082b9df9
Hanpeng Hu
67b2a72e7a49eaea082b9dfa
Hanqi Chen
67b2a72e7a49eaea082b9dfb
Haolong Yan
67b2a72e7a49eaea082b9dfc
Heng Wang
67b2a72e7a49eaea082b9dfd
Hongcheng Guo
67b2a72e7a49eaea082b9dfe
Huilin Xiong
67b2a72e7a49eaea082b9dff
Huixin Xiong
67b2a72e7a49eaea082b9e00
Jiahao Gong
67b2a72e7a49eaea082b9e01
Jianchang Wu
67b2a72e7a49eaea082b9e02
Jiaoren Wu
67b2a72e7a49eaea082b9e03
Jie Wu
67b2a72e7a49eaea082b9e04
Jie Yang
67b2a72e7a49eaea082b9e05
Jiashuai Liu
67b2a72e7a49eaea082b9e06
Jiashuo Li
67b2a72e7a49eaea082b9e07
Jingyang Zhang
67b2a72e7a49eaea082b9e08
Junjing Guo
67b2a72e7a49eaea082b9e09
Junzhe Lin
67b2a72e7a49eaea082b9e0a
Kaixiang Li
67b2a72e7a49eaea082b9e0b
Lei Liu
67b2a72e7a49eaea082b9e0c
Lei Xia
67b2a72e7a49eaea082b9e0d
Liang Zhao
67b2a72e7a49eaea082b9e0e
Liguo Tan
67b2a72e7a49eaea082b9e0f
Liwen Huang
67b2a72e7a49eaea082b9e10
Liying Shi
67b2a72e7a49eaea082b9e11
Ming Li
67b2a72e7a49eaea082b9e12
Mingliang Li
67b2a72e7a49eaea082b9e13
Muhua Cheng
67b2a72e7a49eaea082b9e14
Na Wang
67b2a72e7a49eaea082b9e15
Qiaohui Chen
67b2a72e7a49eaea082b9e16
Qinglin He
67b2a72e7a49eaea082b9e17
Qiuyan Liang
67b2a72e7a49eaea082b9e18
Quan Sun
67b2a72e7a49eaea082b9e19
Ran Sun
67b2a72e7a49eaea082b9e1a
Rui Wang
67b2a72e7a49eaea082b9e1b
Shaoliang Pang
67b2a72e7a49eaea082b9e1c
Shiliang Yang
67b2a72e7a49eaea082b9e1d
Sitong Liu
67b2a72e7a49eaea082b9e1e
Siqi Liu
67b2a72e7a49eaea082b9e1f
Shuli Gao
67b2a72e7a49eaea082b9e20
Tiancheng Cao
67b2a72e7a49eaea082b9e21
Tianyu Wang
67b2a72e7a49eaea082b9e22
Weipeng Ming
67b2a72e7a49eaea082b9e23
Wenqing He
67b2a72e7a49eaea082b9e24
Xu Zhao
67b2a72e7a49eaea082b9e25
Xuelin Zhang
67b2a72e7a49eaea082b9e26
Xianfang Zeng
67b2a72e7a49eaea082b9e27
Xiaojia Liu
67b2a72e7a49eaea082b9e28
Xuan Yang
67b2a72e7a49eaea082b9e29
Yaqi Dai
67b2a72e7a49eaea082b9e2a
Yanbo Yu
67b2a72e7a49eaea082b9e2b
Yang Li
67b2a72e7a49eaea082b9e2c
Yineng Deng
67b2a72e7a49eaea082b9e2d
Yingming Wang
67b2a72e7a49eaea082b9e2e
Yilei Wang
67b2a72e7a49eaea082b9e2f
Yuanwei Lu
67b2a72e7a49eaea082b9e30
Yu Chen
67b2a72e7a49eaea082b9e31
Yu Luo
67b2a72e7a49eaea082b9e32
Yuchu Luo
67b2a72e7a49eaea082b9e33
Yuhe Yin
67b2a72e7a49eaea082b9e34
Yuheng Feng
67b2a72e7a49eaea082b9e35
Yuxiang Yang
67b2a72e7a49eaea082b9e36
Zecheng Tang
67b2a72e7a49eaea082b9e37
Zekai Zhang
67b2a72e7a49eaea082b9e38
Zidong Yang
67b2a72e7a49eaea082b9e39
Binxing Jiao
67b2a72e7a49eaea082b9e3a
Jiansheng Chen
67b2a72e7a49eaea082b9e3b
Jing Li
67b2a72e7a49eaea082b9e3c
Shuchang Zhou
67b2a72e7a49eaea082b9e3d
Xiangyu Zhang
67b2a72e7a49eaea082b9e3e
Xinhao Zhang
67b2a72e7a49eaea082b9e3f
Yibo Zhu
67b2a72e7a49eaea082b9e40
Heung-Yeung Shum
67b2a72e7a49eaea082b9e41
Daxin Jiang
Step-Video-T2V Technical Report: The Practice, Challenges, and Future of
  Video Foundation Model
We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model
with 30B parameters and the ability to generate videos up to 204 frames in
length. A deep compression Variational Autoencoder, Video-VAE, is designed for
video generation tasks, achieving 16x16 spatial and 8x temporal compression
ratios, while maintaining exceptional video reconstruction quality. User
prompts are encoded using two bilingual text encoders to handle both English
and Chinese. A DiT with 3D full attention is trained using Flow Matching and is
employed to denoise input noise into latent frames. A video-based DPO approach,
Video-DPO, is applied to reduce artifacts and improve the visual quality of the
generated videos. We also detail our training strategies and share key
observations and insights. Step-Video-T2V's performance is evaluated on a novel
video generation benchmark, Step-Video-T2V-Eval, demonstrating its
state-of-the-art text-to-video quality when compared with both open-source and
commercial engines. Additionally, we discuss the limitations of current
diffusion-based model paradigm and outline future directions for video
foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval
available at https://github.com/stepfun-ai/Step-Video-T2V. The online version
can be accessed from https://yuewen.cn/videos as well. Our goal is to
accelerate the innovation of video foundation models and empower video content
creators.
Step-Video-T2V is a state-of-the-art text-to-video model with 30B parameters, utilizing a deep compression Variational Autoencoder and a DiT with 3D full attention for high-quality video generation.
Step-Video-T2V Variational Autoencoder Video-VAE DiT 3D full attention Flow Matching Video-DPO Step-Video-T2V-Eval diffusion-based model video foundation models