title: Step-Video-T2V Technical Report: The Practice, Challenges, and Future of
    Video Foundation Model
summary: We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model
  with 30B parameters and the ability to generate videos up to 204 frames in
  length. A deep compression Variational Autoencoder, Video-VAE, is designed for
  video generation tasks, achieving 16x16 spatial and 8x temporal compression
  ratios, while maintaining exceptional video reconstruction quality. User
  prompts are encoded using two bilingual text encoders to handle both English
  and Chinese. A DiT with 3D full attention is trained using Flow Matching and is
  employed to denoise input noise into latent frames. A video-based DPO approach,
  Video-DPO, is applied to reduce artifacts and improve the visual quality of the
  generated videos. We also detail our training strategies and share key
  observations and insights. Step-Video-T2V's performance is evaluated on a novel
  video generation benchmark, Step-Video-T2V-Eval, demonstrating its
  state-of-the-art text-to-video quality when compared with both open-source and
  commercial engines. Additionally, we discuss the limitations of current
  diffusion-based model paradigm and outline future directions for video
  foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval
  available at https://github.com/stepfun-ai/Step-Video-T2V. The online version
  can be accessed from https://yuewen.cn/videos as well. Our goal is to
  accelerate the innovation of video foundation models and empower video content
  creators.
paper_authors._id: 67b2a72e7a49eaea082b9dcf
paper_authors.name: Guoqing Ma
paper_authors._id: 67b2a72e7a49eaea082b9dd0
paper_authors.name: Haoyang Huang
paper_authors._id: 67b2a72e7a49eaea082b9dd1
paper_authors.name: Kun Yan
paper_authors._id: 67b2a72e7a49eaea082b9dd2
paper_authors.name: Liangyu Chen
paper_authors._id: 67b2a72e7a49eaea082b9dd3
paper_authors.name: Nan Duan
paper_authors._id: 67b2a72e7a49eaea082b9dd4
paper_authors.name: Shengming Yin
paper_authors._id: 67b2a72e7a49eaea082b9dd5
paper_authors.name: Changyi Wan
paper_authors._id: 67b2a72e7a49eaea082b9dd6
paper_authors.name: Ranchen Ming
paper_authors._id: 67b2a72e7a49eaea082b9dd7
paper_authors.name: Xiaoniu Song
paper_authors._id: 67b2a72e7a49eaea082b9dd8
paper_authors.name: Xing Chen
paper_authors._id: 67b2a72e7a49eaea082b9dd9
paper_authors.name: Yu Zhou
paper_authors._id: 67b2a72e7a49eaea082b9dda
paper_authors.name: Deshan Sun
paper_authors._id: 67b2a72e7a49eaea082b9ddb
paper_authors.name: Deyu Zhou
paper_authors._id: 67b2a72e7a49eaea082b9ddc
paper_authors.name: Jian Zhou
paper_authors._id: 67b2a72e7a49eaea082b9ddd
paper_authors.name: Kaijun Tan
paper_authors._id: 67b2a72e7a49eaea082b9dde
paper_authors.name: Kang An
paper_authors._id: 67b2a72e7a49eaea082b9ddf
paper_authors.name: Mei Chen
paper_authors._id: 67b2a72e7a49eaea082b9de0
paper_authors.name: Wei Ji
paper_authors._id: 67b2a72e7a49eaea082b9de1
paper_authors.name: Qiling Wu
paper_authors._id: 67b2a72e7a49eaea082b9de2
paper_authors.name: Wen Sun
paper_authors._id: 67b2a72e7a49eaea082b9de3
paper_authors.name: Xin Han
paper_authors._id: 67b2a72e7a49eaea082b9de4
paper_authors.name: Yanan Wei
paper_authors._id: 67b2a72e7a49eaea082b9de5
paper_authors.name: Zheng Ge
paper_authors._id: 67b2a72e7a49eaea082b9de6
paper_authors.name: Aojie Li
paper_authors._id: 67b2a72e7a49eaea082b9de7
paper_authors.name: Bin Wang
paper_authors._id: 67b2a72e7a49eaea082b9de8
paper_authors.name: Bizhu Huang
paper_authors._id: 67b2a72e7a49eaea082b9de9
paper_authors.name: Bo Wang
paper_authors._id: 67b2a72e7a49eaea082b9dea
paper_authors.name: Brian Li
paper_authors._id: 67b2a72e7a49eaea082b9deb
paper_authors.name: Changxing Miao
paper_authors._id: 67b2a72e7a49eaea082b9dec
paper_authors.name: Chen Xu
paper_authors._id: 67b2a72e7a49eaea082b9ded
paper_authors.name: Chenfei Wu
paper_authors._id: 67b2a72e7a49eaea082b9dee
paper_authors.name: Chenguang Yu
paper_authors._id: 67b2a72e7a49eaea082b9def
paper_authors.name: Dapeng Shi
paper_authors._id: 67b2a72e7a49eaea082b9df0
paper_authors.name: Dingyuan Hu
paper_authors._id: 67b2a72e7a49eaea082b9df1
paper_authors.name: Enle Liu
paper_authors._id: 67b2a72e7a49eaea082b9df2
paper_authors.name: Gang Yu
paper_authors._id: 67b2a72e7a49eaea082b9df3
paper_authors.name: Ge Yang
paper_authors._id: 67b2a72e7a49eaea082b9df4
paper_authors.name: Guanzhe Huang
paper_authors._id: 67b2a72e7a49eaea082b9df5
paper_authors.name: Gulin Yan
paper_authors._id: 67b2a72e7a49eaea082b9df6
paper_authors.name: Haiyang Feng
paper_authors._id: 67b2a72e7a49eaea082b9df7
paper_authors.name: Hao Nie
paper_authors._id: 67b2a72e7a49eaea082b9df8
paper_authors.name: Haonan Jia
paper_authors._id: 67b2a72e7a49eaea082b9df9
paper_authors.name: Hanpeng Hu
paper_authors._id: 67b2a72e7a49eaea082b9dfa
paper_authors.name: Hanqi Chen
paper_authors._id: 67b2a72e7a49eaea082b9dfb
paper_authors.name: Haolong Yan
paper_authors._id: 67b2a72e7a49eaea082b9dfc
paper_authors.name: Heng Wang
paper_authors._id: 67b2a72e7a49eaea082b9dfd
paper_authors.name: Hongcheng Guo
paper_authors._id: 67b2a72e7a49eaea082b9dfe
paper_authors.name: Huilin Xiong
paper_authors._id: 67b2a72e7a49eaea082b9dff
paper_authors.name: Huixin Xiong
paper_authors._id: 67b2a72e7a49eaea082b9e00
paper_authors.name: Jiahao Gong
paper_authors._id: 67b2a72e7a49eaea082b9e01
paper_authors.name: Jianchang Wu
paper_authors._id: 67b2a72e7a49eaea082b9e02
paper_authors.name: Jiaoren Wu
paper_authors._id: 67b2a72e7a49eaea082b9e03
paper_authors.name: Jie Wu
paper_authors._id: 67b2a72e7a49eaea082b9e04
paper_authors.name: Jie Yang
paper_authors._id: 67b2a72e7a49eaea082b9e05
paper_authors.name: Jiashuai Liu
paper_authors._id: 67b2a72e7a49eaea082b9e06
paper_authors.name: Jiashuo Li
paper_authors._id: 67b2a72e7a49eaea082b9e07
paper_authors.name: Jingyang Zhang
paper_authors._id: 67b2a72e7a49eaea082b9e08
paper_authors.name: Junjing Guo
paper_authors._id: 67b2a72e7a49eaea082b9e09
paper_authors.name: Junzhe Lin
paper_authors._id: 67b2a72e7a49eaea082b9e0a
paper_authors.name: Kaixiang Li
paper_authors._id: 67b2a72e7a49eaea082b9e0b
paper_authors.name: Lei Liu
paper_authors._id: 67b2a72e7a49eaea082b9e0c
paper_authors.name: Lei Xia
paper_authors._id: 67b2a72e7a49eaea082b9e0d
paper_authors.name: Liang Zhao
paper_authors._id: 67b2a72e7a49eaea082b9e0e
paper_authors.name: Liguo Tan
paper_authors._id: 67b2a72e7a49eaea082b9e0f
paper_authors.name: Liwen Huang
paper_authors._id: 67b2a72e7a49eaea082b9e10
paper_authors.name: Liying Shi
paper_authors._id: 67b2a72e7a49eaea082b9e11
paper_authors.name: Ming Li
paper_authors._id: 67b2a72e7a49eaea082b9e12
paper_authors.name: Mingliang Li
paper_authors._id: 67b2a72e7a49eaea082b9e13
paper_authors.name: Muhua Cheng
paper_authors._id: 67b2a72e7a49eaea082b9e14
paper_authors.name: Na Wang
paper_authors._id: 67b2a72e7a49eaea082b9e15
paper_authors.name: Qiaohui Chen
paper_authors._id: 67b2a72e7a49eaea082b9e16
paper_authors.name: Qinglin He
paper_authors._id: 67b2a72e7a49eaea082b9e17
paper_authors.name: Qiuyan Liang
paper_authors._id: 67b2a72e7a49eaea082b9e18
paper_authors.name: Quan Sun
paper_authors._id: 67b2a72e7a49eaea082b9e19
paper_authors.name: Ran Sun
paper_authors._id: 67b2a72e7a49eaea082b9e1a
paper_authors.name: Rui Wang
paper_authors._id: 67b2a72e7a49eaea082b9e1b
paper_authors.name: Shaoliang Pang
paper_authors._id: 67b2a72e7a49eaea082b9e1c
paper_authors.name: Shiliang Yang
paper_authors._id: 67b2a72e7a49eaea082b9e1d
paper_authors.name: Sitong Liu
paper_authors._id: 67b2a72e7a49eaea082b9e1e
paper_authors.name: Siqi Liu
paper_authors._id: 67b2a72e7a49eaea082b9e1f
paper_authors.name: Shuli Gao
paper_authors._id: 67b2a72e7a49eaea082b9e20
paper_authors.name: Tiancheng Cao
paper_authors._id: 67b2a72e7a49eaea082b9e21
paper_authors.name: Tianyu Wang
paper_authors._id: 67b2a72e7a49eaea082b9e22
paper_authors.name: Weipeng Ming
paper_authors._id: 67b2a72e7a49eaea082b9e23
paper_authors.name: Wenqing He
paper_authors._id: 67b2a72e7a49eaea082b9e24
paper_authors.name: Xu Zhao
paper_authors._id: 67b2a72e7a49eaea082b9e25
paper_authors.name: Xuelin Zhang
paper_authors._id: 67b2a72e7a49eaea082b9e26
paper_authors.name: Xianfang Zeng
paper_authors._id: 67b2a72e7a49eaea082b9e27
paper_authors.name: Xiaojia Liu
paper_authors._id: 67b2a72e7a49eaea082b9e28
paper_authors.name: Xuan Yang
paper_authors._id: 67b2a72e7a49eaea082b9e29
paper_authors.name: Yaqi Dai
paper_authors._id: 67b2a72e7a49eaea082b9e2a
paper_authors.name: Yanbo Yu
paper_authors._id: 67b2a72e7a49eaea082b9e2b
paper_authors.name: Yang Li
paper_authors._id: 67b2a72e7a49eaea082b9e2c
paper_authors.name: Yineng Deng
paper_authors._id: 67b2a72e7a49eaea082b9e2d
paper_authors.name: Yingming Wang
paper_authors._id: 67b2a72e7a49eaea082b9e2e
paper_authors.name: Yilei Wang
paper_authors._id: 67b2a72e7a49eaea082b9e2f
paper_authors.name: Yuanwei Lu
paper_authors._id: 67b2a72e7a49eaea082b9e30
paper_authors.name: Yu Chen
paper_authors._id: 67b2a72e7a49eaea082b9e31
paper_authors.name: Yu Luo
paper_authors._id: 67b2a72e7a49eaea082b9e32
paper_authors.name: Yuchu Luo
paper_authors._id: 67b2a72e7a49eaea082b9e33
paper_authors.name: Yuhe Yin
paper_authors._id: 67b2a72e7a49eaea082b9e34
paper_authors.name: Yuheng Feng
paper_authors._id: 67b2a72e7a49eaea082b9e35
paper_authors.name: Yuxiang Yang
paper_authors._id: 67b2a72e7a49eaea082b9e36
paper_authors.name: Zecheng Tang
paper_authors._id: 67b2a72e7a49eaea082b9e37
paper_authors.name: Zekai Zhang
paper_authors._id: 67b2a72e7a49eaea082b9e38
paper_authors.name: Zidong Yang
paper_authors._id: 67b2a72e7a49eaea082b9e39
paper_authors.name: Binxing Jiao
paper_authors._id: 67b2a72e7a49eaea082b9e3a
paper_authors.name: Jiansheng Chen
paper_authors._id: 67b2a72e7a49eaea082b9e3b
paper_authors.name: Jing Li
paper_authors._id: 67b2a72e7a49eaea082b9e3c
paper_authors.name: Shuchang Zhou
paper_authors._id: 67b2a72e7a49eaea082b9e3d
paper_authors.name: Xiangyu Zhang
paper_authors._id: 67b2a72e7a49eaea082b9e3e
paper_authors.name: Xinhao Zhang
paper_authors._id: 67b2a72e7a49eaea082b9e3f
paper_authors.name: Yibo Zhu
paper_authors._id: 67b2a72e7a49eaea082b9e40
paper_authors.name: Heung-Yeung Shum
paper_authors._id: 67b2a72e7a49eaea082b9e41
paper_authors.name: Daxin Jiang
paper_title: Step-Video-T2V Technical Report: The Practice, Challenges, and Future of
    Video Foundation Model
paper_summary: We present Step-Video-T2V, a state-of-the-art text-to-video pre-trained model
  with 30B parameters and the ability to generate videos up to 204 frames in
  length. A deep compression Variational Autoencoder, Video-VAE, is designed for
  video generation tasks, achieving 16x16 spatial and 8x temporal compression
  ratios, while maintaining exceptional video reconstruction quality. User
  prompts are encoded using two bilingual text encoders to handle both English
  and Chinese. A DiT with 3D full attention is trained using Flow Matching and is
  employed to denoise input noise into latent frames. A video-based DPO approach,
  Video-DPO, is applied to reduce artifacts and improve the visual quality of the
  generated videos. We also detail our training strategies and share key
  observations and insights. Step-Video-T2V's performance is evaluated on a novel
  video generation benchmark, Step-Video-T2V-Eval, demonstrating its
  state-of-the-art text-to-video quality when compared with both open-source and
  commercial engines. Additionally, we discuss the limitations of current
  diffusion-based model paradigm and outline future directions for video
  foundation models. We make both Step-Video-T2V and Step-Video-T2V-Eval
  available at https://github.com/stepfun-ai/Step-Video-T2V. The online version
  can be accessed from https://yuewen.cn/videos as well. Our goal is to
  accelerate the innovation of video foundation models and empower video content
  creators.
paper_ai_keywords: Variational Autoencoder Video-VAE spatial compression temporal compression video reconstruction quality bilingual text encoders DiT 3D full attention Flow Matching latent frames video-based DPO Video-DPO video generation benchmark Step-Video-T2V-Eval diffusion-based model paradigm