title: Training Video Foundation Models with NVIDIA NeMo
summary: Video Foundation Models (VFMs) have recently been used to simulate the real
  world to train physical AI systems and develop creative visual experiences.
  However, there are significant challenges in training large-scale, high quality
  VFMs that can generate high-quality videos. We present a scalable, open-source
  VFM training pipeline with NVIDIA NeMo, providing accelerated video dataset
  curation, multimodal data loading, and parallelized video diffusion model
  training and inference. We also provide a comprehensive performance analysis
  highlighting best practices for efficient VFM training and inference.
paper_authors._id: 67d8d29fbe55b912664641d7
paper_authors.name: Zeeshan Patel
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-25T08:21:30.699Z
paper_authors._id: 638ee4df6251c8bd7abd246e
paper_authors.avatarUrl: /avatars/8d8c4dac7d82cba6dcf884460bdc5328.svg
paper_authors.fullname: Zeeshan Patel
paper_authors.type: user
paper_authors.user: zeeshanp
paper_authors._id: 67d8d29fbe55b912664641d8
paper_authors.name: Ethan He
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-25T08:36:02.115Z
paper_authors._id: 62b20f6df4a72794189248fc
paper_authors.avatarUrl: /avatars/87e1125868616d4f7d6ee1e5ec4499b4.svg
paper_authors.fullname: Ethan He
paper_authors.type: user
paper_authors.user: ethanhe
paper_authors._id: 67d8d29fbe55b912664641d9
paper_authors.name: Parth Mannan
paper_authors._id: 67d8d29fbe55b912664641da
paper_authors.name: Xiaowei Ren
paper_authors._id: 67d8d29fbe55b912664641db
paper_authors.name: Ryan Wolf
paper_authors._id: 67d8d29fbe55b912664641dc
paper_authors.name: Niket Agarwal
paper_authors._id: 67d8d29fbe55b912664641dd
paper_authors.name: Jacob Huffman
paper_authors._id: 67d8d29fbe55b912664641de
paper_authors.name: Zhuoyao Wang
paper_authors._id: 67d8d29fbe55b912664641df
paper_authors.name: Carl Wang
paper_authors._id: 67d8d29fbe55b912664641e0
paper_authors.name: Jack Chang
paper_authors._id: 67d8d29fbe55b912664641e1
paper_authors.name: Yan Bai
paper_authors._id: 67d8d29fbe55b912664641e2
paper_authors.name: Tommy Huang
paper_authors._id: 67d8d29fbe55b912664641e3
paper_authors.name: Linnan Wang
paper_authors._id: 67d8d29fbe55b912664641e4
paper_authors.name: Sahil Jain
paper_authors._id: 67d8d29fbe55b912664641e5
paper_authors.name: Shanmugam Ramasamy
paper_authors._id: 67d8d29fbe55b912664641e6
paper_authors.name: Joseph Jennings
paper_authors._id: 67d8d29fbe55b912664641e7
paper_authors.name: Ekaterina Sirazitdinova
paper_authors._id: 67d8d29fbe55b912664641e8
paper_authors.name: Oleg Sudakov
paper_authors._id: 67d8d29fbe55b912664641e9
paper_authors.name: Mingyuan Ma
paper_authors._id: 67d8d29fbe55b912664641ea
paper_authors.name: Bobby Chen
paper_authors._id: 67d8d29fbe55b912664641eb
paper_authors.name: Forrest Lin
paper_authors._id: 67d8d29fbe55b912664641ec
paper_authors.name: Hao Wang
paper_authors._id: 67d8d29fbe55b912664641ed
paper_authors.name: Vasanth Rao Naik Sabavat
paper_authors._id: 67d8d29fbe55b912664641ee
paper_authors.name: Sriharsha Niverty
paper_authors._id: 67d8d29fbe55b912664641ef
paper_authors.name: Rong Ou
paper_authors._id: 67d8d29fbe55b912664641f0
paper_authors.name: Pallab Bhattacharya
paper_authors._id: 67d8d29fbe55b912664641f1
paper_authors.name: David Page
paper_authors._id: 67d8d29fbe55b912664641f2
paper_authors.name: Nima Tajbakhsh
paper_authors._id: 67d8d29fbe55b912664641f3
paper_authors.name: Ashwath Aithal
paper_title: Training Video Foundation Models with NVIDIA NeMo
paper_summary: Video Foundation Models (VFMs) have recently been used to simulate the real
  world to train physical AI systems and develop creative visual experiences.
  However, there are significant challenges in training large-scale, high quality
  VFMs that can generate high-quality videos. We present a scalable, open-source
  VFM training pipeline with NVIDIA NeMo, providing accelerated video dataset
  curation, multimodal data loading, and parallelized video diffusion model
  training and inference. We also provide a comprehensive performance analysis
  highlighting best practices for efficient VFM training and inference.
paper_ai_summary: A scalable open-source pipeline using NVIDIA NeMo for training and inference of Video Foundation Models addresses challenges in generating high-quality videos.
paper_ai_keywords: Video Foundation Models VFM NVIDIA NeMo video diffusion model multimodal data loading parallelized training performance analysis efficient training