title: Next Block Prediction: Video Generation via Semi-Autoregressive Modeling
summary: Next-Token Prediction (NTP) is a de facto approach for autoregressive (AR)
  video generation, but it suffers from suboptimal unidirectional dependencies
  and slow inference speed. In this work, we propose a semi-autoregressive
  (semi-AR) framework, called Next-Block Prediction (NBP), for video generation.
  By uniformly decomposing video content into equal-sized blocks (e.g., rows or
  frames), we shift the generation unit from individual tokens to blocks,
  allowing each token in the current block to simultaneously predict the
  corresponding token in the next block. Unlike traditional AR modeling, our
  framework employs bidirectional attention within each block, enabling tokens to
  capture more robust spatial dependencies. By predicting multiple tokens in
  parallel, NBP models significantly reduce the number of generation steps,
  leading to faster and more efficient inference. Our model achieves FVD scores
  of 103.3 on UCF101 and 25.5 on K600, outperforming the vanilla NTP model by an
  average of 4.4. Furthermore, thanks to the reduced number of inference steps,
  the NBP model generates 8.89 frames (128x128 resolution) per second, achieving
  an 11x speedup. We also explored model scales ranging from 700M to 3B
  parameters, observing significant improvements in generation quality, with FVD
  scores dropping from 103.3 to 55.3 on UCF101 and from 25.5 to 19.5 on K600,
  demonstrating the scalability of our approach.
paper_authors._id: 67ad5d2f8436e8ea7abb7a15
paper_authors.name: Shuhuai Ren
paper_authors._id: 67ad5d2f8436e8ea7abb7a16
paper_authors.name: Shuming Ma
paper_authors._id: 67ad5d2f8436e8ea7abb7a17
paper_authors.name: Xu Sun
paper_authors._id: 67ad5d2f8436e8ea7abb7a18
paper_authors.name: Furu Wei
paper_title: Next Block Prediction: Video Generation via Semi-Autoregressive Modeling
paper_summary: Next-Token Prediction (NTP) is a de facto approach for autoregressive (AR)
  video generation, but it suffers from suboptimal unidirectional dependencies
  and slow inference speed. In this work, we propose a semi-autoregressive
  (semi-AR) framework, called Next-Block Prediction (NBP), for video generation.
  By uniformly decomposing video content into equal-sized blocks (e.g., rows or
  frames), we shift the generation unit from individual tokens to blocks,
  allowing each token in the current block to simultaneously predict the
  corresponding token in the next block. Unlike traditional AR modeling, our
  framework employs bidirectional attention within each block, enabling tokens to
  capture more robust spatial dependencies. By predicting multiple tokens in
  parallel, NBP models significantly reduce the number of generation steps,
  leading to faster and more efficient inference. Our model achieves FVD scores
  of 103.3 on UCF101 and 25.5 on K600, outperforming the vanilla NTP model by an
  average of 4.4. Furthermore, thanks to the reduced number of inference steps,
  the NBP model generates 8.89 frames (128x128 resolution) per second, achieving
  an 11x speedup. We also explored model scales ranging from 700M to 3B
  parameters, observing significant improvements in generation quality, with FVD
  scores dropping from 103.3 to 55.3 on UCF101 and from 25.5 to 19.5 on K600,
  demonstrating the scalability of our approach.
paper_ai_keywords: semi-autoregressive Next-Block Prediction (NBP) video generation block decomposition bidirectional attention spatial dependencies inference steps FVD scores UCF101 K600 model scales generation quality