title: MagicDriveDiT: High-Resolution Long Video Generation for Autonomous
    Driving with Adaptive Control
summary: The rapid advancement of diffusion models has greatly improved video
  synthesis, especially in controllable video generation, which is essential for
  applications like autonomous driving. However, existing methods are limited by
  scalability and how control conditions are integrated, failing to meet the
  needs for high-resolution and long videos for autonomous driving applications.
  In this paper, we introduce MagicDriveDiT, a novel approach based on the DiT
  architecture, and tackle these challenges. Our method enhances scalability
  through flow matching and employs a progressive training strategy to manage
  complex scenarios. By incorporating spatial-temporal conditional encoding,
  MagicDriveDiT achieves precise control over spatial-temporal latents.
  Comprehensive experiments show its superior performance in generating realistic
  street scene videos with higher resolution and more frames. MagicDriveDiT
  significantly improves video generation quality and spatial-temporal controls,
  expanding its potential applications across various tasks in autonomous
  driving.
paper_authors._id: 67405c15820127a3e9c153e5
paper_authors.name: Ruiyuan Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:20:27.560Z
paper_authors._id: 6360cc93a46f0cdd62e1f687
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6360cc93a46f0cdd62e1f687/PNLw1QrH45HnzWaU_khix.jpeg
paper_authors.fullname: Ruiyuan Gao
paper_authors.type: user
paper_authors.user: flymin
paper_authors._id: 67405c15820127a3e9c153e6
paper_authors.name: Kai Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-24T22:06:54.627Z
paper_authors._id: 65bf3b304b5f8c270d50d6c8
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65bf3b304b5f8c270d50d6c8/iSC_Em-JtfcsKFoGw4KFc.jpeg
paper_authors.fullname: Kai Chen
paper_authors.type: user
paper_authors.user: KaiChen1998
paper_authors._id: 67405c15820127a3e9c153e7
paper_authors.name: Bo Xiao
paper_authors._id: 67405c15820127a3e9c153e8
paper_authors.name: Lanqing Hong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:21:07.437Z
paper_authors._id: 671207b95550f581fcce3cfd
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/wqvv-QQ59dNT1KSpOUQLa.png
paper_authors.fullname: Lanqing HONG
paper_authors.type: user
paper_authors.user: racheltechie
paper_authors._id: 67405c15820127a3e9c153e9
paper_authors.name: Zhenguo Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-11-22T13:21:20.678Z
paper_authors._id: 639944bef7c4648254d124f0
paper_authors.avatarUrl: /avatars/c5cbb87cd51cc1341844f67d42c55151.svg
paper_authors.fullname: lzg
paper_authors.type: user
paper_authors.user: lizhenguo
paper_authors._id: 67405c15820127a3e9c153ea
paper_authors.name: Qiang Xu
paper_title: MagicDriveDiT: High-Resolution Long Video Generation for Autonomous
    Driving with Adaptive Control
paper_summary: The rapid advancement of diffusion models has greatly improved video
  synthesis, especially in controllable video generation, which is essential for
  applications like autonomous driving. However, existing methods are limited by
  scalability and how control conditions are integrated, failing to meet the
  needs for high-resolution and long videos for autonomous driving applications.
  In this paper, we introduce MagicDriveDiT, a novel approach based on the DiT
  architecture, and tackle these challenges. Our method enhances scalability
  through flow matching and employs a progressive training strategy to manage
  complex scenarios. By incorporating spatial-temporal conditional encoding,
  MagicDriveDiT achieves precise control over spatial-temporal latents.
  Comprehensive experiments show its superior performance in generating realistic
  street scene videos with higher resolution and more frames. MagicDriveDiT
  significantly improves video generation quality and spatial-temporal controls,
  expanding its potential applications across various tasks in autonomous
  driving.
paper_ai_summary: MagicDriveDiT enhances video synthesis for autonomous driving by integrating flow matching, progressive training, and spatial-temporal conditional encoding, improving both resolution and control.
paper_ai_keywords: diffusion models controllable video generation DiT architecture flow matching progressive training spatial-temporal conditional encoding video generation quality spatial-temporal controls