title: Mobile Video Diffusion
summary: Video diffusion models have achieved impressive realism and controllability
  but are limited by high computational demands, restricting their use on mobile
  devices. This paper introduces the first mobile-optimized video diffusion
  model. Starting from a spatio-temporal UNet from Stable Video Diffusion (SVD),
  we reduce memory and computational cost by reducing the frame resolution,
  incorporating multi-scale temporal representations, and introducing two novel
  pruning schema to reduce the number of channels and temporal blocks.
  Furthermore, we employ adversarial finetuning to reduce the denoising to a
  single step. Our model, coined as MobileVD, is 523x more efficient (1817.2 vs.
  4.34 TFLOPs) with a slight quality drop (FVD 149 vs. 171), generating latents
  for a 14x512x256 px clip in 1.7 seconds on a Xiaomi-14 Pro. Our results are
  available at https://qualcomm-ai-research.github.io/mobile-video-diffusion/
paper_authors._id: 67598660eff1d97bd2cf2b7b
paper_authors.name: Haitam Ben Yahia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-11T13:57:42.350Z
paper_authors._id: 67598719eff1d97bd2cf6300
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/GoxSruke3IDDZP-vSyqyq.png
paper_authors.fullname: Haitam Ben Yahia/
paper_authors.type: user
paper_authors.user: haitamb
paper_authors._id: 67598660eff1d97bd2cf2b7c
paper_authors.name: Denis Korzhenkov
paper_authors._id: 67598660eff1d97bd2cf2b7d
paper_authors.name: Ioannis Lelekas
paper_authors._id: 67598660eff1d97bd2cf2b7e
paper_authors.name: Amir Ghodrati
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T13:39:02.200Z
paper_authors._id: 65d31d1bfe21569868432e96
paper_authors.avatarUrl: /avatars/195207d241f4eab121e4fed6cf5d578d.svg
paper_authors.fullname: Amir
paper_authors.type: user
paper_authors.user: aghodrati
paper_authors._id: 67598660eff1d97bd2cf2b7f
paper_authors.name: Amirhossein Habibian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-11T13:39:04.250Z
paper_authors._id: 628372aa09aa80237c6d2044
paper_authors.avatarUrl: /avatars/c1e34dee217d73f5539a0807adc12c80.svg
paper_authors.fullname: Amir Habibian
paper_authors.type: user
paper_authors.user: habibian
paper_title: Mobile Video Diffusion
paper_summary: Video diffusion models have achieved impressive realism and controllability
  but are limited by high computational demands, restricting their use on mobile
  devices. This paper introduces the first mobile-optimized video diffusion
  model. Starting from a spatio-temporal UNet from Stable Video Diffusion (SVD),
  we reduce memory and computational cost by reducing the frame resolution,
  incorporating multi-scale temporal representations, and introducing two novel
  pruning schema to reduce the number of channels and temporal blocks.
  Furthermore, we employ adversarial finetuning to reduce the denoising to a
  single step. Our model, coined as MobileVD, is 523x more efficient (1817.2 vs.
  4.34 TFLOPs) with a slight quality drop (FVD 149 vs. 171), generating latents
  for a 14x512x256 px clip in 1.7 seconds on a Xiaomi-14 Pro. Our results are
  available at https://qualcomm-ai-research.github.io/mobile-video-diffusion/
paper_ai_summary: A mobile-optimized video diffusion model reduces computational demands and maintains quality through pruning and adversarial finetuning.
paper_ai_keywords: spatio-temporal UNet MobileVD frame resolution multi-scale temporal representations pruning adversarial finetuning FVD