title: Boximator: Generating Rich and Controllable Motions for Video Synthesis
summary: Generating rich and controllable motion is a pivotal challenge in video
  synthesis. We propose Boximator, a new approach for fine-grained motion
  control. Boximator introduces two constraint types: hard box and soft box.
  Users select objects in the conditional frame using hard boxes and then use
  either type of boxes to roughly or rigorously define the object's position,
  shape, or motion path in future frames. Boximator functions as a plug-in for
  existing video diffusion models. Its training process preserves the base
  model's knowledge by freezing the original weights and training only the
  control module. To address training challenges, we introduce a novel
  self-tracking technique that greatly simplifies the learning of box-object
  correlations. Empirically, Boximator achieves state-of-the-art video quality
  (FVD) scores, improving on two base models, and further enhanced after
  incorporating box constraints. Its robust motion controllability is validated
  by drastic increases in the bounding box alignment metric. Human evaluation
  also shows that users favor Boximator generation results over the base model.
paper_authors._id: 65c06e55987c4be8a5379348
paper_authors.name: Jiawei Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-15T21:21:29.998Z
paper_authors._id: 6788122301d5ba1d3eff23ba
paper_authors.avatarUrl: /avatars/203dc8e1d542be55ea16eafcd7f396ff.svg
paper_authors.fullname: Jiawei Wang
paper_authors.type: user
paper_authors.user: 0nejiawei
paper_authors._id: 65c06e55987c4be8a5379349
paper_authors.name: Yuchen Zhang
paper_authors._id: 65c06e55987c4be8a537934a
paper_authors.name: Jiaxin Zou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-05T12:15:20.773Z
paper_authors._id: 655c485e71ed242b42701fb0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/655c485e71ed242b42701fb0/7d6lHB-wMy7xNj3dWIp5y.jpeg
paper_authors.fullname: Jiaxin Zou
paper_authors.type: user
paper_authors.user: shayeboshi
paper_authors._id: 65c06e55987c4be8a537934b
paper_authors.name: Yan Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-05T12:15:54.454Z
paper_authors._id: 640ef2b2f2d7c41a1e9b6013
paper_authors.avatarUrl: /avatars/556263c07df93942c22cda64e65b2e41.svg
paper_authors.fullname: Yan Zeng
paper_authors.type: user
paper_authors.user: zengyan-97
paper_authors._id: 65c06e55987c4be8a537934c
paper_authors.name: Guoqiang Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-05T12:16:00.898Z
paper_authors._id: 63160a1d29411a6864b0e585
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1668083049513-63160a1d29411a6864b0e585.jpeg
paper_authors.fullname: Guoqiang Wei
paper_authors.type: user
paper_authors.user: weigq
paper_authors._id: 65c06e55987c4be8a537934d
paper_authors.name: Liping Yuan
paper_authors._id: 65c06e55987c4be8a537934e
paper_authors.name: Hang Li
paper_title: Boximator: Generating Rich and Controllable Motions for Video Synthesis
paper_summary: Generating rich and controllable motion is a pivotal challenge in video
  synthesis. We propose Boximator, a new approach for fine-grained motion
  control. Boximator introduces two constraint types: hard box and soft box.
  Users select objects in the conditional frame using hard boxes and then use
  either type of boxes to roughly or rigorously define the object's position,
  shape, or motion path in future frames. Boximator functions as a plug-in for
  existing video diffusion models. Its training process preserves the base
  model's knowledge by freezing the original weights and training only the
  control module. To address training challenges, we introduce a novel
  self-tracking technique that greatly simplifies the learning of box-object
  correlations. Empirically, Boximator achieves state-of-the-art video quality
  (FVD) scores, improving on two base models, and further enhanced after
  incorporating box constraints. Its robust motion controllability is validated
  by drastic increases in the bounding box alignment metric. Human evaluation
  also shows that users favor Boximator generation results over the base model.
paper_ai_keywords: Boximator hard box soft box conditional frame video diffusion models control module self-tracking technique FVD bounding box alignment