title: ReCamMaster: Camera-Controlled Generative Rendering from A Single Video
summary: Camera control has been actively studied in text or image conditioned video
  generation tasks. However, altering camera trajectories of a given video
  remains under-explored, despite its importance in the field of video creation.
  It is non-trivial due to the extra constraints of maintaining multiple-frame
  appearance and dynamic synchronization. To address this, we present
  ReCamMaster, a camera-controlled generative video re-rendering framework that
  reproduces the dynamic scene of an input video at novel camera trajectories.
  The core innovation lies in harnessing the generative capabilities of
  pre-trained text-to-video models through a simple yet powerful video
  conditioning mechanism -- its capability often overlooked in current research.
  To overcome the scarcity of qualified training data, we construct a
  comprehensive multi-camera synchronized video dataset using Unreal Engine 5,
  which is carefully curated to follow real-world filming characteristics,
  covering diverse scenes and camera movements. It helps the model generalize to
  in-the-wild videos. Lastly, we further improve the robustness to diverse inputs
  through a meticulously designed training strategy. Extensive experiments tell
  that our method substantially outperforms existing state-of-the-art approaches
  and strong baselines. Our method also finds promising applications in video
  stabilization, super-resolution, and outpainting. Project page:
  https://jianhongbai.github.io/ReCamMaster/
paper_authors._id: 67d785fa473d4edd330edee1
paper_authors.name: Jianhong Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:47:22.245Z
paper_authors._id: 6530bf50f145530101ec03a2
paper_authors.avatarUrl: /avatars/c61c00c314cf202b64968e51e855694d.svg
paper_authors.fullname: Jianhong Bai
paper_authors.type: user
paper_authors.user: jianhongbai
paper_authors._id: 67d785fa473d4edd330edee2
paper_authors.name: Menghan Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:47:41.792Z
paper_authors._id: 63401c89f81b9d101361f712
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665146415483-63401c89f81b9d101361f712.png
paper_authors.fullname: Richard
paper_authors.type: user
paper_authors.user: menghanxia
paper_authors._id: 67d785fa473d4edd330edee3
paper_authors.name: Xiao Fu
paper_authors._id: 67d785fa473d4edd330edee4
paper_authors.name: Xintao Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:47:51.145Z
paper_authors._id: 60e272ca6c78a8c122b12127
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/60e272ca6c78a8c122b12127/xldEGBzGrU-bX6IwAw0Ie.jpeg
paper_authors.fullname: Xintao Wang
paper_authors.type: user
paper_authors.user: Xintao
paper_authors._id: 67d785fa473d4edd330edee5
paper_authors.name: Lianrui Mu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:47:58.483Z
paper_authors._id: 6672dd6d239ba86f129c5384
paper_authors.avatarUrl: /avatars/6209afb551995b12d5e0d4d95e495694.svg
paper_authors.fullname: Lianrui Mu
paper_authors.type: user
paper_authors.user: Mu437
paper_authors._id: 67d785fa473d4edd330edee6
paper_authors.name: Jinwen Cao
paper_authors._id: 67d785fa473d4edd330edee7
paper_authors.name: Zuozhu Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:48:17.243Z
paper_authors._id: 6458b8d0990172cd1d703715
paper_authors.avatarUrl: /avatars/55f0695e3cb9933c3903fde5a8f740d5.svg
paper_authors.fullname: Zuozhu Liu
paper_authors.type: user
paper_authors.user: Zuozhu
paper_authors._id: 67d785fa473d4edd330edee8
paper_authors.name: Haoji Hu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:48:23.846Z
paper_authors._id: 66c46129d67297a9b93e03c5
paper_authors.avatarUrl: /avatars/cffd8b07fa3655e240efc8e81f99d97d.svg
paper_authors.fullname: Haoji Hu
paper_authors.type: user
paper_authors.user: garland1979
paper_authors._id: 67d785fa473d4edd330edee9
paper_authors.name: Xiang Bai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:48:29.804Z
paper_authors._id: 641790e2f1e86908935d82a0
paper_authors.avatarUrl: /avatars/ced7a137c6344c74b7ac0d5c84833fc8.svg
paper_authors.fullname: Xiang Bai
paper_authors.type: user
paper_authors.user: baixianger
paper_authors._id: 67d785fa473d4edd330edeea
paper_authors.name: Pengfei Wan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-17T14:42:32.134Z
paper_authors._id: 662f93942510ef5735d7ad00
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/662f93942510ef5735d7ad00/ZIDIPm63sncIHFTT5b0uR.png
paper_authors.fullname: magicwpf
paper_authors.type: user
paper_authors.user: magicwpf
paper_authors._id: 67d785fa473d4edd330edeeb
paper_authors.name: Di Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-17T08:48:49.559Z
paper_authors._id: 644c8324f02250233d0d67d9
paper_authors.avatarUrl: /avatars/feb39d281457c1750f3eada3c060a23e.svg
paper_authors.fullname: Di Zhang
paper_authors.type: user
paper_authors.user: dizhang
paper_title: ReCamMaster: Camera-Controlled Generative Rendering from A Single Video
paper_summary: Camera control has been actively studied in text or image conditioned video
  generation tasks. However, altering camera trajectories of a given video
  remains under-explored, despite its importance in the field of video creation.
  It is non-trivial due to the extra constraints of maintaining multiple-frame
  appearance and dynamic synchronization. To address this, we present
  ReCamMaster, a camera-controlled generative video re-rendering framework that
  reproduces the dynamic scene of an input video at novel camera trajectories.
  The core innovation lies in harnessing the generative capabilities of
  pre-trained text-to-video models through a simple yet powerful video
  conditioning mechanism -- its capability often overlooked in current research.
  To overcome the scarcity of qualified training data, we construct a
  comprehensive multi-camera synchronized video dataset using Unreal Engine 5,
  which is carefully curated to follow real-world filming characteristics,
  covering diverse scenes and camera movements. It helps the model generalize to
  in-the-wild videos. Lastly, we further improve the robustness to diverse inputs
  through a meticulously designed training strategy. Extensive experiments tell
  that our method substantially outperforms existing state-of-the-art approaches
  and strong baselines. Our method also finds promising applications in video
  stabilization, super-resolution, and outpainting. Project page:
  https://jianhongbai.github.io/ReCamMaster/
paper_ai_summary: ReCamMaster uses pre-trained text-to-video models to render dynamic scenes of input videos from novel camera trajectories, leveraging a unique video conditioning mechanism and a custom dataset created with Unreal Engine 5.
paper_ai_keywords: camera-controlled generative video re-rendering pre-trained text-to-video models video conditioning mechanism Unreal Engine 5 video stabilization super-resolution outpainting