title: Move-in-2D: 2D-Conditioned Human Motion Generation
summary: Generating realistic human videos remains a challenging task, with the most
  effective methods currently relying on a human motion sequence as a control
  signal. Existing approaches often use existing motion extracted from other
  videos, which restricts applications to specific motion types and global scene
  matching. We propose Move-in-2D, a novel approach to generate human motion
  sequences conditioned on a scene image, allowing for diverse motion that adapts
  to different scenes. Our approach utilizes a diffusion model that accepts both
  a scene image and text prompt as inputs, producing a motion sequence tailored
  to the scene. To train this model, we collect a large-scale video dataset
  featuring single-human activities, annotating each video with the corresponding
  human motion as the target output. Experiments demonstrate that our method
  effectively predicts human motion that aligns with the scene image after
  projection. Furthermore, we show that the generated motion sequence improves
  human motion quality in video synthesis tasks.
paper_authors._id: 6762c4d82faaf11234a44936
paper_authors.name: Hsin-Ping Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-19T18:02:01.287Z
paper_authors._id: 667b2ee8e005e1dbcc76e2e2
paper_authors.avatarUrl: /avatars/8b2f5f997f0ed5ae9f4f274941933c40.svg
paper_authors.fullname: Hsin-Ping Huang
paper_authors.type: user
paper_authors.user: hsinh
paper_authors._id: 6762c4d82faaf11234a44937
paper_authors.name: Yang Zhou
paper_authors._id: 6762c4d82faaf11234a44938
paper_authors.name: Jui-Hsien Wang
paper_authors._id: 6762c4d82faaf11234a44939
paper_authors.name: Difan Liu
paper_authors._id: 6762c4d82faaf11234a4493a
paper_authors.name: Feng Liu
paper_authors._id: 6762c4d82faaf11234a4493b
paper_authors.name: Ming-Hsuan Yang
paper_authors._id: 6762c4d82faaf11234a4493c
paper_authors.name: Zhan Xu
paper_title: Move-in-2D: 2D-Conditioned Human Motion Generation
paper_summary: Generating realistic human videos remains a challenging task, with the most
  effective methods currently relying on a human motion sequence as a control
  signal. Existing approaches often use existing motion extracted from other
  videos, which restricts applications to specific motion types and global scene
  matching. We propose Move-in-2D, a novel approach to generate human motion
  sequences conditioned on a scene image, allowing for diverse motion that adapts
  to different scenes. Our approach utilizes a diffusion model that accepts both
  a scene image and text prompt as inputs, producing a motion sequence tailored
  to the scene. To train this model, we collect a large-scale video dataset
  featuring single-human activities, annotating each video with the corresponding
  human motion as the target output. Experiments demonstrate that our method
  effectively predicts human motion that aligns with the scene image after
  projection. Furthermore, we show that the generated motion sequence improves
  human motion quality in video synthesis tasks.
paper_ai_summary: A diffusion model generates human motion sequences conditioned on scene images and text prompts, showing improved quality in video synthesis.
paper_ai_keywords: diffusion model scene image text prompt human motion sequences video dataset single-human activities