title: NIFTY: Neural Object Interaction Fields for Guided Human Motion
    Synthesis
summary: We address the problem of generating realistic 3D motions of humans
  interacting with objects in a scene. Our key idea is to create a neural
  interaction field attached to a specific object, which outputs the distance to
  the valid interaction manifold given a human pose as input. This interaction
  field guides the sampling of an object-conditioned human motion diffusion
  model, so as to encourage plausible contacts and affordance semantics. To
  support interactions with scarcely available data, we propose an automated
  synthetic data pipeline. For this, we seed a pre-trained motion model, which
  has priors for the basics of human movement, with interaction-specific anchor
  poses extracted from limited motion capture data. Using our guided diffusion
  model trained on generated synthetic data, we synthesize realistic motions for
  sitting and lifting with several objects, outperforming alternative approaches
  in terms of motion quality and successful action completion. We call our
  framework NIFTY: Neural Interaction Fields for Trajectory sYnthesis.
paper_authors._id: 64b4ada9520fa3a154ff746d
paper_authors.name: Nilesh Kulkarni
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T11:10:42.289Z
paper_authors._id: 63a8e340353e10031a90cf40
paper_authors.avatarUrl: /avatars/eee97deed32e1ba366e2216dd159988b.svg
paper_authors.fullname: Nilesh Kulkarni
paper_authors.type: user
paper_authors.user: nileshk
paper_authors._id: 64b4ada9520fa3a154ff746e
paper_authors.name: Davis Rempe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-07-17T11:10:56.980Z
paper_authors._id: 62f9e6790697d2242198e08d
paper_authors.avatarUrl: /avatars/693792462687f5e2485bec9cd6bee16f.svg
paper_authors.fullname: Davis Rempe
paper_authors.type: user
paper_authors.user: drempe
paper_authors._id: 64b4ada9520fa3a154ff746f
paper_authors.name: Kyle Genova
paper_authors._id: 64b4ada9520fa3a154ff7470
paper_authors.name: Abhijit Kundu
paper_authors._id: 64b4ada9520fa3a154ff7471
paper_authors.name: Justin Johnson
paper_authors._id: 64b4ada9520fa3a154ff7472
paper_authors.name: David Fouhey
paper_authors._id: 64b4ada9520fa3a154ff7473
paper_authors.name: Leonidas Guibas
paper_title: NIFTY: Neural Object Interaction Fields for Guided Human Motion
    Synthesis
paper_summary: We address the problem of generating realistic 3D motions of humans
  interacting with objects in a scene. Our key idea is to create a neural
  interaction field attached to a specific object, which outputs the distance to
  the valid interaction manifold given a human pose as input. This interaction
  field guides the sampling of an object-conditioned human motion diffusion
  model, so as to encourage plausible contacts and affordance semantics. To
  support interactions with scarcely available data, we propose an automated
  synthetic data pipeline. For this, we seed a pre-trained motion model, which
  has priors for the basics of human movement, with interaction-specific anchor
  poses extracted from limited motion capture data. Using our guided diffusion
  model trained on generated synthetic data, we synthesize realistic motions for
  sitting and lifting with several objects, outperforming alternative approaches
  in terms of motion quality and successful action completion. We call our
  framework NIFTY: Neural Interaction Fields for Trajectory sYnthesis.
paper_ai_keywords: neural interaction field diffusion model interaction manifold human pose sampling affordance semantics automated synthetic data pipeline pre-trained motion model anchor poses motion quality action completion NIFTY: Neural Interaction Fields for Trajectory sYnthesis