title: Understanding 3D Object Interaction from a Single Image
summary: Humans can easily understand a single image as depicting multiple potential
  objects permitting interaction. We use this skill to plan our interactions with
  the world and accelerate understanding new objects without engaging in
  interaction. In this paper, we would like to endow machines with the similar
  ability, so that intelligent agents can better explore the 3D scene or
  manipulate objects. Our approach is a transformer-based model that predicts the
  3D location, physical properties and affordance of objects. To power this
  model, we collect a dataset with Internet videos, egocentric videos and indoor
  images to train and validate our approach. Our model yields strong performance
  on our data, and generalizes well to robotics data.
paper_authors._id: 64643b887572c66a8e64f216
paper_authors.name: Shengyi Qian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-19T10:27:55.711Z
paper_authors._id: 6445dc7e219dc6142a6a6861
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6445dc7e219dc6142a6a6861/Ak_YV8V1NZpkF5AQnJVlJ.png
paper_authors.fullname: Shengyi Qian
paper_authors.type: user
paper_authors.user: shengyi-qian
paper_authors._id: 64643b887572c66a8e64f217
paper_authors.name: David F. Fouhey
paper_title: Understanding 3D Object Interaction from a Single Image
paper_summary: Humans can easily understand a single image as depicting multiple potential
  objects permitting interaction. We use this skill to plan our interactions with
  the world and accelerate understanding new objects without engaging in
  interaction. In this paper, we would like to endow machines with the similar
  ability, so that intelligent agents can better explore the 3D scene or
  manipulate objects. Our approach is a transformer-based model that predicts the
  3D location, physical properties and affordance of objects. To power this
  model, we collect a dataset with Internet videos, egocentric videos and indoor
  images to train and validate our approach. Our model yields strong performance
  on our data, and generalizes well to robotics data.
paper_ai_keywords: transformer-based model 3D location physical properties affordance Internet videos egocentric videos indoor images robotics data