title: The ObjectFolder Benchmark: Multisensory Learning with Neural and Real
    Objects
summary: We introduce the ObjectFolder Benchmark, a benchmark suite of 10 tasks for
  multisensory object-centric learning, centered around object recognition,
  reconstruction, and manipulation with sight, sound, and touch. We also
  introduce the ObjectFolder Real dataset, including the multisensory
  measurements for 100 real-world household objects, building upon a newly
  designed pipeline for collecting the 3D meshes, videos, impact sounds, and
  tactile readings of real-world objects. We conduct systematic benchmarking on
  both the 1,000 multisensory neural objects from ObjectFolder, and the real
  multisensory data from ObjectFolder Real. Our results demonstrate the
  importance of multisensory perception and reveal the respective roles of
  vision, audio, and touch for different object-centric learning tasks. By
  publicly releasing our dataset and benchmark suite, we hope to catalyze and
  enable new research in multisensory object-centric learning in computer vision,
  robotics, and beyond. Project page: https://objectfolder.stanford.edu
paper_authors._id: 64798ed74be04c76ce451b44
paper_authors.name: Ruohan Gao
paper_authors._id: 64798ed74be04c76ce451b45
paper_authors.name: Yiming Dou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-05T12:12:38.085Z
paper_authors._id: 63a29eb3f30c464227919797
paper_authors.avatarUrl: /avatars/6eb9332ab57ea36fd0d670e06983a319.svg
paper_authors.fullname: Yiming Dou
paper_authors.type: user
paper_authors.user: ymdou
paper_authors._id: 64798ed74be04c76ce451b46
paper_authors.name: Hao Li
paper_authors._id: 64798ed74be04c76ce451b47
paper_authors.name: Tanmay Agarwal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-02T12:46:37.789Z
paper_authors._id: 6479aa2d9763c4ef86ae0ff0
paper_authors.avatarUrl: /avatars/f8ff4a1f55fb9dd17e6527b2f5bb76a6.svg
paper_authors.fullname: Tanmay Agarwal
paper_authors.type: user
paper_authors.user: tanmayxagarwal
paper_authors._id: 64798ed74be04c76ce451b48
paper_authors.name: Jeannette Bohg
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-18T09:55:42.915Z
paper_authors._id: 676272a59865895e253ceee5
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/676272a59865895e253ceee5/Nz51Y-G49SD19lQEjH6Yd.jpeg
paper_authors.fullname: Jeannette Bohg
paper_authors.type: user
paper_authors.user: jbohg
paper_authors._id: 64798ed74be04c76ce451b49
paper_authors.name: Yunzhu Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-04T19:17:35.883Z
paper_authors._id: 64a46562c641afb468a3dc04
paper_authors.avatarUrl: /avatars/bc54e5c13f657251b05bf2725086e56d.svg
paper_authors.fullname: Yunzhu Li
paper_authors.type: user
paper_authors.user: yunzhuli
paper_authors._id: 64798ed74be04c76ce451b4a
paper_authors.name: Li Fei-Fei
paper_authors._id: 64798ed74be04c76ce451b4b
paper_authors.name: Jiajun Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-05T10:06:28.511Z
paper_authors._id: 647b7b8345616c3de3627863
paper_authors.avatarUrl: /avatars/8196364bffed71086ba9a0a4a62027d3.svg
paper_authors.fullname: Jiajun Wu
paper_authors.type: user
paper_authors.user: jiajunwu
paper_title: The ObjectFolder Benchmark: Multisensory Learning with Neural and Real
    Objects
paper_summary: We introduce the ObjectFolder Benchmark, a benchmark suite of 10 tasks for
  multisensory object-centric learning, centered around object recognition,
  reconstruction, and manipulation with sight, sound, and touch. We also
  introduce the ObjectFolder Real dataset, including the multisensory
  measurements for 100 real-world household objects, building upon a newly
  designed pipeline for collecting the 3D meshes, videos, impact sounds, and
  tactile readings of real-world objects. We conduct systematic benchmarking on
  both the 1,000 multisensory neural objects from ObjectFolder, and the real
  multisensory data from ObjectFolder Real. Our results demonstrate the
  importance of multisensory perception and reveal the respective roles of
  vision, audio, and touch for different object-centric learning tasks. By
  publicly releasing our dataset and benchmark suite, we hope to catalyze and
  enable new research in multisensory object-centric learning in computer vision,
  robotics, and beyond. Project page: https://objectfolder.stanford.edu
paper_ai_summary: A benchmark suite and real dataset for multisensory object-centric learning reveal the importance of vision, audio, and touch in various object tasks.
paper_ai_keywords: multisensory object-centric learning object recognition reconstruction manipulation ObjectFolder Benchmark ObjectFolder Real 3D meshes videos impact sounds tactile readings