title: Sonata: Self-Supervised Learning of Reliable Point Representations
summary: In this paper, we question whether we have a reliable self-supervised point
  cloud model that can be used for diverse 3D tasks via simple linear probing,
  even with limited data and minimal computation. We find that existing 3D
  self-supervised learning approaches fall short when evaluated on representation
  quality through linear probing. We hypothesize that this is due to what we term
  the "geometric shortcut", which causes representations to collapse to low-level
  spatial features. This challenge is unique to 3D and arises from the sparse
  nature of point cloud data. We address it through two key strategies: obscuring
  spatial information and enhancing the reliance on input features, ultimately
  composing a Sonata of 140k point clouds through self-distillation. Sonata is
  simple and intuitive, yet its learned representations are strong and reliable:
  zero-shot visualizations demonstrate semantic grouping, alongside strong
  spatial reasoning through nearest-neighbor relationships. Sonata demonstrates
  exceptional parameter and data efficiency, tripling linear probing accuracy
  (from 21.8% to 72.5%) on ScanNet and nearly doubling performance with only 1%
  of the data compared to previous approaches. Full fine-tuning further advances
  SOTA across both 3D indoor and outdoor perception tasks.
paper_authors._id: 67dd5935b4cce80ee64cd98f
paper_authors.name: Xiaoyang Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-21T12:46:08.995Z
paper_authors._id: 643e5d6a1d0e956d94bb3608
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/643e5d6a1d0e956d94bb3608/mTnu7Dts2RmDklHlp9Gqu.jpeg
paper_authors.fullname: Xiaoyang Wu
paper_authors.type: user
paper_authors.user: Gofinge
paper_authors._id: 67dd5935b4cce80ee64cd990
paper_authors.name: Daniel DeTone
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:09:34.106Z
paper_authors._id: 66f1e84ea4d3e85a77bdb545
paper_authors.avatarUrl: /avatars/4e24b0bdd7a8c86ba5f43f4b042817ea.svg
paper_authors.fullname: Daniel DeTone
paper_authors.type: user
paper_authors.user: ddetone
paper_authors._id: 67dd5935b4cce80ee64cd991
paper_authors.name: Duncan Frost
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:09:54.292Z
paper_authors._id: 65fc12c6fc9132a2dfe9e8b3
paper_authors.avatarUrl: /avatars/3d04d79721f4ec6b0b090e0bbbf85f70.svg
paper_authors.fullname: Duncan Frost
paper_authors.type: user
paper_authors.user: frosd01
paper_authors._id: 67dd5935b4cce80ee64cd992
paper_authors.name: Tianwei Shen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:10:00.797Z
paper_authors._id: 66e0950be1bfac4b463fb941
paper_authors.avatarUrl: /avatars/6506d169af3ead9b7d7d30a1013857b1.svg
paper_authors.fullname: Tianwei Shen
paper_authors.type: user
paper_authors.user: tshenaa
paper_authors._id: 67dd5935b4cce80ee64cd993
paper_authors.name: Chris Xie
paper_authors._id: 67dd5935b4cce80ee64cd994
paper_authors.name: Nan Yang
paper_authors._id: 67dd5935b4cce80ee64cd995
paper_authors.name: Jakob Engel
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:10:15.646Z
paper_authors._id: 62861d244713da0266f29f99
paper_authors.avatarUrl: /avatars/819abb2f7d02d65cbaa4a571b8b13169.svg
paper_authors.fullname: Jakob Engel
paper_authors.type: user
paper_authors.user: JakobEngel
paper_authors._id: 67dd5935b4cce80ee64cd996
paper_authors.name: Richard Newcombe
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:10:22.340Z
paper_authors._id: 633ea8814b469e3151bb38f2
paper_authors.avatarUrl: /avatars/245f849cde665f1d7dfec15dead6b880.svg
paper_authors.fullname: Richard Newcombe
paper_authors.type: user
paper_authors.user: MONGIE30
paper_authors._id: 67dd5935b4cce80ee64cd997
paper_authors.name: Hengshuang Zhao
paper_authors._id: 67dd5935b4cce80ee64cd998
paper_authors.name: Julian Straub
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:10:37.008Z
paper_authors._id: 673927a0ab4212713230d456
paper_authors.avatarUrl: /avatars/b90972cefa847649a6acc47e1f100b3b.svg
paper_authors.fullname: Julian Straub
paper_authors.type: user
paper_authors.user: jstraub
paper_title: Sonata: Self-Supervised Learning of Reliable Point Representations
paper_summary: In this paper, we question whether we have a reliable self-supervised point
  cloud model that can be used for diverse 3D tasks via simple linear probing,
  even with limited data and minimal computation. We find that existing 3D
  self-supervised learning approaches fall short when evaluated on representation
  quality through linear probing. We hypothesize that this is due to what we term
  the "geometric shortcut", which causes representations to collapse to low-level
  spatial features. This challenge is unique to 3D and arises from the sparse
  nature of point cloud data. We address it through two key strategies: obscuring
  spatial information and enhancing the reliance on input features, ultimately
  composing a Sonata of 140k point clouds through self-distillation. Sonata is
  simple and intuitive, yet its learned representations are strong and reliable:
  zero-shot visualizations demonstrate semantic grouping, alongside strong
  spatial reasoning through nearest-neighbor relationships. Sonata demonstrates
  exceptional parameter and data efficiency, tripling linear probing accuracy
  (from 21.8% to 72.5%) on ScanNet and nearly doubling performance with only 1%
  of the data compared to previous approaches. Full fine-tuning further advances
  SOTA across both 3D indoor and outdoor perception tasks.
paper_ai_summary: The paper investigates the effectiveness of self-supervised 3D point cloud models, addressing the "geometric shortcut" issue through spatial obscuring and feature enhancement, resulting in strong parameter and data efficient representations.
paper_ai_keywords: self-supervised learning point cloud data linear probing geometric shortcut spatial features self-distillation nearest-neighbor relationships ScanNet 3D indoor perception 3D outdoor perception