title: Learning the 3D Fauna of the Web
summary: Learning 3D models of all animals on the Earth requires massively scaling up
  existing solutions. With this ultimate goal in mind, we develop 3D-Fauna, an
  approach that learns a pan-category deformable 3D animal model for more than
  100 animal species jointly. One crucial bottleneck of modeling animals is the
  limited availability of training data, which we overcome by simply learning
  from 2D Internet images. We show that prior category-specific attempts fail to
  generalize to rare species with limited training images. We address this
  challenge by introducing the Semantic Bank of Skinned Models (SBSM), which
  automatically discovers a small set of base animal shapes by combining
  geometric inductive priors with semantic knowledge implicitly captured by an
  off-the-shelf self-supervised feature extractor. To train such a model, we also
  contribute a new large-scale dataset of diverse animal species. At inference
  time, given a single image of any quadruped animal, our model reconstructs an
  articulated 3D mesh in a feed-forward fashion within seconds.
paper_authors._id: 65979bb1185db94370c57f40
paper_authors.name: Zizhang Li
paper_authors._id: 65979bb1185db94370c57f41
paper_authors.name: Dor Litvak
paper_authors._id: 65979bb1185db94370c57f42
paper_authors.name: Ruining Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-08T08:47:58.291Z
paper_authors._id: 6467cc17e7a6a374fd1a41e5
paper_authors.avatarUrl: /avatars/3cbd4ad5bc17067ee38b13d0cbadd80c.svg
paper_authors.fullname: Ruining Li
paper_authors.type: user
paper_authors.user: rayli
paper_authors._id: 65979bb1185db94370c57f43
paper_authors.name: Yunzhi Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-22T22:28:22.133Z
paper_authors._id: 63793aee2cb50dda39d6e73b
paper_authors.avatarUrl: /avatars/1d7eb1a19ce2e18802e9481200eb03f7.svg
paper_authors.fullname: Yunzhi Zhang
paper_authors.type: user
paper_authors.user: Yunzhi
paper_authors._id: 65979bb1185db94370c57f44
paper_authors.name: Tomas Jakab
paper_authors._id: 65979bb1185db94370c57f45
paper_authors.name: Christian Rupprecht
paper_authors._id: 65979bb1185db94370c57f46
paper_authors.name: Shangzhe Wu
paper_authors._id: 65979bb1185db94370c57f47
paper_authors.name: Andrea Vedaldi
paper_authors._id: 65979bb1185db94370c57f48
paper_authors.name: Jiajun Wu
paper_title: Learning the 3D Fauna of the Web
paper_summary: Learning 3D models of all animals on the Earth requires massively scaling up
  existing solutions. With this ultimate goal in mind, we develop 3D-Fauna, an
  approach that learns a pan-category deformable 3D animal model for more than
  100 animal species jointly. One crucial bottleneck of modeling animals is the
  limited availability of training data, which we overcome by simply learning
  from 2D Internet images. We show that prior category-specific attempts fail to
  generalize to rare species with limited training images. We address this
  challenge by introducing the Semantic Bank of Skinned Models (SBSM), which
  automatically discovers a small set of base animal shapes by combining
  geometric inductive priors with semantic knowledge implicitly captured by an
  off-the-shelf self-supervised feature extractor. To train such a model, we also
  contribute a new large-scale dataset of diverse animal species. At inference
  time, given a single image of any quadruped animal, our model reconstructs an
  articulated 3D mesh in a feed-forward fashion within seconds.
paper_ai_keywords: pan-category deformable 3D animal model Semantic Bank of Skinned Models (SBSM) geometric inductive priors self-supervised feature extractor articulated 3D mesh feed-forward fashion