title: NU-MCC: Multiview Compressive Coding with Neighborhood Decoder and
    Repulsive UDF
summary: Remarkable progress has been made in 3D reconstruction from single-view RGB-D
  inputs. MCC is the current state-of-the-art method in this field, which
  achieves unprecedented success by combining vision Transformers with
  large-scale training. However, we identified two key limitations of MCC: 1) The
  Transformer decoder is inefficient in handling large number of query points; 2)
  The 3D representation struggles to recover high-fidelity details. In this
  paper, we propose a new approach called NU-MCC that addresses these
  limitations. NU-MCC includes two key innovations: a Neighborhood decoder and a
  Repulsive Unsigned Distance Function (Repulsive UDF). First, our Neighborhood
  decoder introduces center points as an efficient proxy of input visual
  features, allowing each query point to only attend to a small neighborhood.
  This design not only results in much faster inference speed but also enables
  the exploitation of finer-scale visual features for improved recovery of 3D
  textures. Second, our Repulsive UDF is a novel alternative to the occupancy
  field used in MCC, significantly improving the quality of 3D object
  reconstruction. Compared to standard UDFs that suffer from holes in results,
  our proposed Repulsive UDF can achieve more complete surface reconstruction.
  Experimental results demonstrate that NU-MCC is able to learn a strong 3D
  representation, significantly advancing the state of the art in single-view 3D
  reconstruction. Particularly, it outperforms MCC by 9.7% in terms of the
  F1-score on the CO3D-v2 dataset with more than 5x faster running speed.
paper_authors._id: 64b75b77da8017900e776d36
paper_authors.name: Stefan Lionar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-19T14:39:15.873Z
paper_authors._id: 63bfb9a54ed1b8fd6883524c
paper_authors.avatarUrl: /avatars/5462264a8cf58cb15fcc5194336d45ac.svg
paper_authors.fullname: Stefan Lionar
paper_authors.type: user
paper_authors.user: slionar
paper_authors._id: 64b75b77da8017900e776d37
paper_authors.name: Xiangyu Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-19T14:39:12.284Z
paper_authors._id: 634e60454677a5891c0902f4
paper_authors.avatarUrl: /avatars/4dc143719afe7686e05b7f2c2c5c1871.svg
paper_authors.fullname: Xiangyu Xu
paper_authors.type: user
paper_authors.user: xjcvcvxj
paper_authors._id: 64b75b77da8017900e776d38
paper_authors.name: Min Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-22T12:01:11.551Z
paper_authors._id: 6565d5fd656151d58b1d9afb
paper_authors.avatarUrl: /avatars/0c416ea74347c6b5bd27f49aec309c48.svg
paper_authors.fullname: Min Lin
paper_authors.type: user
paper_authors.user: mavenlin
paper_authors._id: 64b75b77da8017900e776d39
paper_authors.name: Gim Hee Lee
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-24T07:44:04.007Z
paper_authors._id: 64bc7568b7375f6b84503ce2
paper_authors.avatarUrl: /avatars/d970c9990d9d84ac51ff225c1b26e304.svg
paper_authors.fullname: Gim Hee Lee
paper_authors.type: user
paper_authors.user: gimhee-lee
paper_title: NU-MCC: Multiview Compressive Coding with Neighborhood Decoder and
    Repulsive UDF
paper_summary: Remarkable progress has been made in 3D reconstruction from single-view RGB-D
  inputs. MCC is the current state-of-the-art method in this field, which
  achieves unprecedented success by combining vision Transformers with
  large-scale training. However, we identified two key limitations of MCC: 1) The
  Transformer decoder is inefficient in handling large number of query points; 2)
  The 3D representation struggles to recover high-fidelity details. In this
  paper, we propose a new approach called NU-MCC that addresses these
  limitations. NU-MCC includes two key innovations: a Neighborhood decoder and a
  Repulsive Unsigned Distance Function (Repulsive UDF). First, our Neighborhood
  decoder introduces center points as an efficient proxy of input visual
  features, allowing each query point to only attend to a small neighborhood.
  This design not only results in much faster inference speed but also enables
  the exploitation of finer-scale visual features for improved recovery of 3D
  textures. Second, our Repulsive UDF is a novel alternative to the occupancy
  field used in MCC, significantly improving the quality of 3D object
  reconstruction. Compared to standard UDFs that suffer from holes in results,
  our proposed Repulsive UDF can achieve more complete surface reconstruction.
  Experimental results demonstrate that NU-MCC is able to learn a strong 3D
  representation, significantly advancing the state of the art in single-view 3D
  reconstruction. Particularly, it outperforms MCC by 9.7% in terms of the
  F1-score on the CO3D-v2 dataset with more than 5x faster running speed.
paper_ai_keywords: vision Transformers Transformer decoder Neighborhood decoder Repulsive Unsigned Distance Function (Repulsive UDF) center points occupancy field standard UDFs CO3D-v2 dataset