title: IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and
    Illuminations
summary: Capturing geometric and material information from images remains a
  fundamental challenge in computer vision and graphics. Traditional
  optimization-based methods often require hours of computational time to
  reconstruct geometry, material properties, and environmental lighting from
  dense multi-view inputs, while still struggling with inherent ambiguities
  between lighting and material. On the other hand, learning-based approaches
  leverage rich material priors from existing 3D object datasets but face
  challenges with maintaining multi-view consistency. In this paper, we introduce
  IDArb, a diffusion-based model designed to perform intrinsic decomposition on
  an arbitrary number of images under varying illuminations. Our method achieves
  accurate and multi-view consistent estimation on surface normals and material
  properties. This is made possible through a novel cross-view, cross-domain
  attention module and an illumination-augmented, view-adaptive training
  strategy. Additionally, we introduce ARB-Objaverse, a new dataset that provides
  large-scale multi-view intrinsic data and renderings under diverse lighting
  conditions, supporting robust training. Extensive experiments demonstrate that
  IDArb outperforms state-of-the-art methods both qualitatively and
  quantitatively. Moreover, our approach facilitates a range of downstream tasks,
  including single-image relighting, photometric stereo, and 3D reconstruction,
  highlighting its broad applications in realistic 3D content creation.
paper_authors._id: 6760ee8b8cd4d1c2b6f45f23
paper_authors.name: Zhibing Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-17T08:03:46.852Z
paper_authors._id: 632a80706813868fa4a649e3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/632a80706813868fa4a649e3/MbTsAYGadNwS3-G5vxEnm.jpeg
paper_authors.fullname: Zhibing LI
paper_authors.type: user
paper_authors.user: lizb6626
paper_authors._id: 6760ee8b8cd4d1c2b6f45f24
paper_authors.name: Tong Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:44:25.382Z
paper_authors._id: 62fae9328e137d7c4b896498
paper_authors.avatarUrl: /avatars/1bda39dec585c099417cc9daa9f53c42.svg
paper_authors.fullname: Tong Wu
paper_authors.type: user
paper_authors.user: tongwu2020
paper_authors._id: 6760ee8b8cd4d1c2b6f45f25
paper_authors.name: Jing Tan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:44:07.182Z
paper_authors._id: 65367c40061949598892dbdc
paper_authors.avatarUrl: /avatars/4baf27263841471cbd5f629a8b99424d.svg
paper_authors.fullname: Jing Tan
paper_authors.type: user
paper_authors.user: jingtan
paper_authors._id: 6760ee8b8cd4d1c2b6f45f26
paper_authors.name: Mengchen Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:43:09.687Z
paper_authors._id: 64de20c5808492ba6e65d124
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64de20c5808492ba6e65d124/58IX_TI5vJw73qS1knw56.jpeg
paper_authors.fullname: Zhang Mengchen
paper_authors.type: user
paper_authors.user: Dubhe-zmc
paper_authors._id: 6760ee8b8cd4d1c2b6f45f27
paper_authors.name: Jiaqi Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:42:54.901Z
paper_authors._id: 64638c4d51fa6e63060521b5
paper_authors.avatarUrl: /avatars/c863ace5b1dc788a341bcf4ddbdfaec1.svg
paper_authors.fullname: JIaqi
paper_authors.type: user
paper_authors.user: Jiaqiwang
paper_authors._id: 6760ee8b8cd4d1c2b6f45f28
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-12-17T09:42:07.153Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_title: IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and
    Illuminations
paper_summary: Capturing geometric and material information from images remains a
  fundamental challenge in computer vision and graphics. Traditional
  optimization-based methods often require hours of computational time to
  reconstruct geometry, material properties, and environmental lighting from
  dense multi-view inputs, while still struggling with inherent ambiguities
  between lighting and material. On the other hand, learning-based approaches
  leverage rich material priors from existing 3D object datasets but face
  challenges with maintaining multi-view consistency. In this paper, we introduce
  IDArb, a diffusion-based model designed to perform intrinsic decomposition on
  an arbitrary number of images under varying illuminations. Our method achieves
  accurate and multi-view consistent estimation on surface normals and material
  properties. This is made possible through a novel cross-view, cross-domain
  attention module and an illumination-augmented, view-adaptive training
  strategy. Additionally, we introduce ARB-Objaverse, a new dataset that provides
  large-scale multi-view intrinsic data and renderings under diverse lighting
  conditions, supporting robust training. Extensive experiments demonstrate that
  IDArb outperforms state-of-the-art methods both qualitatively and
  quantitatively. Moreover, our approach facilitates a range of downstream tasks,
  including single-image relighting, photometric stereo, and 3D reconstruction,
  highlighting its broad applications in realistic 3D content creation.
paper_ai_keywords: diffusion-based model intrinsic decomposition surface normals material properties cross-view, cross-domain attention module illumination-augmented, view-adaptive training strategy ARB-Objaverse single-image relighting photometric stereo 3D reconstruction