title: AIM 2024 Sparse Neural Rendering Challenge: Dataset and Benchmark
summary: Recent developments in differentiable and neural rendering have made
  impressive breakthroughs in a variety of 2D and 3D tasks, e.g. novel view
  synthesis, 3D reconstruction. Typically, differentiable rendering relies on a
  dense viewpoint coverage of the scene, such that the geometry can be
  disambiguated from appearance observations alone. Several challenges arise when
  only a few input views are available, often referred to as sparse or few-shot
  neural rendering. As this is an underconstrained problem, most existing
  approaches introduce the use of regularisation, together with a diversity of
  learnt and hand-crafted priors. A recurring problem in sparse rendering
  literature is the lack of an homogeneous, up-to-date, dataset and evaluation
  protocol. While high-resolution datasets are standard in dense reconstruction
  literature, sparse rendering methods often evaluate with low-resolution images.
  Additionally, data splits are inconsistent across different manuscripts, and
  testing ground-truth images are often publicly available, which may lead to
  over-fitting. In this work, we propose the Sparse Rendering (SpaRe) dataset and
  benchmark. We introduce a new dataset that follows the setup of the DTU MVS
  dataset. The dataset is composed of 97 new scenes based on synthetic,
  high-quality assets. Each scene has up to 64 camera views and 7 lighting
  configurations, rendered at 1600x1200 resolution. We release a training split
  of 82 scenes to foster generalizable approaches, and provide an online
  evaluation platform for the validation and test sets, whose ground-truth images
  remain hidden. We propose two different sparse configurations (3 and 9 input
  images respectively). This provides a powerful and convenient tool for
  reproducible evaluation, and enable researchers easy access to a public
  leaderboard with the state-of-the-art performance scores. Available at:
  https://sparebenchmark.github.io/
paper_authors._id: 66f449bbf0eb32a95335e521
paper_authors.name: Michal Nazarczuk
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-09-25T18:24:07.881Z
paper_authors._id: 66f44a3df9252e0f50b59fdb
paper_authors.avatarUrl: /avatars/4234203c77a0e6f594f3de26bfe8c649.svg
paper_authors.fullname: Michal Nazarczuk
paper_authors.type: user
paper_authors.user: michaal94
paper_authors._id: 66f449bbf0eb32a95335e522
paper_authors.name: Thomas Tanay
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-26T09:41:51.930Z
paper_authors._id: 65de232d398a8e9771c63c60
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65de232d398a8e9771c63c60/mw0szrwxxyAwq8fmAht7u.jpeg
paper_authors.fullname: Thomas Tanay
paper_authors.type: user
paper_authors.user: thomas-tanay
paper_authors._id: 66f449bbf0eb32a95335e523
paper_authors.name: Sibi Catley-Chandar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-09-26T09:41:57.714Z
paper_authors._id: 66f42a618547c4e49394655c
paper_authors.avatarUrl: /avatars/094312c81e64eabf8c97f9cd643e1136.svg
paper_authors.fullname: Sibi Catley-Chandar
paper_authors.type: user
paper_authors.user: sibicatleychandar
paper_authors._id: 66f449bbf0eb32a95335e524
paper_authors.name: Richard Shaw
paper_authors._id: 66f449bbf0eb32a95335e525
paper_authors.name: Radu Timofte
paper_authors._id: 66f449bbf0eb32a95335e526
paper_authors.name: Eduardo Pérez-Pellitero
paper_title: AIM 2024 Sparse Neural Rendering Challenge: Dataset and Benchmark
paper_summary: Recent developments in differentiable and neural rendering have made
  impressive breakthroughs in a variety of 2D and 3D tasks, e.g. novel view
  synthesis, 3D reconstruction. Typically, differentiable rendering relies on a
  dense viewpoint coverage of the scene, such that the geometry can be
  disambiguated from appearance observations alone. Several challenges arise when
  only a few input views are available, often referred to as sparse or few-shot
  neural rendering. As this is an underconstrained problem, most existing
  approaches introduce the use of regularisation, together with a diversity of
  learnt and hand-crafted priors. A recurring problem in sparse rendering
  literature is the lack of an homogeneous, up-to-date, dataset and evaluation
  protocol. While high-resolution datasets are standard in dense reconstruction
  literature, sparse rendering methods often evaluate with low-resolution images.
  Additionally, data splits are inconsistent across different manuscripts, and
  testing ground-truth images are often publicly available, which may lead to
  over-fitting. In this work, we propose the Sparse Rendering (SpaRe) dataset and
  benchmark. We introduce a new dataset that follows the setup of the DTU MVS
  dataset. The dataset is composed of 97 new scenes based on synthetic,
  high-quality assets. Each scene has up to 64 camera views and 7 lighting
  configurations, rendered at 1600x1200 resolution. We release a training split
  of 82 scenes to foster generalizable approaches, and provide an online
  evaluation platform for the validation and test sets, whose ground-truth images
  remain hidden. We propose two different sparse configurations (3 and 9 input
  images respectively). This provides a powerful and convenient tool for
  reproducible evaluation, and enable researchers easy access to a public
  leaderboard with the state-of-the-art performance scores. Available at:
  https://sparebenchmark.github.io/
paper_ai_keywords: differentiable rendering neural rendering novel view synthesis 3D reconstruction sparse neural rendering few-shot neural rendering regularisation learnt priors hand-crafted priors sparse rendering DTU M dataset