title: FruitNeRF: A Unified Neural Radiance Field based Fruit Counting
    Framework
summary: We introduce FruitNeRF, a unified novel fruit counting framework that
  leverages state-of-the-art view synthesis methods to count any fruit type
  directly in 3D. Our framework takes an unordered set of posed images captured
  by a monocular camera and segments fruit in each image. To make our system
  independent of the fruit type, we employ a foundation model that generates
  binary segmentation masks for any fruit. Utilizing both modalities, RGB and
  semantic, we train a semantic neural radiance field. Through uniform volume
  sampling of the implicit Fruit Field, we obtain fruit-only point clouds. By
  applying cascaded clustering on the extracted point cloud, our approach
  achieves precise fruit count.The use of neural radiance fields provides
  significant advantages over conventional methods such as object tracking or
  optical flow, as the counting itself is lifted into 3D. Our method prevents
  double counting fruit and avoids counting irrelevant fruit.We evaluate our
  methodology using both real-world and synthetic datasets. The real-world
  dataset consists of three apple trees with manually counted ground truths, a
  benchmark apple dataset with one row and ground truth fruit location, while the
  synthetic dataset comprises various fruit types including apple, plum, lemon,
  pear, peach, and mango.Additionally, we assess the performance of fruit
  counting using the foundation model compared to a U-Net.
paper_authors._id: 66bad38a1caaa1d77c8a080e
paper_authors.name: Lukas Meyer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-13T10:07:10.663Z
paper_authors._id: 66bb2e7d661de4bd46123c75
paper_authors.avatarUrl: /avatars/feb13c6669e44d3f2346527b6d2660be.svg
paper_authors.fullname: Meyer
paper_authors.type: user
paper_authors.user: meyerls
paper_authors._id: 66bad38a1caaa1d77c8a080f
paper_authors.name: Andreas Gilson
paper_authors._id: 66bad38a1caaa1d77c8a0810
paper_authors.name: Ute Schmidt
paper_authors._id: 66bad38a1caaa1d77c8a0811
paper_authors.name: Marc Stamminger
paper_title: FruitNeRF: A Unified Neural Radiance Field based Fruit Counting
    Framework
paper_summary: We introduce FruitNeRF, a unified novel fruit counting framework that
  leverages state-of-the-art view synthesis methods to count any fruit type
  directly in 3D. Our framework takes an unordered set of posed images captured
  by a monocular camera and segments fruit in each image. To make our system
  independent of the fruit type, we employ a foundation model that generates
  binary segmentation masks for any fruit. Utilizing both modalities, RGB and
  semantic, we train a semantic neural radiance field. Through uniform volume
  sampling of the implicit Fruit Field, we obtain fruit-only point clouds. By
  applying cascaded clustering on the extracted point cloud, our approach
  achieves precise fruit count.The use of neural radiance fields provides
  significant advantages over conventional methods such as object tracking or
  optical flow, as the counting itself is lifted into 3D. Our method prevents
  double counting fruit and avoids counting irrelevant fruit.We evaluate our
  methodology using both real-world and synthetic datasets. The real-world
  dataset consists of three apple trees with manually counted ground truths, a
  benchmark apple dataset with one row and ground truth fruit location, while the
  synthetic dataset comprises various fruit types including apple, plum, lemon,
  pear, peach, and mango.Additionally, we assess the performance of fruit
  counting using the foundation model compared to a U-Net.
paper_ai_summary: FruitNeRF uses semantic neural radiance fields to count fruits from unordered images, achieving precise 3D fruit counting without type dependency.
paper_ai_keywords: neural radiance fields semantic neural radiance fields uniform volume sampling cascaded clustering implicit Fruit Field