title: ILIAS: Instance-Level Image retrieval At Scale
summary: This work introduces ILIAS, a new test dataset for Instance-Level Image
  retrieval At Scale. It is designed to evaluate the ability of current and
  future foundation models and retrieval techniques to recognize particular
  objects. The key benefits over existing datasets include large scale, domain
  diversity, accurate ground truth, and a performance that is far from saturated.
  ILIAS includes query and positive images for 1,000 object instances, manually
  collected to capture challenging conditions and diverse domains. Large-scale
  retrieval is conducted against 100 million distractor images from YFCC100M. To
  avoid false negatives without extra annotation effort, we include only query
  objects confirmed to have emerged after 2014, i.e. the compilation date of
  YFCC100M. An extensive benchmarking is performed with the following
  observations: i) models fine-tuned on specific domains, such as landmarks or
  products, excel in that domain but fail on ILIAS ii) learning a linear
  adaptation layer using multi-domain class supervision results in performance
  improvements, especially for vision-language models iii) local descriptors in
  retrieval re-ranking are still a key ingredient, especially in the presence of
  severe background clutter iv) the text-to-image performance of the
  vision-language foundation models is surprisingly close to the corresponding
  image-to-image case. website: https://vrg.fel.cvut.cz/ilias/
paper_authors._id: 67b465600e5142133055d7c1
paper_authors.name: Giorgos Kordopatis-Zilos
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-19T09:04:38.791Z
paper_authors._id: 673c934bdf13003bd11746fd
paper_authors.avatarUrl: /avatars/1aec1157549be85963b39eb54845b695.svg
paper_authors.fullname: Giorgos Kordopatis-Zilos
paper_authors.type: user
paper_authors.user: gkordo
paper_authors._id: 67b465600e5142133055d7c2
paper_authors.name: Vladan Stojnić
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-18T16:39:41.653Z
paper_authors._id: 66a3ae59f33ff23e1c027ccd
paper_authors.avatarUrl: /avatars/216717d547bf785a2b1696171e5f4b11.svg
paper_authors.fullname: Vladan Stojnic
paper_authors.type: user
paper_authors.user: stojnvla
paper_authors._id: 67b465600e5142133055d7c3
paper_authors.name: Anna Manko
paper_authors._id: 67b465600e5142133055d7c4
paper_authors.name: Pavel Šuma
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-19T09:04:41.052Z
paper_authors._id: 67ab9acd412358244419d946
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/tq-eUQMYGdlDZVn15DckY.png
paper_authors.fullname: Pavel Suma
paper_authors.type: user
paper_authors.user: pavelsuma
paper_authors._id: 67b465600e5142133055d7c5
paper_authors.name: Nikolaos-Antonios Ypsilantis
paper_authors._id: 67b465600e5142133055d7c6
paper_authors.name: Nikos Efthymiadis
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-19T11:49:36.494Z
paper_authors._id: 663e63c5902b965ba35a0308
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/663e63c5902b965ba35a0308/OFX5yvY2LL8KORKsiDhAy.jpeg
paper_authors.fullname: Nikos Efthymiadis
paper_authors.type: user
paper_authors.user: nikos-efth
paper_authors._id: 67b465600e5142133055d7c7
paper_authors.name: Zakaria Laskar
paper_authors._id: 67b465600e5142133055d7c8
paper_authors.name: Jiří Matas
paper_authors._id: 67b465600e5142133055d7c9
paper_authors.name: Ondřej Chum
paper_authors._id: 67b465600e5142133055d7ca
paper_authors.name: Giorgos Tolias
paper_title: ILIAS: Instance-Level Image retrieval At Scale
paper_summary: This work introduces ILIAS, a new test dataset for Instance-Level Image
  retrieval At Scale. It is designed to evaluate the ability of current and
  future foundation models and retrieval techniques to recognize particular
  objects. The key benefits over existing datasets include large scale, domain
  diversity, accurate ground truth, and a performance that is far from saturated.
  ILIAS includes query and positive images for 1,000 object instances, manually
  collected to capture challenging conditions and diverse domains. Large-scale
  retrieval is conducted against 100 million distractor images from YFCC100M. To
  avoid false negatives without extra annotation effort, we include only query
  objects confirmed to have emerged after 2014, i.e. the compilation date of
  YFCC100M. An extensive benchmarking is performed with the following
  observations: i) models fine-tuned on specific domains, such as landmarks or
  products, excel in that domain but fail on ILIAS ii) learning a linear
  adaptation layer using multi-domain class supervision results in performance
  improvements, especially for vision-language models iii) local descriptors in
  retrieval re-ranking are still a key ingredient, especially in the presence of
  severe background clutter iv) the text-to-image performance of the
  vision-language foundation models is surprisingly close to the corresponding
  image-to-image case. website: https://vrg.fel.cvut.cz/ilias/