title: Hypencoder: Hypernetworks for Information Retrieval
summary: The vast majority of retrieval models depend on vector inner products to
  produce a relevance score between a query and a document. This naturally limits
  the expressiveness of the relevance score that can be employed. We propose a
  new paradigm, instead of producing a vector to represent the query we produce a
  small neural network which acts as a learned relevance function. This small
  neural network takes in a representation of the document, in this paper we use
  a single vector, and produces a scalar relevance score. To produce the little
  neural network we use a hypernetwork, a network that produce the weights of
  other networks, as our query encoder or as we call it a Hypencoder. Experiments
  on in-domain search tasks show that Hypencoder is able to significantly
  outperform strong dense retrieval models and has higher metrics then reranking
  models and models an order of magnitude larger. Hypencoder is also shown to
  generalize well to out-of-domain search tasks. To assess the extent of
  Hypencoder's capabilities, we evaluate on a set of hard retrieval tasks
  including tip-of-the-tongue retrieval and instruction-following retrieval tasks
  and find that the performance gap widens substantially compared to standard
  retrieval tasks. Furthermore, to demonstrate the practicality of our method we
  implement an approximate search algorithm and show that our model is able to
  search 8.8M documents in under 60ms.
paper_authors._id: 67ab99c8bb44ec714c6a4a96
paper_authors.name: Julian Killingback
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-12T09:16:46.458Z
paper_authors._id: 635582fb6b58fa7cc8701580
paper_authors.avatarUrl: /avatars/a00791ba70a2de40dacac4582307c0f2.svg
paper_authors.fullname: Julian Killingback
paper_authors.type: user
paper_authors.user: jfkback
paper_authors._id: 67ab99c8bb44ec714c6a4a97
paper_authors.name: Hansi Zeng
paper_authors._id: 67ab99c8bb44ec714c6a4a98
paper_authors.name: Hamed Zamani
paper_title: Hypencoder: Hypernetworks for Information Retrieval
paper_summary: The vast majority of retrieval models depend on vector inner products to
  produce a relevance score between a query and a document. This naturally limits
  the expressiveness of the relevance score that can be employed. We propose a
  new paradigm, instead of producing a vector to represent the query we produce a
  small neural network which acts as a learned relevance function. This small
  neural network takes in a representation of the document, in this paper we use
  a single vector, and produces a scalar relevance score. To produce the little
  neural network we use a hypernetwork, a network that produce the weights of
  other networks, as our query encoder or as we call it a Hypencoder. Experiments
  on in-domain search tasks show that Hypencoder is able to significantly
  outperform strong dense retrieval models and has higher metrics then reranking
  models and models an order of magnitude larger. Hypencoder is also shown to
  generalize well to out-of-domain search tasks. To assess the extent of
  Hypencoder's capabilities, we evaluate on a set of hard retrieval tasks
  including tip-of-the-tongue retrieval and instruction-following retrieval tasks
  and find that the performance gap widens substantially compared to standard
  retrieval tasks. Furthermore, to demonstrate the practicality of our method we
  implement an approximate search algorithm and show that our model is able to
  search 8.8M documents in under 60ms.
paper_ai_keywords: vector inner products relevance score neural network learned relevance function small neural network document representation hypernetwork query encoder Hypencoder dense retrieval models reranking models tip-of-the-tongue retrieval instruction-following retrieval tasks approximate search algorithm