Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights
With the increasing numbers of publicly available models, there are probably
pretrained, online models for most tasks users require. However, current model
search methods are rudimentary, essentially a text-based search in the
documentation, thus users cannot find the relevant models. This paper presents
ProbeLog, a method for retrieving classification models that can recognize a
target concept, such as "Dog", without access to model metadata or training
data. Differently from previous probing methods, ProbeLog computes a descriptor
for each output dimension (logit) of each model, by observing its responses on
a fixed set of inputs (probes). Our method supports both logit-based retrieval
("find more logits like this") and zero-shot, text-based retrieval ("find all
logits corresponding to dogs"). As probing-based representations require
multiple costly feedforward passes through the model, we develop a method,
based on collaborative filtering, that reduces the cost of encoding
repositories by 3x. We demonstrate that ProbeLog achieves high retrieval
accuracy, both in real-world and fine-grained search tasks and is scalable to
full-size repositories.
67aef6212c36e4d8bd23740e
Jonathan Kahana
admin_assigned
2025-02-14T12:41:31.020Z
6465fd33dac127ac80f0b334
/avatars/113f02c1b1f8d33d3487daa867afcd3f.svg
Jonathan Kahana
user
jonkahana
67aef6212c36e4d8bd23740f
Or Nathan
claimed_verified
2025-02-20T09:37:48.235Z
67b603c90918c8645fff91e9
/avatars/d590a2055b5553ba6e7487f156aaf06c.svg
Or Nathan
user
OrNathan
67aef6212c36e4d8bd237410
Eliahu Horwitz
claimed_verified
2025-02-14T11:10:52.689Z
630dd4218df86f1e5beb2ed7
https://cdn-avatars.huggingface.co/v1/production/uploads/630dd4218df86f1e5beb2ed7/fKvNWyWv6CVBdbXXUlrYv.jpeg
Eliahu Horwitz
user
Eliahu
67aef6212c36e4d8bd237411
Yedid Hoshen
admin_assigned
2025-02-14T12:41:43.672Z
646cfc3b4220471ca0c56b20
/avatars/19d6ab141ec2cd25c1c3b45fd8f69910.svg
Yedid Hoshen
user
yedid
Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights
With the increasing numbers of publicly available models, there are probably
pretrained, online models for most tasks users require. However, current model
search methods are rudimentary, essentially a text-based search in the
documentation, thus users cannot find the relevant models. This paper presents
ProbeLog, a method for retrieving classification models that can recognize a
target concept, such as "Dog", without access to model metadata or training
data. Differently from previous probing methods, ProbeLog computes a descriptor
for each output dimension (logit) of each model, by observing its responses on
a fixed set of inputs (probes). Our method supports both logit-based retrieval
("find more logits like this") and zero-shot, text-based retrieval ("find all
logits corresponding to dogs"). As probing-based representations require
multiple costly feedforward passes through the model, we develop a method,
based on collaborative filtering, that reduces the cost of encoding
repositories by 3x. We demonstrate that ProbeLog achieves high retrieval
accuracy, both in real-world and fine-grained search tasks and is scalable to
full-size repositories.
ProbeLog is a method for classification model retrieval based on probe logit analysis, offering efficient and scalable search without metadata.
probe logit logit-based retrieval zero-shot retrieval collaborative filtering