Embedding-based statistical inference on generative models
<|reference_start|>Embedding-based statistical inference on generative models: The recent cohort of publicly available generative models can produce human expert level content across a variety of topics and domains. Given a model in this cohort as a base model, methods such as parameter efficient fine-tuning, in-context learning, and constrained decoding have further increased generative capabilities and improved both computational and data efficiency. Entire collections of derivative models have emerged as a byproduct of these methods and each of these models has a set of associated covariates such as a score on a benchmark, an indicator for if the model has (or had) access to sensitive information, etc. that may or may not be available to the user. For some model-level covariates, it is possible to use "similar" models to predict an unknown covariate. In this paper we extend recent results related to embedding-based representations of generative models -- the data kernel perspective space -- to classical statistical inference settings. We demonstrate that using the perspective space as the basis of a notion of "similar" is effective for multiple model-level inference tasks.<|reference_end|>
@article{helm2024embedding-based,
    title={Embedding-based statistical inference on generative models},
    author={Hayden Helm and Aranyak Acharyya and Brandon Duderstadt and Youngser
  Park and Carey E. Priebe},
    journal={arXiv preprint arXiv:2410.01106},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.01106},
    primaryClass={cs.LG stat.ML}
}