Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression
<|reference_start|>Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression: As language models become more general purpose, increased attention needs to be paid to detecting out-of-distribution (OOD) instances, i.e., those not belonging to any of the distributions seen during training. Existing methods for detecting OOD data are computationally complex and storage-intensive. We propose a novel soft clustering approach for OOD detection based on non-negative kernel regression. Our approach greatly reduces computational and space complexities (up to 11x improvement in inference time and 87% reduction in storage requirements) and outperforms existing approaches by up to 4 AUROC points on four different benchmarks. We also introduce an entropy-constrained version of our algorithm, which leads to further reductions in storage requirements (up to 97% lower than comparable approaches) while retaining competitive performance. Our soft clustering approach for OOD detection highlights its potential for detecting tail-end phenomena in extreme-scale data settings.<|reference_end|>
@article{gulati2024out-of-distribution,
    title={Out-of-Distribution Detection through Soft Clustering with Non-Negative
  Kernel Regression},
    author={Aryan Gulati, Xingjian Dong, Carlos Hurtado, Sarath Shekkizhar, Swabha
  Swayamdipta, Antonio Ortega},
    journal={arXiv preprint arXiv:2407.13141},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.13141},
    primaryClass={cs.LG}
}