  In deep neural learning, a discriminator trained on in-distribution (ID)
samples may make high-confidence predictions on out-of-distribution (OOD)
samples. This triggers a significant matter for robust, trustworthy and safe
deep learning. The issue is primarily caused by the limited ID samples
observable in training the discriminator when OOD samples are unavailable. We
propose a general approach for \textit{fine-tuning discriminators by implicit
generators} (FIG). FIG is grounded on information theory and applicable to
standard discriminators without retraining. It improves the ability of a
standard discriminator in distinguishing ID and OOD samples by generating and
penalizing its specific OOD samples. According to the Shannon entropy, an
energy-based implicit generator is inferred from a discriminator without extra
training costs. Then, a Langevin dynamic sampler draws specific OOD samples for
the implicit generator. Lastly, we design a regularizer fitting the design
principle of the implicit generator to induce high entropy on those generated
OOD samples. The experiments on different networks and datasets demonstrate
that FIG achieves the state-of-the-art OOD detection performance.

Machine Learning