Antidistillation Sampling
Frontier models that generate extended reasoning traces inadvertently produce
rich token sequences that can facilitate model distillation. Recognizing this
vulnerability, model owners may seek sampling strategies that limit the
effectiveness of distillation without compromising model performance.
Antidistillation sampling provides exactly this capability. By
strategically modifying a model's next-token probability distribution,
antidistillation sampling poisons reasoning traces, rendering them
significantly less effective for distillation while preserving the model's
practical utility. For further details, see https://antidistillation.com.
6801b77dcb758561ae26997e
Yash Savani
claimed_verified
2025-04-18T09:39:03.683Z
647c0564001553a39c38e79e
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/t5KF0Vp7kCk-5EZRWhG8i.jpeg
Yash Savani
user
yashsavani
6801b77dcb758561ae26997f
Asher Trockman
6801b77dcb758561ae269980
Zhili Feng
6801b77dcb758561ae269981
Avi Schwarzschild
6801b77dcb758561ae269982
Alexander Robey
claimed_verified
2025-04-18T09:39:01.839Z
63458a32d54fb141deda949d
/avatars/fc4b59cd009075ac7987c6cdddbe3fea.svg
Alex Robey
user
arobey1
6801b77dcb758561ae269983
Marc Finzi
6801b77dcb758561ae269984
J. Zico Kolter
Antidistillation Sampling
Frontier models that generate extended reasoning traces inadvertently produce
rich token sequences that can facilitate model distillation. Recognizing this
vulnerability, model owners may seek sampling strategies that limit the
effectiveness of distillation without compromising model performance.
Antidistillation sampling provides exactly this capability. By
strategically modifying a model's next-token probability distribution,
antidistillation sampling poisons reasoning traces, rendering them
significantly less effective for distillation while preserving the model's
practical utility. For further details, see https://antidistillation.com.
Antidistillation sampling modifies a model's next-token probability distribution to disrupt the generation of reasoning traces for distillation without affecting model performance.
antidistillation sampling next-token probability distribution reasoning traces model distillation