Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models
<|reference_start|>Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models: The disconnect between tokenizer creation and model training in language models allows for specific inputs, such as the infamous SolidGoldMagikarp token, to induce unwanted model behaviour. Although such `glitch tokens', tokens present in the tokenizer vocabulary but that are nearly or entirely absent during model training, have been observed across various models, a reliable method to identify and address them has been missing. We present a comprehensive analysis of Large Language Model tokenizers, specifically targeting this issue of detecting under-trained tokens. Through a combination of tokenizer analysis, model weight-based indicators, and prompting techniques, we develop novel and effective methods for automatically detecting these problematic tokens. Our findings demonstrate the prevalence of such tokens across a diverse set of models and provide insights into improving the efficiency and safety of language models.<|reference_end|>
@article{land2024fishing,
    title={Fishing for Magikarp: Automatically Detecting Under-trained Tokens in
  Large Language Models},
    author={Sander Land, Max Bartolo},
    journal={arXiv preprint arXiv:2405.05417},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.05417},
    primaryClass={cs.CL}
}