Agglomerative Token Clustering
<|reference_start|>Agglomerative Token Clustering: We present Agglomerative Token Clustering (ATC), a novel token merging method that consistently outperforms previous token merging and pruning methods across image classification, image synthesis, and object detection & segmentation tasks. ATC merges clusters through bottom-up hierarchical clustering, without the introduction of extra learnable parameters. We find that ATC achieves state-of-the-art performance across all tasks, and can even perform on par with prior state-of-the-art when applied off-the-shelf, i.e. without fine-tuning. ATC is particularly effective when applied with low keep rates, where only a small fraction of tokens are kept and retaining task performance is especially difficult.<|reference_end|>
@article{haurum2024agglomerative,
    title={Agglomerative Token Clustering},
    author={Joakim Bruslund Haurum, Sergio Escalera, Graham W. Taylor, Thomas B.
  Moeslund},
    journal={arXiv preprint arXiv:2409.11923},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.11923},
    primaryClass={cs.CV}
}