CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal Model Inference
<|reference_start|>CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal Model Inference: In response to the rising interest in large multimodal models, we introduce Cross-Attention Token Pruning (CATP), a precision-focused token pruning method. Our approach leverages cross-attention layers in multimodal models, exemplified by BLIP-2, to extract valuable information for token importance determination. CATP employs a refined voting strategy across model heads and layers. In evaluations, CATP achieves up to 12.1X higher accuracy compared to existing token pruning methods, addressing the trade-off between computational efficiency and model precision.<|reference_end|>
@article{liao2024catp:,
    title={CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal
  Model Inference},
    author={Ruqi Liao, Chuqing Zhao, Jin Li, Weiqi Feng},
    journal={arXiv preprint arXiv:2404.08567},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.08567},
    primaryClass={cs.CL cs.AI}
}