Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment
<|reference_start|>Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment: Transformer models have revolutionized AI tasks, but their large size hinders real-world deployment on resource-constrained and latency-critical edge devices. While binarized Transformers offer a promising solution by significantly reducing model size, existing approaches suffer from algorithm-hardware mismatches with limited co-design exploration, leading to suboptimal performance on edge devices. Hence, we propose a co-design method for efficient end-to-end edge deployment of Transformers from three aspects: algorithm, hardware, and joint optimization. First, we propose BMT, a novel hardware-friendly binarized Transformer with optimized quantization methods and components, and we further enhance its model accuracy by leveraging the weighted ternary weight splitting training technique. Second, we develop a streaming processor mixed binarized Transformer accelerator, namely BAT, which is equipped with specialized units and scheduling pipelines for efficient inference of binarized Transformers. Finally, we co-optimize the algorithm and hardware through a design space exploration approach to achieve a global trade-off between accuracy, latency, and robustness for real-world deployments. Experimental results show our co-design achieves up to 2.14-49.37x throughput gains and 3.72-88.53x better energy efficiency over state-of-the-art Transformer accelerators, enabling efficient end-to-end edge deployment.<|reference_end|>
@article{ji2024co-designing,
    title={Co-Designing Binarized Transformer and Hardware Accelerator for
  Efficient End-to-End Edge Deployment},
    author={Yuhao Ji, Chao Fang, Shaobo Ma, Haikuo Shao, Zhongfeng Wang},
    journal={arXiv preprint arXiv:2407.12070},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.12070},
    primaryClass={cs.LG cs.AI}
}