Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml
<|reference_start|>Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml: This study presents an efficient implementation of transformer architectures in Field-Programmable Gate Arrays(FPGAs) using hls4ml. We demonstrate the strategy for implementing the multi-head attention, softmax, and normalization layer and evaluate three distinct models. Their deployment on VU13P FPGA chip achieved latency less than 2us, demonstrating the potential for real-time applications. HLS4ML compatibility with any TensorFlow-built transformer model further enhances the scalability and applicability of this work. Index Terms: FPGAs, machine learning, transformers, high energy physics, LIGO<|reference_end|>
@article{jiang2024low,
    title={Low Latency Transformer Inference on FPGAs for Physics Applications with
  hls4ml},
    author={Zhixing Jiang, Dennis Yin, Yihui Chen, Elham E Khoda, Scott Hauck,
  Shih-Chieh Hsu, Ekaterina Govorkova, Philip Harris, Vladimir Loncar, Eric A.
  Moreno},
    journal={arXiv preprint arXiv:2409.05207},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.05207},
    primaryClass={cs.LG}
}