Ultra Fast Transformers on FPGAs for Particle Physics Experiments
<|reference_start|>Ultra Fast Transformers on FPGAs for Particle Physics Experiments: This work introduces a highly efficient implementation of the transformer architecture on a Field-Programmable Gate Array (FPGA) by using the \texttt{hls4ml} tool. Given the demonstrated effectiveness of transformer models in addressing a wide range of problems, their application in experimental triggers within particle physics becomes a subject of significant interest. In this work, we have implemented critical components of a transformer model, such as multi-head attention and softmax layers. To evaluate the effectiveness of our implementation, we have focused on a particle physics jet flavor tagging problem, employing a public dataset. We recorded latency under 2 $\mu$s on the Xilinx UltraScale+ FPGA, which is compatible with hardware trigger requirements at the CERN Large Hadron Collider experiments.<|reference_end|>
@article{jiang2024ultra,
    title={Ultra Fast Transformers on FPGAs for Particle Physics Experiments},
    author={Zhixing Jiang, Dennis Yin, Elham E Khoda, Vladimir Loncar, Ekaterina
  Govorkova, Eric Moreno, Philip Harris, Scott Hauck, Shih-Chieh Hsu},
    journal={Machine Learning and the Physical Sciences Workshop, NeurIPS 2023},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.01047},
    primaryClass={cs.LG cs.AR hep-ex}
}