KWT-Tiny: RISC-V Accelerated, Embedded Keyword Spotting Transformer
<|reference_start|>KWT-Tiny: RISC-V Accelerated, Embedded Keyword Spotting Transformer: This paper explores the adaptation of Transformerbased models for edge devices through the quantisation and hardware acceleration of the ARM Keyword Transformer (KWT) model on a RISC-V platform. The model was targeted to run on 64kB RAM in bare-metal C using a custom-developed edge AI library. KWT-1 was retrained to be 369 times smaller, with only a 10% loss in accuracy through reducing output classes from 35 to 2. The retraining and quantisation reduced model size from 2.42 MB to 1.65 kB. The integration of custom RISC-V instructions that accelerated GELU and SoftMax operations enabled a 5x speedup and thus ~5x power reduction in inference, with inference clock cycle counts decreasing from 26 million to 5.5 million clock cycles while incurring a small area overhead of approximately 29%. The results demonstrate a viable method for porting and accelerating Transformer-based models in low-power IoT devices.<|reference_end|>
@article{al-qawlaq2024kwt-tiny:,
    title={KWT-Tiny: RISC-V Accelerated, Embedded Keyword Spotting Transformer},
    author={Aness Al-Qawlaq, Ajay Kumar M, Deepu John},
    journal={arXiv preprint arXiv:2407.16026},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.16026},
    primaryClass={cs.AR cs.AI cs.PF}
}