Stochastic Multivariate Universal-Radix Finite-State Machine: a Theoretically and Practically Elegant Nonlinear Function Approximator
<|reference_start|>Stochastic Multivariate Universal-Radix Finite-State Machine: a Theoretically and Practically Elegant Nonlinear Function Approximator: Nonlinearities are crucial for capturing complex input-output relationships especially in deep neural networks. However, nonlinear functions often incur various hardware and compute overheads. Meanwhile, stochastic computing (SC) has emerged as a promising approach to tackle this challenge by trading output precision for hardware simplicity. To this end, this paper proposes a first-of-its-kind stochastic multivariate universal-radix finite-state machine (SMURF) that harnesses SC for hardware-simplistic multivariate nonlinear function generation at high accuracy. We present the finite-state machine (FSM) architecture for SMURF, as well as analytical derivations of sampling gate coefficients for accurately approximating generic nonlinear functions. Experiments demonstrate the superiority of SMURF, requiring only 16.07% area and 14.45% power consumption of Taylor-series approximation, and merely 2.22% area of look-up table (LUT) schemes.<|reference_end|>
@article{feng2024stochastic,
    title={Stochastic Multivariate Universal-Radix Finite-State Machine: a
  Theoretically and Practically Elegant Nonlinear Function Approximator},
    author={Xincheng Feng, Guodong Shen, Jianhao Hu, Meng Li, Ngai Wong},
    journal={arXiv preprint arXiv:2405.02356},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.02356},
    primaryClass={cs.LG cs.AI}
}