Drop-Connect as a Fault-Tolerance Approach for RRAM-based Deep Neural Network Accelerators
<|reference_start|>Drop-Connect as a Fault-Tolerance Approach for RRAM-based Deep Neural Network Accelerators: Resistive random-access memory (RRAM) is widely recognized as a promising emerging hardware platform for deep neural networks (DNNs). Yet, due to manufacturing limitations, current RRAM devices are highly susceptible to hardware defects, which poses a significant challenge to their practical applicability. In this paper, we present a machine learning technique that enables the deployment of defect-prone RRAM accelerators for DNN applications, without necessitating modifying the hardware, retraining of the neural network, or implementing additional detection circuitry/logic. The key idea involves incorporating a drop-connect inspired approach during the training phase of a DNN, where random subsets of weights are selected to emulate fault effects (e.g., set to zero to mimic stuck-at-1 faults), thereby equipping the DNN with the ability to learn and adapt to RRAM defects with the corresponding fault rates. Our results demonstrate the viability of the drop-connect approach, coupled with various algorithm and system-level design and trade-off considerations. We show that, even in the presence of high defect rates (e.g., up to 30%), the degradation of DNN accuracy can be as low as less than 1% compared to that of the fault-free version, while incurring minimal system-level runtime/energy costs.<|reference_end|>
@article{xiang2024drop-connect,
    title={Drop-Connect as a Fault-Tolerance Approach for RRAM-based Deep Neural
  Network Accelerators},
    author={Mingyuan Xiang, Xuhan Xie, Pedro Savarese, Xin Yuan, Michael Maire and
  Yanjing Li},
    journal={arXiv preprint arXiv:2404.15498},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.15498},
    primaryClass={cs.ET}
}