The loss of a few neurons in a brain rarely results in any visible loss of function. However, the insight into what "few" means in this context is unclear. How many random neuron failures will it take to lead to a visible loss of function? In this paper, we address the fundamental question of the impact of the crash of a random subset of neurons on the overall computation of a neural network and the error in the output it produces. We study fault tolerance of neural networks subject to small random neuron/weight crash failures in a probabilistic setting. We give provable guarantees on the robustness of the network to these crashes. Our main contribution is a bound on the error in the output of a network under small random Bernoulli crashes proved by using a Taylor expansion in the continuous limit, where close-by neurons at a layer are similar. The failure mode we adopt in our model is characteristic of neuromorphic hardware, a promising technology to speed up artificial neural networks, as well as of biological networks. We show that our theoretical bounds can be used to compare the fault tolerance of different architectures and to design a regularizer improving the fault tolerance of a given architecture. We design an algorithm achieving fault tolerance using a reasonable number of neurons. In addition to the theoretical proof, we also provide experimental validation of our results and suggest a connection to the generalization capacity problem.
This contribution studies the impact of deletions of random neurons on prediction accuracy of trained architecture, with the application to failure analysis and the specific context of neuromorphic hardware. The manuscript shows that worst-case analysis of failure modes is NP hard and contributes a theoretical analysis of the average case impact of random perturbations with Bernouilli noise on prediction accuracy, as well as a training algorithm based on aggregation. The difficulty of tight bounds comes from the fact that with many layers a neural network can have a very large Lipschitz constant. The average case analysis is based on wide neural networks and an assumption of a form of smoothness in the values of hidden units as the width increases. The improve fitting procedure is done by adding a set of regularizing terms, including regularizing the spectral norm of the layers.
The robustness properties can be interesting to a wider community than that of neuromorphic hardware. In this sense, the manuscript provides interesting content, although I do fear that it is not framed properly. Indeed, the introduction and conclusion mention robustness as a central concern, which it is indeed, but the neuron failures are quite minor in this respect. More relevant questions would be: is the approach introduced here useful to limit the impact of adversarial examples? Does it provide good regularizations that improve generalization? I would believe that the regularization provided are interesting by looking at their expression; in particular the regularization of the operator norm of layers makes a lot of sense. That said, there is some empirical evidence that batch norm achieves similar effects, but with a significantly reduced cost.
Another limitation of the work is that it pushes towards very wide networks and ensemble predictions. These significantly increase the prediction cost and are often frowned upon by applications.
It seems to me that the manuscript has readability issues: the procedure introduced is quite unclear and could not be reimplemented from reading the manuscript (including the supplementary materials). Also, the results in the main part of the manuscript are presented to tersely: I do not understand where in table 2 dropout is varied.
The contributed algorithm has many clauses to tune dynamically the behavior of the regularizations and the architecture. These are very hard to control in theory. They would need strong empirical validation on many different datasets.
It is also very costly, as it involves repeatedly training from scratch a neural network.
The manuscript discusses in several places a median aggregation, which gives robustness properties to the predictor. I must admit that I have not been available to see it in the algorithm. This worries me, because it reveals that I do not understand the approach. The beginning of section 6.1, in the appendix, suggests details that are not understandable from the algorithm description.
Finally, a discussion of the links to dropout would be interesting: both in practice, as dropout can be seen as simulating neuron failure during training, as well as from the point of view of theory, as there has been many attempts to analyze theoretically dropout (starting with Wager NIPS 2013, but more advanced work is found in Gal ICML 2015, Helmbold JMLR 2015, Mianjy ICML 2018).