We introduce backdrop, a flexible and simple-to-implement method, intuitively described as dropout acting only along the backpropagation pipeline. Backdrop is implemented via one or more masking layers which are inserted at specific points along the network. Each backdrop masking layer acts as the identity in the forward pass, but randomly masks parts of the backward gradient propagation. Intuitively, inserting a backdrop layer after any convolutional layer leads to stochastic gradients corresponding to features of that scale. Therefore, backdrop is well suited for problems in which the data have a multi-scale, hierarchical structure. Backdrop can also be applied to problems with non-decomposable loss functions where standard SGD methods are not well suited. We perform a number of experiments and demonstrate that backdrop leads to significant improvements in generalization.
This paper introduces a data dependent strategy to mask parts of the partial derivatives in the chain rule computation. 
Typically with papers proposing modifications of the training regime of the neural network one would expect one of three outcomes:
 - a well justified, mathematically sound method, well tested in simple cases and with some proof of concept results on proper tasks
 - a more heuristic, empirical driven research, where strong results on proper tasks
 - method, however justified, allows us to do something previously impossible, removing some limitations/constraints (like biologically plausible learning etc.)
In its current form paper seems to lack any of these characteristics. On one hand method lacks any guarantees and on the other paper does not present significant improvements under any approved metrics, nor it introduces new which can be properly quantified. In fact, authors explicitly claim that empirical section "Note that in these experiments, the purpose is not to achieve state of the art performance, but to exemplify how backdrop can be used and what measure of performance gains one can expect.".  
With methods like this it is almost obvious that resulting update is not an unbiased gradient estimator of any function. Consequently convergence/learning guarantees that we have for GD or SGD no longer apply. Do authors have any thoughts on how bad can it get? As noted in the text, other methods of "dropping" data (such as dropout) don't have this issue as they still estimate proper gradients. Here, since dropping is done inside the network only on backwards pass, resulting estimates could, in principle, lead to oscilations, divergence and other issues. If these are not encountered in practice it might be interesting to understand why. 
If authors prefer to go through more empirical path, one would expect at least to see some baselines for tasks proposed, rather than comparing Backdrop to SGD. There are many methods that could be applied in scenarios like this, including dozens forms of dropout (which, as authors note, is not aimed at the same goals, but this does not mean that it will not shine under the metrics introduced, as they are non-standard and so - noone tested them in this exact regime).
I am happy to revisit my rating given authors restructure paper towards one of these paths (or other one which is not listed here).