I want to modify the backward of relu, such that i simply pass through the gradients coming from the top rather than 0-ing out the ones where the unit is off.
I looked at this thread 21 and couldn’t get much out of it. Looking here 21,  while this works for making the gradients zero, i am not sure what the components of gradient_input are and which one i should modify to have a pass-through.