Hi @albanD,
Well, to spill the beans a little bit, I’m working on a method that produces a mask on the weights, which is applied during the backward() method, just for experimentation purposes. The feedback alignment paper does something similar, by using a random weight matrix during backpropagation which is asymmetric to the forward weight matrix. Maybe the small example I provided above doesn’t fully represent the complexity of the task… I’m just curious if it can be done. Could I subclass torch.autograd.Function and implement my own backward() method?
Thanks