Hi everyone, I’m trying to port this Torch code (https://github.com/anokland/dfa-torch 47) in Pytorch. It’s the implementation of Direct Feedback Alignment (https://arxiv.org/abs/1609.01596 20).
The logic of he implementation is this: we need to inject the error signal at the output of each leayer, so we will create a custom SequentialSG model, everytime it sees a special module called ErrorFeedback copies the error signal in the grad_output of that module.
Now in each ErrorFeedback module, during the forward pass we simply copy the input to the output (i.e. we do nothing), during the backward pass instead we perform the random projection of the error on the dimension of the hidden layer. To do this, we have to define a new module but also extend torch.autograd. I followed http://pytorch.org/docs/master/notes/extending.html 56
So we have to define a class ErrorFeedbackFunction which inherits from torch.autograd.Function, where we define the two static methods forward and backward and a class which inherits from torch.nn.Module.
Now we just have to define a new nn.Module, which uses our new function in the forward pass.
Now if I try to use it.
I get this error:
TypeError: 'ErrorFeedbackFunction' object does not support indexing

What am I doing wrong? Did I miss some steps in my implementation?