Thank you for your reply.
So backward function always assumes a 1D input. I thought backward network can be considered as the transpose of the forward network in the linear case (fully connected and conv), so I thought backward function should accept an input the size of the output of the layer. That’s why I am puzzled why backward accepts 1D input.
Here is what I want to implement:
According to the chain rule, we can view the process of computing the gradients of loss w.r.t. parameters as the dLoss/dy * dy/dparams. I am building a network that computes the second part i.e. dy/dparams (similar to what happens in the backward function of each module in Pytorch). Now, my question is what would be the input to that network so that it computes the same gradients as loss.backward() calculates for each layer. This way, I would be able to bypass loss.backward() and use my own backward function. But first, for a sanity check, I want to implement backpropagation using this method (feeding gradients computed in loss to a forward network instead of using loss.backward())
Thanks