Thank you, I did as you suggested.
I have a follow up question  - most of the time I try to stick to the torch.nn.\torch.method because I know it will calculate the gradients right. Here I tried to do something a little bit unconventional like using unfold on input and then use different activation function on different patches of the unfolded image.
Or for example using unfold on the hidden layer.
Is there a way to make sure it calculates the gradients right?
I’m familiar with .requires_grad and autograd but I’m not sure if it calculates the gradients as I would thought.
Is there a guide that can help me get intuition about how the gradients are calculated in Pytorch?
Do I need to look explicitly at the gradients and compare it to analytical results?
Thanks a lot