@albanD
So I do have a linear layer in my network. Would this correspond to that? What might be the correct way to do it?
I noticed in your snippet you call backward twice, if this is in reference to my retain_graph=True parameter, when I remove it I get the following error:
RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.
I understand that this is not regular so is there something I should fix there?
Also, thank you so much for your help so far