How the backward() works for torch Variable?
For torch.autograd Variable, there’s the ‘gradient’ input param for the .backward() function. I don’t quite understand what this ‘gradient’ input param stands for, and why this param is needed?
david-leon
smth
Note that if you’re calling it on a loss/cost variable, you don’t need to provide an argument since autograd will assume a tensor of ones.
jekbradbury
apaszke
For example, I have an Embedding variable which is 10,000*100. When I used 10th and 20th embedding, will backward function calculate the gradient only for this two embeddings or will it calculate the gradient for all 10,000 embeddings?
Adoni
smth
I also have trouble with this. It says that the argument of backward should be the gradients w.r.t. the output. But consistently across the documentation it is never mentioned of what the gradient w.r.t. the output (must be a conspiracy to drive me crazy). Clearly you can define the gradient as an operator but then you don’t get numbers (like 1). Can the thing of which you take the gradient w.r.t. the output not be stated concisely? Does it depend on some usecase? If I just have y=f(x) for example how do I derive the 1 that is assumed?
automatisch
Am I getting there?
automatisch
apaszke