Hi, currently I encountered a problem regarding to torch.matmul function. The linear operation in a neural network defined in functional module was output = input.matmul(weight.t()). I tried to play around this, and got confused.
First, I tried to modify it to output = input.detach().matmul(weight.t().detach()), and output = input.data.matmul(weight.t().data). They both gave the wrong training results (i.e. loss didn’t get reduced).
I realized output generated in these ways have the requires_grad setting to false, however, even I manually set this to true using output.requires_grad_(True) after this multiplication, the training still produced the wrong result. Why did this happen?
Also, I noticed the output computed by the modified multiplication is a leaf variable, while the original version is not. Why it that?
Thanks!!