Hi,
I am trying to compute the gradient of output from model with respect to the input, which is a symmetric matrix. I expect the output of gradient calculation to be symmetric. Can someone help explain why the gradient output is not symmetric? This is the code I have been trying.
SM = Variable(matrix,requires_grad=True)
out = model(SM)
output = F.softmax(out)
model.zero_grad()
out_class = output.max(1)[1].data[0]
one_hot_output = torch.zeros(output.size())
one_hot_output[0, output_class] = 1
output.backward(gradient=one_hot_output)
grad_out_in =  SM.grad.data[0]
The output of above code is for example looks like this:
tensor([[[-0.0465, -0.0377, -0.0602,  …,  0.0117,  0.0323, -0.0426],
[ 0.0018, -0.0711,  0.0594,  …,  0.0302, -0.1398,  0.0213],
[-0.0123,  0.0490,  0.0875,  …, -0.0314,  0.0194, -0.0434],
…,
[ 0.0363,  0.0121,  0.0098,  …,  0.0108,  0.0057,  0.0425],
[ 0.0596, -0.0743,  0.0541,  …,  0.0290, -0.1835,  0.1229],
[-0.0395,  0.0283, -0.0428,  …, -0.0294,  0.1277, -0.0793]]]
Using Pytorch 1.4 with cuda 9.2