perhaps it has something to do with how nn.Linear is defined? i.e. y=x@A.T, where ‘@’ is matmul, not the usual y=A@X. so during backprop, dy/dx == grad @ A, whereas if y=A@X, then dy/dx==A.T@grad.
I myself is a new learner too, and I didn’t notice this until now, thanks dude!
BTW, you can test this by using:
import torch
X=torch.tensor([[2., 1.],[1., 0.]],requires_grad=True)
w=torch.tensor([[1., 1.],[0., 1.]],requires_grad=True)
y=torch.nn.Sigmoid()(torch.matmul(X,w.T))
print(f’weight:{w}\n’)
print(f’input:{X}\n’)
print(f’output:{y}\n’)
z = torch.ones_like(y)
dy = torch.autograd.grad(y, X, grad_outputs=z,retain_graph=True, create_graph=True)[0]
print(f’X.grad is:{dy}’)
which gives the same result as your dy. whereas if you used y=torch.nn.Sigmoid()(torch.matmul(w,X)), then dy is the transpose of your dy