Solved by vainaijr in post #3 
              
              
                we could carry following experiment to see what is going on while computing gradient, let us assume we have 
l = nn.Linear(2, 2, bias=False)
list(l.parameters())

[Parameter containing: 
tensor([[-0.0473,  0.0679], 
[-0.6429, -0.5586]], requires_grad=True)] 
input = torch.tensor([[1.3032, 0.5701], […