I have a very simple model, where weights are not updating but biases are. Haven’t been able to figure out why. (full code here 2)
Model:
class MnistModel(nn.Module):
    def __init__(self, batch_size):
        super(MnistModel, self).__init__()
        self.batch_size = batch_size
        self.w = torch.nn.Parameter(torch.empty(batch_size, 784, 10).uniform_(0, 1))
        self.b = torch.nn.Parameter(torch.empty(10).uniform_(0, 1))

    def forward(self, x):
        return torch.bmm(x, self.w) + self.b

and loop:
for raw_data, raw_target in train_loader:
    data = raw_data.view((batch_size, 1, 784))

    logits = model(data)
    zeros[:, :, :] = 0
    zeros[rows, 0, raw_target] = 1
    loss = criterion(logits, zeros.float())

    # Backward and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

when you run this code it will show( biase and weights are taken from model.parameters()):
weight[0][0][0] before training: tensor(0.7576)
Bias[0] before training: tensor(0.3681)

weight[0][0][0] after training:  tensor(0.7576)
Bias[0] after training tensor(-48.1940)