Thanks for the quick reply. The bias value is changing now. Also, I defined the optimizer as optim.SGD([{‘params’:[model.bias], ‘lr’:0.001}], lr=0.001) which will update bias value after each iteration. To update the weights of conv layer as well as the bias value, is optim.SGD(model.parameters(), lr=0.001) the right way to define the optimizer?