If you don’t want to update the bias parameter, you could set the requires_grad attribute of the bias to False and don’t pass it to the optimizer:
lin = nn.Linear(1, 1)
lin.bias.requires_grad = False

optimizer = torch.optim.Adam([lin.weight], lr=1.)
output = lin(torch.randn(1, 1))
output.backward()
lin.bias.grad
> 
lin.weight.grad
> tensor([[-0.0095]])