So… I tried to remove the last linear layer with the softmax and changed the BCELoss to MSELoss and the weights are being updated!
I’m guessing this is a problem from BCELoss and not from my code… Could you check this out @smth @apaszke ?