ptrblck:

Since you are training your model, the loss should decrease. Is that the case or how does the loss change?


I’ve learned that the loss should go down but in my case it prints the same value on each training epoch.
Example:




Train Epoch: 1 [128/640 (20%)]
Loss: 159.35




Train Epoch: 1 [256/640 (40%)]
Loss: 149.98


Train Epoch: 1 [384/640 (60%)]
Loss: 146.86


Train Epoch: 1 [512/640 (80%)]
Loss: 145.30


Train Epoch: 1 [640/640 (100%)]
Loss: 144.37







Train Epoch: 2 [128/640 (20%)]
Loss: 140.62




Train Epoch: 2 [256/640 (40%)]
Loss: 140.62


Train Epoch: 2 [384/640 (60%)]
Loss: 140.62


Train Epoch: 2 [512/640 (80%)]
Loss: 140.62


Train Epoch: 2 [640/640 (100%)]
Loss: 140.62



Also, if I decrease the batch size from 128 to 64, the loss starts lower at 70