Hello Tom,
The loss value seems to go down until it reaches around 2, where it stays around. When the learning rate is 0, nothing changes after training. I am using a SGD optimizer. Learning rate = 0.01 and momentum = 0.9. No matter how low I set the learning rate to, the problem persists.
Thanks,
Rishav