Loss  decreases in case of CPU but remain same in case of GPU.
Following is an Example of loss coming in GPU
Epoch Number= 0 Train Loss= 7.179644451865667 Training Accuracy= 0.004746835443037975
Epoch Number= 0 Validation Loss= 7.178059632265115 Validation Accuracy= 0.0
Epoch Number= 1 Train Loss= 7.177682260923747 Training Accuracy= 0.0
Epoch Number= 1 Validation Loss= 7.176880444152446 Validation Accuracy= 0.0
Epoch Number= 2 Train Loss= 7.176005357428442 Training Accuracy= 0.0
Epoch Number= 2 Validation Loss= 7.17541601688047 Validation Accuracy= 0.0
.
.
.
Thanks
Achyut