Hi,
I also encounter a similar problem, cost of dataset seems normal.
... ...

Epoch: [2][20/296],
Learning_Rate: 0.000492,
Time: 0.9096,       Data:     0.0799,
MIoU: 0.5842,       Accuracy: 0.9174,      Loss: 0.250443
Epoch: [2][40/296],
Learning_Rate: 0.000492,
Time: 0.8930,       Data:     0.0611,
MIoU: 0.5833,       Accuracy: 0.9198,      Loss: 0.249228
Epoch: [2][60/296],
Learning_Rate: 0.000492,
Time: 0.8872,       Data:     0.0547,
MIoU: 0.5920,       Accuracy: 0.9183,      Loss: 0.246295
... ...

I set num_wokers=2 because when I increase it by power 2 it doesn’t have any change. But the Volatile GPU-Util is not stable, it change from 0% to 99%, what could be the potential problem is?
Thank you.