Thanks, thats make sense, so one more thing that’s relates to that.
I have an issue with my network, it mostly takes arround 2 or 3 seconds to load each iteration,
But once in every epochs I’m getting a huge data loading time, like that:

Epoch: [10][1/130]	Time 23355.934 (23355.934)	Data 23355.545 (23355.545)	Loss 5.2440 (5.2440)	Acc 0.008 (0.008)

now this is the 10th Epoch, and until now it worked fine, and it’s repeating with no patterns.
I am using data shuffling.
How would you approach this problem?