I am having the same issue, some times it runs more iterations and some times less if I modify the number of sub-processes (when loading the dataset) it seams it also affects the number of iterations it can perform even though it does not make sense since they are loaded using my CPU and the training is still sequential. Well in my case when I say iterations I mean to say forward passes or batch passes.