-Just run it in a normal training loop (making sure to wrap your net in DataParallel when you create it) and then only call optimizer.step() and zero_grad() after every N steps (where it looks like N is 16 in the above case? if you mean each sub-batch has 64 images). You’ll want to either scale the learning rate or the loss to take into account the fact that the gradients aren’t properly averaged over all the sub-batches, but given that you’re following that paper I suspect you’re already aware of these scalings