But let’s say we count the amount of time it takes for an epoch to finish. That means that it would take the same amount as it would take for a single GPU with a batch size of 64 right? Because we are iterating over the dataset with 64 samples at a time. It’s just that it would converge faster (in less epochs) because the total amount of samples seen in a single epoch is actually 4 times the amount seen when we are using 1 GPU. Am I correct?