Answers.Body: <p>Currently, when using the code on this branch: <a href="https://github.com/benchopt/benchmark_resnet_classif/pull/53" rel="nofollow noreferrer">https://github.com/benchopt/benchmark_resnet_classif/pull/53</a>, I use 35 minutes per epoch to train a ResNet-18 on ImageNet in TensorFlow with a V100 GPU and a batch size of 128, with standard data augmentations.</p>
  <p>I haven't found other mentions of the training times for a ResNet-18 with a standard training policy, so I am just mentioning this to kick off the conversation without claiming that this is the best one can get.</p>
  <h2>EDIT</h2>
  <p>With <code>timm</code> it is possible on a single V100 GPU to reach 0.1s per iteration with 128 batch size (see <a href="https://github.com/rwightman/pytorch-image-models/discussions/1542#discussioncomment-4108700" rel="nofollow noreferrer">this discussion</a> I had with Ross Wightman). In total this gives a 16-minute epoch. I think this is a very good baseline and you can make it even faster with AMP, a larger batch size and of course distributing.
  My implementation <a href="https://github.com/benchopt/benchmark_resnet_classif/pull/53" rel="nofollow noreferrer">here in PyTorch</a>, achieves 22' per epoch, I might be missing some optimizations here and there.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-18T15:36:00.397Z
Answers.LastActivityDate: 2022-11-18T17:42:35.300Z
Answers.LastEditDate: 2022-11-18T17:42:35.300Z
Answers.UserDisplayName: Zaccharie Ramzi
Body: <p>My training of Resnet-18 network on Imagenet using Tesla V100 seems to be quite slow (1 epoch is about 2,5 hours, batch 128). Increasing the number of GPUs does not seem to help.</p>
  <p>What is your training time of Resnet-18/Resnet-50 on Imagenet? How many epochs do you train for to obtain the desired accuracy? I am wondering what I should expect.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-02T15:58:40.260Z
Comments.Text: Do you double the batch size  after increasing the number of gpus? So weird that the training speed still constant
Comments.UserDisplayName: CuCaRot
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-09-12T12:05:11.407Z
Comments.Text: Have you checked the proportion of time being spent in data loading? This sounds like a data loading bottleneck.
Comments.UserDisplayName: Neil Traft
Tags: <machine-learning><training><epochs><image-net>
Title: What is your training time of Resnet-18/Resnet-50 on Imagenet?
UserDisplayName: cerebrou
ThreadText: # What is your training time of Resnet-18/Resnet-50 on Imagenet?
  
  (Asked by: username_0 on 2022-05-02)
  
  My training of Resnet-18 network on Imagenet using Tesla V100 seems to be quite slow (1 epoch is about 2,5 hours, batch 128). Increasing the number of GPUs does not seem to help.
  
  What is your training time of Resnet-18/Resnet-50 on Imagenet? How many epochs do you train for to obtain the desired accuracy? I am wondering what I should expect.
  
  **Tags:** machine-learning, training, epochs, image-net
  > **Comments:**
  > **username_1**: Do you double the batch size  after increasing the number of gpus? So weird that the training speed still constant  
  > **username_2**: Have you checked the proportion of time being spent in data loading? This sounds like a data loading bottleneck.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2022-07-18. Score: 1
  
  Currently, when using the code on this branch: <https://github.com/benchopt/benchmark_resnet_classif/pull/53>, I use 35 minutes per epoch to train a ResNet-18 on ImageNet in TensorFlow with a V100 GPU and a batch size of 128, with standard data augmentations.
  
  I haven't found other mentions of the training times for a ResNet-18 with a standard training policy, so I am just mentioning this to kick off the conversation without claiming that this is the best one can get.
  
  ## EDIT
  
  With `timm` it is possible on a single V100 GPU to reach 0.1s per iteration with 128 batch size (see [this discussion](https://github.com/rwightman/pytorch-image-models/discussions/1542#discussioncomment-4108700) I had with Ross Wightman). In total this gives a 16-minute epoch. I think this is a very good baseline and you can make it even faster with AMP, a larger batch size and of course distributing.
  My implementation [here in PyTorch](https://github.com/benchopt/benchmark_resnet_classif/pull/53), achieves 22' per epoch, I might be missing some optimizations here and there.
  
  
  ---
  
  