Answers.Body: <p>I advice you to always use GPU over CPU for training your models. This is driven by the usage of deep learning methods on images and texts, where the data is very rich.</p>
  
  <p>You must have a GPU suited perfectly for training (e.g. NVIDIA 1080, NVIDIA Titan or higher versions), I wouldn't be surprised to find that your CPU was faster if you don't have a powerful GPU.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-24T18:45:10.377Z
Answers.Text: Yeah, I forgot to mention my gpu is gtx 970, cpu is xeon e-3 1220. Like I said I only train numerical tabular data, no images or text.
Answers.UserDisplayName: GKozinski
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-24T17:21:18.877Z
Answers.LastActivityDate: 2019-08-24T17:21:18.877Z
Answers.UserDisplayName: Varshitha N S
Answers.Body: <p>It depends, if you ve to solve a "simple" problem which does not require CNN or stacked models without multidimensional data and not many multiplications, big long numbers then if you decide to use CNN / stacked architectures AND GPU it is like using a hammer to insert a needle.It will not only spend energy but the computations will do zero padding in memory, you will observe a degradation in speed.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-26T00:25:15.360Z
Answers.LastActivityDate: 2019-08-30T13:48:59.523Z
Answers.LastEditDate: 2019-08-30T13:48:59.523Z
Answers.UserDisplayName: nikolaosmparoutis
Answers.Body: <p>This changes according to your data and complexity of your models. See following article by <a href="https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-deployment-of-deep-learning-models/" rel="noreferrer">microsoft</a>. Their conclusion is </p>
  
  <blockquote>
    <p>The results suggest that the throughput from GPU clusters is always
    better than CPU throughput for all models and frameworks proving that
    GPU is the economical choice for inference of deep learning models.
    ...</p>
    
    <p>It is important to note that, for standard machine learning models
    where number of parameters are not as high as deep learning models,
    CPUs should still be considered as more effective and cost efficient.</p>
  </blockquote>
  
  <p>Since you are training MLP, it can not be thought as standard machine learning model. See my <a href="https://peerj.com/preprints/2838/" rel="noreferrer">preprint, The impact of using large training data set KDD99 on classification accuracy</a>. I compare different machine learning algorithms using weka.</p>
  
  <p><a href="https://i.sstatic.net/H9Ogn.png" rel="noreferrer"><img src="https://i.sstatic.net/H9Ogn.png" alt="enter image description here"></a></p>
  
  <p>As you can see from above image, MLP takes 722 minutes to train while Naive Bayes ~2 minutes. If your data is small and your models parameters are not high, you will see better performance on CPU.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-25T15:10:46.110Z
Answers.Text: You are correct. I run some tests and found out that the networks that are trained faster on GPU are these that are bigger (more layers, more neurons). Smaller ones are faster on CPU. Thanks for clearing this out!
Answers.UserDisplayName: GKozinski
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-02-11T21:25:42.740Z
Answers.Text: Also worth noting that some network organizations may not be well suited to operating on GPUs and CPU may be faster for those.  E.g. cascade correlation networks, where each node can depend on the output of any other node with a lower node number, so it is essentially impossible to calculate them in parallel.
Answers.UserDisplayName: occipita
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-08-25T08:08:53.130Z
Answers.LastActivityDate: 2019-08-25T08:08:53.130Z
Answers.UserDisplayName: Atilla Ozgur
Body: <p>Currently, I am working on a few projects that use feedforward neural networks for regression and classification of simple tabular data. I have noticed that training a neural network using TensorFlow-GPU is often slower than training the same network using TensorFlow-CPU.</p>
  <p>Could something be wrong with my setup/code or is it possible that sometimes GPU is slower than CPU?</p>
  
Tags: <neural-networks><training><tensorflow><gpu>
Title: Is a GPU always faster than a CPU for training neural networks?
UserDisplayName: GKozinski
ThreadText: # Is a GPU always faster than a CPU for training neural networks?
  
  (Asked by: username_0 on 2019-08-24)
  
  Currently, I am working on a few projects that use feedforward neural networks for regression and classification of simple tabular data. I have noticed that training a neural network using TensorFlow-GPU is often slower than training the same network using TensorFlow-CPU.
  
  Could something be wrong with my setup/code or is it possible that sometimes GPU is slower than CPU?
  
  **Tags:** neural-networks, training, tensorflow, gpu
  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2019-08-25. Score: 7 | Accepted answer
  
  This changes according to your data and complexity of your models. See following article by [microsoft](https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-deployment-of-deep-learning-models/). Their conclusion is
  
  > The results suggest that the throughput from GPU clusters is always
  > better than CPU throughput for all models and frameworks proving that
  > GPU is the economical choice for inference of deep learning models.
  > ...
  >
  > It is important to note that, for standard machine learning models
  > where number of parameters are not as high as deep learning models,
  > CPUs should still be considered as more effective and cost efficient.
  
  Since you are training MLP, it can not be thought as standard machine learning model. See my [preprint, The impact of using large training data set KDD99 on classification accuracy](https://peerj.com/preprints/2838/). I compare different machine learning algorithms using weka.
  
  [![enter image description here](https://i.sstatic.net/H9Ogn.png)](https://i.sstatic.net/H9Ogn.png)
  
  As you can see from above image, MLP takes 722 minutes to train while Naive Bayes ~2 minutes. If your data is small and your models parameters are not high, you will see better performance on CPU.
  
  > **Comments:**
  > **username_0**: You are correct. I run some tests and found out that the networks that are trained faster on GPU are these that are bigger (more layers, more neurons). Smaller ones are faster on CPU. Thanks for clearing this out!  
  > **username_4**: Also worth noting that some network organizations may not be well suited to operating on GPUs and CPU may be faster for those.  E.g. cascade correlation networks, where each node can depend on the output of any other node with a lower node number, so it is essentially impossible to calculate them in parallel.  
  
  ---
  
  ### Answer by username_1 on 2019-08-24. Score: 2
  
  I advice you to always use GPU over CPU for training your models. This is driven by the usage of deep learning methods on images and texts, where the data is very rich.
  
  You must have a GPU suited perfectly for training (e.g. NVIDIA 1080, NVIDIA Titan or higher versions), I wouldn't be surprised to find that your CPU was faster if you don't have a powerful GPU.
  
  > **Comments:**
  > **username_0**: Yeah, I forgot to mention my gpu is gtx 970, cpu is xeon e-3 1220. Like I said I only train numerical tabular data, no images or text.  
  
  ---
  
  ### Answer by username_2 on 2019-08-26. Score: 1
  
  It depends, if you ve to solve a "simple" problem which does not require CNN or stacked models without multidimensional data and not many multiplications, big long numbers then if you decide to use CNN / stacked architectures AND GPU it is like using a hammer to insert a needle.It will not only spend energy but the computations will do zero padding in memory, you will observe a degradation in speed.
  
  
  ---
  
  