Answers.Body: <p>From Nvidia www (<a href="https://developer.nvidia.com/discover/lstm" rel="nofollow noreferrer">https://developer.nvidia.com/discover/lstm</a>):</p>
  <blockquote>
  <h3>Accelerating Long Short-Term Memory using GPUs</h3>
  <p>The parallel processing capabilities of GPUs can accelerate the LSTM training and inference processes. GPUs are the de-facto standard for LSTM usage and deliver a 6x speedup during training and <strong>140x higher throughput during inference when compared to CPU implementations</strong>. cuDNN is a GPU-accelerated deep neural network library that supports training of LSTM recurrent neural networks for sequence learning. TensorRT is a deep learning model optimizer and runtime that supports inference of LSTM recurrent neural networks on GPUs. Both cuDNN and TensorRT are part of the NVIDIA Deep Learning SDK.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-09T08:20:57.967Z
Answers.LastActivityDate: 2021-12-17T20:12:35.283Z
Answers.LastEditDate: 2021-12-17T20:12:35.283Z
Answers.UserDisplayName: pasaba por aqui
Answers.Body: <p>I found that there are cuDNN accelerated cells in Keras, for example, <a href="https://keras.io/layers/recurrent/#cudnnlstm" rel="nofollow noreferrer">https://keras.io/layers/recurrent/#cudnnlstm</a>.
  They are very fast. The normal LSTM cells are faster on CPU than on GPU.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-22T06:08:03.157Z
Answers.Text: All the well known deep learning frameworks have gpu accelaration facility for that matter (not only keras). And most of them use cuDNN underneath but exposing different API calls for the user..
Answers.UserDisplayName: varsh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-23T08:52:02.163Z
Answers.Text: Well - the above was just an example. Because normal LSTM Cells are slower...
Answers.UserDisplayName: Dieshe
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-21T16:20:45.313Z
Answers.LastActivityDate: 2021-12-17T20:09:09.530Z
Answers.LastEditDate: 2021-12-17T20:09:09.530Z
Answers.UserDisplayName: Dieshe
Body: <p>I am training LSTM neural networks with Keras on a small mobile GPU. The speed on the GPU is slower than on the CPU. I found some articles that say that it is hard to train LSTMs (and, in general, RNNs) on GPUs because the training cannot be parallelized.</p>
  <p>Is this true? Is LSTM training on large GPUs, like 1080 Ti, faster than on CPUs?</p>
  
Tags: <training><tensorflow><keras><long-short-term-memory><gpu>
Title: Can LSTM neural networks be sped up by a GPU?
UserDisplayName: Dieshe
ThreadText: # Can LSTM neural networks be sped up by a GPU?
  
  (Asked by: username_0 on 2018-07-09)
  
  I am training LSTM neural networks with Keras on a small mobile GPU. The speed on the GPU is slower than on the CPU. I found some articles that say that it is hard to train LSTMs (and, in general, RNNs) on GPUs because the training cannot be parallelized.
  
  Is this true? Is LSTM training on large GPUs, like 1080 Ti, faster than on CPUs?
  
  **Tags:** training, tensorflow, keras, long-short-term-memory, gpu
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2018-07-21. Score: 6 | Accepted answer
  
  I found that there are cuDNN accelerated cells in Keras, for example, <https://keras.io/layers/recurrent/#cudnnlstm>.
  They are very fast. The normal LSTM cells are faster on CPU than on GPU.
  
  > **Comments:**
  > **username_2**: All the well known deep learning frameworks have gpu accelaration facility for that matter (not only keras). And most of them use cuDNN underneath but exposing different API calls for the user..  
  > **username_0**: Well - the above was just an example. Because normal LSTM Cells are slower...  
  
  ---
  
  ### Answer by username_1 on 2018-07-09. Score: 9
  
  From Nvidia www (<https://developer.nvidia.com/discover/lstm>):
  
  > ### Accelerating Long Short-Term Memory using GPUs
  >
  > The parallel processing capabilities of GPUs can accelerate the LSTM training and inference processes. GPUs are the de-facto standard for LSTM usage and deliver a 6x speedup during training and **140x higher throughput during inference when compared to CPU implementations**. cuDNN is a GPU-accelerated deep neural network library that supports training of LSTM recurrent neural networks for sequence learning. TensorRT is a deep learning model optimizer and runtime that supports inference of LSTM recurrent neural networks on GPUs. Both cuDNN and TensorRT are part of the NVIDIA Deep Learning SDK.
  
  
  ---
  
  