Answers.Body: <p>As you read, in keras the input dimensions for a LSTM layer are:  <code>(batch_size, timesteps, input_dim)</code>. Where:</p>
  
  <ul>
  <li><code>batch_size</code>: number of samples (sentences in your case) to compute the loss before running the gradient descent. So number of sentences to train on before compute loss and optimize your model.</li>
  <li><code>timesteps</code>: length of the sequence, in your case length of the sentence</li>
  <li><code>input_dim</code>: the features of your data (the words)</li>
  </ul>
  
  <p>The nice thing about Keras is that you can train with an specific batch size, say <code>batch_size=16</code>, that: helps the model convergence (because it averages the loss of the 16 sentences prediction) and boosts the speed (since the weights are updated only once every 16 sentences).</p>
  
  <p>But then you can infer (or do the predictions) with <code>batch_size=1</code>, meaning, one sentence at a time.</p>
  
  <p>So, if you want to train, specifically with one sentence at a time: <code>batch_size=1</code>. But if you want to take the advantage of using mini-batch then use <code>batch_size=16</code> or higher for training and <code>batch_size=1</code> for inference.</p>
  
  <hr>
  
  <p>BONUS: Keras allows for dynamic batch size change in the models. When a dimension is dynamic in Keras has the value of <code>None</code>, that is why when you do <code>model.summary()</code> you can see: <code>(batch_size, timesteps, input_dim) = (None, 40, 100)</code>. This <code>None</code> allows for different <code>batch_size</code> in training and inference</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-11T11:43:54.650Z
Answers.Text: Thank you very much for your answers: I'll try to implement it.
Answers.UserDisplayName: pairon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-11T20:54:24.563Z
Answers.Text: @JVGC is it right pass to the ```model.fit()```a 3D dataset (list of matrix, each matrix is a list of words)? And if it's right, which should be the dimension of the labels?
Answers.UserDisplayName: pairon
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-12T08:56:09.093Z
Answers.Text: @pairon, as long as the input data have the dimension of the model inputs, it is right. Normally `.fit()` is used for when data can be loaded into memory and there is no data augmentation. On the other hand, `.fit_generator()` is used for when data do not fit in memory. But the `.fit()` input only needs to mach the model inputs.
Answers.UserDisplayName: JVGD
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-11T10:42:00.443Z
Answers.LastActivityDate: 2020-02-11T10:42:00.443Z
Answers.UserDisplayName: JVGD
Body: <p>I would like to build an LSTM to predict the correct words order given a sentence. My dataset is composed of sentences, where each sentence has a variable number of words (each word is embedded). The dataset then is an array of matrices, where each matrix is an array of embedded words.</p>
  
  <p>Now, I'm looking to implement it with Keras but I'm not sure how to fit the necessary parameters wanted by the LSTM layer in Keras, like <code>timesteps</code> and <code>batch_size</code>. </p>
  
  <p>Reading on the web, I notice that <code>timesteps</code> is the length of the sequence, so in my case I believe that corresponds to the length of the sentence. But I want to train my LSTM with one sentence at a time, so would the <code>batch_size</code> be 1?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-05-02T22:14:53.323Z
Comments.Text: I’m closing this question because general programming questions (including how to use an API or library) are off-topic. See our on-topic page: [https://ai.stackexchange.com/help/on-topic](https://ai.stackexchange.com/help/on-topic).
Comments.UserDisplayName: nbro
Tags: <keras><long-short-term-memory>
Title: LSTM implementation in KERAS
UserDisplayName: pairon
ThreadText: # LSTM implementation in KERAS
  
  (Asked by: username_0 on 2020-02-10)
  
  I would like to build an LSTM to predict the correct words order given a sentence. My dataset is composed of sentences, where each sentence has a variable number of words (each word is embedded). The dataset then is an array of matrices, where each matrix is an array of embedded words.
  
  Now, I'm looking to implement it with Keras but I'm not sure how to fit the necessary parameters wanted by the LSTM layer in Keras, like `timesteps` and `batch_size`.
  
  Reading on the web, I notice that `timesteps` is the length of the sequence, so in my case I believe that corresponds to the length of the sentence. But I want to train my LSTM with one sentence at a time, so would the `batch_size` be 1?
  
  **Tags:** keras, long-short-term-memory
  > **Comments:**
  > **username_1**: I’m closing this question because general programming questions (including how to use an API or library) are off-topic. See our on-topic page: [https://ai.stackexchange.com/help/on-topic](https://ai.stackexchange.com/help/on-topic).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-02-11. Score: 0 | Accepted answer
  
  As you read, in keras the input dimensions for a LSTM layer are: `(batch_size, timesteps, input_dim)`. Where:
  
  * `batch_size`: number of samples (sentences in your case) to compute the loss before running the gradient descent. So number of sentences to train on before compute loss and optimize your model.
  * `timesteps`: length of the sequence, in your case length of the sentence
  * `input_dim`: the features of your data (the words)
  
  The nice thing about Keras is that you can train with an specific batch size, say `batch_size=16`, that: helps the model convergence (because it averages the loss of the 16 sentences prediction) and boosts the speed (since the weights are updated only once every 16 sentences).
  
  But then you can infer (or do the predictions) with `batch_size=1`, meaning, one sentence at a time.
  
  So, if you want to train, specifically with one sentence at a time: `batch_size=1`. But if you want to take the advantage of using mini-batch then use `batch_size=16` or higher for training and `batch_size=1` for inference.
  
  ---
  
  BONUS: Keras allows for dynamic batch size change in the models. When a dimension is dynamic in Keras has the value of `None`, that is why when you do `model.summary()` you can see: `(batch_size, timesteps, input_dim) = (None, 40, 100)`. This `None` allows for different `batch_size` in training and inference
  
  > **Comments:**
  > **username_0**: Thank you very much for your answers: I'll try to implement it.  
  > **username_0**: @JVGC is it right pass to the ```model.fit()```a 3D dataset (list of matrix, each matrix is a list of words)? And if it's right, which should be the dimension of the labels?  
  > **username_2**: @pairon, as long as the input data have the dimension of the model inputs, it is right. Normally `.fit()` is used for when data can be loaded into memory and there is no data augmentation. On the other hand, `.fit_generator()` is used for when data do not fit in memory. But the `.fit()` input only needs to mach the model inputs.  
  
  ---
  
  