Answers.Body: <p>I don't see any special characteristic in the problem you're posing. Any LSTM can handle multidimensional inputs (i.e. multiple features). You just need to prepare your data such as they will have shape <strong>[batch_size, time_steps, n_features]</strong>, which is the format required by all main DL libraries (pytorch, keras and tensorflow). </p>
  
  <p>I linked below 2 tutorials that shows how to implement an LSTM for part of speech tagging in Keras and Pytorch. This task is conceptually identical to what you want to achieve: use 2D inputs (i.e. embeddings) to predict the class (i.e. the pos tags) of each element of a sequence (i.e. every single word). For example:</p>
  
  <pre><code># Input sample, 4 steps, encoded in 2D embeddings matrix
  ['we', 'had', 'crispy', 'fries']
  
  # Output predictions, 4 labels
  ['PRP', 'VBD', 'JJ', 'NNS']
  </code></pre>
  
  <p>In your case the predicted label would be 0, 1 or 2 and you would have to encode your data in a matrix of shape <strong>[n_batch, 600, 5]</strong>. The only thing you definetly want to pay attention, since you're using temporal data, is to not shuffle your data at all before the training. </p>
  
  <p>Keras tutorial: <a href="https://nlpforhackers.io/lstm-pos-tagger-keras/" rel="nofollow noreferrer">Build a POS tagger with an LSTM using Keras</a></p>
  
  <p>Pytorch tutorial: <a href="https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html" rel="nofollow noreferrer">LSTM’s in Pytorch</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-10T10:53:58.710Z
Answers.Text: I get the following error -->                                                                                               ValueError: Error when checking target: expected activation_6 to have 4 dimensions, but got array with shape (80, 600, 1)
Answers.UserDisplayName: mgb
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-10T11:19:57.353Z
Answers.Text: could you add your code to the question? It's hard to help you debugging it without seeing it .
Answers.UserDisplayName: Edoardo Guerriero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-10T11:46:03.747Z
Answers.Text: Sure. The 2D matrix is of size (600, 5) 600 rows that represent a value every second and each row shows specific characteristic at that point length, width, shape, color, material.  Each matrix is taken every 2 hrs. I will have per day 12 samples. I collected so far just 160 samples of the same length (600,5).  My training data is the concatenation of all these 2D arrays.
Answers.UserDisplayName: mgb
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-10T00:18:47.683Z
Answers.LastActivityDate: 2020-04-10T00:18:47.683Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>I am trying to train a LSTM, but I have some problems regarding the data representation and feeding it into the model.</p>
  
  <p>My data is a numpy array of three dimensions: One sample consist of a 2D matrix of size (600,5). 600(timesteps) and 5(features). However, I have 160 samples or files that represent the behavior of a user in multiple days. Altogether, my data has a dimension of (160,600,5).</p>
  
  <p>The label set is an array of 600 elements which describes certain patterns of each 2D matrix. The shape of the output should be (600,1).</p>
  
  <p>My question is how can I train the LSTM to the corresponding label set? What would be the best approach to handle this problem? The idea is that the output should be an array of (600,1) with 3 label inside.</p>
  
  <pre><code>Multiple_outputs {0,1,2}
        Output:    0000000001111111110000022222220000000000000
                   -------------600 samples ------------------
  
  Input: (1, 600, 5) 
  Output: (600, 1) 
  Training: (160,600,5)
  </code></pre>
  
  <p>I look forward for some ideas!</p>
  
  <pre><code>dataset(160,600,5)
  
  X_train, X_test, y_train, y_test = train_test_split(dataset[:,:,0:4], dataset[:,:,4:5],test_size = 0.30)
  
  model = Sequential()
  model.add(InputLayer(batch_input_shape = (92,600,5 )))
  model.add(Embedding(600, 128))
  #model.add(Bidirectional(LSTM(256, return_sequences=True)))
  model.add(TimeDistributed(Dense(2)))
  model.add(Activation('softmax'))
  
  model.compile(loss='categorical_crossentropy',
                optimizer=Adam(0.001),
                metrics=['accuracy'])
  
  model.summary()
  
  
  model.fit(X_train,y_train, batch_size=92, epochs=40, validation_split=0.2)
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-06-02T08:22:37.323Z
Comments.Text: it is called [Over-parameterization](https://medium.com/better-ml/what-are-over-parameterized-models-40c53c304367#:~:text=It%20refers%20to%20the%20scenario,in%20an%20over%2Dparameterized%20regime.) - "It refers to the scenario where the number of parameters of the model exceed the size of the training dataset or a similar threshold. Many neural networks are trained in an over-parameterized regime." - there's nothing extra-ordinary in your case for DL
Comments.UserDisplayName: JeeyCi
Tags: <recurrent-neural-networks><long-short-term-memory><multi-label-classification>
Title: How to train a LSTM with multidimensional data
UserDisplayName: mgb
ThreadText: # How to train a LSTM with multidimensional data
  
  (Asked by: username_0 on 2020-04-09)
  
  I am trying to train a LSTM, but I have some problems regarding the data representation and feeding it into the model.
  My data is a numpy array of three dimensions: One sample consist of a 2D matrix of size (600,5). 600(timesteps) and 5(features). However, I have 160 samples or files that represent the behavior of a user in multiple days. Altogether, my data has a dimension of (160,600,5).
  The label set is an array of 600 elements which describes certain patterns of each 2D matrix. The shape of the output should be (600,1).
  My question is how can I train the LSTM to the corresponding label set? What would be the best approach to handle this problem? The idea is that the output should be an array of (600,1) with 3 label inside.
  Multiple_outputs {0,1,2}
        Output:    0000000001111111110000022222220000000000000
                   -------------600 samples ------------------
  
  Input: (1, 600, 5) 
  Output: (600, 1) 
  Training: (160,600,5)
  
  I look forward for some ideas!
  dataset(160,600,5)
  
  X_train, X_test, y_train, y_test = train_test_split(dataset[:,:,0:4], dataset[:,:,4:5],test_size = 0.30)
  
  model = Sequential()
  model.add(InputLayer(batch_input_shape = (92,600,5 )))
  model.add(Embedding(600, 128))
  #model.add(Bidirectional(LSTM(256, return_sequences=True)))
  model.add(TimeDistributed(Dense(2)))
  model.add(Activation('softmax'))
  
  model.compile(loss='categorical_crossentropy',
                optimizer=Adam(0.001),
                metrics=['accuracy'])
  
  model.summary()
  
  
  model.fit(X_train,y_train, batch_size=92, epochs=40, validation_split=0.2)
  
  **Tags:** recurrent-neural-networks, long-short-term-memory, multi-label-classification
  > **Comments:**
  > **username_1**: it is called [Over-parameterization](https://medium.com/better-ml/what-are-over-parameterized-models-40c53c304367#:~:text=It%20refers%20to%20the%20scenario,in%20an%20over%2Dparameterized%20regime.) - "It refers to the scenario where the number of parameters of the model exceed the size of the training dataset or a similar threshold. Many neural networks are trained in an over-parameterized regime." - there's nothing extra-ordinary in your case for DL  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-04-10. Score: 1
  
  I don't see any special characteristic in the problem you're posing. Any LSTM can handle multidimensional inputs (i.e. multiple features). You just need to prepare your data such as they will have shape [batch_size, time_steps, n_features], which is the format required by all main DL libraries (pytorch, keras and tensorflow). 
  I linked below 2 tutorials that shows how to implement an LSTM for part of speech tagging in Keras and Pytorch. This task is conceptually identical to what you want to achieve: use 2D inputs (i.e. embeddings) to predict the class (i.e. the pos tags) of each element of a sequence (i.e. every single word). For example:
  # Input sample, 4 steps, encoded in 2D embeddings matrix
  ['we', 'had', 'crispy', 'fries']
  
  # Output predictions, 4 labels
  ['PRP', 'VBD', 'JJ', 'NNS']
  
  In your case the predicted label would be 0, 1 or 2 and you would have to encode your data in a matrix of shape [n_batch, 600, 5]. The only thing you definetly want to pay attention, since you're using temporal data, is to not shuffle your data at all before the training. 
  Keras tutorial: Build a POS tagger with an LSTM using Keras
  Pytorch tutorial: LSTM’s in Pytorch
  
  > **Comments:**
  > **username_0**: I get the following error -->                                                                                               ValueError: Error when checking target: expected activation_6 to have 4 dimensions, but got array with shape (80, 600, 1)  
  > **username_2**: could you add your code to the question? It's hard to help you debugging it without seeing it .  
  > **username_0**: Sure. The 2D matrix is of size (600, 5) 600 rows that represent a value every second and each row shows specific characteristic at that point length, width, shape, color, material.  Each matrix is taken every 2 hrs. I will have per day 12 samples. I collected so far just 160 samples of the same length (600,5).  My training data is the concatenation of all these 2D arrays.  
  
  ---
  
  