Answers.Body: <p>So you want your network to represent those 3 values at each step as single composite value? I can't think of any better way than utilizing 3 LSTM units but attaching them to same write and read nodes of the enclosing network. In other words your <strong>assumption</strong> that it makes sense to keep all those 3 values together, gets <strong>hardcoded</strong> into your network by making 6 connections (3 connected to read and 3 to write) share their weights. Usually researchers leave it to the network to decide whether to keep such composite values together or separate (by learning read and write weights through backpropagation) and sometimes network chooses to keep them together all on its own like in <a href="https://youtu.be/93rzMHtYT_0?t=531" rel="nofollow noreferrer">https://youtu.be/93rzMHtYT_0?t=531</a> (it can be seen how network resets whole bunch of LSTM units simultaneously ). </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-15T04:34:51.127Z
Answers.LastActivityDate: 2018-01-15T04:34:51.127Z
Answers.UserDisplayName: Andrew Butenko
Body: <p>Lets say you install your LSTM machine on a road between London and Oxford. And it makes observations. A car with 3 people inside drives past it in one direction 21 sec after previously observed car (in any direction) : the input is {3, LO, 21}. A bus with 43 people drives in other direction 11 sec later : {43, OL, 11}</p>
  
  <p>I cant find a definitive general explanation (without references to existing ML packages code) - how your LSTM layer structure should look like to accept those params (there are 3 of them and they are vastly different between themselves but presenting them together at each step is very significant (as opposed to splitting them to 3 streams and feeding then into 3 LSTMs and then pooling the results).</p>
  
  <p>Could someone explain it in a single formula or a drawing?</p>
  
Tags: <machine-learning><long-short-term-memory>
Title: Multi-param LSTM input
UserDisplayName: Boppity Bop
ThreadText: # Multi-param LSTM input
  
  (Asked by: username_0 on 2017-05-28)
  
  Lets say you install your LSTM machine on a road between London and Oxford. And it makes observations. A car with 3 people inside drives past it in one direction 21 sec after previously observed car (in any direction) : the input is {3, LO, 21}. A bus with 43 people drives in other direction 11 sec later : {43, OL, 11}
  
  I cant find a definitive general explanation (without references to existing ML packages code) - how your LSTM layer structure should look like to accept those params (there are 3 of them and they are vastly different between themselves but presenting them together at each step is very significant (as opposed to splitting them to 3 streams and feeding then into 3 LSTMs and then pooling the results).
  
  Could someone explain it in a single formula or a drawing?
  
  **Tags:** machine-learning, long-short-term-memory
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-01-15. Score: 2
  
  So you want your network to represent those 3 values at each step as single composite value? I can't think of any better way than utilizing 3 LSTM units but attaching them to same write and read nodes of the enclosing network. In other words your **assumption** that it makes sense to keep all those 3 values together, gets **hardcoded** into your network by making 6 connections (3 connected to read and 3 to write) share their weights. Usually researchers leave it to the network to decide whether to keep such composite values together or separate (by learning read and write weights through backpropagation) and sometimes network chooses to keep them together all on its own like in <https://youtu.be/93rzMHtYT_0?t=531> (it can be seen how network resets whole bunch of LSTM units simultaneously ).
  
  
  ---
  
  