Answers.Body: <p>Instead of feeding the network the samples at just a single timestamp <span class="math-container">$t$</span>, you should feed it the last <span class="math-container">$n$</span> observations. Since you have two &quot;channels&quot; <span class="math-container">$x_1$</span> and <span class="math-container">$x_2$</span>, your input to the network would have dimensions <span class="math-container">$\texttt{batch_size} \times n \times 2$</span> (or <span class="math-container">$\texttt{batch_size} \times 2 \times n$</span> depending on your framework's conventions). These can then be processed by 1D convolutions, flattened and then the final output is produced by fully connected dense layers.</p>
  <p>Alternatively you could flatten the input to <span class="math-container">$\texttt{batch_size} \times 2n$</span> and use only dense layers, but this is more prone to overfitting especially as <span class="math-container">$n$</span> grows larger.</p>
  <p>Since <span class="math-container">$n$</span> is a hyperparameter, you'll need to run experiments to determine how large it should be. It is best to test it with exponential steps, like <span class="math-container">$1, 2, 4, 8, 16, ...$</span></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-03T14:49:28.183Z
Answers.LastActivityDate: 2022-11-03T14:49:28.183Z
Answers.UserDisplayName: NikoNyrh
Answers.Body: <p>As for using a Neural Network for a multivariate problem I suggest that you investigate an LSTM architecture, that takes into account the temporality of events.</p>
  <p>E.g.</p>
  <pre><code>model = Sequential()  
  model.add(LSTM(200, activation='relu', input_shape=(n_steps_in, n_features)))  
  model.add(RepeatVector(n_steps_out))  
  model.add(LSTM(200, activation='relu', return_sequences=True))  
  model.add(TimeDistributed(Dense(n_features)))  
  model.compile(optimizer='adam', loss='mse') 
  </code></pre>
  <p>I also suggest that you investigate traditional statistics models such as SARIMAX, and test for Autocorrelation and Partial Autocorrelation (the incidence of the last event, and the incidence of the events chain on your actual sample).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-02T07:29:14.007Z
Answers.LastActivityDate: 2022-11-03T06:29:46.620Z
Answers.LastEditDate: 2022-11-03T06:29:46.620Z
Answers.UserDisplayName: Gian Piero Anselmi
Body: <p>I would like to approximate the following relation by a neural network</p>
  <p><span class="math-container">$y = \mathcal{f}(x_1(t),x_2(t))$</span></p>
  <p>Here, I have only one output variable that is a function of 2 other variables which vary in time. Now, I want to be able to predict <span class="math-container">$y$</span>, given any shape of the 2 independent variables in time. For this reason, I have a training set corresponding to different input and output signals. However, I don't know how to make the neural network understand the concept of time which is very important since I expect the solution at time <span class="math-container">$t_k$</span> to be influenced by the previous instants in time. For this reason, I added as input variable the time derivative as</p>
  <p><span class="math-container">$y = \mathcal{f}\left(x_1(t),x_2(t), \dfrac{\partial x_1 (t)}{\partial t}\right)$</span></p>
  <p>This solution seems to work quite well for the fully connected neural network that I'm using. However, I would like to know if there are other ways to treat such problems where the time history is important.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-02T22:38:47.323Z
Comments.Text: Please, put your **specific question** in the title. "Time series prediction" is **not** a question and it's also not specific.
Comments.UserDisplayName: nbro
Tags: <neural-networks><machine-learning>
Title: How to predict time signal based on multi-input signals?
UserDisplayName: Fly
ThreadText: # How to predict time signal based on multi-input signals?
  
  (Asked by: username_0 on 2022-07-02)
  
  I would like to approximate the following relation by a neural network
  
  $y = \mathcal{f}(x\_1(t),x\_2(t))$
  
  Here, I have only one output variable that is a function of 2 other variables which vary in time. Now, I want to be able to predict $y$, given any shape of the 2 independent variables in time. For this reason, I have a training set corresponding to different input and output signals. However, I don't know how to make the neural network understand the concept of time which is very important since I expect the solution at time $t\_k$ to be influenced by the previous instants in time. For this reason, I added as input variable the time derivative as
  
  $y = \mathcal{f}\left(x\_1(t),x\_2(t), \dfrac{\partial x\_1 (t)}{\partial t}\right)$
  
  This solution seems to work quite well for the fully connected neural network that I'm using. However, I would like to know if there are other ways to treat such problems where the time history is important.
  
  **Tags:** neural-networks, machine-learning
  > **Comments:**
  > **username_1**: Please, put your **specific question** in the title. "Time series prediction" is **not** a question and it's also not specific.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2022-11-03. Score: 0
  
  Instead of feeding the network the samples at just a single timestamp $t$, you should feed it the last $n$ observations. Since you have two "channels" $x\_1$ and $x\_2$, your input to the network would have dimensions $\texttt{batch\_size} \times n \times 2$ (or $\texttt{batch\_size} \times 2 \times n$ depending on your framework's conventions). These can then be processed by 1D convolutions, flattened and then the final output is produced by fully connected dense layers.
  
  Alternatively you could flatten the input to $\texttt{batch\_size} \times 2n$ and use only dense layers, but this is more prone to overfitting especially as $n$ grows larger.
  
  Since $n$ is a hyperparameter, you'll need to run experiments to determine how large it should be. It is best to test it with exponential steps, like $1, 2, 4, 8, 16, ...$
  
  
  ---
  
  ### Answer by username_3 on 2022-11-02. Score: 0
  
  As for using a Neural Network for a multivariate problem I suggest that you investigate an LSTM architecture, that takes into account the temporality of events.
  E.g.
  model = Sequential()  
  model.add(LSTM(200, activation='relu', input_shape=(n_steps_in, n_features)))  
  model.add(RepeatVector(n_steps_out))  
  model.add(LSTM(200, activation='relu', return_sequences=True))  
  model.add(TimeDistributed(Dense(n_features)))  
  model.compile(optimizer='adam', loss='mse') 
  
  I also suggest that you investigate traditional statistics models such as SARIMAX, and test for Autocorrelation and Partial Autocorrelation (the incidence of the last event, and the incidence of the events chain on your actual sample).
  
  
  ---
  
  