Answers.Body: <p>I don't think you need to go for aggregation -- this looks like a job for VARIMA, the vector-version of ARIMA. In ARIMA, the output of the sequence at time <span class="math-container">$t$</span>, which can be notated <span class="math-container">$X_t$</span>, is a function of the past inputs <span class="math-container">$\{X_1, X_2, \dots, X_{t-1}\}$</span>. For a univariate <span class="math-container">$AR(k)$</span> process, the corresponding ARIMA model is given by</p>
  <p><span class="math-container">$$ X_t - \sum_{i=1}^k \alpha_i X_{t-i} = \varepsilon_t + \sum_{i=1}^k \theta_{i}\varepsilon_{t-i}$$</span></p>
  <p>with parameters <span class="math-container">$\alpha_i, \theta_i$</span>, inputs <span class="math-container">$X_i$</span>, and i.i.d. zero-mean Gaussian error terms <span class="math-container">$\varepsilon_i$</span>. The generalization of this to multiple variables is thus simply</p>
  <p><span class="math-container">$$\mathbf{x}_t - \sum_{i=1}^k \mathbf{A}_i \mathbf{x}_{t-1} = \mathbf{e}_t + \sum_{i=1}^k \mathbf{\Theta}_i \mathbf{e}_{t-i}$$</span></p>
  <p>for <span class="math-container">$\mathbf{x}_i, \mathbf{e}_i \in \mathbb{R}^n$</span>. Note that before, the parameters were scalars. Now, <span class="math-container">$\mathbf{A}_i, \mathbf{\Theta}_i \in \mathbb{R}^{n\times n}$</span> -- size-<span class="math-container">$n$</span> square matrices.</p>
  <p>It looks like there's a <a href="https://rdrr.io/github/Mthrun/TSAT/man/VARIMA.html" rel="nofollow noreferrer">Github implementation here</a> as well, though I haven't looked closely at this.</p>
  <p>If you're going for an LSTM-based sequence modeling approach, this is even easier -- an LSTM cell can take in input of arbitrary dimensions, so you shouldn't have to make any changes.</p>
  <p>If you'd like to see the math, concretely, the LSTM forward pass equations have the form</p>
  <p><span class="math-container">$$(\cdot)^{(t)} = g(\mathbf{W}^{(\cdot)} x^{(t)} + \mathbf{U}^{(\cdot)}x^{(t)} + \mathbf{b}^{(\cdot)})$$</span></p>
  <p>where <span class="math-container">$\mathbf{W}, \mathbf{U}$</span> are the parameter matrices associated with the inputs and hidden states, respectively, for each gate, and <span class="math-container">$\mathbf{b}$</span> is a bias term. So in the single-variable case, <span class="math-container">$\mathbf{W}, \mathbf{U}, \mathbf{b}$</span> are all scalars; in the multivariate case, <span class="math-container">$\mathbf{W}, \mathbf{U}$</span> are now size-<span class="math-container">$n$</span> square matrices, and <span class="math-container">$\mathbf{b}$</span> is a vector of length <span class="math-container">$n$</span>. <strong>No further modification is needed, and you should be able to just plug-and-play with most deep learning libraries.</strong></p>
  <p>So your inputs would just be the vector of attributes at a particular time step.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-20T16:16:31.950Z
Answers.LastActivityDate: 2021-07-20T16:16:31.950Z
Answers.UserDisplayName: chang_trenton
Body: <p>I have a question</p>
  <p>I have had a dataset below</p>
  <pre><code>             sensor1   sensor2  sensor3 ...
  2021-01-01    1.32       2.2      1.0
  2021-01-02    4.3        2.0      0.8 ...
  ...
  </code></pre>
  <p>I know ARMA model is useful for time-series forecasting</p>
  <p><a href="https://i.sstatic.net/cXzhB.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/cXzhB.png" alt="enter image description here" /></a></p>
  <p>However, how can I use ARMA model for data with multiple attributes ?</p>
  <p>If data with a single attribute can be a input for ARMA model, should I aggregate the attribute set?
  (for example, after normalizing each attribute, I add up all values every rows to transform all attributes to a single attribute)</p>
  
Tags: <long-short-term-memory><datasets><prediction><time-series><forecasting>
Title: How can I use a prediction model (e.g., ARMA model or LSTM) for multi-variate data?
UserDisplayName: Dae-Young Park
ThreadText: # How can I use a prediction model (e.g., ARMA model or LSTM) for multi-variate data?
  
  (Asked by: username_0 on 2021-07-20)
  
  I have a question
  I have had a dataset below
               sensor1   sensor2  sensor3 ...
  2021-01-01    1.32       2.2      1.0
  2021-01-02    4.3        2.0      0.8 ...
  ...
  
  I know ARMA model is useful for time-series forecasting
  
  However, how can I use ARMA model for data with multiple attributes ?
  If data with a single attribute can be a input for ARMA model, should I aggregate the attribute set?
  (for example, after normalizing each attribute, I add up all values every rows to transform all attributes to a single attribute)
  
  **Tags:** long-short-term-memory, datasets, prediction, time-series, forecasting
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-07-20. Score: 1
  
  I don't think you need to go for aggregation -- this looks like a job for VARIMA, the vector-version of ARIMA. In ARIMA, the output of the sequence at time $t$, which can be notated $X\_t$, is a function of the past inputs $\{X\_1, X\_2, \dots, X\_{t-1}\}$. For a univariate $AR(k)$ process, the corresponding ARIMA model is given by
  
  $$ X\_t - \sum\_{i=1}^k \alpha\_i X\_{t-i} = \varepsilon\_t + \sum\_{i=1}^k \theta\_{i}\varepsilon\_{t-i}$$
  
  with parameters $\alpha\_i, \theta\_i$, inputs $X\_i$, and i.i.d. zero-mean Gaussian error terms $\varepsilon\_i$. The generalization of this to multiple variables is thus simply
  
  $$\mathbf{x}\_t - \sum\_{i=1}^k \mathbf{A}\_i \mathbf{x}\_{t-1} = \mathbf{e}\_t + \sum\_{i=1}^k \mathbf{\Theta}\_i \mathbf{e}\_{t-i}$$
  
  for $\mathbf{x}\_i, \mathbf{e}\_i \in \mathbb{R}^n$. Note that before, the parameters were scalars. Now, $\mathbf{A}\_i, \mathbf{\Theta}\_i \in \mathbb{R}^{n\times n}$ -- size-$n$ square matrices.
  
  It looks like there's a [Github implementation here](https://rdrr.io/github/Mthrun/TSAT/man/VARIMA.html) as well, though I haven't looked closely at this.
  
  If you're going for an LSTM-based sequence modeling approach, this is even easier -- an LSTM cell can take in input of arbitrary dimensions, so you shouldn't have to make any changes.
  
  If you'd like to see the math, concretely, the LSTM forward pass equations have the form
  
  $$(\cdot)^{(t)} = g(\mathbf{W}^{(\cdot)} x^{(t)} + \mathbf{U}^{(\cdot)}x^{(t)} + \mathbf{b}^{(\cdot)})$$
  
  where $\mathbf{W}, \mathbf{U}$ are the parameter matrices associated with the inputs and hidden states, respectively, for each gate, and $\mathbf{b}$ is a bias term. So in the single-variable case, $\mathbf{W}, \mathbf{U}, \mathbf{b}$ are all scalars; in the multivariate case, $\mathbf{W}, \mathbf{U}$ are now size-$n$ square matrices, and $\mathbf{b}$ is a vector of length $n$. **No further modification is needed, and you should be able to just plug-and-play with most deep learning libraries.**
  
  So your inputs would just be the vector of attributes at a particular time step.
  
  
  ---
  
  