Body: <p>As the title says, I want to train a Jordan network (i.e. a particular kind of <a href="https://en.wikipedia.org/wiki/Recurrent_neural_network" rel="nofollow noreferrer">recurrent neural network</a>) using a certain number of time series.</p>
  
  <p>Let's say that <span class="math-container">$x_1, x_2, \ldots x_N$</span> are <span class="math-container">$N$</span> input time series (i.e. <span class="math-container">$x_i = [x_{i,1}, x_{i,2}, \ldots, x_{i,T}]$</span>, where <span class="math-container">$T$</span> is the length of the time series) and <span class="math-container">$y_1, y_2, \ldots y_N$</span> (i.e. <span class="math-container">$x_i = [y_{i,1}, y_{i,2}, \ldots, y_{i,T}]$</span>) are the corresponding target time series. </p>
  
  <p>More specifically, the target time series are just sequences of "<span class="math-container">$0$</span>s", which may end with sequences of "<span class="math-container">$1$</span>"s. Here I show you some example:</p>
  
  <p><span class="math-container">$$y_i = [0 ~ 0 ~ 0 \ldots 0 ~ 0 ~ 1 ~ 1 ~ 1 \ldots 1 ~1 ], $$</span>
  <span class="math-container">$$y_i = [0 ~ 0 ~ 0 \ldots 0 ~ 0]. $$</span></p>
  
  <p>This means that I want that my machine "learn to raise" under some situations related to the corresponding inputs <span class="math-container">$x_i$</span>. Indeed, the objective of my network is to "raise" an alarm if "something" happens.</p>
  
  <p>At the moment, my training strategy is the following. I create a new time series which corresponds the <strong>concatenation</strong> of all the available <span class="math-container">$x_i$</span> and <span class="math-container">$y_i$</span>. Let's call the concatenated series <span class="math-container">$X$</span> and <span class="math-container">$Y$</span>. Then I use <span class="math-container">$X$</span> and <span class="math-container">$Y$</span> to train a network.</p>
  
  <p>Here is my problem. If I concatenate, then I also teach to my machine to "drop", since I can have situation like this:</p>
  
  <p><span class="math-container">$$Y = [ \ldots 1 ~ 1 ~ 0 ~ 0 \ldots].$$</span></p>
  
  <p>Is this really a problem? Are there other "training strategies" to be employed so that I avoid this kind of unwanted behaviors?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-14T14:13:50.723Z
Comments.Text: My first thought when reading this was that the issue would be with the Xs instead. By concatenating the Xs, you are artificially including a transition from x_{i,T} to x_{j, 1}. If this transition does actually happen in your data, you might get some interference so its worth exploring your data for that. Otherwise, I think that this approach is fine.
Comments.UserDisplayName: Jaden Travnik
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-14T18:50:59.900Z
Comments.Text: @JadenTravnik you are right, thanks. Anyway, I'm trying to train a neural network since it is very difficult to assess if there is a real transition in the data $x$. Indeed, data $x$ are multidimensional. If there was a "graphical evidence", then maybe machine learning would have been unnecessary.
Comments.UserDisplayName: the_candyman
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-01-11T23:56:28.290Z
Comments.Text: Why a downvote? I think it is stupid to downvote without commenting.
Comments.UserDisplayName: the_candyman
Tags: <neural-networks><machine-learning><training><recurrent-neural-networks>
Title: Train a recurrent neural network by concatenating time series. Is it safe?
UserDisplayName: the_candyman
ThreadText: # Train a recurrent neural network by concatenating time series. Is it safe?
  
  (Asked by: username_0 on 2018-11-13)
  
  As the title says, I want to train a Jordan network (i.e. a particular kind of [recurrent neural network](https://en.wikipedia.org/wiki/Recurrent_neural_network)) using a certain number of time series.
  
  Let's say that $x\_1, x\_2, \ldots x\_N$ are $N$ input time series (i.e. $x\_i = [x\_{i,1}, x\_{i,2}, \ldots, x\_{i,T}]$, where $T$ is the length of the time series) and $y\_1, y\_2, \ldots y\_N$ (i.e. $x\_i = [y\_{i,1}, y\_{i,2}, \ldots, y\_{i,T}]$) are the corresponding target time series.
  
  More specifically, the target time series are just sequences of "$0$s", which may end with sequences of "$1$"s. Here I show you some example:
  
  $$y\_i = [0 ~ 0 ~ 0 \ldots 0 ~ 0 ~ 1 ~ 1 ~ 1 \ldots 1 ~1 ], $$
  $$y\_i = [0 ~ 0 ~ 0 \ldots 0 ~ 0]. $$
  
  This means that I want that my machine "learn to raise" under some situations related to the corresponding inputs $x\_i$. Indeed, the objective of my network is to "raise" an alarm if "something" happens.
  
  At the moment, my training strategy is the following. I create a new time series which corresponds the **concatenation** of all the available $x\_i$ and $y\_i$. Let's call the concatenated series $X$ and $Y$. Then I use $X$ and $Y$ to train a network.
  
  Here is my problem. If I concatenate, then I also teach to my machine to "drop", since I can have situation like this:
  
  $$Y = [ \ldots 1 ~ 1 ~ 0 ~ 0 \ldots].$$
  
  Is this really a problem? Are there other "training strategies" to be employed so that I avoid this kind of unwanted behaviors?
  
  **Tags:** neural-networks, machine-learning, training, recurrent-neural-networks
  
  > **Comments:**
  > **username_1**: My first thought when reading this was that the issue would be with the Xs instead. By concatenating the Xs, you are artificially including a transition from x_{i,T} to x_{j, 1}. If this transition does actually happen in your data, you might get some interference so its worth exploring your data for that. Otherwise, I think that this approach is fine.  
  > **username_0**: @username_1 you are right, thanks. Anyway, I'm trying to train a neural network since it is very difficult to assess if there is a real transition in the data $x$. Indeed, data $x$ are multidimensional. If there was a "graphical evidence", then maybe machine learning would have been unnecessary.  
  > **username_0**: Why a downvote? I think it is stupid to downvote without commenting.  
  
  ---
  
  ## 0 answers
  
  