Answers.Body: <p>Yes, LSTM are ideal for this. For even stronger representational capacity, make your LSTM's multi-layered.
  Using 1-dimensional convolutions in a CNN is a common way to exctract information from time series too, so there's no harm in trying. Typically, you'll test many models out and take the one that has best validation performance.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-05T09:57:26.760Z
Answers.Text: We have tried this models but the loss is extremely big (quintillion scale). What do you suggest to do further ?
Answers.UserDisplayName: estamos
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-05T13:49:52.110Z
Answers.Text: I added a new one , what do you think about it ? Whatever architecture I choose the training time is extremely big . Note that sequence length is 999 and I want to predict the 1000th .
Answers.UserDisplayName: estamos
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-05T03:28:41.793Z
Answers.LastActivityDate: 2020-02-05T03:28:41.793Z
Answers.UserDisplayName: Anon
Body: <p>I have to build a neural network without any architecture limitations which have to predict the next value of a time series.</p>
  
  <p>The dataset is composed of 400.000 values, which are given in hex format. For example</p>
  
  <pre><code>0xbfb22b14
  0xbfb22b10
  0xbfb22b0c
  0xbfb22b18
  0xbfb22b14
  </code></pre>
  
  <p>I think LSTM is suitable for this problem, but I am worried about the length of the input. Would it be a good idea to use CNN?</p>
  
  <pre><code>def structure(step,n_features):
      # define model
      model = Sequential()
      model.add(LSTM(50, activation='relu', return_sequences=True, input_shape=(step, n_features)))
      model.add(Dense(1))
      model.compile(optimizer='adam', loss='mse')
      return model
  </code></pre>
  
  <p>What about this one ?</p>
  
  <pre><code>"model": {
          "loss": "mse",
          "optimizer": "adam",
      "save_dir": "saved_models",
          "layers": [
              {
                  "type": "lstm",
                  "neurons": 999,
                  "input_timesteps": 998,
                  "input_dim": 1,
                  "return_seq": true
              },
              {
                  "type": "dropout",
                  "rate": 0.05
              },
              {
                  "type": "lstm",
                  "neurons": 100,
                  "return_seq": false
              },
              {
                  "type": "dropout",
                  "rate": 0.05
              },
              {
                  "type": "dense",
                  "neurons": 1,
                  "activation": "linear"
              }
  </code></pre>
  
Tags: <neural-networks><deep-learning><prediction><time-series>
Title: What's the best architecture for time series prediction with a long dataset?
UserDisplayName: estamos
ThreadText: # What's the best architecture for time series prediction with a long dataset?
  
  (Asked by: username_0 on 2020-02-02)
  
  I have to build a neural network without any architecture limitations which have to predict the next value of a time series.
  The dataset is composed of 400.000 values, which are given in hex format. For example
  0xbfb22b14
  0xbfb22b10
  0xbfb22b0c
  0xbfb22b18
  0xbfb22b14
  
  I think LSTM is suitable for this problem, but I am worried about the length of the input. Would it be a good idea to use CNN?
  def structure(step,n_features):
      # define model
      model = Sequential()
      model.add(LSTM(50, activation='relu', return_sequences=True, input_shape=(step, n_features)))
      model.add(Dense(1))
      model.compile(optimizer='adam', loss='mse')
      return model
  
  What about this one ?
  "model": {
          "loss": "mse",
          "optimizer": "adam",
      "save_dir": "saved_models",
          "layers": [
              {
                  "type": "lstm",
                  "neurons": 999,
                  "input_timesteps": 998,
                  "input_dim": 1,
                  "return_seq": true
              },
              {
                  "type": "dropout",
                  "rate": 0.05
              },
              {
                  "type": "lstm",
                  "neurons": 100,
                  "return_seq": false
              },
              {
                  "type": "dropout",
                  "rate": 0.05
              },
              {
                  "type": "dense",
                  "neurons": 1,
                  "activation": "linear"
              }
  
  **Tags:** neural-networks, deep-learning, prediction, time-series
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-02-05. Score: 1 | Accepted answer
  
  Yes, LSTM are ideal for this. For even stronger representational capacity, make your LSTM's multi-layered.
  Using 1-dimensional convolutions in a CNN is a common way to exctract information from time series too, so there's no harm in trying. Typically, you'll test many models out and take the one that has best validation performance.
  
  > **Comments:**
  > **username_0**: We have tried this models but the loss is extremely big (quintillion scale). What do you suggest to do further ?  
  > **username_0**: I added a new one , what do you think about it ? Whatever architecture I choose the training time is extremely big . Note that sequence length is 999 and I want to predict the 1000th .  
  
  ---
  
  