Answers.Body: <p>It is a general problem in neural network, when you go deeper, you may have gradient explosion/vanishing problems. You may try to use gradient-clipping for you particular network.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-30T16:18:19.427Z
Answers.Text: Do you have any specific references that highlight this problem?
Answers.UserDisplayName: Haffi112
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-30T17:35:03.513Z
Answers.Text: Check it please: https://www.youtube.com/watch?v=qhXZsFVxGKo
Answers.UserDisplayName: ai-py
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T15:02:23.207Z
Answers.Text: But are you aware of an academic paper on the topic?
Answers.UserDisplayName: Haffi112
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-25T08:33:46.920Z
Answers.Text: Isn't LSTM specifically designed to avoid exploding/vanishing gradients?
Answers.UserDisplayName: NikoNyrh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-30T15:48:58.843Z
Answers.LastActivityDate: 2022-07-30T15:48:58.843Z
Answers.UserDisplayName: ai-py
Body: <p>I'm training an LSTM model for classification on accelerometer data, and I get better results when I downsample the signal to 25 Hz than when I use a 50 Hz signal.</p>
  <p>I use the same time frame of 1.5 seconds. So with a 25 Hz signal, I have 37 data points; with a 50 Hz signal, I have 75 data points as input to my model.</p>
  <p>I think that LSTM models have a harder time dealing with long sequences, which might explain this difference. However, I am not sure about that claim. Are you aware of any publications that go into depth about the limitations of LSTMs for long sequence tasks?</p>
  
Tags: <long-short-term-memory><sequence-modeling>
Title: Limitations of LSTMs
UserDisplayName: Haffi112
ThreadText: # Limitations of LSTMs
  
  (Asked by: username_0 on 2022-07-30)
  
  I'm training an LSTM model for classification on accelerometer data, and I get better results when I downsample the signal to 25 Hz than when I use a 50 Hz signal.
  
  I use the same time frame of 1.5 seconds. So with a 25 Hz signal, I have 37 data points; with a 50 Hz signal, I have 75 data points as input to my model.
  
  I think that LSTM models have a harder time dealing with long sequences, which might explain this difference. However, I am not sure about that claim. Are you aware of any publications that go into depth about the limitations of LSTMs for long sequence tasks?
  
  **Tags:** long-short-term-memory, sequence-modeling
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-07-30. Score: 0
  
  It is a general problem in neural network, when you go deeper, you may have gradient explosion/vanishing problems. You may try to use gradient-clipping for you particular network.
  
  > **Comments:**
  > **username_0**: Do you have any specific references that highlight this problem?  
  > **username_1**: Check it please: https://www.youtube.com/watch?v=qhXZsFVxGKo  
  > **username_0**: But are you aware of an academic paper on the topic?  
  > **username_2**: Isn't LSTM specifically designed to avoid exploding/vanishing gradients?  
  
  ---
  
  