Answers.Body: <ol>
  <li><p><strong>Definitely yes</strong>. I would say Transformers would work wherever LSTM works and even better. The reason is that they <strong>can attend to longer sequences</strong> as an input. In contrast to transformers, in LSTM for example after some sequence length(in your case the signal length) it would lose performance.</p>
  </li>
  <li><p><strong>No</strong>, I don't specifically know projects that are in this domain.But you could have a look at the architecture called <strong>Conformer (<a href="https://arxiv.org/abs/2005.08100" rel="nofollow noreferrer">https://arxiv.org/abs/2005.08100</a>)</strong> (I think they work on audio signals)</p>
  </li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-19T17:33:00.687Z
Answers.Text: Thank you for your answer! 1)  I am aware of the fact, that LSTM have their benefits. Do you have any ressources (paper,benchmarks etc.) which can confirm your statement? I just feel confirmed that it is not a common approach. Maybe I should try it and publish an paper on it ;)
Answers.UserDisplayName: highReynolds
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-20T13:56:16.637Z
Answers.Text: Your welcome! :) I don't remember on top of my head which paper I can show you. But the reasoning is in LSTM because things are recurrent, you need the previous input for calculation and if you do this over a long sequence this leads to vanishing gradients problem therefore decreasing the learning potential. However, because transformer attends to the sentence or input sequence at once it can handle longer sequences better. I hope this helps, if I find a benchmark on this I will share for sure. https://medium.com/mlearning-ai/transformer-implementation-for-time-series-forecasting-a9db2db5c820
Answers.UserDisplayName: Nazmican
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-18T22:30:43.173Z
Answers.LastActivityDate: 2022-01-18T22:30:43.173Z
Answers.UserDisplayName: Nazmican
Body: <p>I am working on a problem regarding the multi-classification of multivariate time signals. So I have multiple signals and try to train an algorithm on them. My current approach is to build a neural network with LSTM-layers and it works pretty fine.</p>
  <p>I have read that LSTMs are pretty outdated because of the transformer architectures. I found some papers about the idea to use them for signal classification (see: <a href="https://arxiv.org/abs/2103.14438" rel="nofollow noreferrer">https://arxiv.org/abs/2103.14438</a>). There was an example on the TensorFlow page regarding univariate signal classification (see <a href="https://keras.io/examples/timeseries/timeseries_transformer_classification/" rel="nofollow noreferrer">https://keras.io/examples/timeseries/timeseries_transformer_classification/</a>). I think it is rather a research question than a common approach for this type of problem.</p>
  <p>To my questions:</p>
  <ul>
  <li><p>Would you recommend implementing a Transformer for this type of problem? Do you think, it is a more &quot;state of the art&quot; approach?</p>
  </li>
  <li><p>Do you know some example projects?</p>
  </li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-10-16T21:08:30.990Z
Comments.Text: Classifying multivariate signals is a difficult task that generally requires more sophisticated methods than the standard transformer. However, if the transformer is able to accurately learn the relationships between the different variables in the signal, it can be an effective classifier.
Comments.UserDisplayName: Faizy
Tags: <classification><transformer><time-series>
Title: Is a Transformer a good choice for multivariate signal classification?
UserDisplayName: highReynolds
ThreadText: # Is a Transformer a good choice for multivariate signal classification?
  
  (Asked by: username_0 on 2022-01-12)
  
  I am working on a problem regarding the multi-classification of multivariate time signals. So I have multiple signals and try to train an algorithm on them. My current approach is to build a neural network with LSTM-layers and it works pretty fine.
  
  I have read that LSTMs are pretty outdated because of the transformer architectures. I found some papers about the idea to use them for signal classification (see: <https://arxiv.org/abs/2103.14438>). There was an example on the TensorFlow page regarding univariate signal classification (see <https://keras.io/examples/timeseries/timeseries_transformer_classification/>). I think it is rather a research question than a common approach for this type of problem.
  
  To my questions:
  
  * Would you recommend implementing a Transformer for this type of problem? Do you think, it is a more "state of the art" approach?
  * Do you know some example projects?
  
  **Tags:** classification, transformer, time-series
  
  > **Comments:**
  > **username_1**: Classifying multivariate signals is a difficult task that generally requires more sophisticated methods than the standard transformer. However, if the transformer is able to accurately learn the relationships between the different variables in the signal, it can be an effective classifier.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-01-18. Score: 0
  
  1. **Definitely yes**. I would say Transformers would work wherever LSTM works and even better. The reason is that they **can attend to longer sequences** as an input. In contrast to transformers, in LSTM for example after some sequence length(in your case the signal length) it would lose performance.
  2. **No**, I don't specifically know projects that are in this domain.But you could have a look at the architecture called **Conformer (<https://arxiv.org/abs/2005.08100>)** (I think they work on audio signals)
  
  > **Comments:**
  > **username_0**: Thank you for your answer! 1)  I am aware of the fact, that LSTM have their benefits. Do you have any ressources (paper,benchmarks etc.) which can confirm your statement? I just feel confirmed that it is not a common approach. Maybe I should try it and publish an paper on it ;)  
  > **username_2**: Your welcome! :) I don't remember on top of my head which paper I can show you. But the reasoning is in LSTM because things are recurrent, you need the previous input for calculation and if you do this over a long sequence this leads to vanishing gradients problem therefore decreasing the learning potential. However, because transformer attends to the sentence or input sequence at once it can handle longer sequences better. I hope this helps, if I find a benchmark on this I will share for sure. https://medium.com/mlearning-ai/transformer-implementation-for-time-series-forecasting-a9db2db5c820  
  
  ---
  
  