<p>Both the architecture have some use cases based on their architecture.</p>
<p><strong>Transformer</strong></p>
<ul>
<li>Transformers are good when there are long-range dependencies, in those cases they outperform the LSTMs.</li>
<li>Main key feature is High dimensional data in those cases self-attention is very useful.</li>
<li>In terms of processing Transformer allows parallel processing during training which leads to faster training.</li>
<li>In terms of feature interaction this model is very efficient in mapping complex feature relations.</li>
</ul>
<p><strong>LSTM</strong></p>
<ul>
<li>It is designed to deal with sequential data and catch dependencies effectively, only for shorter sequences.</li>
<li>When you have limited data for training LSTM generalizes better because of fewer parameters than the Transformer.</li>
<li>It is more suitable for online learning and real-time processing because it processes data sequentially.</li>
<li>It is less demanding in terms of computation.</li>
</ul>
<p><strong>Main differences:</strong></p>
<ul>
<li><p><strong>Mechanism</strong>: The Transformer uses attention to weigh the importance of different parts of the input, which helps to learn global
dependencies, while LSTM processes data sequentially and uses a gate
mechanism for informal flow control, which struggles in long
sequences.</p>
</li>
<li><p><strong>Memory</strong>:LSTM has an inbuilt memory cell state that carries information, which helps in time-series-based patterns while
Transformer doesn't have recursive memory but has positional encoding
to understand sequence order.</p>
</li>
<li><p><strong>Complexity</strong>:Transformer has a high number of params because of self-attention head and FFN (feed-forward network) while LSTM has
fewer params which is beneficial in case of limited data.</p>
</li>
</ul>

CC BY-SA 4.0
2023-12-01T08:56:10.037Z
2023-12-01T08:56:10.037Z
Hiren Namera
<p>I’m working on a time-series classification problem and trying to decide whether to use a Transformer or an LSTM.</p>
<p>From what I’ve learned, Transformers are better suited for capturing long-range dependencies in time-series data vs LSTM. The transformers are an excellent option for processing word data for Natural Language tasks, and frameworks are currently built to represent time-series data, such as TabTransformer and TimeSeriesTransformer models.</p>
<p>But I’m unsure if that’s always the case.</p>
<p>Are there specific use cases where Transformers are better than LSTMs for time-series classification?</p>
<p>What are some key differences between the two architectures in representing time-series data?</p>

<classification><transformer><long-short-term-memory><time-series>
When are Transformers better than LSTMs in time-series tasks such as classification?
Mark Cortejo
# When are Transformers better than LSTMs in time-series tasks such as classification?

(Asked by: username_0 on 2023-12-01)

I’m working on a time-series classification problem and trying to decide whether to use a Transformer or an LSTM.

From what I’ve learned, Transformers are better suited for capturing long-range dependencies in time-series data vs LSTM. The transformers are an excellent option for processing word data for Natural Language tasks, and frameworks are currently built to represent time-series data, such as TabTransformer and TimeSeriesTransformer models.

But I’m unsure if that’s always the case.

Are there specific use cases where Transformers are better than LSTMs for time-series classification?

What are some key differences between the two architectures in representing time-series data?

**Tags:** classification, transformer, long-short-term-memory, time-series


---

## 1 answers

### Answer by username_1 on 2023-12-01. Score: 2

Both the architecture have some use cases based on their architecture.

**Transformer**

* Transformers are good when there are long-range dependencies, in those cases they outperform the LSTMs.
* Main key feature is High dimensional data in those cases self-attention is very useful.
* In terms of processing Transformer allows parallel processing during training which leads to faster training.
* In terms of feature interaction this model is very efficient in mapping complex feature relations.

**LSTM**

* It is designed to deal with sequential data and catch dependencies effectively, only for shorter sequences.
* When you have limited data for training LSTM generalizes better because of fewer parameters than the Transformer.
* It is more suitable for online learning and real-time processing because it processes data sequentially.
* It is less demanding in terms of computation.

**Main differences:**

* **Mechanism**: The Transformer uses attention to weigh the importance of different parts of the input, which helps to learn global
  dependencies, while LSTM processes data sequentially and uses a gate
  mechanism for informal flow control, which struggles in long
  sequences.
* **Memory**:LSTM has an inbuilt memory cell state that carries information, which helps in time-series-based patterns while
  Transformer doesn't have recursive memory but has positional encoding
  to understand sequence order.
* **Complexity**:Transformer has a high number of params because of self-attention head and FFN (feed-forward network) while LSTM has
  fewer params which is beneficial in case of limited data.


---

