<p>What does it mean that the decoder can be parallelized during training?</p>
<p>Let's assume a transformer (with both encoder and decoder) is employed for a time-series prediction. I.e. from the input sequence <code>x_0, ..., x_N</code> we want to predict <code>y_0, ..., y_N</code>. Is this the way that parallelization occurs during training?</p>
<ul>
<li>form the batch <code>[], [y_0], ..., [y_0, ..., y_N-1]</code></li>
<li>feed this batch to the transformer, together with the input sequence</li>
<li>we will obtain the batch <code>Y_0, Y_1, ..., Y_N</code></li>
<li>compare against <code>y_0, ..., y_N</code> and form the loss (*)</li>
</ul>
<p>(*) here, some teacher ratio techiques may be employed, so that more passes may be required</p>

CC BY-SA 4.0
2023-03-12T01:42:26.307Z
Does this answer your question? [Can the decoder in a transformer model be parallelized like the encoder?](https://ai.stackexchange.com/questions/12490/can-the-decoder-in-a-transformer-model-be-parallelized-like-the-encoder)
Minh-Long Luu
CC BY-SA 4.0
2023-03-12T08:06:48.973Z
In fact no, I asked my question after reading that post. @Minh-LongLuu
Lilla
<machine-learning><deep-learning><transformer>
Transformer parallelization during training
Lilla
# Transformer parallelization during training

(Asked by: username_0 on 2023-03-11)

What does it mean that the decoder can be parallelized during training?

Let's assume a transformer (with both encoder and decoder) is employed for a time-series prediction. I.e. from the input sequence `x_0, ..., x_N` we want to predict `y_0, ..., y_N`. Is this the way that parallelization occurs during training?

* form the batch `[], [y_0], ..., [y_0, ..., y_N-1]`
* feed this batch to the transformer, together with the input sequence
* we will obtain the batch `Y_0, Y_1, ..., Y_N`
* compare against `y_0, ..., y_N` and form the loss (\*)

(\*) here, some teacher ratio techiques may be employed, so that more passes may be required

**Tags:** machine-learning, deep-learning, transformer

> **Comments:**
> **username_1**: Does this answer your question? [Can the decoder in a transformer model be parallelized like the encoder?](https://ai.stackexchange.com/questions/12490/can-the-decoder-in-a-transformer-model-be-parallelized-like-the-encoder)  
> **username_0**: In fact no, I asked my question after reading that post. @Minh-LongLuu  

---

## 0 answers

