Body: <p>What does it mean that the decoder can be parallelized during training?</p>
  <p>Let's assume a transformer (with both encoder and decoder) is employed for a time-series prediction. I.e. from the input sequence <code>x_0, ..., x_N</code> we want to predict <code>y_0, ..., y_N</code>. Is this the way that parallelization occurs during training?</p>
  <ul>
  <li>form the batch <code>[], [y_0], ..., [y_0, ..., y_N-1]</code></li>
  <li>feed this batch to the transformer, together with the input sequence</li>
  <li>we will obtain the batch <code>Y_0, Y_1, ..., Y_N</code></li>
  <li>compare against <code>y_0, ..., y_N</code> and form the loss (*)</li>
  </ul>
  <p>(*) here, some teacher ratio techiques may be employed, so that more passes may be required</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-12T01:42:26.307Z
Comments.Text: Does this answer your question? [Can the decoder in a transformer model be parallelized like the encoder?](https://ai.stackexchange.com/questions/12490/can-the-decoder-in-a-transformer-model-be-parallelized-like-the-encoder)
Comments.UserDisplayName: Minh-Long Luu
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-12T08:06:48.973Z
Comments.Text: In fact no, I asked my question after reading that post. @Minh-LongLuu
Comments.UserDisplayName: Lilla
Tags: <machine-learning><deep-learning><transformer>
Title: Transformer parallelization during training
UserDisplayName: Lilla
ThreadText: # Transformer parallelization during training
  
  (Asked by: username_0 on 2023-03-11)
  
  What does it mean that the decoder can be parallelized during training?
  
  Let's assume a transformer (with both encoder and decoder) is employed for a time-series prediction. I.e. from the input sequence `x_0, ..., x_N` we want to predict `y_0, ..., y_N`. Is this the way that parallelization occurs during training?
  
  * form the batch `[], [y_0], ..., [y_0, ..., y_N-1]`
  * feed this batch to the transformer, together with the input sequence
  * we will obtain the batch `Y_0, Y_1, ..., Y_N`
  * compare against `y_0, ..., y_N` and form the loss (\*)
  
  (\*) here, some teacher ratio techiques may be employed, so that more passes may be required
  
  **Tags:** machine-learning, deep-learning, transformer
  > **Comments:**
  > **username_1**: Does this answer your question? [Can the decoder in a transformer model be parallelized like the encoder?](https://ai.stackexchange.com/questions/12490/can-the-decoder-in-a-transformer-model-be-parallelized-like-the-encoder)  
  > **username_0**: In fact no, I asked my question after reading that post. @Minh-LongLuu  
  
  ---
  
  ## 0 answers
  
  