Answers.Body: <p>In seq2seq RNN training, we usually use a technique called &quot;teacher forcing.&quot; With teacher forcing, the actual (ground truth) output word at each timestep is fed as input to the decoder in the next timestep, rather than using the model's previous prediction. This approach helps the model learn faster because it doesn't rely on previous potentially incorrect predictions, which might indeed lead to a cascading effect of errors.</p>
  <p>However, during inference (when actually translating sentences), the model doesn't have access to ground truth outputs. So, it uses its own predictions as input for the next timestep. This can lead to some discrepancies between training and inference, known as &quot;exposure bias.&quot; To mitigate this, a technique called &quot;scheduled sampling&quot; is sometimes used, which involves gradually transitioning from teacher forcing to using the model's own predictions during training.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-21T08:41:14.810Z
Answers.Text: How can we do teacher forcing when the things decoder pass to themselves are hidden states? Do we need to change what they pass themselves (maybe just the last hidden state of the encoder and the raw words output)?
Answers.UserDisplayName: FluidMechanics Potential Flows
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-21T08:59:38.550Z
Answers.Text: You don't need to change what the decoder passes to itself. Instead, you alter the input to the decoder during training.
Answers.UserDisplayName: Hans-Peter Schrei
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-21T09:26:14.960Z
Answers.Text: During training you input words instead of hidden states to the decoder?
Answers.UserDisplayName: FluidMechanics Potential Flows
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-23T10:28:04.400Z
Answers.Text: Oh my bad, you keep the same hidden state but you input the correct word + the hidden state instead of the wrong word + the hidden state?
Answers.UserDisplayName: FluidMechanics Potential Flows
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-23T10:48:13.150Z
Answers.Text: Yes, that's correct. During training with teacher forcing, you keep the same hidden state updating mechanism, but at each time step of the decoder, you input the correct (ground truth) word/token from the target sequence along with the hidden state, instead of using the predicted word/token from the previous time step.
Answers.UserDisplayName: Hans-Peter Schrei
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-21T08:25:15.503Z
Answers.LastActivityDate: 2023-03-21T08:25:15.503Z
Answers.UserDisplayName: Hans-Peter Schrei
Body: <p>How do we train a seq2seq rnn training?</p>
  <p>We input a sentence that needs to be translated. We encode it sequentially. Then the first decoder outputs the first word with probabilities. We do a gradient descent by comparing them with the actual word expected. Then we input to the second decoder the hidden state and it outputs the second word with probabilities. We do another gradient descent. But what if the first hidden state was wrong because it failed to output the right word? Then this second gradient descent is meaningless?</p>
  
Tags: <recurrent-neural-networks><backpropagation><seq2seq>
Title: Seq2seq with RNNs, how is the training loop performed?
UserDisplayName: FluidMechanics Potential Flows
ThreadText: # Seq2seq with RNNs, how is the training loop performed?
  
  (Asked by: username_0 on 2023-03-20)
  
  How do we train a seq2seq rnn training?
  
  We input a sentence that needs to be translated. We encode it sequentially. Then the first decoder outputs the first word with probabilities. We do a gradient descent by comparing them with the actual word expected. Then we input to the second decoder the hidden state and it outputs the second word with probabilities. We do another gradient descent. But what if the first hidden state was wrong because it failed to output the right word? Then this second gradient descent is meaningless?
  
  **Tags:** recurrent-neural-networks, backpropagation, seq2seq
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-03-21. Score: 3 | Accepted answer
  
  In seq2seq RNN training, we usually use a technique called "teacher forcing." With teacher forcing, the actual (ground truth) output word at each timestep is fed as input to the decoder in the next timestep, rather than using the model's previous prediction. This approach helps the model learn faster because it doesn't rely on previous potentially incorrect predictions, which might indeed lead to a cascading effect of errors.
  
  However, during inference (when actually translating sentences), the model doesn't have access to ground truth outputs. So, it uses its own predictions as input for the next timestep. This can lead to some discrepancies between training and inference, known as "exposure bias." To mitigate this, a technique called "scheduled sampling" is sometimes used, which involves gradually transitioning from teacher forcing to using the model's own predictions during training.
  
  > **Comments:**
  > **username_0**: How can we do teacher forcing when the things decoder pass to themselves are hidden states? Do we need to change what they pass themselves (maybe just the last hidden state of the encoder and the raw words output)?  
  > **username_1**: You don't need to change what the decoder passes to itself. Instead, you alter the input to the decoder during training.  
  > **username_0**: During training you input words instead of hidden states to the decoder?  
  > **username_0**: Oh my bad, you keep the same hidden state but you input the correct word + the hidden state instead of the wrong word + the hidden state?  
  > **username_1**: Yes, that's correct. During training with teacher forcing, you keep the same hidden state updating mechanism, but at each time step of the decoder, you input the correct (ground truth) word/token from the target sequence along with the hidden state, instead of using the predicted word/token from the previous time step.  
  
  ---
  
  