Answers.Body: <p>I don't think there's any difference between making predictions when you use or not teacher forcing during training. So, let me describe one way of doing that.</p>
  <p>During testing, as you noticed, you don't know the ground-truth labels, so the only way of predicting a sequence is to feed into the model the predictions for the previous time-steps.</p>
  <p>So, let's say that your model is denoted by <span class="math-container">$f$</span>, which attempts to approximate the language model <span class="math-container">$p(x_t \mid x_{1:t-1})$</span>, a conditional probability distribution over characters.</p>
  <p>To perform inference, you first need to provide the initial character of the sequence, usually, a special character, like <span class="math-container">$\langle \text{start} \rangle$</span>, so your model predicts the first real character of the sequence as follows <span class="math-container">$$f([\langle \text{start} \rangle]) = \hat{x}_1.$$</span> Afterward, you feed to it <span class="math-container">$\hat{x}_1$</span>, in order to produce the next character, i.e. <span class="math-container">$$f([\langle \text{start}\rangle, \hat{x}_1]) = \hat{x}_2,$$</span> and so on, until the model <span class="math-container">$f$</span> produces another special character that denotes the end of the sentence <span class="math-container">$\langle \text{end}\rangle$</span>.</p>
  <p>Now, in practice, rather than all the previously predicted characters, you may just feed the character <span class="math-container">$x_{t-1}$</span> to predict <span class="math-container">$x_t$</span>, and, at the same time, pass the previous last state of the RNN, so something like <span class="math-container">$$f (\hat{x}_{t-1}, h_{t-1}) = \hat{x}_t.$$</span></p>
  <p>The following diagram illustrates the concept.</p>
  <p><a href="https://i.sstatic.net/wgCRz.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/wgCRz.png" alt="enter image description here" /></a></p>
  <p>Here the first char is <span class="math-container">$T$</span> and the model predicts that the next char is <span class="math-container">$e$</span>. Then we pass <span class="math-container">$e$</span> and the previous state of the model to the model to predict <span class="math-container">$n$</span>, and so on.</p>
  <p><a href="https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb" rel="nofollow noreferrer">Here</a> you have a TensorFlow example that generates text with an RNN, uses teacher forcing, and then performs inference in the way I described (of course, there are other details, like using a <a href="https://ai.stackexchange.com/a/32478/2444">temperature</a>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-23T10:44:49.000Z
Answers.Text: Then the architecture of RNN is changing, right?
Answers.UserDisplayName: hanugm
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-23T11:13:54.370Z
Answers.Text: @hanugm No, the architecture doesn't change, at least, not usually. What changes is what you pass as input to the model. The hidden state of a NN can be thought of as containing the information that would allow you to make a prediction based on the previous inputs or history, at least, this is the intuition. If you look at the linked code (TF example), maybe you will get a better sense of what's happening.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-23T11:17:10.197Z
Answers.Text: In that example, basically, you can make the RNN (in that case, a GRU) return you the last state of that layer after having taken the input. Then you use that last state as some kind of "context vector" for predicting the next char, and that GRU produces another "context vector" (i.e. the hidden vector), which you use to make a prediction at the next time step, and so on.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-22T16:56:04.457Z
Answers.LastActivityDate: 2021-12-22T17:01:11.540Z
Answers.LastEditDate: 2021-12-22T17:01:11.540Z
Answers.UserDisplayName: nbro
Body: <p>If an RNN is trained using only the <a href="https://ai.stackexchange.com/a/18007/18758">teacher forcing</a>, then the network takes the actual output from the previous time step as input to the hidden state the next time step.</p>
  <p>We know that the actual outputs cannot be given to the model while testing, then what information passes from a time step to the next time step in the test phase?</p>
  
Tags: <recurrent-neural-networks><testing><teacher-forcing>
Title: How to do testing for an RNN that was trained with teacher forcing only?
UserDisplayName: hanugm
ThreadText: # How to do testing for an RNN that was trained with teacher forcing only?
  
  (Asked by: username_0 on 2021-12-21)
  
  If an RNN is trained using only the [teacher forcing](https://ai.stackexchange.com/a/18007/18758), then the network takes the actual output from the previous time step as input to the hidden state the next time step.
  
  We know that the actual outputs cannot be given to the model while testing, then what information passes from a time step to the next time step in the test phase?
  
  **Tags:** recurrent-neural-networks, testing, teacher-forcing
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-12-22. Score: 1
  
  I don't think there's any difference between making predictions when you use or not teacher forcing during training. So, let me describe one way of doing that.
  
  During testing, as you noticed, you don't know the ground-truth labels, so the only way of predicting a sequence is to feed into the model the predictions for the previous time-steps.
  
  So, let's say that your model is denoted by $f$, which attempts to approximate the language model $p(x\_t \mid x\_{1:t-1})$, a conditional probability distribution over characters.
  
  To perform inference, you first need to provide the initial character of the sequence, usually, a special character, like $\langle \text{start} \rangle$, so your model predicts the first real character of the sequence as follows $$f([\langle \text{start} \rangle]) = \hat{x}\_1.$$ Afterward, you feed to it $\hat{x}\_1$, in order to produce the next character, i.e. $$f([\langle \text{start}\rangle, \hat{x}\_1]) = \hat{x}\_2,$$ and so on, until the model $f$ produces another special character that denotes the end of the sentence $\langle \text{end}\rangle$.
  
  Now, in practice, rather than all the previously predicted characters, you may just feed the character $x\_{t-1}$ to predict $x\_t$, and, at the same time, pass the previous last state of the RNN, so something like $$f (\hat{x}\_{t-1}, h\_{t-1}) = \hat{x}\_t.$$
  
  The following diagram illustrates the concept.
  
  [![enter image description here](https://i.sstatic.net/wgCRz.png)](https://i.sstatic.net/wgCRz.png)
  
  Here the first char is $T$ and the model predicts that the next char is $e$. Then we pass $e$ and the previous state of the model to the model to predict $n$, and so on.
  
  [Here](https://colab.research.google.com/github/tensorflow/text/blob/master/docs/tutorials/text_generation.ipynb) you have a TensorFlow example that generates text with an RNN, uses teacher forcing, and then performs inference in the way I described (of course, there are other details, like using a [temperature](https://ai.stackexchange.com/a/32478/2444)).
  
  > **Comments:**
  > **username_0**: Then the architecture of RNN is changing, right?  
  > **username_1**: @hanugm No, the architecture doesn't change, at least, not usually. What changes is what you pass as input to the model. The hidden state of a NN can be thought of as containing the information that would allow you to make a prediction based on the previous inputs or history, at least, this is the intuition. If you look at the linked code (TF example), maybe you will get a better sense of what's happening.  
  > **username_1**: In that example, basically, you can make the RNN (in that case, a GRU) return you the last state of that layer after having taken the input. Then you use that last state as some kind of "context vector" for predicting the next char, and that GRU produces another "context vector" (i.e. the hidden vector), which you use to make a prediction at the next time step, and so on.  
  
  ---
  
  