Hello, I come up with a specific question about the detachment op we discuss here.
For example, we have a seq2seq model with an attention layer between the encoder and the decoder. According to the common implementations of attention models, the last hidden state of the encoder (say, ‘hn’) is used as the first hidden state of the decoder.
My question is: is it necessary to detach hn from the encoder network?