Hi, I am working on a seq2seq problem, and I am confused about the validation/testing loss.
In seq2seq, during training, in the decoder, I use the current target token to generate the next token and calculate the loss based on the target sequence and my output sequence. During validation, should I use the current generated token to predict the next token? If so, the loss would be much higher than the training loss, and it cannot decrease. The reason is that when using the generated token to as input for next token, it is not that accurate, and I do not use beam search here. Should I use beam search in the validation? Or should I keep validation phase the same setting with training phase (except those dropout things)?
Thanks! Any response would be appreciated.