My model handles the autoregressive part internally in the forward function. During training it doesn’t do any autogressive decoding. During testing it has to autoregressivly decode, but that is handled internally in my implementation so testing is very similar to training. As far as performance goes, in my tasks I couldn’t get it to work as well as a GRU Seq2Seq.