I agree, I don’t think the SOS and EOS tokens are needed in the source sentence either.
Could you say a bit more about your training process? In regards to the auto-regressive part.
Also, if you don’t mind, how did your model end up performing?