Hi @marton-avrios, have a  look at this issue 17, this might be the reason. T5Tokenizer dosen’t add eos  token at the end of text, for now we manually need to add  <s> at the end of both source and target example