Solved by nielsr in post #3 
              
              
                Hi, 
After investigation, it turns out the generate() method currently does not take into account config.decoder.eos_token_id, only config.eos_token_id. 
You can fix it by setting model.config.eos_token_id = 2. 
We will fix this soon.