I don’t think you can use batches for inferencing. In you’re code you have
if decoder_input is torch.tensor([[EOS_token]*batch_size], device=device):
    break

This means that you only stop generating a next word when the most recently predicted word in ALL output sequences of your batch is the EOS_token. That’s arbitrary unlikely. Even if you would that all your output sequences (should) have the same length – and that’s a very big if – that would still assume that your network is so well trained guarantee perfect inference.
Where does target_length in your code come from? Usually that’s some kind of maximum length just in case the decoder will not predict the EOS_token.