Another thing that you can do is sort all the input sentences based on lenght, and then do batches. For example if you have lets say 10 sentences with 10 words long and 10 sentences 30 words long and you use batch size of 10, then you do the first batch with only sentences 10 words long and then next batch with longer sentences.
I believe this could hurt the accuracy, actually I am pretty sure it would but it is excellent for production, leads to around 30 percent speedup.