Is there a computation/speed advantage to batching inputs into `TransformerModel` to reduce its number calls
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

For my particular application, I need to have several `output = TransformerModel(inputIDs)` calls per step, from different datasets. 

so

```
output1 = TransformerModel(inputIDs_dataset1)
output2 = TransformerModel(inputIDs_dataset2)
output3 = TransformerModel(inputIDs_dataset3)
```

Initially I preferred to have these calls separate, as each dataset has a different average and distribution of sequence length, so keeping these separated would decrease the number of paddings I need to do within each batch. 

On the other hand, I imagine that the TransformerModel objects have some optimizations which  would make it overall more computationally efficient just to concatenate all the datasets, and make only one call to the `TransformerModel`.

My intuition is towards the latter approach, but would hear takes from those who designed it. 
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
