If you are using a Dataset and DataLoader (as explained in this tutorial 2), you can set the batch_size to the desired value.
On the other hand, if you are directly working with tensors, you could slice the tensors directly to create the batch.