Yes, you are right.
If you are concerned about the overhead of the (DB) query, you could create the whole batch directly in __getitem__ (with a single query) and set batch_size=1 in your DataLoader.
Would this work for you?