If you don’t need to preprocess the data, you could directly push the tensors to the GPU inside your Dataset or you could create the batch directly from the data via indexing.
The loops might slow down your code. Getting rid of it depends of course on your use case and I don’t know, how embeddings is defined and if you could use a single layer for it.