I suggest you make your data ready before put the data in dataloader.  The “ready data” means they are all vectors number with corresponding labels. So yes, word embeddings, transformations …
The transformation to tensors can be handled later in training. The idea is that  if you are going to train it on GPU, you should put only the current mini-batch on GPU to save its memory. But if your GPU can handle all of your data at once then it’s fine to do the transformation right in the init() method.