I’m implementing an atari pong playing policy gradient agent.
The main loop at the moment is.

Run the policy to gather some experience and save the experience (images, actions, rewards) in a dataset.
Run a single iteration of training over the gathered experience to update the policy.
Throw all the data away and start again.

Up until now, I have just been using cpu training, but now I’d like to push the training to GPU.
When it comes to loading It seems I have a matrix of choices.

Create tensors on CPU, then push them to GPU via pinned memory.
Create tensors directly on GPU.

"1.  Create tensors on the get_item(index) of the DataSet
2.  Create tensors in the collate_batch function of the DataLoader (or write my own DataLoader)
I thought I understood all this, but as it turns out, there are a few gaps in my understanding.  I have 3 questions.
Is there any downside to directly creating GPU tensors in the get_item(index) call on the DataSet.  Is this a bad idea?
What’s the best practice for creating tensors generally, should they created for each item in the dataset, or should the dataset just return what it returns, and the loader take care of creating tensors?
Where is the right place to decide which device a tensor goes to?  The DataSet?  The DataLoader, or the training loop itself?
Any insight appreciated!