Still new to pyTorch, and my local machine has no GPU so I’ve been prepping my network locally until I’m ready to pay for a cloud instance and use a GPU for speed.
I have a large dataset of non-image data which I am converting into image samples so that I may train a VAE with CNN.  I’m struggling with when and which tensors to move on and off of the GPU, which should keep a gradient or whether that is overkill.
The GPU does seem to be working, as I can see it ramp up and down as a batch moves through the training cycle, however I suspect most of my time is wasted trying to build these images on-the-fly (which I need to do for future use case).
are there any helpful examples or guidelines I should be following?  For example, should I first construct an entire numpy array for the values of the image and then convert to a tensor placed on the GPU?  Or should I construct the values entirely with tensors to begin with?  Since I’m generating batches of images, do I wait to place the whole batch on the GPU, or does that not make any difference?  I have hundreds of thousands of data samples and so I cannot effectively keep them all within the GPU memory, so I end up clearing the GPU memory after each mini-batch and the image needs to be regenerated from scratch on the next training round.  There must be a more efficient way to handle all this data.
Does anyone have experience with generating images from sample data during the load phase that could offer advice.
for reference, I’m referencing this paper as a guide for my approach: non-image-data-classification-using-CNN 2