Well if the data don’t fit on GPU, you’ll most likely spend more time sending the data to the GPU than doing actual compute. So I guess you can simply do this on the CPU.
If you do need the GPU, the proposal above looks ok. Maybe use IterableDataset if you have performance issues. (But again if you don’t do anything else with it but a simple mulitplicaition, it is expected that you spend all your time sending stuff to the GPU).