Could you share more information about your profiling and how you’ve narrowed down that the data transfer is indeed the bottleneck?
You could pre-load the entire dataset to the GPU, but would of course use device memory for it besides a slower startup time.