You are not correct when you say that “the whole point of batches will be ruined”. That’s not the case at all. There are many other situations/advantages of using batches.
In terms of performance, you only would iteratively move your batches to your GPU if and only if your dataset does not fit into your GPU’s memory. If it fits, you can just move everything before training so you dont lose time waiting for the data to be moved at every batch iteration.
Moving data before training:

Autoencoder 0, mean loss: 0.049266365027106775, time: 5.8063098430633545
Classifier 0, mean loss: 0.6276001909743748, time: 5.8063098430633545
Autoencoder 1, mean loss: 0.02765635310459747, time: 5.40066180229187
Classifier 1, mean loss: 0.30130706248538836, time: 5.40066180229187
Autoencoder 2, mean loss: 0.025990864579706813, time: 5.43626651763916
Classifier 2, mean loss: 0.24578058288327412, time: 5.43626651763916

Moving one batch at a time

Autoencoder 0, mean loss: 0.05069048443773408, time: 7.177410364151001
Classifier 0, mean loss: 0.5955254633638905, time: 7.177410364151001
Autoencoder 1, mean loss: 0.02769256935619723, time: 6.592367887496948
Classifier 1, mean loss: 0.2981912835336316, time: 6.592367887496948
Autoencoder 2, mean loss: 0.025816337348444504, time: 6.52787971496582
Classifier 2, mean loss: 0.2456918565480948, time: 6.52787971496582