You’re right - I can’t pass few smaller batches, because I need to stack it and have a single graph that considers all of it (so the loss function will consider it all at once)
Thanks for the “model checkpoint” tip - will definitely give it a try!
Edit: moved suggestion below to a separate thread.
wondering - would it make sense to have a mechanism in pytorch, that moves tensors from GPU memory to system ram and back when needed ?
I understand that there will be overhead of the memory allocation and transfer itself, but is it really that bad up to a point that it makes such mechanism worthless?