I have access to a GPU cluster that provides 20 minute slots.
I would like to run the training procedure for as many ‘mini-batches’ (not epochs) as possible in those 20 minutes and save the state. In the next 20 minute slot I would like to start training from the previous state. Can you please provide me with some pointers/example of how to do this? I am a beginner in PyTorch and apologies if this has been discussed elsewhere.