You could most likely add the missing step state manually to the optimizer.
However, I would generally recommend to stick to the latest version so we would need to look into the memory issue. Do you have a code snippet to reproduce this issue and could you post your current setup (PyTorch, CUDA, cudnn, NCCL, versions etc.)?