I think the easiest way would be to use torch.utils.checkpoint 111 to trade memory for compute.
Manually pushing the values to the host and back to the device would probably make your training really slow.
Accumulating gradients won’t help, as the same memory would be needed.