Check out Large Model Support for PyTorch:
Wiki: https://github.com/mtbrandy/pytorch/wiki/Large-Model-Support 123
Code: https://github.com/mtbrandy/pytorch/tree/v1.1.0-LMS 63
This is the contribution proposal for the LMS feature that is currently available in the Watson Machine Learning solution.  For information about that solution, see https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.1/navigation/wmlce_getstarted_pytorch.html 12