Solved by ptrblck in post #2 
              
              
                The recommended way is storing the state_dicts as described in the serialization docs. 
Often a dict is stored with all state_dicts and other attributes as shown in the ImageNet example.