I think the simplest way to do that is to pre-process your state_dict before giving it to the Module.load_state_dict function. That will make sure that your code keeps working with new versions of PyTorch as internal API might change without notice.