You would need to restore the model in the same way, including the last layer change, to be able to properly load the state_dict. Otherwise you’ll get a shape mismatch for this layer in load_state_dict.