I’m not sure what you mean by saving the whole model. You can still save using state dict. Just that you need the original model definition code which is also needed to load state. For the case in my example,
# Original AlexNet
class AlexNet(nn.Module):
    ...

model = AlexNet()
model.load_state_dict(state)

# Replace the final FC layer
classifier = list(model.classifier.children())
model.classifier = nn.Sequential(*classifier[:-1])
model.classifier.add_module(
    '6', nn.Linear(classifier[-1].in_features, num_classes)

As for now, this is the only way without manually changing the PyTorch source code. I hope it clarifies my previous answer.