You can use two different linear layers. You can put a condition like this:
if self.train():
    self.fc3 = nn.Linear(x,5)
else:
    --

That way you can load the weights of this fc layer whenever you want.