To achieve such behaviour, use the (special) self.training flag that is True when the model (the model has to subclass the torch.nn.Module) is in the train mode (model.train()) and False in the model’s eval mode (model.eval()).
And this is a skeleton of the forward pass of the model that outputs different results in the train/eval mode.
def forward(self, x):
    ...
    if self.training:
        ...
        return train_output
    else:
        ...
        return eval_output