I meant: if the only layers in the model, which change their behavior during training/evaluation, are dropout and batchnorm layers, setting model.eval() at the beginning of the script would disable them (as you’ve also said) and you could thus just remove the layers.
However, custom layers can also use the self.training flag internally and thus switch their behavior, but I don’t know if that’s the case in this repository.

Ah ok I understand now! Thank you.

I don’t understand this, as I thought model.eval() is already used during training and evaluation.

Sorry I was not clear - what I had meant was, i switched the eval() flag to train() flag for the training part of the code i.e here: Mean-Shifted-Anomaly-Detection/main.py at a02ac3001f96fcfa2f9b6a5fcb58cb3e9ba7f5b0 · talreiss/Mean-Shifted-Anomaly-Detection · GitHub then I get worser results. Hope I am making myself clear. So basically I do something like this:
def run_epoch(model, train_loader, optimizer, center, device):
    model.train() #this by default was model.eval() according to repo
    total_loss, total_num = 0.0, 0