Dear all,
I have stumbled upon the same error as stated in the description, although I am using model.eval() before predicting labels with my model.
Specifically, I have my trained NN model, in which I use one dropout layer. Then I’m using this network to do optimization over some other parameter (that is not connected to the model) and every time (or almost) I do a forward pass, the prediction changes. I’m adding a snippet of the code where I do my optimization, of the predict() function and of my forward() function. The thing is that when I remove the dropout layer the predictions don’t change…which should be the case with model.eval()… Also, I can’t use torch.no_grad() since I need the gradients for my other parameter. Thank you in advance!
w = input.clone().detach().requires_grad_(True).to(device)
params = [{'params': w}]
optimizer = torch.optim.Adam(params, lr=lr)

def opt(net, w):
        adv_sample = TO_MUL*torch.tanh(w)+TO_ADD
        _, logits = net.predict(adv_sample, logits=True)
        loss, fx = self.loss(const, adv_sample, input, logits[0], target)
        loss.backward()
        optimizer.step()

def predict(samples, logits=False):
        self.eval()
        logs = self.forward(samples)
        if logits:
            return torch.argmax(logs, 1), logs
        softmax = torch.nn.Softmax(dim=1)
        probs = softmax(logs)
        return torch.argmax(probs, 1), probs

def forward(x):
        x = self.conv22(x)
        x = self.mp(x)
        N, C, W, H = (*(x.shape),)
        x = torch.reshape(x, (N, C*W*H))
        x = self.fc1(x)
        x = F.dropout(x, p = .5)
        x = self.fc2(x)
        logits = self.fc3(x) # logits layer
        return logits