Hi. I am trying to perform an adversarial attack on a model, where I have to calculate gradients of loss over inputs, something like this:
model.eval()
for prediction != target:
     prediction = model(input)
     loss = criterion(prediction, target)
     loss.backward()
     input = input - k * (input.grad())

But, following error is encountered Pytorch cudnn RNN backward can only be called in training mode. I tried the solutions posted  here 1, which are:

Call .train() on the rnn module after using model.eval(). Used this as follows:

model.eval()
for prediction != target:
     prediction = model(input)
     loss = criterion(prediction, target)
     model.train()
     loss.backward()
     input = input - k * (input.grad())
     model.eval()

It doesn’t help. The error is still there.


call .eval() only on the necessary modules: I have 2 doubts regarding this solution.
a. My model under consideration is quite big, it is advisable to go for this?
b. Does it affect my prediction accuracy in any way?


disable cudnn via torch.backends.cudnn.enabled = False. Going for this solution, I think I won’t be able to GPU. Without the GPU the running times is 4-5 times to that of time taken for execution with the GPU.


Can you please suggest me workaround/solution? Thanks in advance.