Gradient for input when evaluate
Hi, I want to use a pretrained model to supervise another model, which need to set the batch normalization into eval mode. However, with lstm in my pretrained model, it arises follow message: <SCODE>Traceback (most recent call last):
  File "main.py", line 153, in val
    cost.backward()
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/autograd/variable.py", line 145, in backward
    self._execution_engine.run_backward((self,), (gradient,), retain_variables)
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/autograd/function.py", line 208, in _do_backward
    result = super(NestedIOFunction, self)._do_backward(gradients, retain_variables)
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/autograd/function.py", line 216, in backward
    result = self.backward_extended(*nested_gradients)
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/nn/_functions/rnn.py", line 199, in backward_extended
    self._reserve_clone = self.reserve.clone()
AttributeError: 'CudnnRNN' object has no attribute 'reserve'
<ECODE> Do you have any suggestions? Thanks very much.
meijieru
smth
I see the bug. I’ll fix it today.
apaszke
<SCODE>import torch
import torch.nn as nn
from torch.autograd import Variable

rnn = nn.Sequential(nn.LSTM(10, 10, 2, bidirectional=True))
rnn = rnn.cuda()
rnn.eval()

i = torch.Tensor(2, 1, 10).cuda()
i = Variable(i, requires_grad=True)

o, _ = rnn(i)
o = o.mean()
o.backward(torch.ones(1).cuda())
<ECODE>
meijieru
meijieru
It will be today. I will be building binaries late today.
smth
Thanks very much. (~~~~~~~~~~~~~~~~~~)
meijieru