Hey albanD, this is not what i’m looking for, I would like to optimize over the output of my bottom network, and this is part of my specific use case. I have found a way around this problem by wrapping v.data into a new Variable that is then considered a leaf node by Pytorch:

v_leaf = t.autograd.Variable(v.data, requires_grad=True)
opt = t.optim.SGD([v_leaf], 1e-3)
for i in range(10): opt.zero_grad(); top(v).backward([v]); opt.step()

Although this works I don’t see any reason why I need to use this workaround. One would think you can optimize over any variable as long as it stores it’s gradient. Yes it is a bit weird to optimize over non-leaf nodes but there are use cases out there where this may be useful.