Hi,
You need to set create_graph=True to get the graph constructed when using autograd.grad (and thus the gradients that can require gradients. See doc 78 for more details.
This had a different behavior in 0.3.1 as can be seen in the old doc 8 where this flag was set to True if the gradients provided are not volatile. But volatile has been removed hence this change.