Support for higher order differentiation was just added, but it probably won’t work yet for many architectures because support is still currently in the process of being added for several individual operators in these pulls #1507 5 & #1423 6 and a few others  (e.g. conv 4 is still incompatible with higher order differentiation and currently being modified to allow double differentiation).
To use  higher order differentiation replace retain_variables=True with create_graph=True as outlined in the backward function:
https://github.com/pytorch/pytorch/blob/master/torch/autograd/ 7init.py#L72-L75