Thanks - that makes sense. However, this still doesn’t fix it. Is it possible that the graph survives due to my “external computation of the gradient” that uses a slice of the hidden layer - e.g. if I use
slice_of_mu = hidden[0][0, -num_coords:].cpu().data

as a baseline for these computations?