heatmaps[i, x[i, 1].int(), x[i, 0].int()] = 1 has assert like leaf variable has been moved into the graph interior.

This is just because you modify heatmaps inplace while you created it your self.
You can use new_heatmaps = heatmaps.clone() and use new_heatmaps in the for-loop and your loss computation to avoid that.

it seams difficult for one reason the gradient of x[,].int() is zero

Gradients for integer values are not really possible as you don’t even have a continuous function. You will need to have a softer version of this.
Like make your network output a mask of size 68, 64, 64 that goes through a softmax and use that as the target heatmap?