You are detaching the output tensor in:
predicted_model = VRN(inp)[-1].data.cpu()

by calling the .data attribute (which you shouldn’t call in any way, as it might yield unwanted side effects) and also in:
vol = predicted_model.numpy()

since you are leaving PyTorch and use numpy (Autograd won’t be able to track these operations).
I would assume that:
    loss = lossfct(vol, voxel_array)
    loss.backward()

would raise an error, since you are using numpy arrays, but you might also have re-wrapped the array into a tensor to get rid of the error.
To fix this issue: don’t use the .data attribute and, if possible, use PyTorch operations instead of numpy.
However, if you need to use numpy specific operations, you would have to implement the backward manually via a custom autograd.Function.