Hello,
Yes, this is the right way. You could always wrap the operations inside a function that looks like that:
def to_numpy(tensor):
    return tensor.cpu().detach().numpy()

I do not think a with block would work, and as far as I know, you can’t do those operations inplace (except detach_). The main overhead will be in the .cpu() call, since you have to transfer data from the GPU to the CPU.