sharvil:

I suppose a context manager would shrink the API surface and make code more consistent…


Might be, but I personally don’t really like too many context managers, as the code becomes quite unreadable after the 5th indentation. 
Have also a look at the torch.*_like methods, e.g. torch.zeros_like(tensor).