Yes.
All differentiable functions should be implemented though. Only a few linear algebra functions that could be differentiated are not.
Of course non-differentiable function won’t work.
And the functions like .detach() and torch.no_grad() are built to do things outside of the autograd so they are special cases
.