In that case, yes this is how it is implemented using the new vmap prototype.
You can see the code here: pytorch/functional.py at e91aeb0470f329556f599ac9feb21b0cff31e11c · pytorch/pytorch · GitHub 20