Documenting this would be useful. In particular the fact that there’s global autograd engine shared among Python threads, and that a call into backward will block until all concurrent .backward calls complete. Perhaps here? https://pytorch.org/docs/stable/autograd.html#torch.Tensor.backward 20
Disallowing concurrent backward calls in a process seems fine since Python support for concurrency is not great.