I’m wondering if it’s safe to assume that users will not call my gradient library from multiple Python threads simultaneously
IE, are there reasons to support this use-case?
It seems current PyTorch doesn’t fully support this use case either, thread-global storage for .grad means that two different Python threads can interfere with each other, example https://gist.github.com/yaroslavvb/e53c83c40c8385cd90cdc15c7c61fa63 15