Shenggan:

Or is there any plan to support old version PyTorch for A100?


There is no plan on changing older PyTorch versions to enable CUDA11 and thus new GPU architectures, so you would have to use the latest PyTorch version.



 Shenggan:

is it possible to build from source for old version PyTorch (for example version 1.4) with cuda11?


You could try to cherry-pick all commits mentioning CUDA11 in an older version and try to build it.
However, while it might work, what’s your use case that you need to use an old PyTorch version?