You won’t be able to build PyTorch 1.4.0 with CUDA11 without some cherry-picks, as 1.4.0 was released before CUDA11 was available.
The right approach would be to drop the support for Python2 (alone for the potential security risks) and to update PyTorch to the latest release with CUDA11.
In case that’s not possible, you could search all CUDA11-related commits, try to cherry-pick them into the 1.4.0 branch, and rebuild, although that sounds quite tedious.