I would start with a new and clean conda virtual environment, clone the repository and update all submodules:
git clone --recursive https://github.com/pytorch/pytorch

and install it using the already posted build instructions.
If you see any issues with the build, just post an update here (with the error message from the logs) and we can check what’s wrong.
In case you have some trouble with finding some CUDA libs during the build process, you could also try to use a docker container with an installed CUDA toolkit first and check the installation inside it.