Hi theevann,
I have some question.
Thanks to you, I installed successfully openmpi with cude and tested it by “ompi_info --parsable --all | grep mpi_built_with_cuda_support:value”. (I installed it in ~/lib/openmpi_cuda.)
Also I set PATH and LD_LIBRARY_PATH for it to be first executable and library.
I checked as following:
which mpirun
~/lib/openmpi_cuda/bin/mpirun
But when I install pytorch from source code (https://github.com/pytorch/pytorch 4) with OpenMPI, it doesn’t use the OpenMPI (~/lib/openmpi_cuda).
Instead of it, it refer the OpenMPI installed by “conda install -c conda-forge openmpi” which is NOT CUDA-aware MPI.
So how can I connect the CUDA-aware OpenMPI I installed when I install pytorch (with MPI) from source code?
Thanks.