Hi hyu,
I didn’t try to install pytorch with 2 versions of openmpi.
Maybe that pytorch is taking the first version it finds in the library path … did you try to put the path for your cuda-aware openmpi in the beginning of your paths (PATH and LD_LIBRARY_PATH) ?