Almost  pytorch conda package still requires conda cuda toolkit package, which will bring the following unnecessary goodies with it:
/opt/conda/pkgs/cudatoolkit-9.0-h13b8566_0/lib/libcusolver.so.9.0.176
/opt/conda/pkgs/cudatoolkit-9.0-h13b8566_0/lib/libcusparse.so.9.0.176
/opt/conda/pkgs/cudatoolkit-9.0-h13b8566_0/lib/libcurand.so.9.0.176
/opt/conda/pkgs/cudatoolkit-9.0-h13b8566_0/lib/libnppif.so.9.0.176
/opt/conda/pkgs/cudatoolkit-9.0-h13b8566_0/lib/libcufft.so.9.0.176
/opt/conda/pkgs/cudatoolkit-9.0-h13b8566_0/lib/libcublas.so.9.0.176