So if you get 8, 6, then use TORCH_CUDA_ARCH_LIST="8.6". If you have multiple different cards, you can list all
of them like so TORCH_CUDA_ARCH_LIST="6.1;8.6"
If you need to use the same setup on multiple machines, make a binary wheel:
bash
git clone https://github.com/microsoft/DeepSpeed/
cd DeepSpeed
rm -rf build
TORCH_CUDA_ARCH_LIST="8.6" DS_BUILD_CPU_ADAM=1 DS_BUILD_UTILS=1 \
python setup.py build_ext -j8 bdist_wheel
it will generate something like dist/deepspeed-0.3.13+8cd046f-cp38-cp38-linux_x86_64.whl which now you can install
as pip install deepspeed-0.3.13+8cd046f-cp38-cp38-linux_x86_64.whl locally or on any other machine.
Again, remember to ensure to adjust TORCH_CUDA_ARCH_LIST to the target architectures.
You can find the complete list of NVIDIA GPUs and their corresponding Compute Capabilities (same as arch in this
