Using nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 as your base image should work, you still have to install nccl-dev from http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64/ 44