Thank you. Executing
ln -s /usr/local/nvidia/lib64/libnvidia-ml.so.3xx.xx /lib/x86_64-linux-gnu/libnvidia-ml.so
ldconfig

solved the problem.
nvidia-docker run --rm -it nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 nvidia-smi works fine.