I doubt that you can use GPU+CUDA+cudnn in Docker on Windows. You need “NVIDIA Container Runtime for Docker” 158 which allows you to use the hosts GPU in your container. But on Windows Linux runs in a VM, which in turn has no access to the GPU. See also here: https://github.com/NVIDIA/nvidia-docker/wiki/Frequently-Asked-Questions#is-microsoft-windows-supported 121.
Running deep learning tasks in a VM can be very slow, because the Advanced Vector Extensions (AVX) commands are not available in your virtual host (at least on my Mac), which slows down the learning rate a lot.