@Martin_Mundt

it is not necessary to download CUDNN explicitly by registering for the NVIDIA Dev program and downloading it oneself anymore?

No, not needed.

Also last I checked CUDNN had something like per-person licensing, so how does this work?

CuDNN, like CUDA now has a redistributable clause.