Yes, you are correct in the assumption that you don’t need a local CUDA and cudnn installation, if you are installing the binaries.
The NVIDIA driver should be sufficient.
Are you getting some other CUDA error while running the code or is this error raised randomly?