Your local CUDA installation won’t be used, if you are installing the binaries, which ship with their own CUDA runtime. You would thus only need the appropriate driver.
The error message is raised, if you are using a build, which doesn’t support your GPU architecture, so which GPU are you using?