I had also faced this issue even on single GPU.
I noticed that cublas samples required sudo permission to Initialize.
Also to avoid root permission, I removed the cache files in ~/.nv directory.
Hope this solution helps.