I figured out the reason. GKE COS has CUDA driver installed 450.51.06. For minor compatibility of CUDA 11.1, driver should be at least 450.80.02. CUDA Compatibility :: GPU Deployment and Management Documentation 70