I rolled back to CUDA - 11.2 on Host machine , still when starting the latest tensorflow container , it fails at same .
(pid=1829) 2021-11-08 11:34:42,477 INFO     Processing /app/project/training_0000/raw/segment-10212406498497081993_5300_000_5320_000_with_camera_labels.tfrecord
(pid=1868) 2021-11-08 11:34:43.089321: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected