Bhack:

I see that CUDA has failed to initialize. Your environment is not in good shape.


Well kind of. We use Bright Cluster with Slurm. So on our head node we use a “SBATCH” file (Slurm batch) that calls modules. TF 2.6 is not yet available in Bright’s packages. I used pip to install TF 2.6 on a node in Python 3. So now I exclude the call to the TF module in the SBATCH file and let Slurm auto-magically find that TF 2.6 I installed. It looks like we also needed CUDA 11 or greater. For now it’s running but without the GPU.