Thank you @ptrblck  that is very helpful.
I also dug into NVIDIA’s documentation on MIGs and am still not sure about the following,
Does that mean that if MIG is enabled on even a single GPU in the system then all the others essentially become useless?
Also from NVIDIA’s docs:

CUDA will not enumerate non-MIG GPU if any compute instance is enumerated on any other GPU

So, that seems to me to imply that all non-MIG GPUs become essentially inaccessible, is that right?
Second point it, it should still be possible to access different GPU instances (across different GPUs) with explicitly setting the CUDA_VISIBLE_DEVICES like
CUDA_VISIBLE_DEVICES=MIG-GPU-15b2013a-3a22-6ae2-eae9-967e9bda9007/7/0
It’s just that you cannot set several at a time. But you can use one per process.
Is my understanding right?
Thanks you for your help