Thank you for the great response!
You are right, I hide devices, for example, to show tensorboard, then all GPUs are used (via export CUDA_VISIBLE_DEVICES="" in a console)
…I never fought about consequences of dynamic masking in the code, because didn’t have problems. Thanks