Unfortunately it is very hard to do a general check like this as it will depend a lot on how your manage your cluster with coworkers.
You can use nvidia-smi to check memory usage and/or process running on the different GPUs potentially.
You can check this 12 small tool for example on how to do such thing and adapt it to your needs.