Simply checking whether a GPU is “used” might be dangerous as it might be a race with something else that is contending for a GPU. However, if you are confident about the scheduling of jobs, you can try something like nvidia-smi --query-compute-apps=pid,process_name,used_memory,gpu_bus_id --format=csv.