Extending this short thread: I am performing grid-search of a small LSTM in an 8 GPU instance, and % usage is low. Which one is better to make a more efficient use of resources (given constant model complexity):

Send multiple jobs to all GPUs, or
Send multiple jobs to specific (non-overlapping) GPUs?

What are any pro/cons for each of these options (if at all possible)?