Hi @Yi_Zhang Yes, I would think that this is likely the case and running multiple training jobs will result in more competition for the GPU’s bandwidth and thus overall a slower performance compared to running either job individually.
Are you noticing any extreme slowness/hangs that may be more indicative of a bug?