Ok. I also got this warning before training starts. I don’t think that should be a problem, but would like to know your opinion on it.

Using  2 GPUs!
/usr/local/lib/python3.6/dist-packages/torch/nn/parallel/data_parallel.py:30: UserWarning:
There is an imbalance between your GPUs. You may want to exclude GPU 0 which
has less than 75% of the memory or cores of GPU 1. You can do so by setting
the device_ids argument to DataParallel, or by setting the CUDA_VISIBLE_DEVICES
environment variable.
warnings.warn(imbalance_warn.format(device_ids[min_pos], device_ids[max_pos]))

I tried to reduce the batch size. But still ended up getting crash.