hi guys 
Just wanted to let you know it is not related to only pytorch framework. Happens also with tensorflow.
It feels like modern power supplies can’t handle 300w + 300w simultaneously during training. I got double 2080 ti with another power supply (with cheap 2$ synchronizer) and everything works like a charm.