@ptrblck could you expand on what exclusive process vs shared mode entails? As far as I understand it’s a common practice in compute clusters to have the GPU set up in “exclusive process” mode and that’s not changeable by a user.
How does pytorch work when doing distributed work as opposed to the regular case?