Thanks @ptrblck . My MWE would be a bit convoluted as I am using DDP through pytorch lightning (I don’t think that is the source of this error, however). Could you please post the code snippet that works for you? I can modify that to illustrate the assertion failure.