@ptrblck On single GPU, without DDP, i can achieve a speed of 500Hz (samples processed per second) and 95% GPU utilization. However, this suddenly drops to 50-60 Hz after half epoch, and GPU utilization fluctuates between 0% and 50%. Most of the time it remains 0%. Any idea why this could happen?