@mrshenli
The problem happens regardless of the value of broadcast_buffers.
I managed to narrow down my search for the source of the problem.
The error occurs if I use multiple GPUs per machine AND multiple forward passes of the module before backward().
Otherwise, it works just fine.