Thanks for posting the code!
I can reproduce the issue and stumbled upon this issue 34, which seems to be worked on.
Could you post a link to your use case there and ping for an update?
As a workaround you could call adj_norm.to_dense() for nn.DataParallel or try out DistributedDataParallel 27.