both / either of those weight initialization code snippets should work.
I tried to reproduce your issue locally, so I created a minimal script that wraps your model around DataParallel and runs it. So far I have no luck. Here’s the script I ran: https://gist.github.com/f7e9afbdc561a2cfa9a2c5bdf443aa8b 11
I’d suggest trying to minimize the problem until it shows up, maybe?