I have this problem too.
In my case, this problem occurs only when I wrap the whole model with DataParallel.
@WendyShang
What is the proper usage of flatten_parameters() and what does it do?
Should I call flatten_parameters at initialization or at every forward step?