Sounds like it could be a bug. Could you please provide some random inputs for the forward pass so that I can run it and see what’s up?
I’m curious if the problem is how DataParallel works or the model. What happens if you remove the DataParallel?