first:

dont double post. you’ve posted in the other thread with the same huge code-block. It’s not helpful.

second:

Why do you think DataParallel doesn’t work in backward? of course it does work in backward too.