Hi @daemonslayer, the problem is that you are using model.module to train which is limited to the default device. In DP, you should specify all forward behavior in forward() function and just call model(input) instead of calling model.encode/decode separately.