Solved by richard in post #7 
              
              
                Okay, I repro-ed the problem. You should use register_buffer to add the counts tensor to the Module so that DataParallel will pick it up. The following will fix it: 
import torch.nn as nn
import torch
from torch.autograd import Variable
import math

def main():
    torch.set_default_tensor_type('tor…