I am not sure where exactly this error happens. since it throws error after loss.backward() let me show how i calculate loss
inputs = Variable(inputs).cuda()
net(inputs, truth_boxes, truth_labels, truth_instances)
loss = net.loss(inputs, truth_boxes, truth_labels, truth_instances)
          

I do not use nn.DataParallel  to parallelize the whole model instead i parallelize each module components in the entire model using data_parallel function from nn.parallel.data_parallel because some components are c-wrapper which throws error while i use nn.DataParallel on the model