Hi, I’m trying to distribute the loss among 4 available GPUs. I’m getting below error.
Traceback (most recent call last):
File “gg.py”, line 177, in 
logps = torch.mean(torch.stack(logps))
RuntimeError: All input tensors must be on the same device. Received cuda:0 and cuda:3
Using parallel.py file from thomaswolf. Any suggestion/methods to achieve parallel loss?
Code Snippet
criterion = nn.NLLLoss(reduce=False)
criterion = DataParallelCriterion(criterion, device_ids=[0, 1, 2, 3])
model =DataParallelModel(model, device_ids=[0, 1, 2, 3])
model.to(device)
with torch.no_grad():
for inputs, labels in testloader:
inputs, labels = inputs.to(device), labels.to(device)
#inputs=inputs.unsqueeze(1)
logps = model.forward(inputs)
batch_loss = criterion(logps, labels)
batch_loss=batch_loss.mean()
test_loss += batch_loss.item()
logps = torch.mean(torch.stack(logps))
print(logps)
                # Calculate accuracy
                ps = torch.exp(logps)
                top_p, top_class = ps.topk(1, dim=1)
                equals = top_class == labels.view(*top_class.shape)
                accuracy += torch.mean(equals.type(torch.FloatTensor)).item()