Hi,
In my code, I want to keep the latest input feature map to the layer and subtract it with the input feature map and then update the latest value.
The code is working on a single GPU, but I get the following error when running on multiple GPUs. Any thoughts?

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

class Subtraction(nn.Module):
    def __init__(self, channel,width,height):
        super(Subtraction, self).__init__()
        
        self.latest_input = Variable(torch.zeros(channel,width,height), requires_grad=False).cuda()

    def forward(self, input):
        #difference of feature maps
        output = input - self.latest_input
        self.latest_input = input.clone()
        return output