I have a tensor of shape 12X9X1024. I have created 9 individual fully connected layers which take 12 X 1024 shape array as input. Here 12 is the batch size and 1024 is the input dimension to the fully connected layer.
I planned to append the output of each FCN to a list. Then convert the list of tensors to a tensor using torch.cat. When the code is at out.append(f(tensors[:,i,:]).view(bs,-1,2))
I get this error RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)
class Classifier(nn.Module):

    def __init__(self,hidden_dim):
        super().__init__()
        _,centers = config.get_inds()
        _,counts = np.unique(np.array(centers, dtype= np.int32),return_counts=True)
        self.fcn = []
        self.classes = sum(counts)
        for j in counts:     
            self.fcn.extend([nn.Linear(hidden_dim,2*j)])
    def forward(self,tensors):
        """
            tensor: n_batch X n_center X hidden_dim
        """
        bs = tensors.shape[0]
        out = []
        for i,f in enumerate(self.fcn):
            out.append(f(tensors[:,i,:]).view(bs,-1,2))
        out_tensor = torch.Tensor((bs,self.classes,2),device=tensors.device,requires_grad=True)
        torch.cat(out,dim=1,out=out_tensor)
        out_tensor = F.log_softmax(out_tensor, dim=1)
        return out_tensor

How to make sure the list of tensors are on GPU and has requires_grad=True?