I’m not sure why you want to override the cuda method, but guess you are running into another device mismatch with self.random_matrix, since it’s initialized as a plain list, and tried to fix it with the custom cuda method.
To write device-agnostic code remove all cuda() calls and make sure all modules, parameters, and buffers are properly registered.
E.g. self.random_matrix won’t register the tensors and you would either have to use e.g. nn.ParameterList or register all tensors via self.register_buffer.
This will make sure that calling random_layer.cuda() pushes all internal submodules etc. to the defined device (which is used in nn.DataParallel). The updated module looks generally alright and works fine in this code snippet on 8 GPUs:
class RandomLayer(nn.Module):
    def __init__(self, batch_size, input_dim_list=[], output_dim=1024):
        super(RandomLayer, self).__init__()
        self.input_num = len(input_dim_list)# 2
        self.output_dim = output_dim
        self.random_matrix0 = torch.nn.Parameter(torch.rand(input_dim_list[0], output_dim))
        self.random_matrix1 = torch.nn.Parameter(torch.rand(input_dim_list[1], output_dim))

    def forward(self, x1, x2):
        return_list0 = torch.mm(x1, self.random_matrix0)
        return_list1 = torch.mm(x2, self.random_matrix1)
        return_tensor = return_list0 / math.pow(float(self.output_dim), 1.0 / 2)
        return_tensor = torch.mul(return_tensor, return_list1)
        return return_tensor


class MyModule(nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()
        self.rand = RandomLayer(1, [1, 1])

    def forward(self, x1, x2):
        print(x1.device)
        x = self.rand(x1, x2)
        return x

model = MyModule()
model = nn.DataParallel(model).cuda()
x1 = torch.randn(8, 1).cuda()
x2 = torch.randn(8, 1).cuda()
out = model(x1, x2)
print(out.device)

Note that nn.DataParallel will split the input batch in dim0 and send each chunk to the corresponding GPU. In your current code snippet you are trying to index input_list in dim0, which also seems to be wrong, as it would index it in the (split) batch dimension.
You could thus either pass multiple inputs (as done in my code snippet) or stack and index the input in another dimension.