I want to use multi gpu manually, because the input data size is different.
for (int i = 0; i < param_.gpu_size(); ++i) {
    torch::Device device(torch::kCUDA, i);
    models.push_back(torch::jit::load(MODEL_PATH, device));
    models.back()->eval();
}

And I will create gpu_size threads to run the inference.
Is it the correct way?