Hi, everyone!
I have a PyTorch model generated by torch.jit.trace().save(). According to the information I found online, it didn’t seem to support torch.nn.parallel when the model is generated and saved.
Then, I converted it to TorchScript, loaded, and finally executed the script module in C++.
The whole program is used to detect objects in a video.
But I am not satisfied enough with its runtime (represented by FPS). Since I have two GPUs, I am thinking if it’s possible to execute on multiple GPUs (for example, load 32 frames from the video and let each GPU handle 16). But still, it’s impossible to load the script module directly to multiple GPUs using torch::jit::load(modelPath.c_str()). In this way, I am curious about if it’s possible to use torch::nn::parallel.
Could anyone please give me a clue on how to use torch::nn::parallel in my case? If it’s not possible, are there any other possible ways of executing the script module in C++ on multiple GPUs in parallel? Can I only turn to multiple-GPU programming with CUDA?
Thanks a lot for any possible help!