Thanks  for the suggestion.
I have another issue. I can  run the model inference using C++ , underlying library is ATen. Right now it is running in One GPU. How can I make it run on multiple GPU ? Something similar to dataparallel in PyTorch.
Best regards
JK