You should have a look at the nn.DataParallel 21 and nn.distributed.DistributedDataParallel 21 docs. Both are fairly easy objects to work with. It will basically split your input in the batch dimension across the ids of the GPUs that you pass at initialization.
Note that some model architectures are not able to be parallelized across devices.
Also, if you are just running inference, you may not see any benefit to multi-GPU parallelization. Or even using a GPU at all. You might try running the model after a call to model.eval() if you are experiencing performance issues.