Pytorch does have a support for multiple GPUs, also look into something called as Probabilistic Classification. This technique is mostly used in NLP to predict the upcoming word.So it falls under huge class classification problem.
And you can look here for paralell gpu processing in pytorch
https://pytorch.org/tutorials/beginner/former_torchies/parallelism_tutorial.html 20
I hope this helps!