The terminology between typical GPs lingo and deep learning is a bit different when it comes to inference. For GPs: Inference = find model/hyperparameters (or distributions) “infer them from data” and prediciton: apply model. In DL: Find model = training, inference = apply model.
Any model would have inference and prediction steps, and any framework, too.
The model can be either optimization based (e.g. MAP, VI) or MCMC. It really depends on the details of your problem which is most appropriate.
GPyTorch might, in my view, be more accessible / similar to “classic” GP libraries.
Pyro on the other hand is a generic probabilistic programming framework that happens to be applicable to GPs, too.
But of course I’m biased, long before there was GPyTorch, I had written my own GP library 63 for PyTorch.
Best regards
Thomas