Has been some time since PyTorch 0.2, but I can’t remember it being that slow. It should take a few seconds at most, not minutes. Not sure what’s causing the problem in your case, could be that it’s been a bug in PyTorch 0.2?
PyTorch 0.3 and 0.4 also work with Python 2.7, regardign the compatibility issues you mentioned, the Tesla V100 should work with cuda 8 & 9, so I think so if you can install it  0.3 or 0.4 somehow alongside the 0.2 version, it would help figuring out whether it’s a PyTorch 0.2-specific bug or sth else.