I have built PyTorch successfully with gcc 4.9 - gcc 8. There is a list in the README 56.
PyTorch will make use of multiple threads all right for its ops, it will be very specific circumstances when it’s insufficient.
Best regards
Thomas