I’ve got a similar CPU (Westmere Xeon) and have the same problem.  This appears to be PyTorch issue 43300 (https://github.com/pytorch/pytorch/issues/43300 3).  I’ve tried the latest 1.7.0 nightly build and while I don’t get this problem, I get a different one that’s probably also AVX related.
FWIW, my new issue with the nightly is that NNPACK, while running nnp_initialize returns “unsupported hardware”
The exact error is:
[W NNPACK.cpp:80] Could not initialize NNPACK! Reason: Unsupported hardware.
For a CPU as old as yours and mine, I think we’re going to have to build from source to ensure that no AVX support sneaks in.