Rather than mess with cleanup, I just deleted the folder & re-cloned it. I did:

git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
git pull
git checkout tags/v1.6.0
git pull
python3 setup.py build

and the end result was the same:

…/caffe2/quantization/server/conv_dnnlowp_op.cc: In instantiation of ‘void caffe2::ConvDNNLowPOp<T, ReluFused>::ConvNHWCCore_(const T*, std::vector*) [with T = unsigned char; bool ReluFused = false]’:
…/caffe2/quantization/server/conv_dnnlowp_op.cc:1752:16:   required from here
…/caffe2/quantization/server/conv_dnnlowp_op.cc:1211:55: error: ‘depthwise_3x3x3_per_channel_quantization_pad_1’ was not declared in this scope
depthwise_3x3x3_per_channel_quantization_pad_1(

Any ideas what else I could try?
Thanks again.