My fault, I run pip install -r requirements.txt before the compilation though I removed torch and torchvision indeed, I forgot to remove pytorch-ignite which installed torch 1.6.0.
The code DEFAULT_EXTRA_FILES_MAP = torch._C.ExtraFilesMap() exist only in an older version of pytorch aka 1.6.0 and before and somehow the building procedure didn’t remove it.