Is your work directory in the PyTorch source code dir? If so, could you change it to another workspace?
I would recommend to clean the build (via python setup.py clean), update all submodules and rebuild.
Your Dockerfile doesn’t show the PyTorch installation, which would be interesting to see.