Hi! I just ported a bunch of CFFI extensions from pyTorch 0.4 to the “new” C++ extension framework needed in pyTorch 1.6.
Everything works fine, and it builds using ninja. But it rebuilds everything every time I do the build, and apart from that, the build process now is literally 100x slower than before (now it takes about 2-3 minutes to do a build, and it took 10 seconds before with CFFI   ).
This might be OK for a final deploy but it’s of course horrid for development.
Any hints on both issues? Why does it “clean” the objects every time I run? And why are the individual files compiled so much more slowly now (I guess this last one is due to the massive C++ .h frameworks that now has to be pulled in?). The latter is less of an issue if the first one can get solved.
FWIW, in the build/ folder, I still see all the .o files from the last build under build/temp.linux-x86_64-3.6/ but they aren’t used the next time I build.
My build script is just something like this:
setup(name=‘myext’,
version=‘1.0.0’,
ext_modules=[
CUDAExtension(name=‘my_render’, sources=[‘my_render.cpp’, ‘my_render_gaussian_cuda.cu’]),
],
cmdclass={‘build_ext’: BuildExtension})
and I invoke it with:
MAX_JOBS=6 TORCH_CUDA_ARCH_LIST=Pascal ./build_cudaextensions.py install --prefix=$HOME/.local
I know I can trim the arch list and just set 6.1 for example (Pascal implies 6.0, 6.1 so some speedup per file could be done there).