Yeah, I downgrade the CUDA version to 10.2 and building starts but failed with strange error:
FAILED: confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o 
cd /home/{User}/Projects/AI/pytorch/build/confu-deps/NNPACK && PYTHONPATH=/home/{User}/Projects/AI/pytorch/third_party/python-six:/home/{User}/Projects/AI/pytorch/third_party/python-peachpy /usr/local/bin/python3 -m peachpy.x86_64 -mabi=sysv -g4 -mimage-format=elf -I/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src -I/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma -I/home/{User}/Projects/AI/pytorch/third_party/FP16/include -o /home/{User}/Projects/AI/pytorch/build/confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o /home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/2d-fourier-8x8.py
/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/writer.py:114: ResourceWarning: unclosed file <_io.FileIO name='/home/{User}/Projects/AI/pytorch/build/confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o' mode='wb' closefd=True>
  self.output_file = None
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Traceback (most recent call last):
  File "/opt/python-3.8.2/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/python-3.8.2/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 280, in <module>
    main()
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 267, in main
    execute_script(writers, options.input[0])
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 198, in execute_script
    execute_script(writers, source_filename)
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 198, in execute_script
    execute_script(writers, source_filename)
  File "/home/{User}/Projects/AI/pytorch/third_party/python-peachpy/peachpy/x86_64/__main__.py", line 202, in execute_script
    exec(code, globals())
  File "/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/2d-fourier-8x8.py", line 5, in <module>
    import fft.complex_soa
  File "/home/{User}/Projects/AI/pytorch/third_party/NNPACK/src/x86_64-fma/fft/complex_soa.py", line 7, in <module>
    from common import cos_npi_over_8, sin_npi_over_8, cos_npi_over_4, sin_npi_over_4
ImportError: cannot import name 'cos_npi_over_8' from 'common' (/home/{User}/.local/lib/python3.8/site-packages/common/__init__.py)
[25/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vlrelu/gen/vlrelu-scalar-x4.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vlrelu/gen/vlrelu-scalar-x4.c:13:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[26/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vrnd/gen/vrndd-scalar-libm-x1.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vrnd/gen/vrndd-scalar-libm-x1.c:15:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[27/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-velu/gen/velu-scalar-rr2-p6-x6.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-velu/gen/velu-scalar-rr2-p6-x6.c:14:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[28/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vmulcaddc/gen/c2-minmax-scalar-2x.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vmulcaddc.h:11,
                 from ../third_party/XNNPACK/src/f32-vmulcaddc/gen/c2-minmax-scalar-2x.c:13:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[29/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vrnd/gen/vrndd-scalar-libm-x2.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vrnd/gen/vrndd-scalar-libm-x2.c:15:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[30/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vmulcaddc/gen/c4-minmax-scalar-2x.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vmulcaddc.h:11,
                 from ../third_party/XNNPACK/src/f32-vmulcaddc/gen/c4-minmax-scalar-2x.c:13:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[31/5198] Building C object confu-deps/XNNPACK/CMakeFiles/XNNPACK.dir/src/f32-vrnd/gen/vrndd-scalar-libm-x4.c.o
In file included from ../third_party/XNNPACK/include/xnnpack.h:15:0,
                 from ../third_party/XNNPACK/src/xnnpack/params.h:15,
                 from ../third_party/XNNPACK/src/xnnpack/vunary.h:11,
                 from ../third_party/XNNPACK/src/f32-vrnd/gen/vrndd-scalar-libm-x4.c:15:
../third_party/pthreadpool/include/pthreadpool.h:997:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1003:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1010:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1017:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1026:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../third_party/pthreadpool/include/pthreadpool.h:1037:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "setup.py", line 773, in <module>
    build_deps()
  File "setup.py", line 315, in build_deps
    build_caffe2(version=version,
  File "/home/{User}/Projects/AI/pytorch/tools/build_pytorch_libs.py", line 58, in build_caffe2
    cmake.build(my_env)
  File "/home/{User}/Projects/AI/pytorch/tools/setup_helpers/cmake.py", line 346, in build
    self.run(build_args, my_env)
  File "/home/{User}/Projects/AI/pytorch/tools/setup_helpers/cmake.py", line 141, in run
    check_call(command, cwd=self.build_dir, env=env)
  File "/opt/python-3.8.2/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--target', 'install', '--config', 'Release', '--', '-j', '8']' returned non-zero exit status 1.

Error is:
ImportError: cannot import name 'cos_npi_over_8' from 'common'

But how it’s possible, all submodules updated properly …