Hi!
I’m trying to make custom build of PyTorch Mobile as in Custom Build 9. During build I’m facing this error:
Traceback (most recent call last):
  File "C:/Users/Racek/android/PyTorch_binaries/pytorch/cmake/../aten/src/ATen/gen_vulkan_spv.py", line 8, in <module>
    from tools.codegen.code_template import CodeTemplate
ModuleNotFoundError: No module named 'tools'
CMake Error at cmake/VulkanCodegen.cmake:70 (message):
  Failed to gen spv.h and spv.cpp with precompiled shaders for Vulkan backend
Call Stack (most recent call first):
  caffe2/CMakeLists.txt:6 (include)

What I’ve did:

cloned pytorch repo
set ANDROID_HOME env variable
installed android ndk via Android Studio and set ANDROID_NDK env variable
installed CMake 3.20.0 and added it to PATH

full log:
+++ dirname pytorch/scripts/build_pytorch_android.sh
++ cd pytorch/scripts/..
++ pwd -P
+ PYTORCH_DIR=/c/Users/Racek/android/PyTorch_binaries/pytorch
+ PYTORCH_ANDROID_DIR=/c/Users/Racek/android/PyTorch_binaries/pytorch/android
+ echo PYTORCH_DIR:/c/Users/Racek/android/PyTorch_binaries/pytorch
PYTORCH_DIR:/c/Users/Racek/android/PyTorch_binaries/pytorch
+ source /c/Users/Racek/android/PyTorch_binaries/pytorch/android/common.sh
++ set -eux
++ '[' -z /c/Users/Racek/android/PyTorch_binaries/pytorch ']'
+ check_android_sdk
+ '[' -z 'C:\Users\Racek\AppData\Local\Android\Sdk' ']'
+ '[' '!' -d 'C:\Users\Racek\AppData\Local\Android\Sdk' ']'
+ echo 'ANDROID_HOME:C:\Users\Racek\AppData\Local\Android\Sdk'
ANDROID_HOME:C:\Users\Racek\AppData\Local\Android\Sdk
+ check_gradle
+ GRADLE_PATH=/c/Users/Racek/android/PyTorch_binaries/pytorch/android/gradlew
+ echo GRADLE_PATH:/c/Users/Racek/android/PyTorch_binaries/pytorch/android/gradlew
GRADLE_PATH:/c/Users/Racek/android/PyTorch_binaries/pytorch/android/gradlew
+ parse_abis_list arm64-v8a
+ ABIS_LIST=armeabi-v7a,arm64-v8a,x86,x86_64
+ CUSTOM_ABIS_LIST=false
+ '[' 1 -gt 0 ']'
+ ABIS_LIST=arm64-v8a
+ CUSTOM_ABIS_LIST=true
+ echo ABIS_LIST:arm64-v8a
ABIS_LIST:arm64-v8a
+ echo CUSTOM_ABIS_LIST:true
CUSTOM_ABIS_LIST:true
+ build_android
+ PYTORCH_ANDROID_DIR=/c/Users/Racek/android/PyTorch_binaries/pytorch/android
+ BUILD_ROOT=/c/Users/Racek/android/PyTorch_binaries/pytorch
+ echo BUILD_ROOT:/c/Users/Racek/android/PyTorch_binaries/pytorch
BUILD_ROOT:/c/Users/Racek/android/PyTorch_binaries/pytorch
+ LIB_DIR=/c/Users/Racek/android/PyTorch_binaries/pytorch/android/pytorch_android/src/main/jniLibs
+ INCLUDE_DIR=/c/Users/Racek/android/PyTorch_binaries/pytorch/android/pytorch_android/src/main/cpp/libtorch_include
+ rm -rf /c/Users/Racek/android/PyTorch_binaries/pytorch/android/pytorch_android/src/main/jniLibs
+ mkdir -p /c/Users/Racek/android/PyTorch_binaries/pytorch/android/pytorch_android/src/main/jniLibs
+ rm -rf /c/Users/Racek/android/PyTorch_binaries/pytorch/android/pytorch_android/src/main/cpp/libtorch_include
+ mkdir -p /c/Users/Racek/android/PyTorch_binaries/pytorch/android/pytorch_android/src/main/cpp/libtorch_include
++ echo arm64-v8a
++ tr , '\n'
+ for abi in $(echo "$ABIS_LIST" | tr ',' '\n')
+ echo abi:arm64-v8a
abi:arm64-v8a
+ ANDROID_BUILD_ROOT=/c/Users/Racek/android/PyTorch_binaries/pytorch/build_android_arm64-v8a
++ which ccache
which: no ccache in (C:\Program Files\CMake\bin:C:\Users\Racek\AppData\Local\Android\Sdk\cmake\3.18.1\bin:/c/Users/Racek/bin:/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/mingw64/bin:/usr/bin:/c/Users/Racek/bin:/c/ProgramData/Anaconda3:/c/ProgramData/Anaconda3/Library/mingw-w64/bin:/c/ProgramData/Anaconda3/Library/usr/bin:/c/ProgramData/Anaconda3/Library/bin:/c/ProgramData/Anaconda3/Scripts:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files (x86)/NVIDIA Corporation/PhysX/Common:/c/Program Files/NVIDIA Corporation/NVIDIA NvDLISR:/cmd:/c/Users/Racek/AppData/Local/Microsoft/WindowsApps:/c/Users/Racek/AppData/Local/atom/bin:/usr/bin/vendor_perl:/usr/bin/core_perl)
+ ANDROID_ABI=arm64-v8a
+ BUILD_ROOT=/c/Users/Racek/android/PyTorch_binaries/pytorch/build_android_arm64-v8a
+ /c/Users/Racek/android/PyTorch_binaries/pytorch/scripts/build_android.sh -DANDROID_CCACHE=
Build with ANDROID_ABI[arm64-v8a], ANDROID_NATIVE_API_LEVEL[21]
Bash: GNU bash, version 4.4.23(1)-release (x86_64-pc-msys)
Python: 3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)]
Caffe2 path: /c/Users/Racek/android/PyTorch_binaries/pytorch
Using Android NDK at C:\Users\Racek\AppData\Local\Android\Sdk\ndk\23.0.7123448
Android NDK version: 23
Choose SELECTED_OP_LIST file: /c/Users/Racek/android/PyTorch_binaries/mobilenetv3_small_new.yaml
-- Android: Targeting API '21' with architecture 'arm64', ABI 'arm64-v8a', and processor 'aarch64'
-- Android: Selected unified Clang toolchain
-- The CXX compiler identification is Clang 11.0.5
-- The C compiler identification is Clang 11.0.5
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Users/Racek/AppData/Local/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/windows-x86_64/bin/clang++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Users/Racek/AppData/Local/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Not forcing any particular BLAS to be found
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED - Success
-- std::exception_ptr is supported.
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING - Failed
-- Turning off deprecation warning due to glog.
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS - Failed
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC - Success
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Trying to find preferred BLAS backend of choice: Eigen
CMake Error at cmake/Dependencies.cmake:337 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/pthreadpool

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:340 (set_property):
  set_property could not find TARGET pthreadpool.  Perhaps it has not yet
  been created.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:380 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/cpuinfo

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:385 (set_property):
  set_property could not find TARGET cpuinfo.  Perhaps it has not yet been
  created.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


-- The ASM compiler identification is Clang
-- Found assembler: C:/Users/Racek/AppData/Local/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe
CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:320 (add_subdirectory):
  add_subdirectory given source
  "C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/cpuinfo/deps/clog"
  which is not an existing directory.


CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:324 (set_property):
  set_property could not find TARGET clog.  Perhaps it has not yet been
  created.


CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:342 (add_subdirectory):
  The binary directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/build_android_arm64-v8a/confu-deps/cpuinfo

  is already used to build a source directory.  It cannot be used to build
  source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/cpuinfo

  Specify a unique binary directory name.


CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:352 (add_subdirectory):
  The binary directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/build_android_arm64-v8a/confu-deps/pthreadpool

  is already used to build a source directory.  It cannot be used to build
  source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/pthreadpool

  Specify a unique binary directory name.


CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:378 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/FXdiv

  does not contain a CMakeLists.txt file.


CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:393 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/psimd

  does not contain a CMakeLists.txt file.


CMake Error at aten/src/ATen/native/quantized/cpu/qnnpack/CMakeLists.txt:410 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/FP16

  does not contain a CMakeLists.txt file.


CMake Error at cmake/Dependencies.cmake:468 (set_property):
  set_property could not find TARGET cpuinfo.  Perhaps it has not yet been
  created.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


-- Brace yourself, we are building NNPACK
CMake Error at cmake/External/nnpack.cmake:71 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/NNPACK

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  cmake/Dependencies.cmake:498 (include)
  CMakeLists.txt:603 (include)


CMake Error at cmake/External/nnpack.cmake:76 (set_property):
  set_property could not find TARGET nnpack.  Perhaps it has not yet been
  created.
Call Stack (most recent call first):
  cmake/Dependencies.cmake:498 (include)
  CMakeLists.txt:603 (include)


CMake Error at cmake/External/nnpack.cmake:77 (set_property):
  set_property could not find TARGET pthreadpool.  Perhaps it has not yet
  been created.
Call Stack (most recent call first):
  cmake/Dependencies.cmake:498 (include)
  CMakeLists.txt:603 (include)


CMake Error at cmake/External/nnpack.cmake:78 (set_property):
  set_property could not find TARGET cpuinfo.  Perhaps it has not yet been
  created.
Call Stack (most recent call first):
  cmake/Dependencies.cmake:498 (include)
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:528 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/XNNPACK

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:532 (set_property):
  set_property could not find TARGET XNNPACK.  Perhaps it has not yet been
  created.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


-- VULKAN_INCLUDE_DIR:C:/Users/Racek/AppData/Local/Android/Sdk/ndk/23.0.7123448/sources/third-party/vulkan/src/include
-- Vulkan_ANDROID_NDK_WRAPPER_DIR:C:/Users/Racek/AppData/Local/Android/Sdk/ndk/23.0.7123448/sources/third_party/vulkan/src/common
CMake Warning at cmake/Dependencies.cmake:732 (message):
  Turning USE_FAKELOWP off as it depends on USE_FBGEMM.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:866 (add_subdirectory):
  The binary directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/build_android_arm64-v8a/confu-deps/fp16

  is already used to build a source directory.  It cannot be used to build
  source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/FP16

  Specify a unique binary directory name.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


-- Using third party subdirectory Eigen.
-- Could NOT find pybind11 (missing: pybind11_DIR)
-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR)
-- Using third_party/pybind11.
-- pybind11 include dirs: C:/Users/Racek/android/PyTorch_binaries/pytorch/cmake/../third_party/pybind11/include
CMake Warning at cmake/Dependencies.cmake:1263 (message):
  Not using CUDA/ROCM, so disabling USE_NCCL.  Suppress this warning with
  -DUSE_NCCL=OFF.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Warning at cmake/Dependencies.cmake:1383 (message):
  Metal is only used in ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:1779 (add_subdirectory):
  The source directory

    C:/Users/Racek/android/PyTorch_binaries/pytorch/third_party/fmt

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


CMake Error at cmake/Dependencies.cmake:1788 (set_target_properties):
  set_target_properties Can not find target to add properties to:
  fmt-header-only
Call Stack (most recent call first):
  CMakeLists.txt:603 (include)


-- Not using libkineto in a mobile build.
-- Performing Test HAS_WERROR_FORMAT
-- Performing Test HAS_WERROR_FORMAT - Success
-- Performing Test HAS_WERROR_CAST_FUNCTION_TYPE
-- Performing Test HAS_WERROR_CAST_FUNCTION_TYPE - Success
-- Performing Test HAS_VST1
-- Performing Test HAS_VST1 - Success
-- Performing Test HAS_VLD1
-- Performing Test HAS_VLD1 - Success
-- Looking for backtrace
-- Looking for backtrace - not found
-- Could NOT find Backtrace (missing: Backtrace_LIBRARY Backtrace_INCLUDE_DIR)
-- don't use NUMA
INFOUse default op dependency graph .yaml file for custom build with dynamic dispatch.
-- Custom build with op registration whitelist: aten::Int;aten::_adaptive_avg_pool2d;aten::_addmv_impl_;aten::_batch_norm_impl_index;aten::_cat;aten::_coalesced_;aten::_convolution;aten::_convolution_nogroup;aten::_copy_from;aten::_empty_affine_quantized;aten::_empty_per_channel_affine_quantized;aten::_fused_dropout;aten::_indices;aten::_local_scalar_dense;aten::_make_per_tensor_quantized_tensor;aten::_mkldnn_reshape;aten::_mkldnn_transpose;aten::_mkldnn_transpose_;aten::_nnpack_available;aten::_nnpack_spatial_convolution;aten::_nnz;aten::_sparse_coo_tensor_with_dims_and_tensors;aten::_unsafe_view;aten::_values;aten::adaptive_avg_pool2d;aten::add;aten::add_;aten::addmm;aten::addmm_;aten::arange;aten::as_strided;aten::as_strided_;aten::baddbmm_;aten::batch_norm;aten::bernoulli_;aten::bmm;aten::cat;aten::clamp;aten::clamp_;aten::clamp_max;aten::clamp_min;aten::clamp_min_;aten::clone;aten::contiguous;aten::convolution_overrideable;aten::copy_;aten::copy_sparse_to_sparse_;aten::cudnn_batch_norm;aten::cudnn_convolution;aten::cudnn_convolution_transpose;aten::dense_dim;aten::dequantize;aten::detach;aten::div;aten::div_;aten::dot;aten::dropout;aten::empty;aten::empty_like;aten::empty_meta;aten::empty_strided;aten::eq;aten::equal;aten::expand;aten::fill_;aten::hardtanh_;aten::index_select;aten::int_repr;aten::is_coalesced;aten::is_floating_point;aten::is_leaf;aten::is_nonzero;aten::is_same_size;aten::item;aten::matmul;aten::max;aten::maximum;aten::mean;aten::min;aten::minimum;aten::miopen_batch_norm;aten::miopen_convolution;aten::miopen_convolution_transpose;aten::miopen_depthwise_convolution;aten::mkldnn_adaptive_avg_pool2d;aten::mm;aten::mul;aten::mul_;aten::mv;aten::narrow;aten::native_batch_norm;aten::ne;aten::nonzero;aten::ones;aten::permute;aten::q_per_channel_axis;aten::q_per_channel_scales;aten::q_per_channel_zero_points;aten::q_scale;aten::q_zero_point;aten::qscheme;aten::quantize_per_tensor;aten::relu_;aten::reshape;aten::resize_;aten::resize_as_;aten::select;aten::set_;aten::size;aten::slice;aten::slow_conv3d;aten::slow_conv3d_forward;aten::slow_conv_dilated2d;aten::slow_conv_dilated3d;aten::slow_conv_transpose2d;aten::slow_conv_transpose3d;aten::sparse_dim;aten::sparse_resize_and_clear_;aten::squeeze;aten::squeeze_;aten::sum;aten::t;aten::thnn_conv2d;aten::thnn_conv2d_forward;aten::thnn_conv_depthwise2d;aten::thnn_conv_depthwise2d_forward;aten::to;aten::transpose;aten::transpose_;aten::unsqueeze;aten::unsqueeze_;aten::view;aten::zero_;aten::zeros;aten::zeros_like;prim::NumToTensor

Traceback (most recent call last):
  File "C:/Users/Racek/android/PyTorch_binaries/pytorch/cmake/../aten/src/ATen/gen_vulkan_spv.py", line 8, in <module>
    from tools.codegen.code_template import CodeTemplate
ModuleNotFoundError: No module named 'tools'
CMake Error at cmake/VulkanCodegen.cmake:70 (message):
  Failed to gen spv.h and spv.cpp with precompiled shaders for Vulkan backend
Call Stack (most recent call first):
  caffe2/CMakeLists.txt:6 (include)


-- Configuring incomplete, errors occurred!
See also "C:/Users/Racek/android/PyTorch_binaries/pytorch/build_android_arm64-v8a/CMakeFiles/CMakeOutput.log".
See also "C:/Users/Racek/android/PyTorch_binaries/pytorch/build_android_arm64-v8a/CMakeFiles/CMakeError.log".

At this point I have no idea how to fix it. Can you help me? I can post CMakeError.log and CMakeOutput.log if needed.