Hi Guys, sorry to take so long before giving a response, I was off for few weeks, but here I am back.
So I am currently building CPU and GPU version of Torch as static library.
First of all the first time I try I was using lib torch 1.3.1 and 1.4 but same if I was able to build successfully my perf was extremely bad so I decide to move to a more recent version:
So I am currently using this checkout 246b208e4f60d6894ffaeeed8c9746e331fa6f6f : I am almost sure a most recent version should work too.
In order to build I try both calling CMake directly and using Python script and both work for me:
-DBUILD_CAFFE2_MOBILE=OFF;
-DBUILD_SHARED_LIBS=OFF;
-DCAFFE2_USE_MSVC_STATIC_RUNTIME=ON;
-DMSVC_Z7_OVERRIDE=OFF;
-DUSE_CUDA=OFF/ON;
-DBUILD_PYTHON=OFF;
-DBUILD_TEST=OFF;

One important point I decide to use Visual Studio 2019 too build, when using the script python I decide to force two part:

Disable python : HERE 17

Force Visual studio 2019: HERE 17


I am sure there are better way to bypass those value, but for me it was enough like that.
After that the build was going smooth and I add all the static library ready then for linking here what I use:
For CPU only:
        WHOLE_LIBRARIES
        torch;
        torch_cpu;

        LIBRARIES
        clog
        cpuinfo;
        libprotobuf;
        libprotoc;
        onnx_proto;
        onnx;
        c10;
        fbgemm;
        dnnl;
        foxi_loader;
        caffe2_protos;
        Caffe2_perfkernels_avx;
        Caffe2_perfkernels_avx2;
        Caffe2_perfkernels_avx512;

For GPU:
        WHOLE_LIBRARIES
        torch_cpu;
        torch_cuda;
        c10_cuda;

        LIBRARIES
        asmjit
        c10
        c10_cuda
        Caffe2_perfkernels_avx
        Caffe2_perfkernels_avx2
        Caffe2_perfkernels_avx512
        caffe2_protos
        clog
        cpuinfo_internals
        cpuinfo
        dnnl
        fbgemm
        foxi_loader
        mkldnn
        onnx_proto
        onnx
        libprotobuf
        libprotoc
        torch

LIBRARIES
TARGET_LINK_LIBRARIES(${__NAME}${_SUFFIX} INTERFACE ${LIBNAME})
WHOLE_LIBRARIES
TARGET_LINK_LIBRARIES(${__NAME}${_SUFFIX} INTERFACE -WHOLEARCHIVE:${LIBNAME})