For building LibTorch with make, no, and I’d think you’d see limited interest in supporting building libtorch without cmake.
Now, using a prebuilt libtorch in other programs is pretty straightforward even without CMake, I have written a trivial C++ program running a model on an image and displaying (the sample program shown on my slides  https://lernapparat.de/pytorch-jit-android/ 31) and compiled that with
g++ run.cpp -I /usr/local/lib/python3.6/dist-packages/torch/lib/include/  -L /usr/local/lib/python3.6/dist-packages/torch/lib/ -ltorch -lcaffe2 -lc10 -lpthread -lX11

(where the pthread possibly and X11certainly are CImg dependencies rather than those of PyTorch).
Best regards
Thomas