A better way is to get the include_path and lib_path like as given here: https://github.com/pytorch/extension-ffi/issues/8#issuecomment-300812066 21 and then directly linking against the libraries present in lib_path. You can first get it via python, and then set them as env variables of your shell script that is calling cmake. Inside cmake, you can read this env variables.
This way it is guaranteed to not link against a [Lua]Torch installation.