@shyney7
Once you tell cmake to do CMAKE_TOOLCHAIN_FILE, cmake enter into cross-compiling mode, your library search root will be set here:
SET(CMAKE_FIND_ROOT_PATH ${QNX_TARGET})
This will override your CMAKE_PREFIX_PATH.
I am not sure how many other things will be override if specifying toolchain file, you can try specify your torch root in toolchain file, something like SET(CMAKE_FIND_ROOT_PATH $(QNX_TARGET), $(TORCH_ROOT)). or you can put torch root into system root and then set SET(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE BOTH)