This problem is caused by ABI incompatibility. Particularly, the libtorch binaries from PyTorch website are built using the old ABI, which is not binary compatible with ROS libraries in this case.
If you build libtorch from source using the new ABI, then it’s binary compatible with ROS libraries and there won’t be any linking error.