Do you use the same PyTorch distribution (i.e. libtorch cmake from /usr/local/lib/python3.x/dist-packages/torch/share/cmake or somesuch)?
In the end, the same libtorch should behave the same way…
Best regards
Thomas