Hello there, interesting project going here.
I just finished to test an integration btw Unity and Libtorch following this recipe: https://www.goodai.com/neural-networks-in-unity-using-native-libraries/ 25.
Then, I would suggest to build an external library similar to the link and then just import the built library into UE4. I would try this also for a project later.
Let me know if it works.
Best