If you use float32 model - the most straightforward will be to use our Vulkan backend. ( tutorials/vulkan_workflow.rst at master · pytorch/tutorials · GitHub 7 )

You might found that not all the operators for your model are yet supported.
Please message them here if you find it.
You can use Vulkan backend from java, specifying Device.VULKAN during model loading
Module module = Module.load("$PATH", Device.VULKAN)

Since 1.8 version Vulkan backend is included in our main gradle artifact (org.pytorch:pytorch_android), that you can skip libtorch building process.

For quantized model is better to follow Android NNAPI way.
You do not need custom build of pytorch and can use our default maven artifact.