There is a similar issue on Github right now: Android API to run model on Vulkan backend · Issue #57893 · pytorch/pytorch · GitHub 14
In that thread I linked a MobileNet V2 model that you can try. If that works, I would suggest re-building pytorch with USE_VULKAN=ON as mentioned in the thread.