Thanks for sharing your experience!
I typically work on image regression models that consume quite heavy computation.
While waiting for the official mobile GPU support, I guess your suggestion is the most practical way at the time being.
By the way, I found that GPU & NNAPI acceleration might be possible if we use onnx runtime.

  
    
    
    Network inference on GPU under Android Mobile
  
  
    I recently read about ONNX runtime and it might be an alternative. 
PyTorch could export models to ONNX format and run the model with ONNX runtime. 
https://pytorch.org/docs/stable/onnx.html 
According to the onnxruntime, NNAPI is supported including CPU and GPU inference. 
I didn’t try it but looks like it is the only Android gpu-enabled accelerator for now.
  


I will try both ways