@Mr-G
The latest published nightlies are 1.8.0-SNAPSHOT
300ms and 700ms are very off what we are seeing in our benchmarks.
In the HelloWorld tutorial model preparation is naive and missing performance optimizations.
We have a separate tutorial for mobile optimizations: Pytorch Mobile Performance Recipes — PyTorch Tutorials 1.8.0 documentation 4
The main steps will be to use optimie_for_mobile for model preparation:
import torch
import torchvision
from torch.utils.mobile_optimizer import optimize_for_mobile

model = torchvision.models.mobilenet_v2(pretrained=True)
model.eval()

torchscript_model = torch.jit.script(model)
torchscript_model_optimized = optimize_for_mobile(torchscript_model)
torch.jit.save(torchscript_model_optimized, "model.pt")

and to use channels_last memory format input tensor:
Tensor inputNHWC = Tensor.fromBlob(dataNHWC, inputShape, MemoryFormat.CHANNELS_LAST);

ChannelsLast memory format is not integrated with TensorImageUtils yet.
Once [android][utils] Support ChannelsLast in TensorImageUtils by IvanKobzarev · Pull Request #48990 · pytorch/pytorch · GitHub PR will be landed - it will be available as argument for it:
    TensorImageUtils.imageYUV420CenterCropToFloatBuffer(
        image.getImage(),
        rotationDegrees,
        TENSOR_WIDTH,
        TENSOR_HEIGHT,
        TensorImageUtils.TORCHVISION_NORM_MEAN_RGB,
        TensorImageUtils.TORCHVISION_NORM_STD_RGB,
        mInputTensorBuffer,
        0,
        MemoryFormat.CHANNELS_LAST);