Bhack:

XNNPACK


Thanks for those links. I couldn’t figure out how to pass xnnpack in the list of delegates in Python but I’ve rebuilt TF with tflite_with_xnnpack=true. With float16 quantization I’m able to get comparable performance. Int8 quantization didn’t yield any performance increase in my basic test. Though accuracy did seem to suffer but that is likely due to the fact they were trained on super small training set. I think I’ve got enough to go on now though - thank you for your help!