Honestly, my current full model is a little complex to refactor into colab now as it is written into bits of classes. It was saved into SavedModels, loaded back for outputs verification, then converted using tf.lite.TFLiteConverter.from_saved_model(). There were some issues regarding space optimisation/quantisation as well, but I will probably explore after I at least get the model running on mobile.
I was trying to get around the issue of tflite’s currently unsupported initialisation of vocab.txt into lookup tables specifically for ‘hashtablev2’ ops. This is required for Bert’s wordpiece tokeniser. So i went and rewritten the lookup table ops parts, replace it with tflite conversion friendly code. It’s a python dict that initialises vocabs into a normal TensorArray that tflite is able to convert and operate on.
For the Bert’s wordpiece tokeniser,  there is a preprocessing step which they referred as Basic Tokeniser. This BasicTokeniser is largely what’s on the gist model i shared earlier, apart from a missing ‘nfd’ normalisation. I have thought about it if i can rewrite the tf_text ops into normal tf ops or even on mobile before it enters the model (for ndf case.). But i found that it’s non-trivial to rewrite some regex delimited splits ops. That’s why i’m stuck with using some of the tf_text ops for now.
I did briefly considered if tflite support library would help, but I’m looking for an eventual solution that works with iOS as well, so i didn’t dig further.
I felt i’m so close to making it work on mobile by just having a working binary, that’s why i’m still working around this approach.