The good thing about training a tagger model based on ALBERT is it will likely be supported in TFLite. TensorFlow Hub also provides MobileBERT checkpoints, another model supported in TFLite.
Tagging @lgusm for more details.