We haven’t yet added ALBERT and  DeBERTa-v2 in Flax yet. ALBERT should be easy to add though.
ELECTRA (which is already available in Flax) is also a good option since it’s much more sample efficient. But we don’t have a pre-training script for electra yet.