Machine Learning, Deep Learning, Naturallanguageprocessing.
scenarios. If you want to do custom vocabularies, WordPiece is not actually open sourced but there are other similar methods (such as, confusingly, SentencePiece that sounds like a different layer of abstraction but isn’t). See the value of WordPieces and discussion of WordPiece vs SentencePiece