  We present a subword regularization method for WordPiece, which uses a
maximum matching algorithm for tokenization. The proposed method,
MaxMatch-Dropout, randomly drops words in a search using the maximum matching
algorithm. It realizes finetuning with subword regularization for popular
pretrained language models such as BERT-base. The experimental results
demonstrate that MaxMatch-Dropout improves the performance of text
classification and machine translation tasks as well as other subword
regularization methods. Moreover, we provide a comparative analysis of subword
regularization methods: subword regularization with SentencePiece (Unigram),
BPE-Dropout, and MaxMatch-Dropout.

Computation and Language