Hello. I have a pretrained RoBERTa model on fairseq, which contains dict.txt, model.pt, sentencepiece.bpe.model.
I have found a way to convert a fairseq checkpoint to huggingface format in https://github.com/huggingface/transformers/blob/master/src/transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py 36
Howerver, I couldn’t find a similar method to convert the tokenizer in fairseq sentencepiece.bpe.model to huggingface’s format.
Is there any existing solution? Or do I have to convert it by myself?
Thanks.