Coversion between tokenizers
I want to convert GPT2 tokens to BERT tokens. Is there any API that can directly convert between the Tokenizers?
GPT-2 and BERT have very different tokenization mechanism. What do you mean by "convert between the tokenizers"? What do you want to do?<|||||>I want to use Bert as encoder and GPT2 as decoder. Then I want to evaluate the generated text with another Bert as discriminator (like the technique [here](https://arxiv.org/pdf/1703.00955.pdf)). I want the decoder to generate text just based on the context vector (refer to the above link). So I don't think I can use EncoderDecoderModel (am I right?)