Fix: encoder and decoder in tokenizer
#13
by
JingweiZuo
- opened
Hi!
When evaluating the RWKV-v5-Eagle-7B-HF model, I found the errors as shown below. This is mostly caused by the tokenizer. In the code, the encoder and decoder are reversed, as discussed here https://huggingface.co/RWKV/v5-Eagle-7B-HF/discussions/9#65d4dc35f9cbfa798c4be4b3
This PR is raised to fix this issue.
Thanks,