When I used the add_special_tokens function in the BertTokenizer, it assigns 2 different tokens with the same ID. Is this done on purpose?
We would need a bit more information to understand the issue. A reproducible code example would be even better.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
