Slight hack to fix the decoder error - I tried editing ProphetNetTokenizer itself, specifically the load_vocab method and changed the encoding to ‘latin-1’.

image998×420 40 KB

The edit is made here 2, I did it locally so I’m not sure whether this could be done for Colab notebooks.
This stackoverflow page 3 gave some suggestions for a fix, of which the latin-1 encoding seemed to work.
Caveat: I haven’t tried using the tokenizer any further than just getting the ProphetNetTokenizer.from_pretrained() step working.
It gave a warning of "Special tokens have been added in the vocabulary, make sure the associated word embedding are fine-tuned or trained.", so there may be some extra steps needed after.
The overall issue is something that can’t be encoded to utf-8 in the vocabulary used. Hope this helps