IndexError: Out of range: piece id is out of range
Using Exllama & Exllamav2 I get the following error with this model in the Text-generation-webui
Traceback (most recent call last):
File "C:\AI\text-generation-webui\modules\text_generation.py", line 384, in generate_reply_custom
for reply in shared.model.generate_with_streaming(question, state):
File "C:\AI\text-generation-webui\modules\exllamav2.py", line 119, in generate_with_streaming
decoded_text = self.tokenizer.decode(ids[:, initial_len:])[0]
File "C:\AI\text-generation-webui\installer_files\env\lib\site-packages\exllamav2\tokenizer.py", line 92, in decode
texts.append(self.tokenizer.Decode(seq))
File "C:\AI\text-generation-webui\installer_files\env\lib\site-packages\sentencepiece_init_.py", line 780, in Decode
return self.DecodeIds(input)
File "C:\AI\text-generation-webui\installer_files\env\lib\site-packages\sentencepiece_init.py", line 337, in _DecodeIds
return _sentencepiece.SentencePieceProcessor__DecodeIds(self, ids)
IndexError: Out of range: piece id is out of range.
Output generated in 4.94 seconds (53.20 tokens/s, 263 tokens, context 1071, seed 587620518)
Same issue here, the error is coming from the sentencepiece library, perhaps something with the vocabulary size isn't set correctly.
Is that related to this at all? Traceback (most recent call last):
File "c:\Users\Scott\VSC_Source\LLM_DB\EXLLAMAv2y production.py", line 181, in
output = generator.generate_simple(prompt,settings,max_new_tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Scott\anaconda3\envs\ELLAMA\Lib\site-packages\exllamav2\generator\base.py", line 60, in generate_simple
token, _ = ExLlamaV2Sampler.sample(logits, gen_settings, self.sequence_ids, random.random())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Scott\anaconda3\envs\ELLAMA\Lib\site-packages\exllamav2\generator\sampler.py", line 63, in sample
if settings.token_bias is not None: logits += settings.token_bias
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The size of tensor a (32032) must match the size of tensor b (32002) at non-singleton dimension 1