mradermacher

/

model_requests

like 59

Hmm, I attempted this back in april already but it failed to convert. I'll try again and see what happens.

Yeah, it seems the model is broken: check_tensor_dims: tensor 'token_embd.weight' has wrong shape; expected 4096, 32002, got 6144, 32000, 1, 1

Does it actually work with transformers? I would expect not. Possibly it is some vocabulary problem.

OK, I saw somebody hosted this on poe.com, but I don't have a rig to run it without quantize so I didn't try myself. But I will try to ask that person how he hosts it, if there's some update I will come back here. Thanks again!

The most common issues are lkaxer checks in transformers, and sometimes there are multiple vocabularies, and transformers chosing the working one. Without a fix in the model itself, it's unlikely for llama to be able to quantize it.

@DazzlingXeno i removed the other repo - this is the discussion of this model, see the error message, above.

Cheers