Error loading model
#3
by
ThamaluM
- opened
error loading model: llama.cpp: tensor 'layers.0.attention.wk.weight' has wrong shape; expected 8192 x 8192, got 8192 x 1024
Got this error. Loaded using llama-cpp-python in Linux. Python3.11, llama-cpp-python 0.1.77
(model:
llama-2-70b.ggmlv3.q2_K.bin)
Thanks for the answer.. I found the following instruction and it worked.
Loading llama-2 70b
Llama2 70b must set the n_gqa parameter (grouped-query attention factor) to 8 when loading:llm = Llama(model_path="./models/70B/ggml-model.bin", n_gqa=8)
Hello, it says that it is valid with "langchain," but when I try to use it with "RetrievalQA," it gives the following error:
ValidationError: 1 validation error for LLMChain
llm value is not a valid dict (type=type_error.dict)