invalid magic number 00000000
from llama_cpp import Llama
llm = Llama(model_path='./llama-2-70b-chat.Q5_K_S.gguf', n_ctx=2048, n_gpu_layers=10)
ggml_init_cublas: found 4 CUDA devices:
Device 0: NVIDIA GeForce GTX TITAN X, compute capability 5.2
Device 1: NVIDIA GeForce GTX TITAN X, compute capability 5.2
Device 2: NVIDIA GeForce GTX TITAN X, compute capability 5.2
Device 3: NVIDIA GeForce GTX TITAN X, compute capability 5.2
gguf_init_from_file: invalid magic number 00000000
error loading model: llama_model_loader: failed to load model from ./llama-2-70b-chat.Q5_K_S.gguf
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "", line 1, in
File "/home/developer/mambaforge/envs/CodeLlama/lib/python3.10/site-packages/llama_cpp/llama.py", line 340, in init
assert self.model is not None
AssertionError
sha256sum matches.
same issue here
same issue
the same
Same issue with llama.cpp
$ ./main -m ./models/llama-2-70b-chat.Q5_K_S.gguf -n 256 --repeat_penalty 1.0 --color -i -r "User:"
Log start
main: build = 2240 (4cb4d8b2)
main: built with MSVC 19.39.33520.0 for x64
main: seed = 1709215679
gguf_init_from_file: invalid magic characters ''
llama_model_load: error loading model: llama_model_loader: failed to load model from ./models/llama-2-70b-chat.Q5_K_S.gguf
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model './models/llama-2-70b-chat.Q5_K_S.gguf'
main: error: unable to load model
Confirmed same issue for me
gguf_init_from_file: invalid magic characters ''
same issue for me. after splitting and trying to merge or load
I had the same problem:Log start
main: build = 2690 (3b8f1ec4)
main: built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for x86_64-linux-gnu
main: seed = 1713578690
gguf_init_from_file: invalid magic characters ''
llama_model_load: error loading model: llama_model_loader: failed to load model from ./models/7B/ggml-model-q4_0.gguf
llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model './models/7B/ggml-model-q4_0.gguf'
main: error: unable to load model
@lcc666 ggml models do not work with llama.cpp or anything that uses it(text generation web ui, ctransformers, ollama and more). Use gguf variants, they are higher quality and better.