lmstudio-community

/

Meta-Llama-3.1-8B-Instruct-GGUF

like 206

If you update to the newest LM Studio you're going to want to pull the new .gguf yes, it'll work better especially with long context

Is anyone else having issues with the lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF in LM Studio.
I am seeing this for all the 3.1 8B versions.
I get the following error when I try to load in LM Studio:
"llama.cpp error: 'done_getting_tensors: wrong number of tensors; expected 292, got 291'"
I am using LM Studio Version 0.2.28 which is reporting as the current version.

@SWVAI9 you need to grab the newest one on the website:

https://lmstudio.ai/

0.2.29 is available for download there

Thank you very much for your guidance. Working perfectly with the new version.

Hey there, i am still having issues with the "expected 292, got 291" error. I have upgraded to LM studio 0.2.29 but it still wont load the model.
Im on Arch Linux , the normal model 3 works just fine but not the 3.1. Any recommendations?

@SuohLaevatein sounds like you need to update the model, delete the one you have locally and download again

I've tried doing so, but no matter which version of the model i choose , download or redownload there is no difference sadly... Are there maybe some presets or is something cached and is preventing the loading of new models?

When running from LM Studio (latest version 0.2.29) with the updated model this worked for me.

When running from Ollama (latest release version 0.3.0) with the updated model, I was still getting this error.

Ollama Logs

2024-07-30 13:44:40 llm_load_tensors: ggml ctx size =    0.27 MiB
2024-07-30 13:44:40 llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 292, got 291
2024-07-30 13:44:40 llama_load_model_from_file: exception loading model
2024-07-30 13:44:40 terminate called after throwing an instance of 'std::runtime_error'
2024-07-30 13:44:40   what():  done_getting_tensors: wrong number of tensors; expected 292, got 291
2024-07-30 13:44:41 time=2024-07-30T17:44:41.027Z level=ERROR source=sched.go:443 msg="error loading llama server" error="llama runner process has terminated: error loading model: done_getting_tensors: wrong number of tensors; expected 292, got 291\nllama_load_model_from_file: exception loading model"

The fix that worked for me was updating to the prerelease version of Ollama 0.3.1 that was just released. I no longer get this error now.

I tried everything but it's not answering same questions correctly in LM Studio as it does in llama.cpp. I'm using same Q4_K_M.gguf by bartowski with this template
"<|begin_of_text|><|start_header_id|>system<|end_header_id|>
Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>"
So, something is wrong with lm studio templates. I tried all Llama 3, Llama 3 v2. Also this system promt instead of default. In llama.cpp it 100% always answers my reasoning question correctly at temperature "0". (just like LMSYS Chatbot Arena)

@SuohLaevatein

can you check if this thread resolves it for you? https://x.com/LMStudioAI/status/1818646952252244389

I same error that tensor not match when using beta lm studio(I cant send screenshot)

Does this still supports function calling? Normally llama3.1 8b can call functions but that gguf does not enter functions.