302 19 352

John Leimgruber III PRO

ubergarm

https://www.paypal.com/donate/?hosted_button_id=HU59345BZVSUA

AI & ML interests

Open LLMs and Astrophotography image processing.

Recent Activity

new activity about 1 hour ago

unsloth/Qwen3-Coder-Next-GGUF:Feb 4: Qwen3-Coder-Next GGUFs reuploaded - much better outputs! (Still in progress)

liked a model about 2 hours ago

DevQuasar/Qwen.Qwen3-Coder-Next-GGUF

liked a model about 2 hours ago

bartowski/Qwen_Qwen3-Coder-Next-GGUF

View all activity

Organizations

New activity in unsloth/Qwen3-Coder-Next-GGUF about 1 hour ago

Feb 4: Qwen3-Coder-Next GGUFs reuploaded - much better outputs! (Still in progress)

🤗 👍 11

#5 opened 1 day ago by

danielhanchen

New activity in moonshotai/Kimi-K2.5 about 6 hours ago

performance of quantized models

#61 opened about 12 hours ago by

darvec

New activity in unsloth/Kimi-K2.5-GGUF about 7 hours ago

Original model is just 600gb

#2 opened 8 days ago by

jpsequeira

New activity in ubergarm/Kimi-K2-Instruct-GGUF about 8 hours ago

A question

#9 opened 8 days ago by

toyelit736

New activity in noctrex/GLM-4.7-Flash-MXFP4_MOE-GGUF 3 days ago

Feedback from running in LM Studio 0.39.3 with v1.103.2 of llama.cpp

#1 opened 16 days ago by

spanspek

New activity in ubergarm/Kimi-K2-Thinking-GGUF 3 days ago

moonshotai/Kimi-K2.5

🔥 3

#16 opened 9 days ago by

gghfez

New activity in ubergarm/Kimi-K2-Instruct-GGUF 3 days ago

Kimi 2.5

#10 opened 7 days ago by

alain401

New activity in ubergarm/GLM-4.7-GGUF 3 days ago

Stable run on 2x RTX 5090 and 2 Xeon E5 2696 V4 and DDR4 with ik_llama.cpp - 6.1 t/s on IQ4_K and 5.1 t/s on IQ5_K, opencode works with this

👍 1

#5 opened about 1 month ago by

martossien

New activity in ubergarm/Qwen3-30B-A3B-Instruct-2507-GGUF 3 days ago

Imatrix calibration data

#3 opened 7 days ago by

bobchenyx

New activity in ubergarm/GLM-4.7-Flash-GGUF 3 days ago

Reporting successful run of MXFP4 using ik_llama cpp

🔥 🚀 2

#4 opened 10 days ago by

nimishchaudhari

New activity in ubergarm/GLM-4.7-Flash-GGUF 15 days ago

question about mxfp4

#3 opened 15 days ago by

koifish12

New activity in zai-org/GLM-4.7-Flash 15 days ago

Why does the KV cache occupy so much GPU memory?

#21 opened 16 days ago by

yyg201708

New activity in ubergarm/GLM-4.7-Flash-GGUF 15 days ago

Re-cooking imatrix and quants with updated ik/llama.cpp PR

🚀 1

#1 opened 16 days ago by

ubergarm

New activity in zai-org/GLM-4.7-Flash 16 days ago

Cannot run vLLM on DGX Spark: ImportError: libcudart.so.12

#18 opened 17 days ago by

yyg201708

Performance Discussion

👀 2

#1 opened 17 days ago by

IndenScale

Enormous KV-cache size?

👍 ➕ 6

#3 opened 17 days ago by

nephepritou

New activity in ArtusDev/requests-exl 18 days ago

[QUANTING UPDATE]

❤️ 👍 4

#28 opened 21 days ago by

ArtusDev

New activity in ubergarm/Devstral-Small-2-24B-Instruct-2512-GGUF 18 days ago

Mistral 3 large wuant

👍 1

#1 opened 19 days ago by

facedwithahug

New activity in ubergarm/DeepSeek-V3.2-Speciale-GGUF 18 days ago

QuIP - 2 bit quantised as good as 16 bit

#5 opened 23 days ago by

infinityai

New activity in msievers/gemma-3-1b-it-qat-q4_0-gguf 22 days ago

Thanks for sharing your work!

❤️ 2

#1 opened 23 days ago by

ubergarm

John Leimgruber III PRO

AI & ML interests

Recent Activity

Organizations

ubergarm's activity

Feb 4: Qwen3-Coder-Next GGUFs reuploaded - much better outputs! (Still in progress)

performance of quantized models

Original model is just 600gb

A question

Feedback from running in LM Studio 0.39.3 with v1.103.2 of llama.cpp

moonshotai/Kimi-K2.5

Kimi 2.5

Stable run on 2x RTX 5090 and 2 Xeon E5 2696 V4 and DDR4 with ik_llama.cpp - 6.1 t/s on IQ4_K and 5.1 t/s on IQ5_K, opencode works with this

Imatrix calibration data

Reporting successful run of MXFP4 using ik_llama cpp

question about mxfp4

Why does the KV cache occupy so much GPU memory?

Re-cooking imatrix and quants with updated ik/llama.cpp PR

Cannot run vLLM on DGX Spark: ImportError: libcudart.so.12

Performance Discussion

Enormous KV-cache size?

[QUANTING UPDATE]

Mistral 3 large wuant

QuIP - 2 bit quantised as good as 16 bit

Thanks for sharing your work!