DevQuasar/all-hands.openhands-lm-32b-v0.1-ep3-GGUF
Text Generation
•
Updated
DevQuasar
community
Verified
AI & ML interests
Open-Source LLMs, Local AI Projects: https://pypi.org/project/llm-predictive-router/
Recent Activity
View all activity
DevQuasar's activity
csabakecskemeti
updated
a
model
2 minutes ago
csabakecskemeti
updated
a
model
4 minutes ago
csabakecskemeti
published
a
model
4 minutes ago
Need to rerun conversion
1
#1 opened about 6 hours ago
by
sydneyfong
csabakecskemeti
published
a
model
about 6 hours ago
csabakecskemeti
updated
a
model
about 6 hours ago
csabakecskemeti
published
a
model
about 6 hours ago
csabakecskemeti
updated
2
models
about 6 hours ago
Update model metadata to set pipeline tag to the new `text-ranking`
#1 opened about 13 hours ago
by
tomaarsen
Update model metadata to set pipeline tag to the new `text-ranking`
1
#1 opened about 13 hours ago
by
tomaarsen
Update model metadata to set pipeline tag to the new `text-ranking`
#1 opened about 13 hours ago
by
tomaarsen
csabakecskemeti
updated
a
model
about 7 hours ago
Saw you in the Level1Techs video!
1
#1 opened about 7 hours ago
by
Mdubbya
csabakecskemeti
updated
a
model
about 14 hours ago
csabakecskemeti
posted
an
update
10 days ago
Post
3331
I'm collecting llama-bench results for inference with a llama 3.1 8B q4 and q8 reference models on varoius GPUs. The results are average of 5 executions.
The system varies (different motherboard and CPU ... but that probably that has little effect on the inference performance).
https://devquasar.com/gpu-gguf-inference-comparison/
the exact models user are in the page
I'd welcome results from other GPUs is you have access do anything else you've need in the post. Hopefully this is useful information everyone.
The system varies (different motherboard and CPU ... but that probably that has little effect on the inference performance).
https://devquasar.com/gpu-gguf-inference-comparison/
the exact models user are in the page
I'd welcome results from other GPUs is you have access do anything else you've need in the post. Hopefully this is useful information everyone.
csabakecskemeti
posted
an
update
12 days ago
Post
2360
Managed to get my hands on a 5090FE, it's beefy
| llama 8B Q8_0 | 7.95 GiB | 8.03 B | CUDA | 99 | pp512 | 12207.44 ± 481.67 |
| llama 8B Q8_0 | 7.95 GiB | 8.03 B | CUDA | 99 | tg128 | 143.18 ± 0.18 |
Comparison with others GPUs
http://devquasar.com/gpu-gguf-inference-comparison/
| llama 8B Q8_0 | 7.95 GiB | 8.03 B | CUDA | 99 | pp512 | 12207.44 ± 481.67 |
| llama 8B Q8_0 | 7.95 GiB | 8.03 B | CUDA | 99 | tg128 | 143.18 ± 0.18 |
Comparison with others GPUs
http://devquasar.com/gpu-gguf-inference-comparison/
csabakecskemeti
posted
an
update
15 days ago
Post
1799
GTC new model announcement now from Nvidia
nvidia/Llama-3_3-Nemotron-Super-49B-v1
GGUFs:
DevQuasar/nvidia.Llama-3_3-Nemotron-Super-49B-v1-GGUF
Enjoy!
nvidia/Llama-3_3-Nemotron-Super-49B-v1
GGUFs:
DevQuasar/nvidia.Llama-3_3-Nemotron-Super-49B-v1-GGUF
Enjoy!
csabakecskemeti
posted
an
update
20 days ago
Post
565
Cohere Command-a Q2 quant
DevQuasar/CohereForAI.c4ai-command-a-03-2025-GGUF
6.7t/s on a 3gpu setup (4080 + 2x3090)
(q3, q4 currently uploading)
DevQuasar/CohereForAI.c4ai-command-a-03-2025-GGUF
6.7t/s on a 3gpu setup (4080 + 2x3090)
(q3, q4 currently uploading)
csabakecskemeti
posted
an
update
26 days ago
Post
822
Fine tuning on the edge. Pushing the MI100 to it's limits.
QWQ-32B 4bit QLORA fine tuning
VRAM usage 31.498G/31.984G :D
QWQ-32B 4bit QLORA fine tuning
VRAM usage 31.498G/31.984G :D
-
- 4 replies