Quanitization request - hermes 405b
If possible I'd like to see this gguf quantized.
Consider this very low priority.
https://huggingface.co/michaelv182/Hermes-3-Llama-3.1-405B-Uncensored-FL2
1bit and/or 2bit are all i'll request as that's all that will fit in 128Gb RAM.
On a side note, i'm seeking good 50B-400B models.
As you can see from https://huggingface.co/michaelv182/Hermes-3-Llama-3.1-405B-Uncensored-FL2/commits/main this seams to be an unmodified clone of my https://huggingface.co/nicoboss/Hermes-3-Llama-3.1-405B-Uncensored model.
We alreay quantized this one under:
- Static quants: https://huggingface.co/mradermacher/Hermes-3-Llama-3.1-405B-Uncensored-GGUF
- Weighted/imatrix quants: https://huggingface.co/mradermacher/Hermes-3-Llama-3.1-405B-Uncensored-i1-GGUF
On a side note, i'm seeking good 50B-400B models.
I highly recommend you use this one instead. This is a reasoner finetuned version of Hermes-3-Llama-3.1-405B-Uncensored and it is in my opinion far more intelligent. Its probably the best model I've ever created togethet with Guilherme34 and is uncensored as well if you use the Dirty D system prompt:
- Static quants: https://huggingface.co/mradermacher/Nature-Reason-1-AGI-GGUF
- Weighted/imatrix quants: https://huggingface.co/mradermacher/Nature-Reason-1-AGI-i1-GGUF
Here some good 100B to 500B base models beside Llama 405B in random order:
- https://huggingface.co/baidu/ERNIE-4.5-VL-424B-A47B-Base-PT
- https://huggingface.co/baidu/ERNIE-4.5-300B-A47B-PT
- https://huggingface.co/MiniMaxAI/MiniMax-M1-80k
- https://huggingface.co/Qwen/Qwen3-235B-A22B
- https://huggingface.co/nvidia/Llama-3_1-Nemotron-Ultra-253B-v1
- https://huggingface.co/rednote-hilab/dots.llm1.inst
- https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct
- https://huggingface.co/databricks/dbrx-instruct
- https://huggingface.co/Snowflake/snowflake-arctic-instruct
- https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1
- https://huggingface.co/ai21labs/AI21-Jamba-Large-1.5
- https://huggingface.co/huihui-ai/DeepSeek-V3-0324-Pruned-Coder-411B
- https://huggingface.co/mistralai/Mistral-Large-Instruct-2407
- https://huggingface.co/tiiuae/falcon-180B
- https://huggingface.co/xai-org/grok-1
As you can see from https://huggingface.co/michaelv182/Hermes-3-Llama-3.1-405B-Uncensored-FL2/commits/main this seams to be an unmodified clone of my https://huggingface.co/nicoboss/Hermes-3-Llama-3.1-405B-Uncensored model.
I did a search before putting in the request and didn't see it. No clue why i didn't see it then.
edit: 2bit is 149Gb... darn too big... :( At least for 128Gb RAM.
On a side note, i'm seeking good 50B-400B models.
I highly recommend you use this one instead. This is a reasoner finetuned version of Hermes-3-Llama-3.1-405B-Uncensored and it is in my opinion far more intelligent.
Here some good 100B to 500B base models beside Llama 405B in random order:
Thanks, i have been grabbing stuff as it crops up but it's easy to miss some. I'll look them over and grab the most promising (and preferably uncensored ones). Thanks a bunch!